hive sql有个比较强大的功能就是可以使用外界脚本来完成map/reduce。用法如下：
TRANSFORM(….) USING ‘….’ AS (….)。
USING里面可以使用php脚本来处理，具体使用方法看下面这个例子。
①、建一个表：
<code>hive> CREATE TABLE <span>`member`</span>( > <span>`id`</span><span>int</span>, > <span>`user_name`</span><span>string</span>, > <span>`passwd`</span><span>string</span> > ) > row <span>format</span> delimited > fields terminated by <span>"\t"</span><span>//这句很必要，不然下面的文件导入会导致所有的值都变成null </span> > stored as textfile;</code>
②、准备如下数据 /tmp/member.dat
<code>1 zhangsan zs10242 lisi ls19913 wangwu ww20014 liumang lm12345 linxing lx1990</code>
③、导入数据
<code>hive> <span>load</span> data <span>local</span> inpath <span>'/tmp/member.dat'</span> into <span>table</span> member;</code>
可以看到：
<code>hive> <span>select</span> * <span>from</span> member;OK<span>1</span> zhangsan zs1024<span>2</span> lisi ls1991<span>3</span> wangwu ww2001<span>4</span> liumang lm1234<span>5</span> linxing lx1990</code>
数据已经准备好了 ，现在我们相对第三列做一次md5加密。我们使用php脚本来处理，php脚本（/tmp/changePasswd.php）代码如下：
<code><span><span><?php</span><span>while</span>(!feof(STDIN)){ <span>$line</span> = rtrim(fgets(STDIN), <span>"\n"</span>); <span>//逐行读取</span><span>if</span> (<span>empty</span>(<span>$line</span>)) <span>continue</span>; <span>//空，跳出当次循环</span><span>$data</span> = explode(<span>"\t"</span>, <span>$line</span>); <span>//将切分出来的数组保存起来，下面判断使用</span><span>foreach</span>(<span>$data</span><span><i>本文@来#源gaodai$ma#com搞$$代**码网</i><strong>搞代gaodaima码</strong>as</span> &<span>$val</span>){ <span>$val</span> = md5(<span>$val</span>); } <span>unset</span>(<span>$val</span>); <span>echo</span> implode(<span>"\t"</span>, <span>$data</span>) . <span>"\n"</span>;}</span></code>
④、将php脚本加入到hive中。
<code>hive> <span>add</span> file /tmp/changePasswd<span>.php</span><span>;</span></code>
⑤、使用php脚本来执行map/reduce
<code>hive> insert overwrite table member > select TRANSFORM(<span>`i</span>d<span>`,</span><span>`u</span>ser_name<span>`,</span><span>`p</span>asswd<span>`)</span> using <span>"/usr/bin/php changePasswd.php"</span> > as (<span>`i</span>d<span>`,</span><span>`u</span>ser_name<span>`,</span><span>`p</span>asswd<span>`)</span> from member<span>;</span></code>
最后，我们可以看到，passwd那一列的数据已经改变：
<code>hive> <span>select</span> * <span>from</span> member;OK<span>1</span> zhangsan d03eed89429cc3006cc279322c2800c5<span>2</span> lisi <span>063401506</span>c9d9f0e49a706e3779b7428<span>3</span> wangwu ac5a8109dbbb46c9f69ffd5fc93c11f8<span>4</span> liumang fda8b97fd723bdbf6a754812b5ecec27<span>5</span> linxing <span>4035378</span>ace8936e93d95aa77e7e224d4</code>
版权声明：本文为博主原创文章，转载请注明出处。
以上就介绍了hive 使用php脚本来完成map/reduce，包括了方面的内容，希望对PHP教程有兴趣的朋友有所帮助。
搞代码网（gaodaima.com）提供的所有资源部分来自互联网，如果有侵犯您的版权或其他权益，请说明详细缘由并提供版权或权益证明然后发送到邮箱[email protected]，我们会在看到邮件的第一时间内为您处理，或直接联系QQ：872152909。本网站采用BY-NC-SA协议进行授权
转载请注明原文链接：hive 使用php脚本来完成map/reduce
转载请注明原文链接：hive 使用php脚本来完成map/reduce
