How do I compile a Hive UDF
12,710
The following works for me, but I suspect the details will vary depending on your installation and what your source code does:
export CLASSPATH=/usr/lib/hive/lib/hive-exec-0.9.0.15.jar:/usr/lib/hadoop/hadoop-core.jar
Author by
nickponline
Updated on June 08, 2022Comments
-
nickponline almost 2 years
I am trying to compile this UDF:
package com.dataminelab.hive.udf; import org.apache.hadoop.hive.ql.exec.UDF; import org.apache.hadoop.io.Text; import java.security.*; /** * Calculate md5 of the string */ public final class Md5 extends UDF { public Text evaluate(final Text s) { if (s == null) { return null; } try { MessageDigest md = MessageDigest.getInstance("MD5"); md.update(s.toString().getBytes()); byte[] md5hash = md.digest(); StringBuilder builder = new StringBuilder(); for (byte b : md5hash) { builder.append(Integer.toString((b & 0xff) + 0x100, 16).substring(1)); } return new Text(builder.toString()); } catch (NoSuchAlgorithmException nsae) { System.out.println("Cannot find digest algorithm"); System.exit(1); } return null; } }
Trying to compile with:
javac Md5.java
But I get:
Md5.java:2: package org.apache.hadoop.hive.ql.exec does not exist import org.apache.hadoop.hive.ql.exec.UDF; ^ Md5.java:3: package org.apache.hadoop.io does not exist import org.apache.hadoop.io.Text;
I assume these are in a jar file somewhere but I'm not sure where hadoop install them to so I can't add them to my classpath. Does anyone know the default location or how to find out?
-
xenodevil about 5 yearsOn a HortonWorks distribution, I had to use
hadoop-common.jar
instead ofhadoop-core.jar