Tuesday, March 10, 2015

Custom Java UDA implementation in Apache Drill 0.7



CUSTOM JAVA UDA IMPLEMENTATION IN APACHE DRILL


PREREQUISITE
The following software is to be installed.
  • Ubuntu 13.10
  • Java 1.7
  • Maven 3.0.4
  • Git
  • Eclipse Kepler with Maven Plugins
  • Apache Drill 0.7.0 (Installed in an embedded mode)
DOWNLOAD AND COMPILE SOURCE CODE OF DRILL
Download and build the source code of Apache Drill 0.7.0 and 0.8.0 from git.
  • Apache Drill 0.8.0
mvn clean install
  • Apache Drill 0.7.0
mvn clean install
SET UP DRILL DEVELOPMENT ENVIRONMENT IN ECLIPSE
  • Open Eclipse and Right click on Package Explorer and select Import.
  • Navigate to Maven and select Existing Maven Projects to Import the compiled source code of Drill 0.7.0
  • In case of any error in project, fix the errors.
  • Create a new Maven project (e.g., groupID - com.udf, artifactID - udf, version - 0.0.1-SNAPSHOT and package - jar)
  • Add the following lines in pom.xml file.
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.apache.drill.exec</groupId>
<artifactId>drill-java-exec</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</dependencyManagement>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>2.4</version>
<executions>
<execution>
<id>attach-sources</id>
<!-- <phase>verify</phase> -->
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
  • Create a Java class and develop your UDA
CLEAN, COMPILE AND BUILD JAR WITH DEPENDENCIES
  • To Clean - Right click on your Maven project and select Run as -> Maven Clean
  • Goto Project in Eclipse and select clean
  • To compile and build Jar - Right click on your Maven project and select Run as -> Maven Install.
  • Now the jar file would be available in target folder under your project.
DEPLOY THE CUSTOM JAR IN APACHE DRILL
  • Stop the drillbit service
  • Copy the jar files (yourprojectname-0.0.1-SNAPSHOT-sources.jar and yourprojectname-0.0.1-SNAPSHOT.jar) from the target directory of your project and paste to $DRILL_HOME/jars/classb/ location.
  • Start the drillbit service.
  • Make sure that your jar is loaded successfully into the Drill by verifying the drill log files.
TEST YOUR CUSTOM UDA FUNCTION IN WEB UI OF THE APACHE DRILL

1 comments:

Post a Comment