Monday, May 25, 2015

Oozie job failed "Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster"

Problem: 
Oozie launcher mapper failed with the below error:
Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

Solution:
The above error indicates that yarn classpath does not include the jars containing the required MRAppMaster class on the Oozie server / client.  You will need to setup Yarn application client libraries with updated yarn-site.xml containing correct yarn.application.classpath. Once you have ensured that yarn client & yarn-site.xml is setup correctly, restart Oozie server so that it can pick up the changes. This is not problem with Oozie (can be with any similar utility), but oozie fails due to incorrect configuration.
Example:
<property>
 <name>yarn.application.classpath</name>
 <value>$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,$HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,$HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,$HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*,$USS_HOME/*,$USS_CONF</value>
</property>

where all the above variables are setup correctly using the corresponding <daemon>-config.sh scripts. Ex: hdfs-config.sh

1 comment: