[문제 해결] sqoop import export시 java.lang.ClassNotFoundException

문제 원인

필요한 jar 파일이 없어서

2023-01-12 15:05:08,700 [main] ERROR org.apache.sqoop.tool.ImportTool  - Import failed: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
	at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:50)
	at org.apache.sqoop.hive.HiveImport.getHiveArgs(HiveImport.java:337)
	at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:324)
	at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:281)
	at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:213)
	at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:349)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:549)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:633)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:251)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:260)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:269)
	at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:165)
	at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:155)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
	at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:47)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:412)
	at org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:54)
	at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:225)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
	at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:219)
	at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:155)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
	at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:143)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:44)
	... 33 more

 

문제 해결

sharelib에 접속해 jar 파일을 반영해줘야하므로 우선 ssh로 서버 접속을 한다

root로 접속

#hdfs 권한 계정으로 바꾼다
su - hdfs

# 필요한 lib가 있는 곳으로 가 cp를 해준다
cd /usr/lib/hive/lib
cp hive-exec-3.1.2.jar /home/oozie

cd /home/oozie

# 적절하게 권한 변경
chown hdfs:oozie hive-exec-3.1.2.jar

su - oozie

# 반드시 hdfs 명령어를 사용해서 sharelib 경로에 넣어준다
hdfs dfs -put hive-exec-3.1.2.jar /user/oozie/share/lib/lib_20221107183124/sqoop/

 

이후에 우지 sharelib 갱신 반영을 반드시 해줘야 한다

갱신 반영 방법 포스트 

https://about2weeks.tistory.com/37