Yarn不接受任务的问题
Yarn不接受任务的问题
现象描述
提交Spark任务时,报如下错误。
Exception in thread "main" org.apache.spark.SparkException: Yarn application already ended,might be killed or not able to launch application master. at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApp(YarnClientSchedulerBackend.scala:111) at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:87) at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:141) at org.apache.spark.SparkContext.<init>(SparkContext.scala:323) at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:28) at org.apache.spark.examples.SparkPi.main(SparkPi.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:332) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:79) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
可能原因
Yarn出现问题,不能支持Spark on Yarn。
定位思路
无。
处理步骤
建议观察Application Master的Web UI报错信息,并联系Yarn相关人员定位。