spark运行写入doris的任务报错park.DorisStreamLoad: Doris BE’s response cannot map to schema
在CDH的集群上跑spark任务,运行报错:
ERROR spark.DorisStreamLoad: Doris BE’s response cannot map to schema. res: “Access denied for user ‘default_cluster:hdfs@10.0.10.101’ (using password: YES)”
org.apache.doris.shaded.com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of org.apache.doris.spark.rest.models.BackendV2
(although at least one Creator exists): no String-argument constructor/factory method to deserialize from String value (‘Access denied for user ‘default_cluster:hdfs@10.0.10.101′ (using password: YES)’)
at [Source: (String)””Access denied for user ‘default_cluster:hdfs@10.0.10.101’
查了一下,连接doris的用户名和密码没错。后来发现,spark-submit的参数–conf spark.user=hdfs 影响了,把这个参数去掉就正常了。
关注公众号“大模型全栈程序员”回复“大数据面试”获取800页左右大数据面试宝典 ,回复“大数据”获取多本大数据电子书