出现Not attempting to re-login since the last re-login was attempted less than 600 seconds before异常

现象描述

安全模式下客户端提交代码时，Driver端日志报错如下。

14/09/20 15:29:31 INFO SparkUI: Started SparkUI at http://linux-18:23000
14/09/20 15:29:31 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/09/20 15:29:35 WARN UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
14/09/20 15:29:38 WARN UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
14/09/20 15:29:43 WARN UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
14/09/20 15:29:46 WARN UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.

或者报如下错误：

Caused by: GSSException: No valid credentials provided (Mechanism level: Clock skew too great (37))
        at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:770)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
        at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
        ... 80 more
Caused by: KrbException: Clock skew too great (37)
        at sun.security.krb5.KrbKdcRep.check(KrbKdcRep.java:88)
        at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:87)
        at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:259)
        at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:270)
        at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:302)
        at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:120)
        at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458)
        at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693)
        ... 83 more

可能原因

Kerberos认证时会对客户端节点和服务端节点上的时间进行认证，如果时间差超过一定阈值（默认为5分钟），则拒绝客户端的连接。

定位思路

使用date命令分别查看客户端以及服务端的日期及时间，如果两者相差5分钟以上，则为时间不同步引起的问题。

处理步骤

1. 在客户端安装ntp服务进行时间同步。
2. 手动修改客户端上的时间，使之与服务端一致。