2014-10-15 11:37:08,098 INFO org.apache.hadoop.ha.ActiveStandbyElector: Trying to re-establish ZK session
2014-10-15 11:37:08,102 INFO org.apache.zookeeper.ZooKeeper: Session: 0x34911de9796000f closed
2014-10-15 11:37:09,102 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=a1.hadoop.xxx.com:2181,hadoop5.xxx.com:2181,hadoop4.xxx.com:2181 sessionTimeout=10000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@28b35e23
2014-10-15 11:37:09,104 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server hadoop4.xxx.com/172.16.3.19:2181. Will not attempt to authenticate using SASL (unknown error)
2014-10-15 11:37:09,104 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to hadoop4.xxx.com/172.16.3.19:2181, initiating session
2014-10-15 11:37:09,107 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server hadoop4.xxx.com/172.16.3.19:2181, sessionid = 0x34911de97960012, negotiated timeout = 10000
2014-10-15 11:37:09,108 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2014-10-15 11:37:09,109 INFO org.apache.hadoop.ha.ActiveStandbyElector: Session connected.
2014-10-15 11:37:09,110 INFO org.apache.hadoop.conf.Configuration: found resource yarn-site.xml at file:/var/run/cloudera-scm-agent/process/4352-yarn-RESOURCEMANAGER/yarn-site.xml
2014-10-15 11:37:09,115 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=yarn OPERATION=refreshAdminAcls TARGET=AdminService RESULT=SUCCESS
2014-10-15 11:37:09,115 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Already in standby state
2014-10-15 11:37:09,115 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=yarn OPERATION=transitionToStandby TARGET=RMHAProtocolService RESULT=SUCCESS
2014-10-15 11:37:56,578 INFO org.apache.hadoop.ha.ActiveStandbyElector: Checking for any old active which needs to be fenced...
2014-10-15 11:37:56,586 INFO org.apache.hadoop.ha.ActiveStandbyElector: Old node exists: 0a067961726e524d1205726d323038
2014-10-15 11:37:56,586 INFO org.apache.hadoop.ha.ActiveStandbyElector: Writing znode /yarn-leader-election/yarnRM/ActiveBreadCrumb to indicate that the local node is the most recent active...
2014-10-15 11:37:56,589 INFO org.apache.hadoop.conf.Configuration: found resource yarn-site.xml at file:/var/run/cloudera-scm-agent/process/4352-yarn-RESOURCEMANAGER/yarn-site.xml
2014-10-15 11:37:56,596 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=yarn OPERATION=refreshAdminAcls TARGET=AdminService RESULT=SUCCESS
2014-10-15 11:37:56,596 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to active state
2014-10-15 11:37:56,597 WARN org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=yarn OPERATION=transitionToActive TARGET=RMHAProtocolService RESULT=FAILURE DESCRIPTION=Exception transitioning to active PERMISSIONS=All users are allowed
2014-10-15 11:37:56,597 WARN org.apache.hadoop.ha.ActiveStandbyElector: Exception handling the winning of election
org.apache.hadoop.ha.ServiceFailedException: RM could not transition to Active
at org.apache.hadoop.yarn.server.resourcemanager.EmbeddedElectorService.becomeActive(EmbeddedElectorService.java:118)
at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:804)
at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:415)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:599)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Caused by: org.apache.hadoop.ha.ServiceFailedException: Error when transitioning to Active mode
at org.apache.hadoop.yarn.server.resourcemanager.AdminService.transitionToActive(AdminService.java:291)
at org.apache.hadoop.yarn.server.resourcemanager.EmbeddedElectorService.becomeActive(EmbeddedElectorService.java:116)
... 4 more
Caused by: org.apache.hadoop.service.ServiceStateException: RMActiveServices cannot enter state STARTED from state STOPPED
at org.apache.hadoop.service.ServiceStateModel.checkStateTransition(ServiceStateModel.java:129)
at org.apache.hadoop.service.ServiceStateModel.enterState(ServiceStateModel.java:111)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:190)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:928)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:968)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:965)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:965)
at org.apache.hadoop.yarn.server.resourcemanager.AdminService.transitionToActive(AdminService.java:282)
... 5 more