Client cannot authenticate via:[TOKEN, KERBEROS]

Question

I'm using YarnClient to programmatically start a job. The cluster i'm running on has been kerberos-ized.

Normal map reduce jobs submitted via "yarn jar examples.jar wordcount..." work.

The job i'm trying to submit programmatically, does not. I get this error:

14/09/04 21:14:29 ERROR client.ClientService: Error happened during application submit: Application application_1409863263326_0002 failed 2 times due to AM Container for appattempt_1409863263326_0002_000002 exited with exitCode: -1000 due to: Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "yarn-c1-n1.clouddev.snaplogic.com/10.184.28.108"; destination host is: "yarn-c1-cdh.clouddev.snaplogic.com":8020; .Failing this attempt.. Failing the application. 14/09/04 21:14:29 ERROR client.YClient: Application submission failed

The code looks something like this:

ClientContext context = createContextFrom(args);
YarnConfiguration configuration = new YarnConfiguration();
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(configuration);
ClientService client = new ClientService(context, yarnClient, new InstallManager(FileSystem.get(configuration)));
LOG.info(Messages.RUNNING_CLIENT_SERVICE);
boolean result = client.execute();

I had thought that perhaps adding something to the effect of:

yarnClient.getRMDelegationToken(new Text(InetAddress.getLocalHost().getHostAddress()));

Could perhaps assuage my woes, but that doesn't seem to help either. Any help would be greatly appreciated.

Solution

Alright, well after hours and hours and hours we have this figured out. For all following generations of coders, forever plagued by hadoop's lack of documentation:

You must grab the tokens from UserGroupInformation object with a call to get credentials. Then you must set the tokens on the ContainerLaunchContext.