python - boto issue with IAM role

Question

I'm trying to use AWS' recently announced "IAM roles for EC2" feature, which lets security credentials automatically get delivered to EC2 instances. (see http://aws.amazon.com/about-aws/whats-new/2012/06/11/Announcing-IAM-Roles-for-EC2-instances/).

I've set up an instance with an IAM role as described. I can also get (seemingly) proper access key / credentials with curl.

However, boto fails to do a simple call like "get_all_buckets", even though I've turned on ALL S3 permissions for the role.

The error I get is "The AWS Access Key Id you provided does not exist in our records"

However, the access key listed in the error matches the one I get from curl.

Here is the failing script, run on an EC2 instance with an IAM role attached that gives all S3 permissions:

import urllib2
import ast
from boto.s3.connection import S3Connection

resp=urllib2.urlopen('http://169.254.169.254/latest/meta-data/iam/security-credentials/DatabaseApp').read()
resp=ast.literal_eval(resp)
print "access:" + resp['AccessKeyId']
print "secret:" + resp['SecretAccessKey']
conn = S3Connection(resp['AccessKeyId'], resp['SecretAccessKey'])
rs= conn.get_all_buckets()

Answer 1

If you are using boto 2.5.1 or later it's actually much easier than this. Boto will automatically find the credentials in the instance metadata for you and use them as long as no other credentials are found in environment variables or in a boto config file. So, you should be able to simply do this on the EC2 instance:

>>> import boto
>>> c = boto.connect_s3()
>>> rs = c.get_all_buckets()

The reason that your manual approach is failing is that the credentials associated with the IAM Role is a temporary session credential and consists of an access_key, a secret_key and a security_token and you need to supply all three of those values to the S3Connection constructor.