Proper access policy for Amazon Elastic Search Cluster

Question

I've recently started using the new Amazon Elasticsearch Service and I can't seem to figure out the access policy I need so that I can only access the services from my EC2 instances that have a specific IAM role assigned to them.

Here's an example of the access policy I currently have assigned for the ES domain:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::[ACCOUNT_ID]:role/my_es_role",
        ]
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-east-1:[ACCOUNT_ID]:domain/[ES_DOMAIN]/*"
    }
  ]
}

But as I said, this doesn't work. I log into the EC2 instance (which has the my_es_role role attached to it) and attempt to run a simple curl call on the "https://*.es.amazonaws.com" end point, I get the following error:

{"Message":"User: anonymous is not authorized to perform: es:ESHttpGet on resource: arn:aws:es:us-east-1:[ACCOUNT_ID]:domain/[ES_DOMAIN]/“}

Does anyone know what I have to change in the access policy in order for this to work?

Solution 1

You can lock access down to IAM-only, but how will you view Kibana in your browser? You could setup a proxy (see Gist and/or NPM module) or enable both IAM and IP-based access for viewing results.

I was able to get both IAM access IP-restricted access with the following Access Policy. Note the order is important: I could not get it working with the IP-based statement before the IAM statement.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::xxxxxxxxxxxx:root"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-west-2:xxxxxxxxxxxx:domain/my-elasticsearch-domain/*"
    },
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-west-2:xxxxxxxxxxxx:domain/my-elasticsearch-domain/*",
      "Condition": {
        "IpAddress": {
          "aws:SourceIp": [
            "192.168.1.0",
            "192.168.1.1"
          ]
        }
      }
    }
  ]
}

My EC2 instance has an instance profile with the arn:aws:iam::aws:policy/AmazonESFullAccess policy. Logstash should sign requests using the logstash-output-amazon-es output plugin. Logstash running on my EC2 instance includes an output section like this:

output {
    amazon_es {
        hosts => ["ELASTICSEARCH_HOST"]
        region => "AWS_REGION"
    }
    # If you need to do some testing & debugging, uncomment this line:
    # stdout { codec => rubydebug }
}

I can access Kibana from the two IPs in the access policy (192.168.1.0 and 192.168.1.1).

Solution 2

According to AWS doc and as you (and I) just tested, you cannot restrict access to an AWS ES domain to a role/account/user/... and simply cURL it!

Standard clients, such as curl, cannot perform the request signing that is required of identity-based access policies. You must use an IP address-based access policy that allows anonymous access to successfully perform the instructions for this step. (http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg-search.html)

So you have basically two solutions:

• change your access policy and restrict it to IP(s), I think you cannot use private IP because your ES cluster does not seems to belong to your VPC (default or not). Please use the public IP
• sign your request: http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains.html#es-managedomains-signing-service-requests

Signing your request is probably the best solution if you want to keep your access policy as is (which is more flexible than restricting to an IP), but it seems to be a bit more complex. I haven't tried so far and I cannot find any doc to help.

Solution 3

A bit late to the party, but I was able to deal with the exact same issue by adding signature to my requests.

If you use Python (like I do), you can use the following library to make it particularly easy to implement: https://github.com/DavidMuller/aws-requests-auth

It worked perfectly for me.

Solution 4

You just need to full user name in elastic search policy.

In this case, you can get your full user name from the error message itself. In my case: "arn:aws:sts::[ACCOUNT_ID]:assumed-role/[LAMBDA_POLICY_NAME]/[LAMBDA_NAME]"

    {
        "Version": "2012-10-17",
        "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "AWS": [
              "arn:aws:sts::xxxxxxxxxxxx:assumed-role/[lambda-role]/[full-lambda-name]"
            ]
          },
          "Action": "es:*",
          "Resource": "arn:aws:es:[region]:xxxxxxxxxxxxx:domain/[elasticsearch-domain-name]/*"
        }
      ]

    }

View more solutions

Comments

[Cyril Duchon-Doris]

Beware, ElasticSearch access policy changes take a long while to apply, unlike other IAM changes that are almost instantaneous. It's easy to just click "apply" and switch tab without noticing the "Processing..."

[Anant Gupta]

I used my laptop's public IP address and tried accessing the endpoint with curl/browser, but I'm still getting the User:anonymous error.

[nemo]

i am dealing with the same problem. and I have noticed that processing the changes by aws elasticsearch takes a loooong time.

[Pete]

Set an Access Policy with two statements: one for IAM access to write logs, the other with IP-restricted access to view KIbana. See my answer for details

[Garreth McDaid]

Hi, you only need to use the plugin if you are using an IAM-based policy. You can use the standard elasticsearch plugin in Logstash if your access policy is based on IP addresses. You don't need an instance profile in that case either. Also, the ES service is not available in VPCs. You have to use public ip addresses to connect. Not sure if your references to 192.168 addresses are substitutions for something else, but may mislead.

[Pete]

The aws:SourceIp's in my example are intended to be your personal workstation IP so you can use Kibana. The IAM-restricted access enables one or more EC2 instances to write to Elasticsearch without worrying about which IPs belong to a particular instance or CIDR block.

[sventechie]

It is worth noting that limiting to the private IP CIDR range of your VPC does not seem to work. ES does not operate within the VPC or something.

[Balmipour]

I wondered if "loooong" meant minutes, hours, or days. Looks like its 10-15 minutes. You can see that when checking the status of your ES (green 'active' if update is complete, else, something like an orange 'preparing'.

[gmajivu]

I had the same problem and after searching I found this handy library.

[alph486]

Upvote for mention of private IPs not working because not in a VPC.

[Robert Calhoun]

Thanks for giving a sample policy in your answer; I'd been unable to get Kibana past the dreaded "User: anonymous" error until I switched aws:SourceIp from a scalar value to an array, as in the example you gave. (I'm CIDR notation, if that helps out anyone else.) The whole process of setting policies for AWS ES would be less frustrating if every single policy change didn't put the cluster into the mysterious "processing" state for 20 minutes as the policy is carefully inscribed on stone tablets, or whatever it is they are doing.