How to append a value to list attribute on AWS DynamoDB?

Question

I'm using DynamoDB as an K-V db (cause there's not much data, I think that's fine) , and part of 'V' is list type (about 10 elements). There's some session to append a new value to it, and I cannot find a way to do this in 1 request. What I did is like this:

item = self.list_table.get_item(**{'k': 'some_key'})
item['v'].append('some_value')
item.partial_save()

I request the server first and save it after modified the value. That's not atomic and looks ugly. Is there any way to do this in one request?

Solution 1

The following code should work with boto3:

table = get_dynamodb_resource().Table("table_name")
result = table.update_item(
    Key={
        'hash_key': hash_key,
        'range_key': range_key
    },
    UpdateExpression="SET some_attr = list_append(some_attr, :i)",
    ExpressionAttributeValues={
        ':i': [some_value],
    },
    ReturnValues="UPDATED_NEW"
)
if result['ResponseMetadata']['HTTPStatusCode'] == 200 and 'Attributes' in result:
    return result['Attributes']['some_attr']

The get_dynamodb_resource method here is just:

def get_dynamodb_resource():
    return boto3.resource(
            'dynamodb',
            region_name=os.environ['AWS_DYNAMO_REGION'],
            endpoint_url=os.environ['AWS_DYNAMO_ENDPOINT'],
            aws_secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'],
            aws_access_key_id=os.environ['AWS_ACCESS_KEY_ID'])

Solution 2

You can do this in 1 request by using the UpdateItem API in conjunction with an UpdateExpression. Since you want to append to a list, you would use the SET action with the list_append function:

SET supports the following functions:

...

• list_append (operand, operand) - evaluates to a list with a new element added to it. You can append the new element to the start or the end of the list by reversing the order of the operands.

You can see a couple examples of this on the Modifying Items and Attributes with Update Expressions documentation:

• The following example adds a new element to the FiveStar review list. The expression attribute name #pr is ProductReviews; the attribute value :r is a one-element list. If the list previously had two elements, [0] and [1], then the new element will be [2].

  SET #pr.FiveStar = list_append(#pr.FiveStar, :r)
  

• The following example adds another element to the FiveStar review list, but this time the element will be appended to the start of the list at [0]. All of the other elements in the list will be shifted by one.

  SET #pr.FiveStar = list_append(:r, #pr.FiveStar)
  

The #pr and :r are using placeholders for the attribute names and values. You can see more information on those on the Using Placeholders for Attribute Names and Values documentation.

Solution 3

I would look at update expressions: http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.Modifying.html#Expressions.Modifying.UpdateExpressions.ADD

Should be doable with an ADD, although not sure what the support in boto is for this.

Solution 4

@LaserJesus 's answer is correct. However, using boto3 directly is kind of a pain, hard to maintain, and not at all reusable. dynamof abstracts that junk away. Using dynamof appending an item to a list attribute would look like:

from functools import partial
from boto3 import client
from dynamof.executor import execute
from dynamof.operations import update
from dynamof.attribute import attr

client = client('dynamodb', endpoint_url='http://localstack:4569')
db = partial(execute, client)

db(update(
  table_name='users',
  key={ 'id': user_id },
  attributes={
      'roles': attr.append('admin')
  }))

disclaimer: I wrote dynamof

View more solutions

Comments

[kishorer747]

thank you! this works perfectly. Although i tried using ADD but that did not work. I read in the docs that they recommend using ADD. When i tried to use ADD, i got operator mismatch Error in UpdateExression.

[MuntingInsekto]

Very useful! Sorry i can only do +1 :(

[Jonathan]

does the list item have to exist? or can I just use this and it'll init some_attr for me as an empty list if it did not exist?

[YTerle]

If you need to create a list if it's not exist, please see stackoverflow.com/questions/34951043/…

[petey]

Totally with vnpnlz on this (am also sorry I can only do +1).

[neiloth]

Is this best practise? I'm asking in terms of capacity units. I'm appending a very tiny object to a list, but writing capacity is calculated based on whole item size. What should I do if I want to append 50byte to a 300kb item? Do I have to pay writing capacity of 300 kb?