Reference material: http://boto3.readthedocs.io/en/latest/reference/services/dynamodb.html

Recently, due to the need of work, we have studied the dynamoDB part of boto3, and have some experience. Here we summarize it.

The first is the boto3 installation, which runs on a machine equipped with python and pip

sudo pip install boto3  

 

In the official website documents, boto3 provides the following interfaces to interact with dynamoDB:

batch_get_item()
batch_write_item()
can_paginate()
create_table()
delete_item()
delete_table()
describe_limits()
describe_table()
describe_time_to_live()
generate_presigned_url()
get_item()
get_paginator()
get_waiter()
list_tables()
list_tags_of_resource()
put_item()
query()
scan()
tag_resource()
untag_resource()
update_item()
update_table()
update_time_to_live()

To put it plainly, it means adding, deleting, checking and modifying tables and records. This article mainly describes the interfaces I have recently used.

 

To use boto3 in python, you have to import boto3 first. Of course, this is nonsense. For ease of use, I first wrote a configuration file in json format, as follows:

{
    "region_name":"xxx",
    "aws_access_key_id":"xxx",
    "aws_secret_access_key":"xxx"
}

 

Then it encapsulates a class dedicated to manipulating dynamoDB, which currently has nothing at all.

class dynamodb_operation():

It requires a way to read json files:

    def load_json(self,path):
        try:
            with open(path) as json_file:
                data = json.load(json_file)
        except Exception as e:
            print 'ERROR: no such file like ' + path
            exit(-1)
        else:
            return data

Since the file may not be in json format, I just want him to report an error and exit. If you don't want it to quit, change it in except.

Then, I want this class to have a private member, client, which establishes the connection when I instantiate the object. So, I have the following initialization methods:

    def __init__(self,path):
        conf = self.load_json(path)
        self.client = boto3.client('dynamodb',region_name=conf['region_name'],aws_access_key_id=conf['aws_access_key_id'], aws_secret_access_key=conf['aws_secret_access_key'])

It corresponds to the previous configuration file.

With this foundation, you can encapsulate the methods you want to use. The instructions of various methods on the official website will not be copied.

List all tables in dynamoDB

    def list_all_table(self):
        page=1
        LastEvaluationTableName = ""
        while True:
            if page == 1:
                response = self.client.list_tables()
            else:
                response = self.client.list_tables(
                        ExclusiveStartTableName=LastEvaluationTableName
                        )
            TableNames = response['TableNames']
            for table in TableNames:
                print table
            if response.has_key('LastEvaluatedTableName'):
                LastEvaluationTableName = response["LastEvaluatedTableName"]
            else:
                break
            page += 1

The list_table() method can only get the table names of up to 100 tables at a time, and each time it returns, the key is the table name of the last table with the value of "LastEvaluatedTableName", which can be used as a parameter for the next request. In this way, all table names can be obtained by circular invocation. If there are no tables behind, there will be no LastEvaluatedTableName in response. Here I just want to print the name of the form to the terminal, if you want to save it, it is also possible.

2. Get information about a table

    def get_table_desc_only(self,table):
        try:
            response = self.client.describe_table(TableName=table)
        except Exception as e:
            print 'ERROR: no such table like ' + table
            exit(-1)
        else:
            return response["Table"]

Here, the response["Table"] is simply returned locally, without any other processing.

If I want to know the size of a table, I can:

    def get_table_size(self,table):
        response = self.get_table_desc_only(table)
        stastic = {}
        stastic['TableSizeBytes'] = response['TableSizeBytes']
        stastic['ItemCount'] = response['ItemCount']
        return stastic

If you want to know other information, and only want to know those information, you can also write the corresponding method.

3. Create a table

    def create_table(self,tablename,keySchema,attributeDefinitions,provisionedThroughput):
        table = self.client.create_table(
                TableName=tablename,
                KeySchema=keySchema,
                AttributeDefinitions=attributeDefinitions,
                ProvisionedThroughput=provisionedThroughput
                )

        # Wait until the table exists.
        self.client.get_waiter('table_exists').wait(TableName=tablename)

        response = self.client.describe_table(TableName=tablename)
        print response

This is creating a table without an index. It takes time to create the surface, so get_waiter() method is used.

4. Insert data

    def put_item(self,tableName,item):
        try:
            self.client.put_item(
                    TableName=tableName,
                    Item=item
                    )
        except Exception as e:
            print 'ERROR: put item fail. msg: ' + str(e)
            exit(-1)
        else:
            return

The encapsulated method needs to pass in a json with the correct format, and the key corresponds to the table. For example:

{'uid':{'N':'999'},'aid':{'N':'999'},'sid':{'N':'999'},'ksid':{'N':'999'}}

5. Delete tables

    def delete_table(self,table):
        try:
            self.client.delete_table(
                    TableName=table
                    )   
        except Exception as e:
            print 'ERROR: delete table ' + table + ' fail. msg: ' + str(e)
        else:
            print 'delete table ' + table + ' succ'

 

To be continued...