Tutorial 1: Serialization

introduction

This tutorial will show you how to create simple Web API (restful) code with good cohesion. In the process, it will introduce the various components that make up the REST framework and give you a comprehensive understanding of how everything comes together.

Explain several terms:

api: API (Application Programming Interface) is some predefined function, or refers to the agreement of different components of a software system [1]. The goal is to provide applications and developers with the ability to access a set of routines based on a piece of software or hardware without access to the source code or understanding the details of the internal working mechanism.

Api endpoints: API endpoints usually refer to service providers that provide a specific subset of functions in the same interface. Different endpoints can have different protocols and formats.

Serialization: Converting pytho instance objects or data objects into json format data or xml, etc.

Deserialization: An inverse process of serialization.

New Virtual Environment Viralenv

Before doing anything else, we will create a new virtual environment using venv... This will ensure that our package configuration is well isolated from any other project we are working on.

python3 -m venv env
source env/bin/activate

Now that we are in a virtual environment, we can install our package requirements.

pip install django
pip install djangorestframework
pip install pygments  # We'll be using this for the code highlighting

Note: To exit the virtual environment at any time, just type deactivate. For more information, search for the use of python virtual environments.

start

Well, we're ready for coding. To get started, let's create a new project to deal with.

cd ~
django-admin startproject tutorial
cd tutorial

After that, we can create an application to create a simple Web API.

python manage.py startapp snippets

We need to add our new snippets application and rest_framework application INSTALLED_APPS... Let's edit the tutorial/settings.py file:

INSTALLED_APPS = [
...
'rest_framework',
'snippets.apps.SnippetsConfig',
]
Okay, we're ready.

Create a model to use

For the purposes of this tutorial, we will first create a simple Snippet model for storing code segments. Continue editing the snippets/models.py file.

from django.db import models
from pygments.lexers import get_all_lexers
from pygments.styles import get_all_styles

LEXERS = [item for item in get_all_lexers() if item[1]]
LANGUAGE_CHOICES = sorted([(item[1][0], item[0]) for item in LEXERS])
STYLE_CHOICES = sorted([(item, item) for item in get_all_styles()])


class Snippet(models.Model):
    created = models.DateTimeField(auto_now_add=True)
    title = models.CharField(max_length=100, blank=True, default='')
    code = models.TextField()
    linenos = models.BooleanField(default=False)
    language = models.CharField(choices=LANGUAGE_CHOICES, default='python', max_length=100)
    style = models.CharField(choices=STYLE_CHOICES, default='friendly', max_length=100)

    class Meta:
        ordering = ['created']

We also need to create an initial migration for the code snippet model and synchronize the database for the first time.

python manage.py makemigrations snippets
python manage.py migrate

Create serializer classes

The first thing we need to start with the Web API is to provide a way to serialize and deserialize code snippet instances into representations, such as json... We can do this by declaring serializers that are very similar to Django's forms. Add the following to the snippets directory name serializers.py.

from rest_framework import serializers
from snippets.models import Snippet, LANGUAGE_CHOICES, STYLE_CHOICES


class SnippetSerializer(serializers.Serializer):
    id = serializers.IntegerField(read_only=True)
    title = serializers.CharField(required=False, allow_blank=True, max_length=100)
    code = serializers.CharField(style={'base_template': 'textarea.html'})
    linenos = serializers.BooleanField(required=False)
    language = serializers.ChoiceField(choices=LANGUAGE_CHOICES, default='python')
    style = serializers.ChoiceField(choices=STYLE_CHOICES, default='friendly')

    def create(self, validated_data):
        """
        Create and return a new `Snippet` instance, given the validated data.
        """
        return Snippet.objects.create(**validated_data)

    def update(self, instance, validated_data):
        """
        Update and return an existing `Snippet` instance, given the validated data.
        """
        instance.title = validated_data.get('title', instance.title)
        instance.code = validated_data.get('code', instance.code)
        instance.linenos = validated_data.get('linenos', instance.linenos)
        instance.language = validated_data.get('language', instance.language)
        instance.style = validated_data.get('style', instance.style)
        instance.save()
        return instance

The first part of the serializer class defines the serialized/deserialized fields. This create() and update() methods define how to create or modify fully mature instances when invoked. serializer.save()

The serialized class is very similar to Django's Form class and contains similar validation flags in various fields, such as required, max_length and default.

Field flags can also control how serializers are displayed in certain situations, such as when rendered as HTML. The {base_template':'textarea.html'} above is equivalent to using widget=widgets.Textarea on Django Form class. This is particularly useful for controlling how browsable API s are displayed, as we will see later in this tutorial.

In fact, we can also use the ModelSerializer class, as we will see later, but now we will keep the definition of serializer clear.

Using serializers

Before we go any further, we'll be familiar with how to use our new serializer class. Let's go into the Django shell.

python manage.py shell
Okay, let's import the necessary packages first, and then let's create a few code snippets to use.

from snippets.models import Snippet
from snippets.serializers import SnippetSerializer
from rest_framework.renderers import JSONRenderer
from rest_framework.parsers import JSONParser

snippet = Snippet(code='foo = "bar"\n')
snippet.save()

snippet = Snippet(code='print("hello, world")\n')
snippet.save()
//We now have several fragment instances available. Let's look at one example of serialization.

serializer = SnippetSerializer(snippet)
serializer.data
# {'id': 2, 'title': '', 'code': 'print("hello, world")\n', 'linenos': False, 'language': 'python', 'style': 'friendly'}

So far, we have converted the model instance to a Python native data type. To complete the serialization process, we present the data to json.

content = JSONRenderer().render(serializer.data)
content
# b'{"id": 2, "title": "", "code": "print(\\"hello, world\\")\\n", "linenos": false, "language": "python", "style": "friendly"}'

Deserialization is similar. First, we parse the flow into a Python local data type.

import io

stream = io.BytesIO(content)
data = JSONParser().parse(stream)

Then, we restore these native data types to a fully populated object instance.

serializer = SnippetSerializer(data=data)
serializer.is_valid()
# True
serializer.validated_data
# OrderedDict([('title', ''), ('code', 'print("hello, world")\n'), ('linenos', False), ('language', 'python'), ('style', 'friendly')])
serializer.save()
# <Snippet: Snippet object>

Note that the API has many similarities with the use of forms. This similarity is better understood when we begin to write views that use serializers.

We can also serialize query sets rather than single model instances. To do this, we just need to add a many=True to mark the flag as a serializer parameter.

serializer = SnippetSerializer(Snippet.objects.all(), many=True)
serializer.data
# [OrderedDict([('id', 1), ('title', ''), ('code', 'foo = "bar"\n'), ('linenos', False), ('language', 'python'), ('style', 'friendly')]), OrderedDict([('id', 2), ('title', ''), ('code', 'print("hello, world")\n'), ('linenos', False), ('language', 'python'), ('style', 'friendly')]), OrderedDict([('id', 3), ('title', ''), ('code', 'print("hello, world")'), ('linenos', False), ('language', 'python'), ('style', 'friendly')])]

Using Model Serializer

Our SnippetSerializer class copies a lot of information, which is also included in the Snippet model. It would be better if we could keep the code a little more concise.

In the same way as Django provides Form classes and ModelForm classes, the REST framework consists of two classes, the Serializer class and the ModelSerializer class.

Let's see how to use the ModelSerializer class. Open the file snippets/serializers.py and replace SnippetSerializer again to initialize in the following way.

class SnippetSerializer(serializers.ModelSerializer):
    class Meta:
        model = Snippet
        fields = ['id', 'title', 'code', 'linenos', 'language', 'style']

A good property of a serializer is that you can check it by printing the representations of all fields in the serializer instance. Open the Django shell command line and enter: python manage.py shell, and then try the following:

from snippets.serializers import SnippetSerializer
serializer = SnippetSerializer()
print(repr(serializer))
# SnippetSerializer():
#    id = IntegerField(label='ID', read_only=True)
#    title = CharField(allow_blank=True, max_length=100, required=False)
#    code = CharField(style={'base_template': 'textarea.html'})
#    linenos = BooleanField(required=False)
#    language = ChoiceField(choices=[('Clipper', 'FoxPro'), ('Cucumber', 'Gherkin'), ('RobotFramework', 'RobotFramework'), ('abap', 'ABAP'), ('ada', 'Ada')...
#    style = ChoiceField(choices=[('autumn', 'autumn'), ('borland', 'borland'), ('bw', 'bw'), ('colorful', 'colorful')...

It's important to remember that ModelSerializer classes don't do anything particularly magical, they're just shortcuts to creating serialized classes:

A set of fields that are automatically determined.
Simple default implementation of class. create() and update() methods.

Writing regular Django views using serializers

Let's see how to write some API views using the new serialized classes. Currently, we will not use any other features of the REST framework, just write the view as a normal Django view.

Edit the snippets/views.py file and add the following.

from django.http import HttpResponse, JsonResponse
from django.views.decorators.csrf import csrf_exempt
from rest_framework.parsers import JSONParser
from snippets.models import Snippet
from snippets.serializers import SnippetSerializer

The root of our API will be a view that supports listing all existing fragments or creating a new code fragment.

@csrf_exempt
def snippet_list(request):
    """
    List all code snippets, or create a new snippet.
    """
    if request.method == 'GET':
        snippets = Snippet.objects.all()
        serializer = SnippetSerializer(snippets, many=True)
        return JsonResponse(serializer.data, safe=False)

    elif request.method == 'POST':
        data = JSONParser().parse(request)
        serializer = SnippetSerializer(data=data)
        if serializer.is_valid():
            serializer.save()
            return JsonResponse(serializer.data, status=201)
        return JsonResponse(serializer.errors, status=400)

Note that since we want to be able to send views from clients that never use CSRF tokens, we need to mark views as csrf_exempt... This is not what you usually want to do. The REST Framework View actually uses more reasonable operations, but it's enough now.

We also need a view corresponding to a single fragment that can be used to retrieve, update, or delete code fragments.

@csrf_exempt
def snippet_detail(request, pk):
    """
    Retrieve, update or delete a code snippet.
    """
    try:
        snippet = Snippet.objects.get(pk=pk)
    except Snippet.DoesNotExist:
        return HttpResponse(status=404)

    if request.method == 'GET':
        serializer = SnippetSerializer(snippet)
        return JsonResponse(serializer.data)

    elif request.method == 'PUT':
        data = JSONParser().parse(request)
        serializer = SnippetSerializer(snippet, data=data)
        if serializer.is_valid():
            serializer.save()
            return JsonResponse(serializer.data)
        return JsonResponse(serializer.errors, status=400)

    elif request.method == 'DELETE':
        snippet.delete()
        return HttpResponse(status=204)

Finally, we need to link these endpoints. Create snippets/urls.py files:

from django.urls import path
from snippets import views

urlpatterns = [
    path('snippets/', views.snippet_list),
    path('snippets/<int:pk>/', views.snippet_detail),
]

We also need to connect to the root urlconf in the tutorial/urls.py file, including the URL of our fragment application.

from django.urls import path, include

urlpatterns = [
    path('', include('snippets.urls')),
]

It is noteworthy that we have not yet properly handled some marginal cases. If we send a malformed json or send a request in a way that the view cannot handle, we will get a 500 "server error" response. At present, however, we need not consider this issue.

Testing our first Web API

Now, we can start a sample server that serves snippets of code.

Exit from the shell.

quit()
Start the Django development server

python manage.py runserver

Validating models...

0 errors found
Django version 1.11, using settings 'tutorial.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

In another terminal window, we can test the server.

We can use curl or httpie... Httpie is a user-friendly HTTP client written in Python. Let's install it. Or use postman
You can install httpie using pip:

pip install httpie

Finally, we can get a list of all the fragments:

http http://127.0.0.1:8000/snippets/

HTTP/1.1 200 OK
...
[
  {
    "id": 1,
    "title": "",
    "code": "foo = \"bar\"\n",
    "linenos": false,
    "language": "python",
    "style": "friendly"
  },
  {
    "id": 2,
    "title": "",
    "code": "print(\"hello, world\")\n",
    "linenos": false,
    "language": "python",
    "style": "friendly"
  }
]

Or we can get a specific fragment by referring to its id:

http http://127.0.0.1:8000/snippets/2/

HTTP/1.1 200 OK
...
{
  "id": 2,
  "title": "",
  "code": "print(\"hello, world\")\n",
  "linenos": false,
  "language": "python",
  "style": "friendly"
}

Similarly, you can display the same json by accessing these URL s in a Web browser.

What steps have we taken?

So far, we've done well. We have a serialized API very similar to Django's Forms API, as well as some common Django views.

Apart from services, our api view currently does nothing special. In response to json, although we still want to clean up some errors in edge handling, it is a well-functioning Web api.

We'll learn how to start improving the api and response DjangoRestFramework series (2): request object and response object.