Building a Serverless Backend with AWS API Gateway and DynamoDB
When building a modern application, having a serverless backend that scales easily and can handle high traffic is essential for many applications. Without it, it’ll be nearly impossible to scale on demand and you’ll find yourself playing catch up on maintenance of your assets in perpetuity.
Amazon Web Services (AWS)’s API Gateway and DynamoDB are two services that work together to create a serverless backend that is reliable, cost-effective, and easy to manage. We have extensive experiencing deploying solutions within both services and have gathered our best knowledge and steps for setting up these resources effectively, providing you with the benefits of the cost savings and security cloud environments provide users.
API Gateway
API Gateway is a fully managed service that makes it easy to create, publish, and manage APIs at any scale. These APIs allow technology assets to communicate between one another.
It allows developers to create RESTful APIs that can integrate with a wide range of AWS services or any HTTP endpoint.
API Gateway provides a number of out of the box features such as security, caching, throttling, monitoring, and logging that technologists previously had to account for and configure themselves.
RESTful APIs
REST: Representational State Transfer; a set of principles defining how web standards should be used to create your services.
In a RESTful API, API resources are represented as URLs, and clients can use HTTP methods such as GET, POST, PUT, DELETE, etc. to communicate and interact with those resources.
For example, a client could send a GET request to the URL "https://example.com/api/users" to retrieve a list of users, or a POST request to "https://example.com/api/users" to create a new user.
Benefits of using RESTful APIs can include…
Scalability; These APIs can handle a large number of requests and can be easily distributed across multiple servers
Flexibility; These APIs aren’t tied to specific technology or programming language, making them easy to use for a wide variety of clients and servers
Reusability; These APIs are built on top of HTTP which is a widely-used and accepted protocol, making it reusable in other applications
Statelessness; These APIs mandate that each request contains all necessary information for the server to process it, making them more simple and reliable
Security; These APIs can be secured using standard authentication and encryption methods (such as HTTPS), protecting data as it’s communicated between assets
DynamoDB
DynamoDB is a fast and flexible NoSQL database service that provides high performance at any scale. It is fully managed and can automatically scale tables up or down to handle any amount of traffic or data.
Similar to API gateway, DynamoDB comes with a number out of the box features such as encryption, backup and restore, and global tables for multi-region replication, making it a reliable and security option to store your data.
Check out some of our other blogs on DynamoDB for more information about this service!
So how do you set up these services?
Setting up your backend infrastructure with API Gateway and DynamoDB is fairly easy. There are four main steps:
Step One: Design the API
Before you dive into setting up your infrastructure, take some time to ideate how your API should function. You should look at identifying the resources, data models, and methods required to effectively build out your API.
For example, let’s say you want to build an API that allows users to view or manage a to-do list. This resembles the baseline functionality of many different ticketing tools used by enterprises, such as Cherwell, JIRA, TFS, etc.
Planning for your API
When designing an API, there are several considerations to keep in mind when it comes to users, tasks, and lists.
User Authentication:
How are your users going to log into your API to access the resources you’ve made available?
You want to make sure you’ll implementing appropriate security measures, such as:
Password Protection
Multi-Factor Authentication
Integration with Third Party Authorization Services
User Authorization:
Once you've figured out who the user is, you need to identify what they need to have access to.
One of the most prevailing direct loss exposure scenarios for our clients is poorly provisioned accounts giving excessive access to users. This loss comes to fruition in the form of efficiency loss, and data integrity and confidentiality loss to either intentional or unintentional actors with permission to read, modify, or delete your assets.
This involves setting up Role-Based Access Control (RBAC) or Attribute-Based Access Control (ABAC).
Essentially, you want to make sure that the person access the data is doing so to complete the task they’re supposed to.
Good: A user who is supposed to share data with another account logs in, pulls a copy of the data, and sends it through a secure file transfer method to another account
Bad: A user realizes that another business use case can be solved by sharing the same data they share with one person with another business area. In an effort to set this up expeditiously, the user just pulls and sends the file to the business area with existing credentials.
Data Storage and Retrieval:
For any tasks or lists, you’ll need to determine how to store and retrieve that data from your DynamoDB.
To design the schema and understand what you can query, you should do the following:
Identify the Access Patterns:
Before designing your schema, you should identify the primary access patterns for your data. Access patterns are the different ways in which you will need to query your data.
For example, if you're building a to-do list app, your access patterns might include querying all tasks for a particular user, retrieving completed tasks, or retrieving tasks by due date.
Define the Primary Key:
In DynamoDB, the primary key is used to uniquely identify items in a table.
There are two types of primary keys: partition keys and composite keys.
Partition keys are used to partition data across multiple physical partitions, while composite keys are used to sort data within a partition.
You should choose the type of primary key that best matches your access patterns.
Define the Attributes:
In DynamoDB, each item in a table can have a different set of attributes.
Attributes are the properties of the item that you want to store in the table.
You should define the attributes that are relevant to your access patterns.
Define the Indexes:
In DynamoDB, you can create secondary indexes to support additional access patterns beyond the primary key.
You can create either global secondary indexes (which can be queried across all partitions) or local secondary indexes (which can be queried within a partition).
Understand Querying Capabilities:
Once you've designed your schema, you should understand the querying capabilities of DynamoDB.
DynamoDB supports several types of queries, including partition key queries, composite key queries, and index queries.
You should choose the query type that best matches your access patterns.
API Design:
When designing your API, you'll want to follow RESTful principles and create a clear, consistent structure for accessing resources.
This will help to make your API easy to use and understand for developers who are integrating with it.
API Documentation:
It’s important to provide clear and comprehensive documentation for your API, including details on how to authenticate, authorize, and access resources. This will help developers who are integrating with your API to get up and running quickly.
You should consider making versions of the documents below to help explain the purpose and most efficient uses of your API:
Getting Started Guide: A getting started guide provides an overview of how to use the API, including how to authenticate, how to make requests, and how to handle responses. It should also include information on any required dependencies or libraries.
API Reference: An API reference provides detailed documentation on each endpoint, including the HTTP method, URL, request parameters, and response schema. It should also include any possible error responses and their associated status codes.
Authentication and Authorization Guide: An authentication and authorization guide provides detailed documentation on how to authenticate with the API, including the types of authentication supported (e.g. API key, OAuth), and how to obtain and use access tokens. It should also include information on how to authorize access to resources.
Sample Code and SDKs: Sample code and SDKs provide pre-built code that developers can use to integrate with the API. This can help to reduce the time and effort required to integrate with the API and ensure that best practices are followed.
Rate Limiting and Throttling Guide: A rate limiting and throttling guide provides information on any limitations or quotas for API usage, as well as guidelines for how to handle rate limiting and throttling errors.
Error Handling Guide: An error handling guide provides information on how to handle errors returned by the API, including how to interpret error codes and messages.
Versioning Guide: A versioning guide provides information on how the API is versioned and how developers can ensure compatibility with future versions.
Release Notes: Release notes provide a summary of changes made to the API, including bug fixes, feature enhancements, and breaking changes.
Support and Troubleshooting Guide: A support and troubleshooting guide provides information on how to contact support, how to troubleshoot common issues, and how to report bugs or issues with the API.
Step Two: Create the DynamoDB Tables
Once the API design is finalized, the next step is to create the DynamoDB tables required to store and retrieve data. We can use the AWS Management Console to create these.
For example, we can create a "Tasks" table with a "TaskId" primary key and a "UserId" sort key to store all the tasks associated with a user. We can also create a "Users" table with a "UserId" primary key to store information about users.
Step Three: Create the API Gateway
There are a number of different actions that can be performed once you’ve set up the gateway:
We can create an API with a "Tasks" resource and a "GET" method to retrieve all the tasks associated with a user.
We can create a "POST" method to create a new task
We can create a "PUT" method to update an existing task
Finally, we can create a "DELETE" method to delete a task. We can then create a "Users" resource with a "GET" method to retrieve user information.
Step Four: Integrate the API Gateway with DynamoDB
After creating the API Gateway, we can integrate it with DynamoDB using Velocity Template Language (VTL) mapping templates. VTL mapping templates enable us to transform data between the API Gateway and DynamoDB.
For example, we can create a VTL mapping template to transform data between the API Gateway and DynamoDB for the "Tasks
As the development process continued, it was important to ensure that the API Gateway was secure and protected against potential attacks. Security measures such as HTTPS encryption and AWS Identity and Access Management (IAM) roles were implemented to restrict access to the API and the DynamoDB tables.
VTL Mapping Templates
VTL mapping templates are a powerful scripting tool that allows developers to transform data in real-time within AWS AppSync and other AWS services. They enable the mapping of data between two distinct models, utilizing a syntax that is similar to other programming languages but with unique features that make it easy to manipulate data. By defining a set of rules once, VTL mapping templates can significantly reduce the amount of code needed and reduce the risk of errors. To learn more visit AWS documentation .
I’ve set it up, now what?
If you still have questions or want to learn more, read our other blogs or reach out today to learn how we can help!
