Amazon Web Services' DynamoDB is a highly scalable NoSQL database service that can handle massive amounts of data and traffic.

However, like any service, DynamoDB is not immune to misconfiguration or misuse that can lead to unexpected cost overruns. Inefficient configurations of your DynamoDB instance can result in some unintended risk to, through…

Excessive usage/spending
Resource usage inefficiencies
Billing inaccuracies
Inability to accurately conduct capacity planning

Keep reading to learn more about the most common mistakes people make when they’re setting up and managing a Dynamo DB resource, and how to avoid falling into the same traps.

The Importance of Partition Keys

One of the most important factors to consider when designing a DynamoDB table is the partition key.

The partition key determines the physical location of data in the database, and affects how DynamoDB distributes data across partitions. Poor partition key design can result in inefficient query performance and increased costs.

One common mistake we see is to utilize a high cardinality attribute, such as a Universally Unique Identifier (UUID), as the partition key. Users may want to utilize high cardinality attributes because they can provide good data distribution across partitions due to their high degree of variability leading them to distribute data evenly across partitions. However, there are a few reasons why using these may lead to performance issues and increased costs.

Hot partitions:

When using a high cardinality attribute as a partition key, such as UUID, there is a risk of creating "hot partitions," where a small number of partitions receive a disproportionate amount of traffic. This can cause performance issues and can limit the scalability of your DynamoDB table.

Increased costs:

Using a high cardinality attribute as a partition key can also lead to increased costs because it can cause DynamoDB to create more partitions than necessary. This is because DynamoDB creates a new partition every time a new partition key value is encountered, so if you use a high cardinality attribute as a partition key, you may end up with many partitions that are mostly empty, which can increase your storage and read/write costs.

Query performance:

Another issue with using a high cardinality attribute like UUID as a partition key is that it can negatively impact query performance. Since partition keys determine the physical location of the data within DynamoDB, using a high cardinality attribute can result in more scattered data across partitions, making it more difficult for DynamoDB to efficiently retrieve the data you need.

To avoid this mistake, it's important to choose a partition key that evenly distributes data across partitions. This can be achieved by choosing a key with low cardinality, such as a country code or date, or by using composite keys that include a high cardinality attribute along with a low cardinality attribute.

Overprovisioning Throughput

Another common mistake is overprovisioning throughput. DynamoDB allows you to provision read and write capacity units for your tables, which determine the maximum amount of traffic your table can handle. Overprovisioning throughput can lead to increased costs without providing any significant benefits in terms of query performance.

We strongly recommend regular monitoring to make sure you’re not overprovisioning throughput by utilizing CloudWatch. Cloudwatch is an Amazon Web Services Product that can be used to monitor the resource capacity and usage metrics for your cloud environment. Cloudwatch can be used most effectively by employing the following tactics for monitoring efforts:

Determine your baseline usage:

The first step is to determine your baseline usage for your DynamoDB tables. This will help you understand the minimum amount of read and write capacity units you need to provision for your tables. You can use the CloudWatch Metrics tab to view the Consumed Read Capacity Units and Consumed Write Capacity Units metrics for your tables. These metrics show you how many capacity units are being consumed by your table over a given time period.

Set alarms:

Once you have determined your baseline usage, you can set CloudWatch alarms to monitor your read and write capacity units. This will help you detect any spikes in usage that may indicate that you are overprovisioning your throughput.

To set alarms, navigate to the Alarms tab in the CloudWatch console and click the "Create Alarm" button. From there, you can select the Consumed Read Capacity Units or Consumed Write Capacity Units metric and set the threshold at which you want the alarm to trigger.

Adjust capacity:

If you receive an alarm notification that your usage has exceeded your provisioned capacity, you can adjust your capacity to better match your usage.

You can do this by navigating to the Tables tab in the DynamoDB console and selecting the table you want to adjust. From there, you can click the "Capacity" tab and adjust your read and write capacity units accordingly.

Monitor and adjust:

Finally, it's important to continuously monitor your usage and adjust your capacity as needed. You can use the CloudWatch metrics and alarms to track your usage and ensure that you are not overprovisioning your throughput.

If you find that you are consistently hitting your provisioned capacity limits, you may need to consider optimizing your application or adjusting your partition key to improve data distribution across partitions.

Not Using On-Demand Capacity Mode

DynamoDB offers two capacity modes: provisioned and on-demand.

Provisioned capacity mode requires you to specify a fixed amount of read and write capacity units for your table, and can cause issues if your traffic patterns are unpredictable.

On-demand capacity mode, on the other hand, automatically scales to handle your table's traffic and charges you only for the actual usage.

Transitioning from Provisioned Capacity mode to On-Demand Capacity mode in DynamoDB is a straightforward process that involves verifying your account's eligibility, switching to On-Demand mode, adjusting your capacity settings, and monitoring your usage to ensure that you're not incurring unexpected costs.

Verify your account is eligible:

Before transitioning to On-Demand Capacity mode, you need to make sure that your AWS account is eligible.

You can check your account's eligibility by visiting the DynamoDB console and selecting the table you want to modify.

From there, click on the "Capacity" tab and look for the "Billing Mode" section. If your account is eligible, you will see the option to switch to On-Demand Capacity mode.

Switch to On-Demand Capacity mode:

Once you have verified that your account is eligible, you can switch to On-Demand Capacity mode by visiting the DynamoDB console and selecting the table you want to modify. From there, click on the "Capacity" tab and look for the "Billing Mode" section. Click on the "Edit" button, select "On-Demand," and then click "Save."

Adjust capacity settings:

Once you have switched to On-Demand Capacity mode, DynamoDB will automatically handle the capacity provisioning for your table. You will no longer need to manually provision capacity units or monitor usage to avoid overprovisioning.

However, it's important to note that On-Demand Capacity mode may result in higher costs if your usage patterns are unpredictable or if you experience sudden spikes in traffic.

To manage costs, you may need to adjust your application's query patterns or consider switching back to Provisioned Capacity mode if your usage becomes more predictable.

Monitor your usage:

After switching to On-Demand Capacity mode, it's important to monitor your usage to ensure that you're not incurring unexpected costs. You can use the CloudWatch metrics and alarms to track your usage and adjust your application's query patterns as needed.

Using Global Secondary Indexes (GSIs) Improperly

GSIs are a powerful feature in DynamoDB that allow you to create indexes on non-key attributes, which can improve query performance. However, improper use of GSIs can lead to increased costs and decreased query performance.

One common mistake is to overuse GSIs by creating too many indexes on a table. Each GSI requires additional storage and provisioned throughput, which can result in increased costs without providing significant query performance improvements.

In order to avoid these mistakes, it’s important to make sure that the indexes being used on each table are there because they’re necessary to complete business functions, optimize efficiencies, or reduce cost in your AWS cloud environment.

A user of AWS might create too many indexes on a table in DynamoDB because they believe that having more indexes will improve the performance of their queries.

While it's true that indexes can improve query performance by allowing you to retrieve data based on different attributes, creating too many indexes can lead to over-usage of GSIs, which can have a negative impact on performance and increase costs. Here are a few reasons why:

Increased write costs:

When you create a GSI, DynamoDB creates a new index that stores a copy of the data in your table. This can significantly increase your write costs because you'll be writing to both the table and the GSI. If you create too many GSIs, you'll end up paying for a lot of extra write capacity that you may not actually need.

Increased storage costs:

Each GSI that you create also requires additional storage, which can increase your storage costs. If you create too many GSIs, you may end up paying for a lot of extra storage that you may not actually need.

Decreased query performance:

While indexes can improve query performance, creating too many GSIs can actually decrease performance. This is because each index adds overhead to read and write operations, which can slow down your queries.

Additionally, if you're querying a GSI that's not frequently used, you may end up wasting read capacity units that could be better used for other queries.

Increased maintenance overhead:

Finally, creating too many indexes can also increase your maintenance overhead. Each index requires additional resources to maintain and update, which can add complexity to your application and increase the time and effort required to maintain your DynamoDB tables.

Not Using Time to Live (TTL) Properly

TTL is a feature in DynamoDB that allows you to automatically delete expired items from your table. This can be useful for managing the size of your table and reducing storage costs.

One common mistake is to set the TTL too aggressively, which can result in frequent deletes and unnecessary write capacity usage.

On the other hand, setting the TTL too conservatively can result in increased storage costs due to a larger number of expired items that are not deleted.

To avoid this mistake, it's important to carefully consider the appropriate TTL value for your use case. You can use CloudWatch metrics to monitor the number of expired items and adjust the TTL accordingly.

Using TTL

To use TTL properly in DynamoDB, you'll need to determine which items should have TTL, enable TTL for your table, create a TTL attribute, set the TTL attribute for each item, and monitor expired items to ensure that they are being properly deleted.

By following these steps, you can use TTL to automatically delete expired items and manage the lifecycle of your data in DynamoDB.

Determine which items should have TTL:

The first step is to determine which items in your DynamoDB table should have TTL. TTL is used to automatically delete expired items from your table, so you'll need to decide which items have a finite lifespan and should be deleted when they expire.

For example, you might want to use TTL to automatically delete session data or temporary files after a certain amount of time.

Enable TTL for your table:

Once you've identified which items should have TTL, you can enable TTL for your table by visiting the DynamoDB console, selecting the table you want to modify, and clicking on the "Time to Live" tab. From there, you can select the attribute that will determine when items expire and set the TTL value in seconds.

Create a TTL attribute:

If you don't already have an attribute in your table that can be used to determine when items expire, you'll need to create one. This attribute should be a number that represents the expiration time of each item in Unix epoch time format.

Set the TTL attribute for each item:

Once you have a TTL attribute, you'll need to set it for each item that should have TTL.

You can do this by updating each item with the current timestamp plus the TTL value in seconds. For example, if your TTL value is 3600 seconds (1 hour), you would set the TTL attribute to the current timestamp plus 3600.

Monitor expired items:

After you've enabled TTL for your table, DynamoDB will automatically delete expired items based on the TTL attribute. However, it's important to monitor expired items to ensure that they are being properly deleted.

You can use the CloudWatch metrics and alarms to monitor expired items and ensure that your TTL settings are working as expected.

Not Using DynamoDB Streams Properly

DynamoDB Streams is a feature in DynamoDB that allows you to capture data modification events in real-time.

Streams can be used for a variety of use cases, including data replication, real-time analytics, and trigger-based processing. However, improper use of Streams can lead to increased costs and decreased query performance.

One common mistake is to enable Streams on tables that do not require it. Streams can increase the cost of table writes and can result in decreased query performance if not used properly.

To avoid this mistake, it's important to carefully consider the use cases for Streams and enable it only for tables that require it.

You can also configure Streams to capture only the necessary data modification events to reduce the cost and improve query performance.

Architecting for DynamoDB requires careful consideration of many factors, including partition keys, provisioned throughput, capacity modes, GSIs, TTL, and Streams.

By avoiding the common mistakes outlined in this post and following best practices, you can ensure optimal query performance and cost efficiency for your DynamoDB applications.

Common Mistakes People Make Architecting for DynamoDB That Lead to Cost Overruns