Our goal in this series is to help clarify and provide understanding about how workloads are priced in the cloud for our customers. We’ve previously written about Virtual Machines, Local SSD, and Data Warehouses. In this post, we'll expand on what we've written about running NoSQL databases on Google Compute Engine by covering pricing of managed NoSQL databases in the cloud.
For some background on what we mean when we say "NoSQL," have a look at our first post. NoSQL databases are defined more by what they aren’t than what they are. They aren't traditional, relational databases. They each have different features that solve different problems, so discussing them as if they're interchangeable doesn't quite work. However, there are broader categories that we can use to group NoSQL databases.
There are three groups that represent the most popular applications: wide-column databases, key-value stores, and document databases. For example, MongoDB is a document database and Cassandra is a wide-column database. In this post, we will examine two managed databases that fall into two of the three categories: Google Cloud Bigtable and Amazon DynamoDB.
Overview of the Databases
Google Cloud Bigtable was introduced in 2015 as a new, managed NoSQL database on Google Cloud Platform. Bigtable started out as an internal tool, much like BigQuery began life as the internal tool, Dremel. Internally, Google has used Bigtable for years, and we released a white paper describing it in 2006. Bigtable is one of the databases that started the NoSQL database market. Commonly referred to as a type of wide-column-store database, the two generally defining features of this class are:
-
They store data along columns, rather than rows.
-
They use super-column families, which are groupings of columns. These groupings are usually defined when the schema is created, and they provide some structure to the schema.
Otherwise, wide-column databases, and Cloud Bigtable specifically, are similar in function to other key-value stores. Cloud Bigtable’s focus is to provide a fast, massively scalable database that is fully managed for our customers.
Amazon DynamoDB was introduced in 2012 as Amazon’s second, managed NoSQL service. The first, named SimpleDB, had been around since 2007. DynamoDB is a key-value store, which allows for schemaless data storage, with all the data being stored as a value and an indexed key. DynamoDB is based on the principles of Dynamo, a system created and used internally at Amazon, and described in a paper published in 2007.
Pricing The Databases
Cloud Bigtable has two pricing parameters: price per node and price for amount of data stored. The number of nodes is straightforward and the price for storage only depends on the type of storage you chose: solid-state drive (SSD) or hard-disk drive (HDD). Since Cloud Bigtable is currently in beta, the only choice is SSD, so we must use that for our comparison.
DynamoDB sets pricing based on the amount of data stored. It doesn’t have a concept of nodes for pricing, though it does store data across multiple partitions on the backend, so it also prices on both read- and write-operations-per-second. However, there are three ways that AWS charges for reads and writes:
-
DynamoDB allows you to read data that is either consistent or eventually consistent, the difference being the cost—you pay half for eventually consistent reads.
-
You can pay the on-demand rate for reads and writes.
-
You can pay for DynamoDB Reserved Capacity (RC).
With RC pricing, you pay a certain amount of money up front and get a lower rate for the amount of capacity that you use. Like Reserved Instances for EC2 and Redshift, you have the choice of locking in the RC pricing for one year or three years.
Sizing The Databases
Because we are comparing these databases, we should establish the parameters for how we are comparing them.
The minimum-size cluster for Cloud Bigtable is three nodes. Each node provides 10,000 queries per second (QPS), which gives a total of 30,000 QPS for a three node cluster. Since this pricing is scalar, meaning three times the number of nodes will be three times the cost, a cluster of this size and performance is a reasonable comparison point.
Because any of the 30,000 queries could be a read or a write for Cloud Bigtable, we need to decide how the queries will be split up on DynamoDB; we need to specify those numbers in order to price them. For the sake of simplicity, we will simply split them in half into 15,000 reads and 15,000 writes.
Finally, we need to consider the amount of data we’re storing. Neither of the services have any kind of tiered pricing for data storage, so, for the sake of this example, let’s just use 10 TB of data stored for each.
Pricing Comparison
We completed these calculations on September 4, 2015, and have included the output prices in this post. It's possible that pricing or the calculator changed following the publishing of this post.
Additionally, while AWS does provide a calculator for DynamoDB, it does not include RC pricing. We have calculated these prices and included them when necessary.
All prices below, unless otherwise stated, are monthly costs.
Cloud Bigtable
This amount represents $1,423.50 for the nodes and $1,740.80 for storage.
DynamoDB, on-demand
This amount represents $8,550.13 for provisioned capacity and $2,803.75 for storage.
Just running on demand, Cloud Bigtable clearly is far more cost effective. It’s more than $1000 cheaper for storage and DynamoDB is 6 times the cost for operations! When making these kind of comparisons, it can be hard to draw performance conclusions because workloads can vary significantly. In this case, the performance characteristics of these services are built into the pricing. So we can say that for similarly performing databases, Cloud Bigtable is less than a third the cost of DynamoDB. We say "similarly" because they’re not exactly the same. With Cloud Bigtable, you have the flexibility to run any kind of operation you want and it’s the same price. So whether or not your application is read-heavy or write-heavy, it will cost that amount. DynamoDB, on the other hand, has different prices for the different operations. Writes are more expensive than reads, and consistent reads are more expensive than eventually consistent reads.
Let’s see how much of a difference it would make to change all the reads to eventually consistent. This would certainly change the performance of your DynamoDB usage, but let’s see if it drops the price enough to be competitive.
DynamoDB, on-demand, with eventually consistent reads
This amount represents $7837.62 for provisioned capacity and again $2803.75 for storage.
Using only eventually consistent reads doesn’t make an appreciable difference to the overall cost as compared to Cloud Bigtable. We also looked at doing 100% reads and even 100% eventually consistent reads. While both those scenarios are still cheaper, they are not cheaper than Cloud Bigtable and are also unrealistic use cases, so we don't list them here.
However, DynamoDB does offer RC pricing. So let’s take a look at that too.
DynamoDB, 1-year Reserved Capacity
Estimate:
Monthly: $4,480.89
Upfront: $26,955.00
Effective monthly: $6,727.14
The effective monthly cost is $3,923.39 for provisioned capacity and, again, $2803.75 for storage. This lowers your DynamoDB costs by nearly half and the actual provisioned capacity cost to less than half. However, this is still more expensive than the monthly cost of of Cloud Bigtable, and you have to pay more than $25,000 up front for the privilege of locking in your capacity.
Finally, let’s take a look at the 3-year RC Pricing.
DynamoDB, 3-year Reserved Capacity
Estimate:
Monthly: $3,867.03
Upfront: $32,346.00
Effective Monthly: $4,765.53
The effective monthly cost includes $1,961.78 for provisioned capacity and, again, $2,803.75 for storage costs. So even after you pay over $32,000 up front and lock your usage pattern in for the next three years, you are still paying over $500 more for the performance and over $1,000 more for the storage.
If you look back to our first post in this series on understanding cloud pricing, we wrote about how there’s a cost of capital investment. Most businesses pay about 7% per year cost of capital. That would add nearly $7,000 ($6,792) to the overall cost of running DynamoDB.
Summary
We hope you’ve found this useful. Pricing services across different cloud platforms can be challenging, and our goal here is to help clarify how to do that. You can see that Cloud Bigtable is a cost-effective option for running NoSQL workloads, without any upfront investment or long-term commitments. Beyond just the value that Cloud Bigtable provides, it also delivers high performance and can scale to whatever your needs are. If you want to learn more, check out our documentation, read up on understanding Cloud Bigtable performance, or read through our in depth guide to schema design for time series data.
