You have many times heard about the two words when people talk about Cosmos DB, Latency and Throughput.

Latency means the end to end time required by your user to see the data. Latency should be as minimum as possible for better user experience. The latency will be low, when the data (and or application) is near to the location of user. So, the latency can be lowered by Geo-replication feature of Cosmos DB.

Throughput is measured in terms of how many requests per seconds can be processed by an application. For Cosmos DB, when you configure for throughput, it asks for number of request units per second. Please note that request unit per second is not same as requests per second.

In this article, we will see what is request unit, how it is related to requests per second and how to calculate it.

What is Request Unit ?

Every request to the Cosmos DB has different needs for resources. It may need more or less memory, it may need more or less computational units.

The cost of all database operations is normalized by Azure Cosmos DB and is expressed by Request Units (or RUs, for short). You can think of RUs per second as the currency for throughput. RUs per second is a rate-based currency.  A request unit (RU) abstracts the system resources such as CPU, IOPS, and memory that are required to perform the database operations supported by Azure Cosmos DB.

The cost to read a 1 KB item is 1 Request Unit (or 1 RU). A minimum of 10 RU/s is required to store each 1 GB of data.

So it may happen that one request to Cosmos DB, may be consuming 2 RU, while other request to Cosmos DB may consume 4 RU. That’s why I said earlier, number of requests per second is not same as number of request units per seconds.

But this is how request units are calculated by Azure. How will you know how many request units are required for your application ?

Estimating request units

If you have not created a Cosmos DB account, you may want to follow steps in my previous blog to create the Cosmos DB account.

If you have created the account already, then you know that when you want to add new container in the Cosmos DB account, you need to specify the RU per seconds.

You need to be very cautious while specifying this input. If you exhaust the RU while processing incoming requests, the Cosmos DB will throttle the requests. Throttle meaning, it would mark the request as failed.

But the question is how can you calculate right request units for your application.

You can obviously play with request units per second value during load testing and try to find out optimal value for your application.

But if you do not have any idea about the number and if you start with the first number 400, it may take a while to come to the right number of RUs.

There is a capacity calculator for Cosmos DB which can caluculate the approximate capacity. You need to provide size of your database and how have you configured the Cosmos DB account (e.g. how many regions you are replicating the Cosmos DB and if you have enabled multi-master scenario, what is size of database).

This planner gives you approximate capacity to start with. Then you can analyze the results of your load testing to come to the desired RU per second value.

Then you can go to the Cosmos DB account, and select metrics from the left side menu. This panel will show you the number of requests which exceeded the capacity.

Options for setting throughput

You can configure throughput (i.e. RU per second) value either on database level or on container level.

Setting throughput at container level

While creating new container, you can specify the throughput at the container level. The throughput is exclusively reserved for the container.

This is most frequently used option. The throughput specified at container level gets evenly distributed among the physical partitions.

The partition key results in creating logical partitions of the data in a given container. One or more of the logical partitions reside on a physical partition server.

Assuming that you have chosen a partition key in such a way that distributes logical partitions evenly among the physical partitions, then the throughput will also be evenly distributed among the logical partitions.

You can set the container level throughput while creating the container as shown in below diagram:

Setting throughput at database level

When you specify the throughput at database level, the throughput is shared among all the containers. These containers are sometimes are called as shared database containers.

If you specify the throughput at container level, then the throughput at database levels is ignored.

Because all containers within a database share the resources available on a machine, you naturally do not get predictable performance on any specific container. 

In fact, if we have to be very specific, then the database level throughput is shared among the partition sets. Every partition set contains 25 containers at max.

So if you have set 2500 RUs per second at database level and you have 25 containers, then every container may have maximum throughput of 1k RUs per second. When 26th container is created, then a new partition set is created and then the throughput is divided equally among the two partition sets. So first 25 containers have on ly 1250 RUs in total available for sharing.

If you want to specify the throughput at the database level, then you can select the database from Data Explorer as show in below snapshot. Then you can select Scale option to change the throughput.

Setting throughput at both levels

This we already have discussed in previous section. Let’s say you have one database “DB1” and it has throughput of “X” RUs per second.

This DB1 has five containers, C1, C2, C3, C4, and C5. But C2 has a dedicated throughput specified while creating the container, then the throughput specified at database level would be shared only among remaining 4 containers. C2 will have dedicated throughput.

Please note that you have to select a checkbox “Provision dedicated throughput for this container“. This can be done only while creating a new container.

Billing Of Cosmos DB Account

Now, that when we are discussing about Cosmos DB throughput, there is one important aspect. The billing of Cosmos DB account depends on two things, storage and access rates (in terms of RUs per second)

Most of the billing is actually decided by RUs and storage costs are very less as compared to access costs. You can check that using Cosmos DB capacity calculator.

I hope this article has helped you to understand the concept of RUs. Let me know your thoughts.