In-Memory Key-Value Stores in NoSQL Databases

By Adam Fowler

Businesses must ensure that their data is durable by keeping it safe on disk. However, in-memory databases implemented with NoSQL are also useful in some instances, such as the following:

  • High-speed caching: To remove read workloads from the database of record, to reduce the cost of hardware and software licenses, and to prevent distributed denial-of-service (DDoS) attacks from affecting a live system.

  • Transient data holding: Data that isn’t very important and that has a known lifespan. A typical web application’s details on users’ sessions are good examples.

  • Analysis before storage: For example, large memory systems are used by scientists to analyze stellar observations. Most of the time, telescopes are looking at a blank bit of sky — no need to store that data! Scientists analyze the data quickly in memory, and store only what’s useful.

Because of their uncomplicated nature, many in-memory databases are also key-value stores. In-memory use also lends itself to high-speed applications. Retrieving a record using its unique key is the quickest way to retrieve data, so key-value stores and in-memory databases are a natural fit.

Redis began as an in-memory database. Indeed, Redis can still be used in this way. Redis does operate as a single process, though, so you need to run multiple instances of Redis on each server in order to get full utilization of its resources.

Hazelcast is an in-memory NoSQL database that replicates its data to other Hazelcast nodes in the cluster. It is an open-source product, but it’s also offers a commercial product (Hazelcast Enterprise) with more features. Hazelcast is used within the commercial version of the OrientDB triple store. OrientDB uses Hazelcast in order to provide high availability. Hazelcast effectively provides replication of OrientDB’s data structures.

Hazelcast’s community website has an excellent section on using Hazelcast as a NoSQL key-value store. The main page of this website also provides basic code examples for several use cases.

Hazelcast targets the in-memory data grid market. This is where computer resources (RAM and CPU) are shared in large grids used to power applications. Financial services often use data grids. Traditional commercial options include Oracle Coherence, VMWare Pivotal Gemfire, and Software AG Terracotta. Hazelcast is a good open-source alternative for these products.