Scaling Redis™ Memory, Data Management and Operations

Scaling Redis™ Memory, Data Management and Operations

RedisConf19 in San Francisco is April 2-3, so I thought I’d take this opportunity to provide a quick overview on how Ultrastar® products can help reduce the costly problem of scaling Redis data sets, conduct critical Redis data management and operations, and provide some storage tidbits on Apache Hadoop®-as-a-Service when Redis is used as a frontend to Hadoop.

Memory and NVMe™ Storage Demands of Redis

Scaling Redis in-memory and running the underlying Redis data management and operations functions – persistent storage, backup, containerized microservices, etc. – are headaches for Redis administrators and cloud architects.

Redis requires a lot of DRAM since it’s an in-memory database in order to operate at peak performance levels.  Failure to provide enough DRAM for growing Redis data sets puts digital initiatives at a disadvantage.

Just as important to plenty of DRAM, is the underlying performance of Redis data management and operations using NVMe SSD storage. NVMe SSD storage provide the necessary tier for Redis data management and operational functions like data persistence, backups, running containerized microservices, disaster recovery, multi-cloud architectures, and more.

Cloud Architects, DevOps, Analytics teams, Digital Platform managers, and other critical Redis administrator roles stand to benefit by understanding how the Ultrastar portfolio can reduce Redis infrastructure costs through memory drives and how NVMe SSD storage options can be the basis for critical and reliable infrastructure.

Overview

A quick overview table on how Ultrastar products solve Redis data growth and management, and Hadoop, followed by further details and resources on each product:

Ultrastar DC ME200 Memory Extension Drive

  • Scale Redis In-Memory data sets at a better TCO than physical DRAM-based clusters
  • Redis performance depends on total available memory for larger and larger data sets

 

Ultrastar DC SN200 NVMe SSDs – AIC HH-HL and 2.5” U.2

  • Low-latency NVMe storage for Redis data persistence, ephemeral storage, (e.g., replication files, logs), backups, etc.
  • Running Redis-related containerized microservices and Redis data operations with high dependability and predictable performance is mission-critical

 

Ultrastar DC SN630 NVMe SSD – 2.5” U.2

  • Economies-of-scale NVMe SSD storage for Disaster Recovery sites, multi-cloud architectures, analytics, and more
  • The balance of NVMe storage performance with storage capacity is critical to keep Redis infrastructure costs in check

 

Ultrastar Capacity-Optimized HDDs – various capacities

  • HDFS for Hadoop-as-a-Service hard disk drive capacity storage when Redis is used as a frontend cache to Hadoop
  • Reliable HDDs for critical Hadoop HDFS performance

 

Ultrastar DC ME200 Memory Extension Drive – Scaling Redis Data Sets at a Better TCO

Scaling Redis memory/DRAM configurations across more than a few nodes has the following problems:

• DRAM cost is expensive, particularly for 64GiB and 128GiB DIMMs

• Nodes packed with DIMMs, for the sake of scaling, lead to woefully poor CPU utilization and an inefficient core:DRAM ratio, hence a waste of computing resources

• Managing large compute clusters has inherent CAPEX and OPEX costs

Thankfully, there is a new method to address the Redis scalability problems for growing data sets. We recently announced the Ultrastar DC ME200 Memory Extension Drive, a solution to augment physical DRAM to create virtual memory pools. It delivers near-DRAM performance at a far lower cost and can alleviate the pains and cost of scaling memory configurations for Redis.

Comparison of two Ultrastar memory configurations to a baseline configuration of the same dual socket system with 1.5TB of DDR4 DIMMs . Based on internal testing. Performance may vary based on operating environment.

We’ve documented Redis benchmark performance for SET and GET 1kB and 100kB message sizes using the DC ME200 Memory Extension in this 2-page pdf available here.

2-Minute Video – What is Memory Expansion?

High-Performance Redis Data Management and Running Containerized Microservices Using Low-Latency NVMe SSDs

Redis data persistence, ephemeral storage, running containerized microservices, and other Redis data management functions, all demand the low latency storage performance of NVMe SSDs.

NVMe is one of the latest standardized technologies for high performance requirements. If you are not familiar with what exactly NVMe is, our technical guide is available here. It’s a key technology you need to understand for high-performance and data-intensive workloads. Here are two great examples from our portfolio, and the type of workloads they are best fit for Redis.

Ultrastar DC SN200 NVMe SSDs – Reliable and Low-Latency storage for Redis Data Management and Operations

Low-latency NVMe performance means your critical Redis data management features complete in a timely manner. The Ultrastar DC SN200 series is the top-end Ultrastar NVMe SSDs for reliable and ultra-low latency. It is a logical choice where low response time is critical.

Whether spanning Redis data across DRAM and SSDs, or Redis backups (snapshots and/or append operations), the Ultrastar DC SN200 series provides the necessary performance for Redis data management and running containers. When running Redis-related containerized microservices on Ultrastar DC SN200 you can be assured with predictable and reliable performance of the random storage I/O driven by containers, to ensure optimal end-user experience.

The solution can be used in both an Add-in Card (AIC) HH-HL and U.2 NVMe format.

Economies of Scale NVMe SSD – Ultrastar DC SN630 NVMe SSD

The advantages of NVMe come in many different options. For economies of scale NVMe storage requirements, a solution like the Ultrastar DC SN630 NVMe SSD can address capacity requirements, with drives up to 7.68TB[1], and the reliability for other critical Redis data management requirements.

For example, Redis Disaster Recovery sites, analytics, and multi-cloud architectures make the Ultrastar DC SN630 a fantastic solution for the capacity storage of Redis NVMe SSD requirements.

Get to know the Ultrastar DC SN630 NVMe SSD by downloading the data sheet here.

Getting the Most Capacity for Hadoop-as-a-Service with Redis Frontend

Redis as a frontend cache to Hadoop-as-a-Service can accelerate the ordinary performance of HDFS on HDDs. Yet a scalable HDFS capacity tier requires both reliable and high capacity HDDs to achieve the optimal cost and power efficiency.

HDD technology is advancing by great leaps, and it’s what’s enabling big data today. Our portfolio of Ultrastar HDDs include both helium-based models and air-filled models of varying storage capacity points and interfaces. These solutions have a long track record of proven quality and performance used by major OEMs, systems integrators, and leading cloud service providers to power their big data services.

At RedisConf19? Visit us at Booth #19

Make sure to stop by booth #19 at the event. We’d be glad to discuss how Ultrastar Memory Extension Drives and the other Ultrastar products can help enable your scalable Redis strategies and the following:

  • How Ultrastar memory works with Intel® Xeon® processors and the minimum amount of physical DRAM per node
  • Our published Redis benchmark of SET and GET 1kB and 100kB message sizes
  • The Ultrastar memory drive POC kit
  • Ultrastar NVMe SSDs for persistent storage, ephemeral storage
  • Redis high available, hybrid cloud and multi-cloud architectures
  • Redis Eventual Consistency, Strong Eventual Consistency, and Casual Consistency
  • Backup (snapshots and append) and Disaster Recovery
  • Storage options for running containerized microservices and running containers
  • Redis caching for HDD-based Hadoop-as-a-Service and Data Lakes

Learn More

• Get to know our data center NVMe SSD solutions

• Read the solution brief: How to scale Redis Memory Configurations for larger data stores and caching

[1] One gigabyte (GB) is equal to 1,000MB (one billion bytes) and one terabyte (TB) is equal to 1,000GB (one trillion bytes) when referring to solid-state capacity. Accessible capacity will vary from the stated capacity due to formatting and partitioning of the drive, the computer’s operating system, and other factors.

Related Stories

Net-Zero Supply Chain