Architecture Maps

Redis Internals

Interactive architecture map of Redis's internal subsystems — single-threaded event loop, memory management, RDB/AOF persistence, cluster gossip protocol, replication, streams, Lua scripting, modules, and pub/sub.

Open Source (BSD) Since 2009 Single-Threaded Event Loop In-Memory Data Store 10+ Data Structures
01

Architecture Overview

Redis is an in-memory data structure store used as a database, cache, message broker, and streaming engine. Its single-threaded event loop processes commands sequentially, eliminating lock contention and delivering sub-millisecond latency at millions of operations per second.

1M+
Ops/sec (single node)
<1ms
Avg Latency
10+
Data Structures
16384
Hash Slots (Cluster)
Redis High-Level Architecture
graph TD
    CLIENT["Client
(RESP Protocol)"] --> EVENTLOOP["Event Loop
(ae library)"] EVENTLOOP --> CMDPROC["Command
Processor"] CMDPROC --> DS["Data Structures
(dict, ziplist, skiplist...)"] DS --> MEM["Memory
Manager (jemalloc)"] CMDPROC --> PERSIST["Persistence
(RDB + AOF)"] CMDPROC --> REPL["Replication
Engine"] CMDPROC --> PUBSUB["Pub/Sub
Engine"] CMDPROC --> LUA["Lua
Scripting"] CMDPROC --> MODULES["Module
API"] REPL --> REPLICA["Replica
Nodes"] EVENTLOOP --> CLUSTER["Cluster Bus
(Gossip)"] style CLIENT fill:#22143d,stroke:#6366f1,color:#e9e0ff style EVENTLOOP fill:#2a0015,stroke:#ef4444,color:#fef3c7 style CMDPROC fill:#1a0f30,stroke:#f59e0b,color:#fef3c7 style DS fill:#1a0f30,stroke:#ef4444,color:#fef3c7 style MEM fill:#1a0f30,stroke:#dc2626,color:#fef3c7 style PERSIST fill:#1a0f30,stroke:#f59e0b,color:#fef3c7 style REPL fill:#1a0f30,stroke:#14b8a6,color:#e9e0ff style PUBSUB fill:#1a0f30,stroke:#d946ef,color:#e9e0ff style LUA fill:#1a0f30,stroke:#eab308,color:#fef3c7 style MODULES fill:#1a0f30,stroke:#3b82f6,color:#e9e0ff style REPLICA fill:#22143d,stroke:#14b8a6,color:#e9e0ff style CLUSTER fill:#1a0f30,stroke:#6366f1,color:#e9e0ff
02

Event Loop & Threading Model

Redis uses a single-threaded event loop built on its own ae library, which wraps epoll (Linux), kqueue (macOS/BSD), or select. The main thread handles all command processing. Since Redis 6.0, I/O threads can be enabled to parallelize network read/write, but command execution remains single-threaded.

Event Loop Processing Cycle
graph LR
    AE["ae Event
Loop"] --> FE["File Events
(socket I/O)"] AE --> TE["Time Events
(serverCron)"] FE --> READ["Read Query
Buffer"] READ --> PARSE["Parse RESP
Protocol"] PARSE --> LOOKUP["Command
Lookup Table"] LOOKUP --> EXEC["Execute
Command"] EXEC --> WRITE["Write Reply
Buffer"] WRITE --> AE TE --> CRON["serverCron
(100ms default)"] CRON --> BG["Background
Tasks"] style AE fill:#2a0015,stroke:#ef4444,color:#fef3c7 style FE fill:#1a0f30,stroke:#f97316,color:#fef3c7 style TE fill:#1a0f30,stroke:#f97316,color:#fef3c7 style READ fill:#22143d,stroke:#6366f1,color:#e9e0ff style PARSE fill:#22143d,stroke:#6366f1,color:#e9e0ff style LOOKUP fill:#22143d,stroke:#6366f1,color:#e9e0ff style EXEC fill:#1a0f30,stroke:#ef4444,color:#fef3c7 style WRITE fill:#22143d,stroke:#6366f1,color:#e9e0ff style CRON fill:#1a0f30,stroke:#eab308,color:#fef3c7 style BG fill:#22143d,stroke:#14b8a6,color:#e9e0ff
I/O Threading (Redis 6.0+)

With io-threads enabled, the main thread distributes pending client reads/writes to a pool of I/O threads. Each I/O thread handles network serialization in parallel, but command execution still occurs on the main thread. This preserves atomicity guarantees while improving throughput on multi-core machines under high network load.

File Events

Socket accept, read, and write handlers. Each connected client has a query buffer and reply buffer managed by the event loop. Readable events trigger command parsing; writable events flush response buffers.

networking.c, ae.c

Time Events

The serverCron function fires every 100ms (configurable via hz). It handles: active key expiration sampling, client timeout checks, replication heartbeats, cluster pings, lazy-free tasks, and memory defragmentation.

server.c:serverCron()

Background Threads

Three background threads (BIO) handle slow tasks: closing file descriptors, fsyncing AOF, and lazy-freeing large objects. These never touch the main data dictionary, avoiding contention.

bio.c (BIO_CLOSE_FILE, BIO_AOF_FSYNC, BIO_LAZY_FREE)
03

Memory Management

Redis stores all data in memory, using jemalloc as the default allocator. Memory efficiency is critical: Redis employs compact encodings, shared integer objects, and multiple eviction policies to manage its memory footprint. The maxmemory directive sets an upper bound, and eviction policies determine what happens when it is reached.

Memory Layout & Eviction Flow
graph TD
    ALLOC["jemalloc
Allocator"] --> DICT["Main Dict
(hash table)"] ALLOC --> EXPIRES["Expires Dict
(TTL tracking)"] ALLOC --> OBJ["redisObject
(type+encoding+ptr)"] OBJ --> STR["SDS Strings"] OBJ --> LIST["Listpacks /
Quicklists"] OBJ --> HASH["Listpacks /
Hash Tables"] OBJ --> ZSET["Listpacks /
Skiplists"] OBJ --> SET["Listpacks /
Hash Tables"] subgraph Eviction["Eviction Pipeline"] CHECK["maxmemory
Check"] --> POLICY["Eviction
Policy"] POLICY --> LRU["LRU / LFU
Approximation"] POLICY --> TTL["Volatile-TTL
Shortest TTL"] POLICY --> RAND["Random
Sampling"] LRU --> EVICT["Free Key"] TTL --> EVICT RAND --> EVICT end DICT --> CHECK style ALLOC fill:#2a0015,stroke:#dc2626,color:#fef3c7 style DICT fill:#1a0f30,stroke:#ef4444,color:#fef3c7 style EXPIRES fill:#1a0f30,stroke:#f97316,color:#fef3c7 style OBJ fill:#22143d,stroke:#f59e0b,color:#fef3c7 style STR fill:#22143d,stroke:#6366f1,color:#e9e0ff style LIST fill:#22143d,stroke:#6366f1,color:#e9e0ff style HASH fill:#22143d,stroke:#6366f1,color:#e9e0ff style ZSET fill:#22143d,stroke:#6366f1,color:#e9e0ff style SET fill:#22143d,stroke:#6366f1,color:#e9e0ff style CHECK fill:#1a0f30,stroke:#dc2626,color:#fef3c7 style POLICY fill:#1a0f30,stroke:#ef4444,color:#fef3c7 style LRU fill:#22143d,stroke:#f97316,color:#fef3c7 style TTL fill:#22143d,stroke:#f97316,color:#fef3c7 style RAND fill:#22143d,stroke:#f97316,color:#fef3c7 style EVICT fill:#2a0015,stroke:#dc2626,color:#fef3c7

Eviction Policies

Policy Scope Algorithm Use Case
allkeys-lru All keys Approximated LRU (sampling) General-purpose cache
allkeys-lfu All keys Approximated LFU (counter decay) Frequency-biased cache
volatile-lru Keys with TTL Approximated LRU Mixed persistent + cache
volatile-ttl Keys with TTL Shortest remaining TTL Time-sensitive data
noeviction N/A Returns OOM error Database (no data loss)
LRU Approximation

Redis does not maintain a true LRU linked list. Instead, each key's redisObject stores a 24-bit clock field (LRU mode) or an 8-bit logarithmic frequency counter + 16-bit decrement time (LFU mode). On eviction, Redis samples maxmemory-samples random keys (default 5) and evicts the best candidate. Increasing the sample size improves approximation quality at the cost of CPU.

04

Data Structures & Encodings

Every Redis value is wrapped in a redisObject (16 bytes: type, encoding, LRU/LFU clock, refcount, pointer). Each logical type supports multiple internal encodings that Redis switches between automatically based on element count and size thresholds.

Type-to-Encoding Mapping
graph LR
    STRING["STRING"] --> INT["int
(integer value)"] STRING --> EMBSTR["embstr
(≤ 44 bytes)"] STRING --> RAW["raw
(SDS)"] LISTT["LIST"] --> LP1["listpack
(≤ 128 elements)"] LISTT --> QL["quicklist
(linked listpacks)"] SETT["SET"] --> LP2["listpack
(≤ 128 elements)"] SETT --> INTSET["intset
(all integers)"] SETT --> HT1["hashtable"] HASHT["HASH"] --> LP3["listpack
(≤ 128 fields)"] HASHT --> HT2["hashtable"] ZSETT["ZSET"] --> LP4["listpack
(≤ 128 elements)"] ZSETT --> SKIP["skiplist +
hashtable"] style STRING fill:#2a0015,stroke:#ef4444,color:#fef3c7 style LISTT fill:#2a0015,stroke:#f97316,color:#fef3c7 style SETT fill:#2a0015,stroke:#eab308,color:#fef3c7 style HASHT fill:#2a0015,stroke:#22c55e,color:#fef3c7 style ZSETT fill:#2a0015,stroke:#6366f1,color:#e9e0ff style INT fill:#22143d,stroke:#ef4444,color:#fef3c7 style EMBSTR fill:#22143d,stroke:#ef4444,color:#fef3c7 style RAW fill:#22143d,stroke:#ef4444,color:#fef3c7 style LP1 fill:#22143d,stroke:#f97316,color:#fef3c7 style QL fill:#22143d,stroke:#f97316,color:#fef3c7 style LP2 fill:#22143d,stroke:#eab308,color:#fef3c7 style INTSET fill:#22143d,stroke:#eab308,color:#fef3c7 style HT1 fill:#22143d,stroke:#eab308,color:#fef3c7 style LP3 fill:#22143d,stroke:#22c55e,color:#fef3c7 style HT2 fill:#22143d,stroke:#22c55e,color:#fef3c7 style LP4 fill:#22143d,stroke:#6366f1,color:#e9e0ff style SKIP fill:#22143d,stroke:#6366f1,color:#e9e0ff

SDS (Simple Dynamic String)

Redis's string type. Binary-safe with O(1) length lookup, pre-allocated capacity to reduce realloc calls, and five header sizes (sdshdr5 through sdshdr64) chosen by string length to minimize overhead.

core sds.c

Listpack

Compact, sequential, memory-efficient encoding for small collections. Replaced ziplist in Redis 7.0. Entries are variable-length encoded inline with no per-entry pointers, yielding excellent cache locality.

core listpack.c

Skip List

Probabilistic data structure backing sorted sets. O(log N) search, insert, and delete. Each node has a randomized tower of forward pointers (up to 32 levels). Used alongside a dict for O(1) score lookup by member.

core t_zset.c

Dict (Hash Table)

Incrementally-rehashing hash table. Maintains two tables; during rehash, each operation migrates a bucket from the old table to the new one. This spreads resize cost across many commands rather than a single blocking pause.

core dict.c
05

Persistence: RDB & AOF

Redis offers two complementary persistence mechanisms. RDB creates point-in-time snapshots as compact binary files. AOF logs every write operation for replay-based recovery. Since Redis 7.0, a hybrid AOF format combines both for faster restarts with minimal data loss.

Persistence Pipeline
graph TD
    CMD["Write
Command"] --> AOF_BUF["AOF
Buffer"] CMD --> DIRTY["Dirty Counter
(+1)"] AOF_BUF --> FSYNC{"fsync
Policy"} FSYNC -->|always| EVERY["fsync every
write"] FSYNC -->|everysec| SEC["fsync every
1 second"] FSYNC -->|no| OS["OS decides
when to flush"] EVERY --> AOF_FILE["AOF File
(appendonly.aof)"] SEC --> AOF_FILE OS --> AOF_FILE DIRTY --> BGSAVE{"BGSAVE
Trigger"} BGSAVE --> FORK["fork()"] FORK --> CHILD["Child Process
Writes RDB"] CHILD --> RDB_FILE["RDB File
(dump.rdb)"] FORK --> COW["Copy-on-Write
Parent continues"] subgraph Rewrite["AOF Rewrite (BGREWRITEAOF)"] RWFORK["fork()"] --> RWCHILD["Child rewrites
AOF from memory"] RWCHILD --> NEWFILE["New AOF
File"] end AOF_FILE -.-> RWFORK style CMD fill:#2a0015,stroke:#ef4444,color:#fef3c7 style AOF_BUF fill:#1a0f30,stroke:#f59e0b,color:#fef3c7 style DIRTY fill:#1a0f30,stroke:#f97316,color:#fef3c7 style FSYNC fill:#22143d,stroke:#eab308,color:#fef3c7 style EVERY fill:#22143d,stroke:#22c55e,color:#fef3c7 style SEC fill:#22143d,stroke:#22c55e,color:#fef3c7 style OS fill:#22143d,stroke:#6366f1,color:#e9e0ff style AOF_FILE fill:#1a0f30,stroke:#f59e0b,color:#fef3c7 style BGSAVE fill:#22143d,stroke:#dc2626,color:#fef3c7 style FORK fill:#1a0f30,stroke:#dc2626,color:#fef3c7 style CHILD fill:#22143d,stroke:#ef4444,color:#fef3c7 style RDB_FILE fill:#1a0f30,stroke:#ef4444,color:#fef3c7 style COW fill:#22143d,stroke:#14b8a6,color:#e9e0ff style RWFORK fill:#1a0f30,stroke:#f59e0b,color:#fef3c7 style RWCHILD fill:#22143d,stroke:#f59e0b,color:#fef3c7 style NEWFILE fill:#1a0f30,stroke:#eab308,color:#fef3c7

RDB Snapshots

Binary dump of the entire dataset. Triggered by save rules (e.g., "900 1" = snapshot if 1+ key changed in 900 seconds) or manual BGSAVE. Uses fork() + copy-on-write for non-blocking background saves. Compact and fast to load.

rdb rdb.c

AOF (Append-Only File)

Sequential log of every write command in RESP format. Three fsync policies trade off durability vs. throughput. AOF rewrite (BGREWRITEAOF) compacts the log by generating the minimal set of commands from current memory state.

aof aof.c

Hybrid AOF (7.0+)

Multi-part AOF: a base RDB snapshot plus incremental AOF diffs. On restart, Redis loads the RDB portion first (fast binary load), then replays only the incremental commands. Combines RDB speed with AOF durability.

hybrid aof.c
Copy-on-Write Hazard

When BGSAVE or BGREWRITEAOF forks, the child shares the parent's memory pages via COW. If the workload is write-heavy during the fork, the parent's page modifications trigger COW page copies, potentially doubling memory usage briefly. Operators monitor INFO persistence for latest_fork_usec and rdb_last_cow_size to detect this.

06

Replication

Redis uses asynchronous, leader-based replication. A single primary accepts writes and propagates them to one or more replicas. Full synchronization sends an RDB snapshot; partial resynchronization uses a replication backlog buffer to send only missed commands after brief disconnections.

Replication Handshake & Data Flow
sequenceDiagram
    participant R as Replica
    participant P as Primary

    R->>P: PING
    P->>R: PONG
    R->>P: AUTH password
    P->>R: OK
    R->>P: REPLCONF listening-port 6380
    R->>P: REPLCONF capa eof psync2
    P->>R: OK
    R->>P: PSYNC replid offset
    alt Full Resync
        P->>R: +FULLRESYNC replid offset
        P->>R: RDB snapshot (bulk transfer)
        P->>R: Buffered writes during RDB
    else Partial Resync
        P->>R: +CONTINUE
        P->>R: Backlog commands since offset
    end
    loop Steady State
        P->>R: Command stream (async)
        R->>P: REPLCONF ACK offset
    end
                

Replication Backlog

The primary maintains a fixed-size circular buffer (repl-backlog-size, default 1MB) of recent write commands. When a replica reconnects, it sends its last known replication offset. If that offset is still in the backlog, Redis performs a partial resync (PSYNC), sending only the missed commands. If the offset has been overwritten, a full resync is triggered.

Diskless Replication

With repl-diskless-sync yes, the primary streams the RDB directly to replica sockets without writing to disk first. The child process created by fork() serializes the dataset into the socket. This is beneficial when disk I/O is the bottleneck but network bandwidth is available.

07

Cluster & Gossip Protocol

Redis Cluster provides automatic data sharding across multiple nodes with built-in failover. The keyspace is divided into 16,384 hash slots distributed across primaries. Nodes communicate via a binary gossip protocol on a dedicated cluster bus port (data port + 10000).

Cluster Topology & Hash Slot Distribution
graph TD
    subgraph Shard1["Shard 1 (Slots 0-5460)"]
        P1["Primary A"]
        R1A["Replica A1"]
        R1B["Replica A2"]
        P1 --> R1A
        P1 --> R1B
    end

    subgraph Shard2["Shard 2 (Slots 5461-10922)"]
        P2["Primary B"]
        R2A["Replica B1"]
        P2 --> R2A
    end

    subgraph Shard3["Shard 3 (Slots 10923-16383)"]
        P3["Primary C"]
        R3A["Replica C1"]
        P3 --> R3A
    end

    P1 <--->|"Gossip
(cluster bus)"| P2 P2 <--->|"Gossip
(cluster bus)"| P3 P1 <--->|"Gossip
(cluster bus)"| P3 CLIENT2["Client"] -->|"CRC16(key)
mod 16384"| REDIRECT{"Slot
Routing"} REDIRECT -->|"slots 0-5460"| P1 REDIRECT -->|"slots 5461-10922"| P2 REDIRECT -->|"slots 10923-16383"| P3 style P1 fill:#2a0015,stroke:#ef4444,color:#fef3c7 style P2 fill:#2a0015,stroke:#ef4444,color:#fef3c7 style P3 fill:#2a0015,stroke:#ef4444,color:#fef3c7 style R1A fill:#22143d,stroke:#14b8a6,color:#e9e0ff style R1B fill:#22143d,stroke:#14b8a6,color:#e9e0ff style R2A fill:#22143d,stroke:#14b8a6,color:#e9e0ff style R3A fill:#22143d,stroke:#14b8a6,color:#e9e0ff style CLIENT2 fill:#22143d,stroke:#6366f1,color:#e9e0ff style REDIRECT fill:#1a0f30,stroke:#f59e0b,color:#fef3c7
Gossip Protocol & Failure Detection
graph LR
    NODE_A["Node A"] -->|"PING (random node)"| NODE_B["Node B"]
    NODE_B -->|"PONG + gossip data"| NODE_A
    NODE_A -->|"PING"| NODE_C["Node C"]

    subgraph FailureDetect["Failure Detection"]
        PFAIL["PFAIL
(local suspicion)"] --> GOSSIP_SPREAD["Gossip spreads
PFAIL flags"] GOSSIP_SPREAD --> MAJORITY["Majority of
primaries agree?"] MAJORITY -->|Yes| FAIL["FAIL
(cluster-wide)"] FAIL --> FAILOVER["Replica
Promotion"] end NODE_A -.-> PFAIL style NODE_A fill:#1a0f30,stroke:#6366f1,color:#e9e0ff style NODE_B fill:#1a0f30,stroke:#6366f1,color:#e9e0ff style NODE_C fill:#1a0f30,stroke:#6366f1,color:#e9e0ff style PFAIL fill:#22143d,stroke:#f97316,color:#fef3c7 style GOSSIP_SPREAD fill:#22143d,stroke:#eab308,color:#fef3c7 style MAJORITY fill:#1a0f30,stroke:#f59e0b,color:#fef3c7 style FAIL fill:#2a0015,stroke:#dc2626,color:#fef3c7 style FAILOVER fill:#1a0f30,stroke:#14b8a6,color:#e9e0ff
MOVED & ASK Redirections

When a client sends a command to the wrong node, it receives a -MOVED slot host:port redirect for permanently migrated slots, or -ASK slot host:port during active slot migration. Smart clients cache the slot-to-node mapping and update it on redirections, minimizing hops.

08

Pub/Sub

Redis Pub/Sub provides fire-and-forget message broadcasting. Publishers send messages to channels; all subscribed clients receive them in real-time. Messages are not persisted or buffered: if no subscriber is listening, the message is lost. Since Redis 7.0, sharded pub/sub routes messages by hash slot for cluster-aware broadcasting.

Pub/Sub Message Flow
graph TD
    PUB["Publisher
PUBLISH channel msg"] --> SERVER["Redis
Server"] SERVER --> CH["Channel
Subscription Dict"] SERVER --> PAT["Pattern
Subscription List"] CH --> SUB1["Subscriber 1
(exact match)"] CH --> SUB2["Subscriber 2
(exact match)"] PAT --> SUB3["Subscriber 3
(glob pattern)"] subgraph Sharded["Sharded Pub/Sub (7.0+)"] SPUB["SPUBLISH
channel msg"] --> SLOT_NODE["Node owning
CRC16(channel) slot"] SLOT_NODE --> SSUB1["Shard
Subscriber 1"] SLOT_NODE --> SSUB2["Shard
Subscriber 2"] end style PUB fill:#1a0f30,stroke:#d946ef,color:#e9e0ff style SERVER fill:#2a0015,stroke:#ef4444,color:#fef3c7 style CH fill:#22143d,stroke:#d946ef,color:#e9e0ff style PAT fill:#22143d,stroke:#d946ef,color:#e9e0ff style SUB1 fill:#22143d,stroke:#6366f1,color:#e9e0ff style SUB2 fill:#22143d,stroke:#6366f1,color:#e9e0ff style SUB3 fill:#22143d,stroke:#6366f1,color:#e9e0ff style SPUB fill:#1a0f30,stroke:#14b8a6,color:#e9e0ff style SLOT_NODE fill:#1a0f30,stroke:#f59e0b,color:#fef3c7 style SSUB1 fill:#22143d,stroke:#14b8a6,color:#e9e0ff style SSUB2 fill:#22143d,stroke:#14b8a6,color:#e9e0ff

Channel Subscriptions

Stored in a dict mapping channel name to a linked list of subscribing clients. SUBSCRIBE and UNSUBSCRIBE add/remove entries. PUBLISH iterates the list and writes the message to each client's output buffer.

pubsub pubsub.c

Pattern Subscriptions

PSUBSCRIBE registers glob patterns (e.g., news.*). On PUBLISH, Redis checks all patterns against the channel name using stringmatchlen(). Pattern matching is O(N) in the number of patterns.

pubsub util.c

Sharded Pub/Sub

SSUBSCRIBE/SPUBLISH route messages to the cluster node owning the channel's hash slot. Unlike global pub/sub, messages only cross nodes that own the relevant slot, reducing cluster bus traffic.

cluster pubsub.c
09

Streams

Redis Streams (introduced in 5.0) provide an append-only log data structure with consumer groups, inspired by Apache Kafka. Unlike Pub/Sub, stream entries are persisted, acknowledged, and can be replayed. Each entry has a unique ID (timestamp-sequence) and contains field-value pairs.

Stream Consumer Group Architecture
graph LR
    PROD["Producer
XADD"] --> STREAM["Stream
(Radix Tree +
Listpacks)"] STREAM --> CG1["Consumer Group A
(last-delivered-id)"] STREAM --> CG2["Consumer Group B
(last-delivered-id)"] CG1 --> C1["Consumer 1
XREADGROUP"] CG1 --> C2["Consumer 2
XREADGROUP"] CG1 --> PEL1["PEL
(Pending Entries)"] C1 -->|XACK| PEL1 C2 -->|XACK| PEL1 PEL1 -->|timeout| XCLAIM["XCLAIM /
XAUTOCLAIM"] style PROD fill:#1a0f30,stroke:#22c55e,color:#fef3c7 style STREAM fill:#2a0015,stroke:#ef4444,color:#fef3c7 style CG1 fill:#1a0f30,stroke:#22c55e,color:#fef3c7 style CG2 fill:#1a0f30,stroke:#14b8a6,color:#e9e0ff style C1 fill:#22143d,stroke:#6366f1,color:#e9e0ff style C2 fill:#22143d,stroke:#6366f1,color:#e9e0ff style PEL1 fill:#22143d,stroke:#f97316,color:#fef3c7 style XCLAIM fill:#22143d,stroke:#eab308,color:#fef3c7

Stream Internals

Internally, a stream is a radix tree where keys are entry IDs (128-bit: 64-bit ms timestamp + 64-bit sequence). Leaf nodes contain listpacks holding the actual field-value data. This structure provides efficient range queries (XRANGE) and trimming (XTRIM) with memory-efficient storage.

Each consumer group tracks a last-delivered-id and a Pending Entries List (PEL) implemented as a radix tree mapping entry IDs to consumer names and delivery timestamps. XACK removes entries from the PEL. XCLAIM and XAUTOCLAIM transfer unacknowledged entries to different consumers for failure recovery.

Streams vs. Pub/Sub vs. Lists

Pub/Sub: fire-and-forget, no persistence, fan-out. Lists: LPUSH/BRPOP gives queue semantics but no consumer groups or replay. Streams: persistent, replayable, consumer groups with acknowledgment, multiple independent readers, automatic ID assignment. Streams are the right choice for reliable message processing with at-least-once delivery.

10

Lua Scripting

Redis embeds a Lua 5.1 interpreter to execute server-side scripts atomically. A Lua script runs to completion without interleaving from other commands, enabling multi-key atomic operations without distributed locks. Since Redis 7.0, Functions provide a persistent, named, and replicable alternative to ad-hoc EVAL scripts.

Lua Execution Pipeline
graph TD
    CLIENT3["Client"] -->|"EVAL script
numkeys keys args"| EVALCMD["EVAL
Command"] CLIENT3 -->|"EVALSHA sha1
numkeys keys args"| EVALSHA["EVALSHA
(cached script)"] EVALSHA --> CACHE["Script Cache
(dict: SHA1 → Lua function)"] EVALCMD --> SHA["SHA1
hash script"] SHA --> CACHE CACHE --> LUAVM["Lua 5.1 VM"] LUAVM -->|"redis.call()"| CMDEXEC["Redis Command
Execution"] LUAVM -->|"redis.pcall()"| CMDEXEC CMDEXEC --> RESP_REPLY["Reply to
Lua"] RESP_REPLY --> LUAVM LUAVM --> RESULT["Return Value
to Client"] subgraph Functions["Functions (7.0+)"] FCALL["FCALL name
numkeys keys args"] --> LIB["Function
Library"] LIB --> LUAVM2["Lua VM
(persistent)"] end style CLIENT3 fill:#22143d,stroke:#6366f1,color:#e9e0ff style EVALCMD fill:#1a0f30,stroke:#eab308,color:#fef3c7 style EVALSHA fill:#1a0f30,stroke:#eab308,color:#fef3c7 style CACHE fill:#22143d,stroke:#f59e0b,color:#fef3c7 style SHA fill:#22143d,stroke:#f97316,color:#fef3c7 style LUAVM fill:#2a0015,stroke:#eab308,color:#fef3c7 style CMDEXEC fill:#1a0f30,stroke:#ef4444,color:#fef3c7 style RESP_REPLY fill:#22143d,stroke:#14b8a6,color:#e9e0ff style RESULT fill:#22143d,stroke:#6366f1,color:#e9e0ff style FCALL fill:#1a0f30,stroke:#eab308,color:#fef3c7 style LIB fill:#22143d,stroke:#f59e0b,color:#fef3c7 style LUAVM2 fill:#1a0f30,stroke:#eab308,color:#fef3c7

Atomicity Guarantee

While a Lua script executes, no other command can run. This provides transactional semantics without explicit MULTI/EXEC. However, long-running scripts block the server entirely, so a configurable timeout (lua-time-limit, default 5s) triggers a SCRIPT KILL option.

lua eval.c

Script Caching

EVAL computes a SHA1 hash and caches the compiled Lua function. Subsequent calls via EVALSHA skip re-parsing. SCRIPT LOAD pre-caches scripts. The cache persists until SCRIPT FLUSH or server restart.

lua scripting.c

Functions (7.0+)

Named, replicable function libraries loaded via FUNCTION LOAD. Unlike EVAL, functions are persisted to AOF and replicated to replicas. They support multiple named functions per library, flags for determinism control, and are the recommended replacement for EVAL in production.

functions functions.c
11

Module System

The Redis Module API (introduced in 4.0, significantly expanded since) allows loading shared-object plugins that extend Redis with new commands, data types, and capabilities. Modules have full access to Redis internals through a stable C API, enabling products like RediSearch, RedisJSON, RedisGraph, and RedisTimeSeries.

Module API Layers
graph TD
    MOD["Loaded Module
(.so / .dylib)"] --> API["Module API
(redismodule.h)"] API --> CMDS["Custom
Commands"] API --> TYPES["Custom Data
Types"] API --> EVENTS["Event
Notifications"] API --> KEYSPACE["Keyspace
Access"] API --> THREADS["Thread-Safe
Contexts"] API --> FILTERS["Command
Filters"] API --> CLUSTER_API["Cluster
Messages"] API --> CONFIGS["Custom
Configs"] subgraph Notable["Notable Modules"] SEARCH["RediSearch
(full-text search)"] JSON["RedisJSON
(JSON documents)"] TSDB["RedisTimeSeries
(time-series data)"] BLOOM["RedisBloom
(probabilistic)"] end CMDS --> SEARCH TYPES --> JSON TYPES --> TSDB TYPES --> BLOOM style MOD fill:#1a0f30,stroke:#3b82f6,color:#e9e0ff style API fill:#2a0015,stroke:#ef4444,color:#fef3c7 style CMDS fill:#22143d,stroke:#3b82f6,color:#e9e0ff style TYPES fill:#22143d,stroke:#3b82f6,color:#e9e0ff style EVENTS fill:#22143d,stroke:#3b82f6,color:#e9e0ff style KEYSPACE fill:#22143d,stroke:#3b82f6,color:#e9e0ff style THREADS fill:#22143d,stroke:#3b82f6,color:#e9e0ff style FILTERS fill:#22143d,stroke:#3b82f6,color:#e9e0ff style CLUSTER_API fill:#22143d,stroke:#3b82f6,color:#e9e0ff style CONFIGS fill:#22143d,stroke:#3b82f6,color:#e9e0ff style SEARCH fill:#22143d,stroke:#22c55e,color:#fef3c7 style JSON fill:#22143d,stroke:#f59e0b,color:#fef3c7 style TSDB fill:#22143d,stroke:#f97316,color:#fef3c7 style BLOOM fill:#22143d,stroke:#d946ef,color:#e9e0ff
Module API Capabilities

Modules can: register new commands with custom ACL categories, define new data types with RDB save/load callbacks, subscribe to keyspace notifications, create background threads with thread-safe contexts, intercept commands via filters, send/receive custom cluster bus messages, register custom configuration parameters, and block clients waiting for async operations. The API surface has grown to 400+ functions.

12

Glossary & Acronyms

Key terms and acronyms used throughout this architecture map.

ACL Access Control List (user permissions)
AOF Append-Only File (write log persistence)
BIO Background I/O threads (fsync, close, lazy-free)
COW Copy-on-Write (fork memory sharing)
CRC16 Hash function for cluster slot assignment
LFU Least Frequently Used (eviction policy)
LRU Least Recently Used (eviction policy)
OOM Out of Memory
PEL Pending Entries List (stream consumer tracking)
PFAIL Possible Failure (local node suspicion)
PSYNC Partial Synchronization (replication protocol)
RDB Redis Database (binary snapshot format)
RESP Redis Serialization Protocol (client wire format)
SDS Simple Dynamic String (binary-safe string type)
SHA1 Secure Hash Algorithm 1 (script caching key)
TTL Time to Live (key expiration)
Diagram
100%
Scroll to zoom · Drag to pan · Esc to close