redis-packhashMillions of keys, a fraction of the memory

Install ​

npm install redis-packhash

The whole API in one screen ​

import { PackHash } from "redis-packhash";
import Redis from "ioredis";

const store = new PackHash(new Redis(), {
  namespace: "users",
  expectedKeys: 1_000_000,
});

// A profile cache keyed by user UUID. Pass the bare UUID, not "user:<uuid>":
// the namespace already scopes it, and the key is stored as a hash field, so
// its bytes count against Redis's 64-byte listpack value limit just as the
// value's do. Values are strings; serialize and parse on your side.
const id = "f47ac10b-58cc-4372-a567-0e02b2c3d479";

await store.set(id, JSON.stringify({ name: "John Doe", plan: "pro" }));

const raw = await store.get(id);
//=> '{"name":"John Doe","plan":"pro"}'

await store.has(id); //=> true
await store.del(id); //=> true

Why bucket at all ​

Redis keeps small hashes in listpack, a flat encoding with almost no per-entry overhead, renamed from ziplist in Redis 7.0. It promotes a hash to a far more expensive hash table as soon as either threshold is crossed: more than hash-max-listpack-entries fields (512 by default), or any single field name or value over hash-max-listpack-value (64 bytes). So you never use listpack; you keep each hash small enough to stay in it. That is the entire trick:

store.set("f47ac10b-58cc-4372-a567-0e02b2c3d479", val)
        │
        ├─ bucket = fnv1a(key) % 2605  → 1342
        ├─ HSET  users:1342  "f47ac10b-58cc-4372-a567-0e02b2c3d479"  val
        │                    └─ 36-byte field, under the 64-byte listpack limit
        └─ each "users:N" hash stays a listpack → compact memory

Storing a million values as a million top-level keys pays per-key overhead a million times. Storing them as fields inside ~2,600 listpack-encoded hashes does not.

How it works, in full →  ·  Get started →

Requirements ​

Node 22 or newer. Zero runtime dependencies. Both ESM and CommonJS builds ship in the package, with TypeScript types included.