Redis in Production for Django: Persistence, Memory Limits and Eviction, Security and Sentinel

Most Django deployments point the cache, sessions, Celery and Channels at one Redis with default settings, and that setup behaves until memory fills up. Then one of two things happens. Redis evicts keys and Celery tasks disappear without an error, or it refuses writes and every cache call starts raising. The way out is to decide what each workload needs from Redis and set persistence, memory policy, access control and failover to match.

Four jobs, four sets of requirements

Redis does different kinds of work for a Django project, and those jobs want opposite settings:

A single maxmemory-policy can't serve both a cache that should forget old keys and a queue that must never lose any. Logical databases (/0, /1, …) don't help, because every database in an instance shares one memory limit, one eviction policy, one persistence configuration and one event loop. A cache that fills up will evict Celery keys in database 1 to make room for page fragments in database 0.

Recommendation: run at least two instances, an ephemeral one for the cache and Channels and a durable one for Celery and sessions. If you really can only have one instance, put each role in its own logical DB so that cache.clear() (which runs FLUSHDB) doesn't wipe your queue, set noeviction, and accept that a full cache will cause write errors rather than evicting keys.

Running two instances on one host

On Ubuntu 24.04 the redis-server package reads /etc/redis/redis.conf. The Debian packaging also ships a redis-server@.service template that reads /etc/redis/redis-<name>.conf, so a second instance only needs its own config file, port, data directory and pidfile:

sudo install -d -o redis -g redis -m 750 /var/lib/redis-cache
sudo cp /etc/redis/redis.conf /etc/redis/redis-cache.conf
# edit port, dir, pidfile, logfile, then:
sudo systemctl enable --now redis-server@cache

In this guide the durable instance listens on 6379 and the cache on 6380. Give each one a maxmemory value, and make sure the two values plus fork overhead fit in RAM with room left for the OS.

Persistence: RDB, AOF, or nothing

RDB takes point-in-time snapshots. Redis forks, the child process writes a compact dump.rdb, and the file is renamed into place atomically. Restores are fast and the files are easy to back up, but everything written since the last snapshot is lost in a crash. AOF logs every write. With appendfsync everysec a crash loses at most about one second of writes, and the throughput cost is small. always fsyncs on every write and is rarely worth the latency. Redis 7 stores the AOF as a multi-part set in appenddirname and writes the base file in RDB format, so rewrites stay compact.

# /etc/redis/redis.conf  (durable: Celery + sessions)
port 6379
dir /var/lib/redis
save 3600 1 300 100 60 10000
appendonly yes
appendfilename "appendonly.aof"
appenddirname "appendonlydir"
appendfsync everysec
aof-use-rdb-preamble yes
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
stop-writes-on-bgsave-error yes

Both mechanisms fork. The RDB save rules above mean "snapshot after 3600 seconds if at least one key changed, after 300 seconds if 100 changed, after 60 seconds if 10,000 changed". On a busy queue the last rule fires about once a minute. AOF rewrites also fork, and on slow disks the everysec fsync can stall while a rewrite floods the disk. no-appendfsync-on-rewrite yes avoids those stalls, but it lets the durability window grow beyond one second during a rewrite. On a Celery broker, leave it at the default no and put the data directory on decent storage. The broker's working set is small, so its disks rarely need to be fast.

For a pure cache, turn persistence off completely. After a restart the cache starts empty, which is correct behaviour for a cache. You also avoid fork pauses and disk I/O, and a restart doesn't make Redis spend time loading gigabytes of stale fragments.

# /etc/redis/redis-cache.conf  (ephemeral: cache + channels)
port 6380
dir /var/lib/redis-cache
pidfile /run/redis/redis-server-cache.pid
logfile /var/log/redis/redis-server-cache.log
save ""
appendonly no

Keep RDB enabled on the cache only if a cold cache would overload your database, for example with expensive aggregates or a large traffic spike after a deploy. In that case a snapshot taken every hour or so is enough.

maxmemory and eviction

If you don't set maxmemory, Redis keeps growing until the kernel's OOM killer ends the process. Set a limit on every instance, leaving headroom because a fork during BGSAVE can temporarily need extra memory for copy-on-write pages.

# cache instance
maxmemory 2gb
maxmemory-policy allkeys-lru
maxmemory-samples 10

# durable instance
maxmemory 1gb
maxmemory-policy noeviction

Why eviction loses Celery tasks. Kombu stores each queue as a Redis list (the key celery by default) and tracks in-flight messages in the unacked hash and unacked_index sorted set. These keys have no TTL. Under allkeys-lru Redis can evict the whole queue list in one step, and every pending task goes with it. Neither the producer nor the worker gets an error, and nothing appears in the Celery logs. volatile-* policies only look at keys with a TTL, so they wouldn't touch the queue, but they also can't free memory once the TTL keys run out, and at that point they fail the same way noeviction does. With noeviction, a full Redis rejects writes with OOM command not allowed when used memory > 'maxmemory'. apply_async() then raises, you get alerted, and no data is lost. For a queue, that failure mode is the one you want.

Sizing maxmemory: a worked example

Take an 8 GB VM running both instances next to the Django app servers. The OS, page cache and a monitoring agent need about 1 GB. Gunicorn workers and Celery workers need another 2 to 3 GB. That leaves about 4 GB for Redis. The cache instance never forks because it has no persistence, so its limit can be close to what it will actually use: 2 GB. The durable instance forks for every snapshot and every AOF rewrite. In the worst case, when every page is written during the fork, copy-on-write can double its footprint, so give it 1 GB and keep the other gigabyte as headroom. maxmemory counts dataset bytes, not RSS. Allocator fragmentation and client output buffers come on top, and that's another reason not to set the limit to the machine's total RAM.

Base the queue's size on its worst backlog, not its steady state. If a nightly import can queue 500,000 tasks averaging 1.5 KB each (Celery's JSON envelope plus arguments), that backlog is about 750 MB. Measure it with MEMORY USAGE celery during a real backlog rather than guessing. And pass primary keys to tasks rather than whole model instances. Serialising a large object into every message is the most common reason Celery brokers grow unexpectedly.

LRU, LFU and the TTL question

allkeys-lru evicts the key that hasn't been accessed for the longest time, approximated by sampling maxmemory-samples keys. allkeys-lfu evicts the key used least often. LFU works better when a small set of hot keys (the navigation menu, a price list) sits alongside a long tail of one-off keys (per-user fragments, search result pages), because a burst of one-off keys can't push the hot set out. If you can't tell which fits, start with LRU and compare hit ratios. Avoid the volatile-* policies on a cache. Django's TIMEOUT: None writes keys without a TTL, and those keys can then never be evicted.

Result backend keys expire according to CELERY_RESULT_EXPIRES (one day by default). If you don't read results, set ignore_result=True on the task so they don't pile up next to your queue.

Django, Celery and Channels settings

# myproject/settings.py
import os

REDIS_CACHE_URL = os.environ["REDIS_CACHE_URL"]      # redis://django-app:PASS@127.0.0.1:6380
REDIS_DURABLE_URL = os.environ["REDIS_DURABLE_URL"]  # redis://django-app:PASS@127.0.0.1:6379

CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": f"{REDIS_CACHE_URL}/0",
        "KEY_PREFIX": "myproject",
        "TIMEOUT": 300,
        "OPTIONS": {
            "socket_connect_timeout": 2,
            "socket_timeout": 2,
            "health_check_interval": 30,
        },
    },
    "sessions": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": f"{REDIS_DURABLE_URL}/1",
        "KEY_PREFIX": "sess",
    },
}

SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "sessions"

CELERY_BROKER_URL = f"{REDIS_DURABLE_URL}/2"
CELERY_RESULT_BACKEND = f"{REDIS_DURABLE_URL}/3"
CELERY_RESULT_EXPIRES = 3600
CELERY_BROKER_TRANSPORT_OPTIONS = {"visibility_timeout": 3600}

CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "channels_redis.core.RedisChannelLayer",
        "CONFIG": {"hosts": [f"{REDIS_CACHE_URL}/1"]},
    },
}

The built-in backend passes unrecognised OPTIONS through to the redis-py connection pool, so the socket timeouts above apply to every connection. Unlike some third-party backends, it doesn't swallow errors. If the cache is down, cache.get() raises redis.exceptions.ConnectionError, and short timeouts make that happen quickly so requests don't hang. If you'd rather sessions survive a Redis restart without depending on AOF, use backends.cached_db, which costs one database write per session save.

Network exposure, ACLs and dangerous commands

Redis doesn't protect itself on an open network. Anyone who can reach an unauthenticated instance can run CONFIG SET dir and write files to the host. Security comes in layers.

Bind and protected mode

bind 127.0.0.1 -::1          # or: bind 127.0.0.1 10.0.0.11 (private interface only)
protected-mode yes
tcp-keepalive 300

Never bind to a public address. When Redis must be reachable from other hosts, bind to the private interface and restrict it at the firewall as well:

sudo ufw allow from 10.0.0.0/24 to any port 6379 proto tcp
sudo ufw allow from 10.0.0.0/24 to any port 26379 proto tcp
sudo ufw deny 6379/tcp

ACL users

Disable the default user and create one user per role. Keep the users in an ACL file so that ACL SAVE can persist runtime changes. Redis won't start if aclfile is combined with inline user directives in redis.conf.

# redis.conf
aclfile /etc/redis/users.acl

# /etc/redis/users.acl  (owner redis, mode 640)
user default off
user admin on >CHANGE_ME_ADMIN ~* &* +@all
user django-app on >CHANGE_ME_APP ~* &* +@all -@admin -@dangerous +flushdb +info
user backup on >CHANGE_ME_BACKUP ~* +@all

&* grants access to Pub/Sub channels. Redis 7 defaults to acl-pubsub-default resetchannels, so without it Celery's Pub/Sub-based features and channels_redis fail with permission errors. FLUSHDB is added back to the app user because Django's cache.clear() calls it. Leave it out if you never clear the cache from code. You can manage users at runtime and then persist the changes:

redis-cli --user admin --askpass
> ACL SETUSER django-app on >NEW_PASSWORD
> ACL SAVE
> ACL LOG 10

If a client breaks after you tighten permissions, ACL LOG shows each denied command and the user that tried it.

KEYS, FLUSHALL and friends

KEYS * is O(N) and blocks the single-threaded event loop. On a few million keys that can stall every client for seconds. Use SCAN instead. FLUSHALL deletes every database, including the queue. The ACL above denies @dangerous (which covers KEYS, FLUSHALL, CONFIG, DEBUG) to application users. The older rename-command FLUSHALL "" still works, but it applies to all users, including administrators, and it confuses tooling. On Redis 7, use ACLs instead.

TLS

If Redis traffic crosses a network you don't fully control, enable TLS: set port 0, tls-port 6379, tls-cert-file, tls-key-file and tls-ca-cert-file, then use rediss:// URLs in Django and Celery. TLS adds CPU cost and a certificate rotation process, so on a private network behind a firewall many teams choose not to use it.

Kernel settings and why they matter

# /etc/sysctl.d/60-redis.conf
vm.overcommit_memory = 1
net.core.somaxconn = 1024

sudo sysctl --system

vm.overcommit_memory=1: BGSAVE and AOF rewrites use fork(). The child shares memory with the parent through copy-on-write, so it needs little real memory, but under the default heuristic the kernel may still refuse the fork. The log then shows Can't save in background: fork: Cannot allocate memory, and with stop-writes-on-bgsave-error yes the instance starts rejecting writes.

Transparent huge pages: With THP enabled, a single write after a fork can copy a 2 MB page instead of a 4 KB one. That inflates memory use and causes latency spikes during every snapshot. Redis logs a warning at startup if THP is on. Recent Redis versions disable it for their own process (disable-thp yes). To disable it system-wide, run echo never | sudo tee /sys/kernel/mm/transparent_hugepage/enabled from a boot-time systemd unit.

Monitoring: the numbers that matter

redis-cli -p 6380 --user admin --askpass INFO memory | grep -E 'used_memory_human|used_memory_rss_human|maxmemory_human|mem_fragmentation_ratio'
redis-cli -p 6380 --user admin --askpass INFO stats | grep -E 'evicted_keys|keyspace_hits|keyspace_misses|rejected_connections'
redis-cli -p 6379 --user admin --askpass INFO persistence | grep -E 'rdb_last_bgsave_status|aof_last_write_status|rdb_last_save_time'

• evicted_keys should be zero on the durable instance. Anything else means the policy is wrong. On the cache a steady rate is normal. A sudden jump means the working set no longer fits.
• Hit ratio is keyspace_hits / (keyspace_hits + keyspace_misses). A cache below roughly 80% is usually too small, has TTLs that are too short, or uses keys that are too specific to be reused.
• mem_fragmentation_ratio well above 1.5 means RSS is much larger than the data. Consider activedefrag yes.
• rdb_last_bgsave_status:err is an emergency on the durable instance.

To diagnose latency and memory problems:

redis-cli --latency-history -i 5      # round-trip latency sampled every 5 seconds
redis-cli --bigkeys                   # largest key per type, uses SCAN
redis-cli --memkeys                   # largest keys by bytes
redis-cli SLOWLOG GET 10              # commands slower than slowlog-log-slower-than (µs)
redis-cli MEMORY USAGE celery         # bytes held by the default Celery queue
redis-cli LLEN celery                 # backlog depth (run against the broker DB with -n 2)

Set slowlog-log-slower-than 10000 (10 ms) and slowlog-max-len 256. A KEYS call or a DEL on a huge hash will show up in the slowlog. For large keys, use UNLINK instead of DEL so the memory is freed in a background thread.

A realistic incident, reconstructed from these numbers

A shop runs one Redis with maxmemory 1gb and allkeys-lru for its cache, sessions and Celery. A marketing email goes out at 09:00 and traffic triples. Each product page caches a fragment for every combination of currency and filter, so the cache keyspace goes from 300,000 to 2 million keys in twenty minutes. used_memory reaches the limit at 09:12 and evicted_keys starts climbing by thousands a second. At 09:14 customers begin reporting being logged out, because their session keys were idle for a few minutes and became eviction candidates. At 09:20 the order-confirmation email task stops arriving. The web process had pushed those tasks, but the celery list was evicted while the workers were busy, and the workers now see an empty queue. There are no errors in the error tracker or in the Celery logs.

The fix was the one this guide describes. The cache moved to its own instance with allkeys-lfu and a limit sized to the working set. Sessions and Celery stayed on a noeviction instance with AOF. Alerts were added for evicted_keys > 0 on the durable instance and for a hit ratio below 70% on the cache. Orders placed between 09:12 and 09:40 were found by comparing the order table with the email log, and their emails were sent again with a management command, which is why tasks should be idempotent and driven by database state.

High availability with Sentinel

Sentinel watches a primary and its replicas and promotes a replica if the primary fails. Run three Sentinels on three separate hosts (for example the two Redis hosts and an app server) with a quorum of 2. The quorum is the number of Sentinels that must agree the primary is down. The failover itself also needs a majority of all Sentinels to elect a leader, so two Sentinels can't survive losing one of them.

# replica: /etc/redis/redis.conf on 10.0.0.12
replicaof 10.0.0.11 6379
masteruser replica-user
masterauth CHANGE_ME_REPLICA
min-replicas-to-write 1
min-replicas-max-lag 10

# /etc/redis/sentinel.conf on each of 10.0.0.11-13 (must be writable by redis)
port 26379
sentinel monitor mymaster 10.0.0.11 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1
sentinel auth-user mymaster sentinel-user
sentinel auth-pass mymaster CHANGE_ME_SENTINEL

Sentinel rewrites its own config file to record the current topology, so don't manage that file with a tool that overwrites it on every run. Replication is asynchronous, so a failover can still lose recently acknowledged writes. min-replicas-to-write limits how much can be lost if the primary gets cut off by a network partition. On the data nodes, the replica and Sentinel users need limited ACLs:

user replica-user on >CHANGE_ME_REPLICA +psync +replconf +ping
user sentinel-user on >CHANGE_ME_SENTINEL allchannels +multi +slaveof +ping +exec +subscribe +config|rewrite +role +publish +info +client|setname +client|kill +script|kill

Connecting Django through Sentinel

The built-in RedisCache has no Sentinel support. Its LOCATION list means "first server takes writes, the rest serve reads". It doesn't discover a new primary after a failover. Teams usually handle this in one of three ways:

• Don't make the cache HA. A pure cache can be a single instance. If it dies you get cache misses and errors, and you restart it. Keep Sentinel for the durable instance.
• A local TCP proxy on each app host, with a health check that sends INFO replication and expects role:master. Django keeps pointing at 127.0.0.1, and the proxy follows the primary.
• django-redis with its Sentinel client, which asks the Sentinels for the primary on each new connection:

from urllib.parse import quote

REDIS_PW = quote(os.environ["REDIS_APP_PASSWORD"], safe="")
DJANGO_REDIS_CONNECTION_FACTORY = "django_redis.pool.SentinelConnectionFactory"
SENTINELS = [("10.0.0.11", 26379), ("10.0.0.12", 26379), ("10.0.0.13", 26379)]

CACHES = {
    "sessions": {
        "BACKEND": "django_redis.cache.RedisCache",
        # "mymaster" is the Sentinel service name, not a hostname;
        # the credentials are used for the primary Sentinel returns
        "LOCATION": f"redis://django-app:{REDIS_PW}@mymaster/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.SentinelClient",
            "SENTINELS": SENTINELS,
            "CONNECTION_POOL_CLASS": "redis.sentinel.SentinelConnectionPool",
        },
    },
}

Celery (kombu) understands sentinel:// URLs directly. List every Sentinel, separated by semicolons, and name the service in the transport options. The credentials in the URL are for the Redis primary; if the Sentinels themselves require a password, pass it separately with sentinel_kwargs in the transport options:

def sentinel_url(db):
    return ";".join(f"sentinel://django-app:{REDIS_PW}@{host}:{port}/{db}" for host, port in SENTINELS)

CELERY_BROKER_URL = sentinel_url(2)
CELERY_BROKER_TRANSPORT_OPTIONS = {"master_name": "mymaster", "visibility_timeout": 3600}
CELERY_RESULT_BACKEND = sentinel_url(3)
CELERY_RESULT_BACKEND_TRANSPORT_OPTIONS = {"master_name": "mymaster"}

What a failover looks like from the application

A failover isn't invisible. With down-after-milliseconds 5000, about five seconds pass before Sentinels even mark the primary as down, and then they elect a leader, promote a replica and reconfigure the others. Expect 10 to 30 seconds during which writes fail. Connections to the old primary are dropped or return READONLY You can't write against a read only replica. Django requests that use sessions will return 500s during that window unless you catch redis.exceptions.ConnectionError in middleware and degrade gracefully. Celery workers reconnect automatically. A task that was fetched but not yet acknowledged may be delivered again after visibility_timeout expires, and a task written just before the crash may be missing if it never reached the replica. Both cases lead to the same rule: tasks must be idempotent, and anything that must happen should be recorded in PostgreSQL first, with Celery used only to deliver it.

Don't use Sentinel on a single host or with only two nodes. Three Sentinels on one machine protect you against nothing, and a two-node setup can't keep a majority through a network partition. If you can't run three separate failure domains, a single durable instance with AOF, good monitoring and a tested restore procedure is more honest than a failover setup that can't actually fail over.

For your own code, redis-py's redis.sentinel.Sentinel(...).master_for("mymaster", username=..., password=...) returns a client that follows failovers. Test failover before you depend on it: run redis-cli -p 26379 SENTINEL FAILOVER mymaster while a Celery worker is running.

Backing up RDB files

An RDB file is the best backup format even on an AOF instance. It's a single self-contained file, and because Redis writes the snapshot to a temp file and renames it, the file on disk is always consistent. You can pull a fresh snapshot over the network without touching the server's disk:

#!/usr/bin/env bash
# /srv/app/bin/redis-backup.sh  (run from cron or a systemd timer)
set -euo pipefail
STAMP=$(date +%Y%m%d-%H%M%S)
DEST=/srv/backups/redis
export REDISCLI_AUTH="$(cat /srv/app/secrets/redis-backup)"
redis-cli -h 127.0.0.1 -p 6379 --user backup --rdb "$DEST/durable-$STAMP.rdb"
redis-check-rdb "$DEST/durable-$STAMP.rdb" > /dev/null
find "$DEST" -name 'durable-*.rdb' -mtime +14 -delete

You could instead run BGSAVE, wait until rdb_bgsave_in_progress:0, and copy /var/lib/redis/dump.rdb. Either way, copy the file off the host. There's no reason to back up the cache instance.

Restore gotcha: when appendonly yes is set, Redis loads the AOF at startup and ignores dump.rdb. To restore, stop Redis, set appendonly no, put the RDB in dir owned by redis, and start it. Then run CONFIG SET appendonly yes so Redis rebuilds the AOF from memory, and CONFIG REWRITE once aof_rewrite_in_progress returns to 0.

Common mistakes

• One instance with allkeys-lru for everything. Under memory pressure the queue list gets evicted and pending Celery tasks disappear without any error.
• Relying on logical DBs for isolation. They share memory, the eviction policy and persistence. A FLUSHALL during debugging deletes the queue.
• No maxmemory at all. The OOM killer ends Redis at the worst possible moment, and on the durable instance it can take unpersisted writes with it.
• Keeping the default vm.overcommit_memory. BGSAVE fails, stop-writes-on-bgsave-error kicks in, and every write fails.
• Restoring an RDB while AOF is enabled. Redis starts from the old AOF and silently ignores the backup you just copied into place.
• Pointing RedisCache at one Sentinel-managed node and expecting it to follow a failover. It keeps writing to the old primary, which is now a read-only replica.

Verify it

# policies and persistence per instance
redis-cli -p 6379 --user admin --askpass CONFIG GET maxmemory-policy   # noeviction
redis-cli -p 6380 --user admin --askpass CONFIG GET maxmemory-policy   # allkeys-lru
redis-cli -p 6380 --user admin --askpass CONFIG GET save               # ""

# default user disabled, app user cannot run KEYS
redis-cli -p 6379 PING                                                  # NOAUTH / auth error
redis-cli -p 6379 --user django-app --askpass KEYS '*'                  # NOPERM

# listening only on private/loopback addresses
sudo ss -ltnp | grep redis

# kernel
sysctl vm.overcommit_memory                                             # 1
cat /sys/kernel/mm/transparent_hugepage/enabled                         # [never]

# Django and Celery reach the right instances
python manage.py shell -c "from django.core.cache import cache; cache.set('ping', 1, 10); print(cache.get('ping'))"
celery -A myproject inspect ping

# Sentinel sees the primary and replicas
redis-cli -p 26379 SENTINEL get-master-addr-by-name mymaster
redis-cli -p 26379 SENTINEL replicas mymaster

Redis is fine in production when each instance is set up for its job. The cache is allowed to forget, the queue has to keep everything, the network is closed off and the kernel settings allow the fork. Once the checks above pass and evicted_keys on the durable instance stays at zero, you'll know about memory problems from alerts rather than from users asking why their tasks never ran.