Skip to content

PostgreSQL WAL segment layout#

2024-08-05

Refresher on how PostgreSQL organizes its write-ahead log on disk, mostly because I got a question about pg_wal/ size limits and realized I couldn't answer without looking it up again.

Segment files#

WAL is a sequence of fixed-size segment files (default 16 MB, configurable at initdb via --wal-segsize). Files live in PGDATA/pg_wal/ and are named by a 24-hex-digit sequence:

000000010000000A000000B7
├──┬──┴──┬──┴──┬──┴──┬──
│  │     │     │
│  │     │     └── segment number within logical file (LSN low bits)
│  │     └────────  logical file number (LSN high bits)
│  └──────────────  timeline ID

The pair (logical file number, segment number) maps deterministically to an LSN. Segment N starts at LSN N * 16 MB (for default segment size).

Timeline#

The timeline counter increments every time a standby is promoted or a recovery from PITR diverges from the primary. This is how the same LSN can mean different things after a failover — you have to specify which timeline you're talking about.

WAL insertion path#

  1. A transaction generates an XLogRecord (an opcode + rmgr ID + payload).
  2. The record gets appended to an in-memory buffer under XLogInsertLock (partitioned into 8 locks by default).
  3. On commit, XLogFlush() blocks until the WAL is durable on disk.
  4. Background walwriter process writes buffered WAL to disk periodically to reduce commit-time latency spikes.

Practical notes#

  • wal_keep_size (formerly wal_keep_segments) is how much WAL to retain for replicas that fall behind. Set to 0 and rely on replication slots + archive if you have them.
  • max_wal_size triggers a checkpoint, it does not cap disk usage. A slow replica or a busy archive command can still explode pg_wal/.
  • Replication slots without wal_keep_size will pin WAL forever if the consumer disappears. Disk fills, DB stops accepting writes.

Reference#

  • src/backend/access/transam/xlog.c — the actual implementation.
  • The PostgreSQL docs page on WAL Configuration.