IRRD Modded

IRRD (Internet Routing Registry Daemon) v4 -- modified for production deployment at whois.koetsier.org.

Overview

This is a fork of IRRD v4 with significant modifications focused on memory optimisation, real-time streaming, and operational telemetry for high-scale IRR deployments.

What Changed

Memory Optimisation

Memory usage reduced from ~8.5 GB to ~2.9 GB peak.

Rust Preload Daemon (irrd-preloadd)

Replaced per-process Python preload dictionaries (~200-250 MB per worker) with a single shared Rust daemon communicating over Unix Domain Sockets. A single preloader service serves all IRRD worker processes.

NRTM Response Streaming

• NRTMGenerator.generate_stream() yields individual chunks instead of building the entire response in memory.
• WhoisQueryResponse supports lazy iterators via a result_stream parameter.
• Whois worker writes streaming responses with on-demand buffering:
  • IRRD mode: buffers to SpooledTemporaryFile (1 MB max in-memory) to preserve A{len}\n protocol framing.
  • RIPE mode: writes chunks directly with trailing newlines.

Mirror Update Parser Streaming

• MirrorUpdateFileImportParser.run_import() streams objects one at a time.
• Eliminated objs_from_file list and file_objs_by_pk dictionary of RPSLObject instances.
• New objects are inserted immediately; retained objects store only rendered text, RPKI status, and scope filter status as strings (not full parsed objects).

New Query Commands

Command	Description
!FNO-SCOPE-FILTER	Disables scope filtering for the remainder of the connection
!FNO-ROUTE-PREFERENCE-FILTER	Disables route preference filtering for the remainder of the connection
-V <agent> <query>	Sets user agent for the connection (RIPE -V flag equivalent)

Real-Time Event Streaming

• irrd/storage/event_stream.py -- Redis Stream-based event publishing for real-time database change notifications.
• WebSocket clients can consume updates via AsyncEventStreamRedisClient (uses coredis).
• Supports full_reload and journal_extended operation types.

Telemetry

• irrd/server/telemetry.py -- Redis-based request counters for whois (TCP/HTTP) and GraphQL.
• Counters exposed under irrd:telemetry:whois_requests:{tcp|http} and irrd:telemetry:graphql_requests.

Configuration Enhancements

Setting Group	Validations Added
webui	auth_failure_rate_limit parsed via limits.parse()
auth	set_creation mode validation, autnum_authentication modes, prefix_required booleans
auth	password_hashers availability validation
scopefilter	Prefix and ASN scope filter configuration validation
route_object_preference	update_timer numeric validation
suspension	Requires authoritative mode enabled
nrtm4	server_snapshot_frequency range (1-24 hours)

Architecture

+-------------------+     UDS (JSON)     +--------------------+
|  irrd-preloadd    |<------------------|  UdsPreloader      |
|  (Rust)           |                    |  (Python)          |
+-------------------+                    +--------------------+
                                          |
                           +--------------+--------------+
                           |                             |
                     +-----v------+                +-----v------+        +-----v------+
                     |  Redis     |                |  PostgreSQL|        |   NRTM /   |
                     |  Stream    |                |  Database  |        |  Whois API |
                     |  (events)  |                |            |        |            |
                     +------------+                +------------+        +------------+

Project Structure

irrd/                          # Modified IRRD Python codebase
+-- irrd/                      # Core application (memory-optimised)
|   +-- mirroring/             # NRTM streaming, parser streaming
|   +-- server/                # Whois streaming, telemetry, GraphQL
|   +-- storage/               # UDS preloader, event stream
|   +-- conf/                  # Extended config validation
+-- irrd-preloadd/             # Rust preload daemon
|   +-- src/                   # main.rs, server.rs, data.rs, redis.rs
|   +-- Cargo.toml             # Rust project manifest
|   +-- Cargo.lock             # Dependency lockfile
|   +-- irrd-preloadd.service  # systemd unit
+-- README.md                  # This file
+-- CHANGELOG.md               # Version history

Deployment

Prerequisites

• Python 3.13+
• PostgreSQL 14+
• Redis (with Unix socket)
• Rust toolchain (for irrd-preloadd)

Building irrd-preloadd

cd irrd-preloadd
cargo build --release
sudo cp target/release/irrd-preloadd /usr/local/bin/
sudo systemctl enable --now irrd-preloadd.service

IRRD Configuration

The modified IRRD supports additional settings:

preload = {
    "uds_socket_path": "/run/irrd/irrd-preloadd.sock",
}

scopefilter = {
    "prefixes": [],
    "asns": [],
}

route_object_preference = {
    "update_timer": 3600,
}

License

This project retains the original BSD 2-Clause license from IRRD v4. See LICENSE for details.

Acknowledgements

IRRD v4 was originally commissioned by NTT in 2018 and designed/developed by Reliably Coded. Modifications by Sebastiaan Koetsier for whois.koetsier.org.

---

Status: Production -- whois.koetsier.org Original upstream: irrdnet/irrd Version: 4.x (forked 2026-05-12)