←Projects

Affix

2026

A zero-dependency C++20 library for CPU core pinning, thread placement, NUMA-aware allocation, and runtime performance observability.

C++20Linux sysfsPOSIXHardware PMUCMakeGitHub

When tuning low-latency software or high-throughput data pipelines, pinning threads to dedicated CPU cores is standard practice. But in most codebases, placement is managed through ad-hoc calls to sched_setaffinity or pthread_setaffinity_np. More importantly, developers rarely measure whether pinning actually improved anything or simply introduced cache contention and imbalance.

I built Affix to give C++20 applications a clean abstraction for hardware topology discovery, core placement, NUMA memory binding, and—crucially—runtime observability to verify whether a placement policy actually reduces tail latency and context switches.

The library has zero external dependencies, compiles down into public headers and platform shims, and can be integrated into existing codebases in a couple of lines.

Hardware topology discovery

Before deciding where a thread should run, the application needs an accurate picture of the underlying processor architecture.

On startup, Affix queries the operating system (using Linux sysfs or Darwin sysctl/Mach interfaces) and builds an in-memory SystemTopology object. It maps logical CPU IDs to physical cores, SMT siblings, NUMA nodes, and cache hierarchies. It also detects kernel-isolated cores configured via the Linux isolcpus boot parameter, as well as heterogeneous core types—distinguishing Performance (P) cores from Efficient (E) cores on hybrid architectures.

Discovered topology and placement masks
Filter Policy:
NUMA Node 0 (Local RAM: 16 GB)Package 0
CPU 0P-core
Phys 0 • SMT [0,1]
CPU 1P-core
Phys 0 • SMT [0,1]
CPU 2isolcpus
Phys 1 • SMT [2,3]
CPU 3isolcpus
Phys 1 • SMT [2,3]
NUMA Node 1 (Local RAM: 16 GB)Package 0
CPU 4P-core
Phys 2 • SMT [4]
CPU 5P-core
Phys 3 • SMT [5]
CPU 6E-core
Phys 4 • SMT [6]
CPU 7E-core
Phys 5 • SMT [7]
Affix detects physical cores, SMT siblings, NUMA memory nodes, and isolated cores. Policies can target single CPUs, full physical cores, or discovered P-cores.

Having this topology mapped out allows policies to be expressed declaratively:

  • Policy::pinned(core_id): Restricts execution to a specific logical CPU.
  • Policy::physical_core(id): Pins across an entire physical core and all its SMT siblings.
  • Policy::performance_cores(): Binds exclusively to discovered P-cores.
  • Policy::isolated_core(): Binds directly to an isolcpus core, falling back to a P-core if none are isolated.
  • Policy::numa_node(node_id): Restricts execution to the CPUs belonging to a specific NUMA domain.

Scoped placement and RAII

One common issue with thread affinity is state leakage. When a worker in a thread pool pins itself for a specific task, that affinity mask often persists indefinitely, causing subsequent unrelated tasks to contend on the same core.

To avoid this, Affix provides ScopedPlacement. It captures the thread's existing affinity mask and scheduling policy on entry, applies the target placement, and automatically restores the previous state when the guard leaves scope.

#include <affix/affix.hpp>

void process_critical_batch() {
    // Pin to core 2 for this scope only; restored automatically on exit
    affix::ScopedPlacement guard(affix::Policy::pinned(2));

    affix::MetricsSnapshot metrics;
    {
        affix::ScopedProbe probe(&metrics);
        execute_matching_engine_step();
    }

    if (metrics.involuntary_context_switches > 0) {
        // Core was preempted by kernel or higher-priority thread
    }
}

Affix also supports in-situ self-pinning via affix::this_thread::set_placement(), retrofitting existing std::thread handles via apply_placement(), and elevating real-time FIFO scheduling priorities (SCHED_FIFO / SCHED_RR) to prevent kernel preemption on critical paths.

Scoped placement lifecycle
Step 1 of 4:Default OS Scheduling
// Thread running unconstrained
// Subject to kernel migrations & preemption
Thread Placement
Migrating (CPU 0 ⇄ 5)
Affinity State
Mask: 0xFF (All CPUs)
OS Preemption
Voluntary: 48, Involuntary: 12
ScopedPlacement records the thread's starting affinity mask and restores it when leaving scope, preventing thread pool state pollution while isolating critical loops.

Three-tiered observability

Pinning a thread without measuring performance often leads to false assumptions. For example, pinning two compute-heavy threads to SMT siblings on the same physical core can degrade throughput due to shared L1/L2 caches and ALUs.

Affix includes an observability probe designed to measure execution metrics without allocating memory on the hot path:

  • Tier 0 (Clocks): High-resolution wall-clock duration and thread CPU time measured via clock_gettime(CLOCK_THREAD_CPUTIME_ID).
  • Tier 1 (Scheduler): Voluntary and involuntary OS context switches measured via getrusage(RUSAGE_THREAD), along with physical CPU core migration detection using sched_getcpu().
  • Tier 2 (Hardware PMU): Microarchitectural counters read directly from Linux perf_event_open(), including CPU cycles, instructions retired, IPC, L1 cache misses, and Last-Level Cache (LLC) misses.
The three-tiered telemetry stack
OS Kernel Scheduling DetailSource: getrusage(RUSAGE_THREAD) & sched_getcpu()

Preemption counters and physical CPU core migration detection.

Snapshot Fields:
voluntary_context_switchesinvoluntary_context_switchescpu_migrations
Telemetry is split into clocks, kernel scheduling context switches, and hardware PMU counters. Hardware counters gracefully fall back if running in unprivileged containers.

If hardware performance counters are disabled by the kernel (perf_event_paranoid restrictions) or unavailable in a virtualized container, Tier 2 gracefully reports zero while Tier 0 and Tier 1 continue operating without throwing exceptions or crashing.

NUMA memory allocation

On multi-socket servers, accessing memory attached to a remote NUMA node incurs a significant latency penalty over the interconnect bus (such as AMD Infinity Fabric or Intel UPI).

To prevent cross-socket thrashing, Affix includes a NUMA-aware STL allocator affix::numa::NumaAllocator<T> that binds heap allocations directly to the local NUMA node using mbind:

#include <affix/affix.hpp>
#include <vector>

// Allocate order book records strictly on NUMA Node 0
std::vector<Order, affix::numa::NumaAllocator<Order>> order_book(
    affix::numa::NumaAllocator<Order>(/*numa_node=*/0)
);

You can also bind existing raw memory pages in place with affix::numa::bind_memory().

Placement advisor and benchmarks

Affix includes an automated placement advisor (affix::evaluate_placement) that runs an A/B trial of candidate policies against a workload function. It evaluates each policy across repeated iterations (including configurable warmup cycles), gathers latency percentiles (P50, P90, P99), and recommends the best placement option based on minimal tail latency and context switches.

The accompanying CLI (affix-cli) and benchmark binary (affix-bench) allow inspecting machine topologies in human-readable or JSON formats, and measuring latency stability when competing against noisy neighbor threads.

Implementation and limits

Affix is written in C++20 and built with CMake. Public headers are separated from platform implementations, keeping client build times fast and avoiding leaking OS headers into user code.

On Linux, full feature support is available (including isolcpus, NUMA memory binding, SCHED_FIFO, and hardware PMUs). On Darwin (macOS), thread affinity is treated as a scheduling hint through Mach thread affinity tags, and NUMA node distinctions are unified under a single memory architecture. Elevating to real-time FIFO scheduling requires CAP_SYS_NICE privileges on Linux.