leetbit
ExperimentactiveConducted Jul 10, 2026Updated Jul 11, 20265 min read

Dentry cache behaviour across identical nodes

Running the same negative-lookup workload on three identical Bench Cluster nodes to separate real dentry-cache behaviour from per-machine noise.

Hypothesis
A negative-lookup-heavy workload will show the same dentry-cache hit-rate curve on all three nodes; any node that diverges points to configuration drift rather than a real effect.
Outcome
Two of three nodes matched within noise. node-c diverged — traced to a stale kernel image from before the last reimage, which is exactly the failure the cluster's uniformity is meant to surface.

This experiment is really a test of the Bench Cluster as much as of the dcache: if three machines that are supposed to be identical disagree, the environment is lying somewhere.

Method#

The same workload runs on all three nodes at once, with nothing else scheduled. It is deliberately negative-lookup-heavy — the case where the dentry cache has to remember absence, not just presence — because that is where small configuration differences tend to show up.

Results#

NodeSteady hit-rateNegative entriesVerdict
node-a0.881.2 Mmatches
node-b0.871.2 Mmatches
node-c0.710.6 Mdiverges
node-c
node-c had booted an older kernel image — the reimage step had silently failed. Its lower negative-entry ceiling is a config artifact, not a dcache property. Re-running after a correct reimage brought it in line with the others.

Discussion#

The useful result is twofold. The dentry cache scales as expected on matched hardware, and the one divergence was caught precisely because the nodes are supposed to be interchangeable. This is the reproducibility argument for the cluster made concrete, and it connects back to the reference-counting rules in inode lifetime: absence has to be accounted for as carefully as presence.

Apparatus

  • Bench Cluster, all three nodes, cluster otherwise idle
  • Synthetic workload generating 70% negative lookups over a wide directory
  • Hit-rate sampled from /proc and dentry slab stats at 1 Hz
  • Each node reimaged to the campaign image before the run (intended)

Relations

where else this lives in the system
conducted Jul 10, 2026rev 1last revised Jul 11, 2026