Compliance Detection Subsystem Rework

Overview

Our detection subsystem was originally built around a dependency graph that propagated resource changes into monitor updates. It worked through the development cycle but started putting too much pressure on the write path under production load, creating ingestion latency and reliability risk.

I owned the investigation by tracing the async path until the write amplification was visible. Detection updates were fanning out into per-entity work that tuning alone would not fix, because the subsystem shape was doing unnecessary work by design.

What we did

I worked with a platform staff engineer on the redesign, while I drove the domain-event shape, payload enrichment, and read-path reconfiguration.

We turned dependency updates into domain events about monitor-resource changes. With domain events, the consumer layer could scale separately from dependency-graph updates. We also added payload enrichment so downstream evaluation logic could reuse the enriched data instead of recomputing it.

As part of the rework, I reconfigured the read path onto a maintained monitor-resource model over 5M+ rows. Instead of reconstructing a monitor's resources on every query, the model updates incrementally through the event pipeline. Query logic got simpler, and the data layer gained an explicit control point for monitoring and tuning.

A platform partner helped scope the immediate write-path fix. That handled the short-term pressure while the domain-event shape gave us independent event scaling and a clearer read model.

Collaboration

The architecture work was a partnership with a platform staff engineer, and the platform team owned the event-infrastructure pieces the redesign depended on.

Outcome

• Write amplification moved out of the per-entity dependency-update path.
• The read path stopped reconstructing monitor resources from scratch on every request.
• Detection updates became easier to monitor and tune because the bottleneck moved to an explicit data-layer view over 5M+ rows.

Observability showed where the system was hurting, but the durable fix came from stepping back far enough to change the subsystem shape, not just the slow line.