August 6, 2026 · Case study

Two scrapers changed their output on the same week. Triage took one question.

Monitoring flagged schema drift on two Apify Actors: expected output fields had disappeared while both runs still exited green and both datasets still parsed. The useful question was not whether the runs succeeded, but whether the output still served the pipeline that consumed it. This account omits source identities, field names and operational details; the lesson does not require publishing a dependency map.

What the alerts said

The checks reported missing expected fields under the schema_drift failure class. Run-status checks alone would not have explained this change: the processes completed successfully, but the output no longer matched its previous shape.

A single drifted run can be noise: a bad sample, a partial response, a temporary upstream issue. The next scheduled run reported the identical field lists on both scrapers. Two consecutive runs with the same missing fields upgraded the verdict from possibly transient to a persistent output change.

The triage question nobody starts with

The instinct when a vendor drifts is to open the vendor's changelog, or their issue tracker, or a support ticket. All of that is the slow path, and most of it is someone else's queue. The fast path is one question about your own code: which of the missing fields does our pipeline actually read?

Reviewing the consumers showed that the missing fields were either unused or already available through other inputs. The output change was real, but it did not require a customer-facing repair. The distinction came from reviewing what the pipeline needed, not from the green run status.

Total remediation: update the stored schema baselines so the new shape is treated as normal and the alerts go quiet. No code shipped and no customer-facing data was affected. That decision followed the consumer review, not an assumption that green status meant the output was safe.

Severity lives in the consumer, not the vendor

The uncomfortable lesson in this incident is that the vendor cannot tell you how bad a drift is, and neither can the monitor. Both scrapers genuinely lost fields; the monitor was right to call it high severity, because a missing usually-present field is exactly what a breaking change looks like from the outside. Whether it actually breaks anything is decided entirely by which fields your pipeline reads. The same drift that was harmless for this pipeline could be a production incident for another team that depends on the missing fields.

That is why the triage order matters: confirm the drift is real (two consecutive runs), then interrogate your own readers, and only then, if you do read a lost field, go looking for where the data went. When you do, there are three endings. The field was renamed or moved, which is a small reader fix. The field moved behind an input option, common when scraper vendors gate detail levels to cut cost, which is a one-line input change. Or the field is genuinely gone, which is not a code fix at all but a product decision: degrade, re-source, or drop the feature that needed it.

The counterfactual, priced

Without the fingerprint comparison, this incident has a different shape. The fields go hollow silently. Downstream tables accept the empty values, because most schemas do. Nothing pages. Weeks later a report looks thin, someone traces it backward through the pipeline, and the diagnosis that took one question now takes a day of archaeology, starting from the symptom instead of the cause. That delayed version is the normal way teams discover schema drift, and it is the expensive one even when, as here, the drift turns out not to matter.

At the public Store price, a drift check is $0.02 per Actor per run, about $0.61 a month on a daily schedule. The mechanics of the fingerprint-and-diff pattern, including the sampling traps that make homemade detectors page you at 3 a.m., are in the schema drift field guide; watching a Store actor you did not build, including failed runs and maintainer staleness, has its own guide.

One caveat we are keeping in our own back pocket, and you should too: vendors that silently drop fields tend to keep drifting. A future loss of a field the consumer does need can turn the same kind of alert into a real incident. The monitor does not know the difference. The triage question does, and it takes minutes when you can start from the exact fields and the exact date they vanished.