How This Publication Verifies a Number

This page describes how alldone.jobs turns a corpus of job postings into a published finding, and what a reader should distrust if any step below is missing from a given article. It is written to be checked, not taken on faith. Every article on this site links here and states which parts of this page it relied on.

The corpus is a set of job postings collected by Olive's own acquisition pipeline: employer applicant-tracking-system feeds, aggregator boards, and a government job-listings API, deduplicated and stored as one record per observed posting. It is not a survey of employers and it is not a census of vacancies. It is a record of what employers chose to post online, where, and when. Every claim on this site is a claim about that record, stated as a claim about that record — not restated as a claim about the labor market at large without the qualification that step requires.

1. Flow, not stock

A posting can be counted two ways: as a stock — every posting that is currently live, however long it has been open — or as a flow — only the postings that are newly observed in a given window. alldone.jobs reports flow. A stock count rises when postings simply stay open longer, falls when a platform prunes stale listings, and is inflated by postings that were never real hiring intent in the first place (see §9). A flow count, restricted to postings newly observed in a window and deduplicated against everything seen before, is a cleaner measure of the rate at which employers are initiating new hiring activity.

The unit a flow counts is a distinct vacancy-location, not a raw posting row and not a raw content cluster. Precisely, the dedup key is the pair

( content fingerprint , location )where content fingerprint = COALESCE(clusterKey, dedupContentHash, _id)and location = COALESCE(zip5, city, ·),

and each distinct key is counted once, with its month assigned from the **earliest postedAt** observed under that key. clusterKey is the corpus's own fingerprint (normalized title + normalized employer + normalized location); dedupContentHash is a content-only fingerprint (title + description) that, on its own, is identical for the same posting text at different sites. Keying on the content fingerprint alone therefore collapses a chain's identical listing across every city it hires in — in this corpus one national restaurant chain's 30,540 rows collapse to 459 content clusters, a 66× collapse, while a typical single-site employer collapses barely at all. That collapse is differential across employers and across months, and applying it silently flattens a share series. Pairing the content fingerprint with location fixes both directions at once: identical content at the same location in the window collapses to one vacancy (killing same-location re-posting churn and the same opening arriving through more than one source), while identical content at different locations is counted separately (a cook posted in Tucson and the same text posted in Phoenix are two vacancies — the standard Lightcast / Burning Glass convention). Rows carrying neither clusterKey nor dedupContentHash (about two-thirds of US rows) cannot be matched to anything by content and so fall back to their row id and stand alone — the honest choice, since the corpus only collapses what it can prove is identical.

Every published series states which of the two — flow or stock — it is showing; where a series is flow, the dedup key (the vacancy-location key above) and the window are stated alongside it. This vacancy-location definition is analysis-local: it is computed over the raw posting table, because the pipeline's own canonical_postings view keys on the content fingerprint alone and has therefore already collapsed the per-location splits this measure restores.

2. Normalization: share-of-total and seasonal adjustment

Raw counts are not compared across time or geography without adjustment. Two forms of adjustment are used, and an article that makes a trend claim from raw counts alone does not meet this site's bar.

The first is share-of-total: a category's postings are expressed as a fraction of all postings observed in the same geography and window, rather than as an absolute count. This removes the effect of the corpus itself growing or shrinking — from new sources being added, from a source's own scraping cadence changing, or from an industry-wide rise in postings that never convert to hires (§9) — none of which are labor-market signal, and all of which move raw counts in ways unrelated to the claim being made.

The second is seasonal adjustment. Many occupation categories have a predictable annual cycle independent of any economic shock — food service is the clearest example, and the one this site's first published finding turns on (§3). A raw month-over-month comparison in a seasonal category will read a normal seasonal swing as a trend. Where the site's own data history is too short to compute its own seasonal factors, published seasonal factors from the Bureau of Labor Statistics' Current Employment Statistics program are used instead, and that substitution is stated explicitly rather than left implicit.

3. The CES seasonal norm, and what "decline" means

Food-service employment does not move flat across a calendar year. Using CES seasonal factors for food services and drinking places (NAICS 722), averaged 2017–2025, the month-by-month deviation from the annual average is:

Food-service employment normally rises roughly five to six percentage points from January to June. A finding that food-service postings "fell" over a January-to-June window, stated on raw counts, is not evidence of a downturn unless the reader is also told that a rise of five to six points was the expected baseline. This site states such findings as a shortfall against that baseline — postings fell short of the normal seasonal ramp by some number of points — and does not use the word "decline" for a raw-count comparison inside a season where the norm itself is a rise.

4. Employer-concentration sensitivity

A finding that a category's postings fell in a given metro can be produced by one employer's hiring freeze rather than a market-wide pattern. Every metro-level finding on this site is reported twice: once with all employers included, and once with the five largest employers in that category and geography removed. Where the finding holds in both versions, the article says so. Where it does not — where the effect is driven by the top five employers — the article says that instead, and names which employers are excluded from the second version. This check is run regardless of whether any single employer appears dominant on inspection, because concentration at the level of a whole occupation category (a small number of national chains accounting for a majority of that category's postings, for instance) can produce a market-wide-looking pattern from a small number of corporate decisions even when no single metro cell looks concentrated on its own.

5. Coverage: what online postings do and do not represent

Job postings collected online are not a representative sample of hiring activity across all occupations. An OECD occupation-level assessment of web-scraped postings data (Cammeraat & Squicciarini, 2021) found systematic over-representation of business, financial, computer, and healthcare occupations, and systematic under-representation of education, construction, production, transportation, and — directly relevant to this site's first published finding — food preparation and serving occupations. A later OECD comparison against official vacancy statistics found the same pattern by sector: high coverage in professional services, information technology, and finance; low coverage in hospitality, agriculture, and roles typically recruited by word of mouth or in person rather than through an online posting.

The consequence for this site's work: a raw comparison of this corpus's absolute posting counts against government levels data (CES, JOLTS) is not made, because the corpus does not claim to observe the same universe those programs measure. Claims are stated as claims about the within-corpus trend or share, with the corpus's own coverage limitation named alongside any claim touching an under-covered sector such as food service.

6. Causal language

This site's designs are descriptive: cross-sectional and time-series comparisons within a single corpus, not randomized or quasi-experimental designs with an identified counterfactual. Where the data window and design permit it, a pre-trend is shown alongside the post-period the finding covers, so a reader can judge whether the pattern predates the event the article attributes it to. No article on this site claims that a factor "caused" an outcome from a descriptive design of this kind. The language used is "consistent with," "coincides with," or "precedes" — language that states what the data show without asserting a mechanism the design cannot identify.

7. Suppression

A published data cell — a given occupation category, in a given geography, in a given window — is withheld from publication, and rendered as insufficient data rather than a number or a blank, under either of two conditions: fewer than 30 postings underlie the cell, or a single employer accounts for more than half of the postings in the cell. The first threshold exists because small counts produce unstable percentages that read as volatility unrelated to any real pattern. The second exists because a cell dominated by one employer is a statement about that employer, not about a labor market, and publishing it without suppression would misrepresent a single company's staffing decision as a regional or occupational trend. This is a stricter, separate check from the sensitivity reporting in §4: a cell just below the fifty-percent threshold is published, but still carries the with/without-top-five view described there.

8. Vintage

Every article states the date of the data snapshot its numbers are drawn from, referencing the dated manifest the snapshot was built from. A dataset download attached to an article reproduces the numbers published in that article exactly, from that same snapshot — not from a later, updated pull of the corpus. Where a later article revisits the same question with fresher data, it is published as a new article citing its own, later vintage, rather than silently updating the numbers behind an earlier one.

9. Ghost postings

A meaningful share of postings on any online job board do not correspond to an employer's active intent to hire in the near term — postings kept open for pipeline-building, postings never actually filled, and postings duplicated or reposted for visibility. Independent estimates of this share across recruiting platforms range roughly from one-fifth to one-third of active postings, and the share is not believed to be stable over time. This is a primary reason this site reports flow rather than stock (§1): a newly observed posting, deduplicated against prior observations, is less susceptible to inflation from postings that stay open indefinitely without ever converting to a hire than a stock count of everything currently live is. It is not a complete mitigation — a posting can be a first-observed ghost posting as easily as a first-observed real one — and no claim on this site treats flow counts as free of this contamination, only as less exposed to it than a stock count would be.

10. Source-mix drift

The corpus is assembled from multiple acquisition sources, and the relative share of postings contributed by any one source can shift for reasons that have nothing to do with the labor market — a source changing its own scraping cadence, a feed going temporarily stale, a new source being added to the pipeline. A shift of this kind can produce a change in posting counts or category composition that looks like a labor-market signal but is a measurement artifact of the pipeline. The share of postings contributed by each source is monitored on an ongoing basis; a move of more than five percentage points in any one source's share, month over month, is treated as a flag requiring review before any finding touching that period is published, and any pipeline change of this kind is logged as a dated annotation on the affected series rather than left unmarked in the underlying trend.

11. Source scope

The corpus's postings are not evenly spread across the sources that feed it. In this site's first production run against the real corpus (2026-07-12, Feb–Jun 2026 window), a single source — DirectEmployers, an applicant-tracking-system distribution network used by large U.S. employers — accounted for 93–99% of all US postings in every month observed. Every other individual source (aggregator boards and smaller ATS feeds among them) contributed under 1.5% each. Food-service share computed from DirectEmployers postings alone, and food-service share computed from every other source combined, moved in opposite directions across the same window — a spike-and-settle pattern from DirectEmployers against a monotonic rise from everything else. The non-DirectEmployers panel is too thin, in both volume and duration, to independently support a trend claim, and it disagrees in direction with the DirectEmployers-dominated headline series when it is thick enough to compare at all.

Because of this, every claim this site publishes from this corpus is scoped explicitly to direct employer postings from large U.S. employers (the DirectEmployers network) — stated that way, not restated as a claim about "the labor market," "employers" generally, or "online job postings" at large. A source-mix table (each source's share of postings, by month) is published alongside every article that makes a trend or share claim, so the scope is checkable rather than taken on the article's word. This standard is stricter than, and in addition to, §10's drift monitor: §10 catches a source's share moving; this standard states that even a stable, undrifting source mix this concentrated changes what a finding is a claim about in the first place.

12. Occupational coding: the SOC crosswalk and the honest-grain policy

Every exposure measure this site joins — Eloundou, Felten AIOE, the Anthropic Economic Index — is keyed to a Standard Occupational Classification (SOC) code, and postings carry only a free-text title. A title→SOC crosswalk therefore sits between the corpus and every occupation-level claim, and its error is a first-class source of measurement error, stated here rather than left implicit.

The crosswalk was audited on a 500-title random sample, each title's assigned SOC judged CORRECT / WRONG / AMBIGUOUS by a separate judge agent (2026-07-12). The measured precision, stated at the grain a claim commits to:

Two facts drive the policy. First, a raw 6-digit code is right only about two-thirds of the time (68.4% strict), and — surprisingly — an exact string match to a title bank is not the safest tier: exact matches clear only 57.9% strict, because one surface title (“Server,” “Associate”) routinely denotes different occupations. Committing every posting to its 6-digit code would overstate precision. Second, the same audit re-scored at coarser grain recovers most of that error: crediting a match whose first-four SOC digits are right lifts resolved precision to ≈84%, and the single boolean the first published finding actually turns on — “is this posting in SOC major group 35, food preparation and serving” — is right 95.0% of the time on resolved cases (19 of 20; small n — only 22 of 500 audited titles land in group 35, plus one food-service title mapped outside it, so this rate carries a wide interval).

The honest-grain policy follows: use a 6-digit code only where the match quality earns it, and fall back to minor-group aggregates otherwise. Concretely, each posting is tagged with an exposure_join_level — soc6 for exact / exact_v2 matches and high-confidence adjudicated or company-pair overrides, minor for nearest-neighbor (knn), minor-group (group), and medium/low adjudicated matches. A soc6 posting joins the 6-digit exposure score; a minor posting joins the minor-group mean of that score (an unweighted mean over the 6-digit occupations in the group — the reference table carries no posting-volume weights, and weighting by ONET’s occupation count would reward a taxonomy artifact rather than employment). Exposure scores are themselves smooth within a minor group, so this coarsening costs little signal while buying the precision the strict 6-digit grain does not have. Every published number states the grain it was computed at, and the ≥95% precision bar in the PRD is read as ≥95% at the grain used for the claim*, not at 6-digit precision the audit does not support.

Finally, the food-service universe this site’s first finding rests on is not defined by the crosswalk alone. It is defined three independent ways in parallel — a semantic classifier on embedded titles, a title-regex family, and the SOC major-group-35 flag — and a finding must survive all runnable definitions (triangulation, not a fallback chain). Definition (c) uses only the coarse, well-measured major-group-35 boolean above, never a 6-digit food-service claim. A reader who distrusts the crosswalk can drop it and still has two independent definitions holding the same universe.

13. Window policy: the primary window and greyed context months

Not every month a corpus covers can carry a claim. This corpus is assembled from nightly ingest batches, and the first months of its history are shaped by the pipeline coming online rather than by the labor market. Two facts fix which window a finding may rest on.

First, February 2026 is not a stable baseline. Eighty-seven percent of its in-window postings arrive on a single ingest day (2026-02-21), and a batch-day jackknife — dropping each of the highest-volume ingest days one at a time and recomputing — moves February's food-service share by enough to flip the sign of a February-anchored comparison. A month whose headline number depends on which single ingest day happened to land cannot anchor a seasonal baseline.

Second, March 2026 is an ingest burn-in artifact. A single ~2.17-million- cluster batch dump lands in March as the pipeline backfills, so March's composition reflects the backfill, not March hiring.

The policy that follows: the primary window every published claim is made on is April 1 – June 28, 2026. February and March are still computed and still drawn — rendered greyed as documented context — but they are never load-bearing, no seasonal baseline is anchored in them, and no headline sentence turns on them. The seasonal baseline for a within-primary-window comparison is April (the first primary-window month). The batch-day jackknife of §-flow above remains mandatory on the primary window: every published monthly share ships with its drop-one-ingest-day min/max band, and a claim that does not survive that band is not published. When the corpus accumulates more post-burn-in history, a later article may widen the primary window and states that it did so, under §8's vintage rule, rather than silently restating an earlier window.

The drift program, and why it is not published

This publication runs a second research program that is deliberately not on this site: a skill-demand drift detector, which reads the same corpus for evidence that the skills employers ask for inside a role are changing, and crystallizes each candidate into a named, magnitude-scored, region-scoped, citation-backed statement. It inherits every standard above. It is disclosed here, and withheld, for the same reason those standards exist.

The drift surface is gated on two population-scale signal-validity bars: at least sixteen of twenty stratified artifacts must be independently defensible (on the current thirteen-artifact population, at least eleven of thirteen), and at least three of five ground-truth backtest shifts must be detected. Neither bar is met on the current corpus window, and the program does not pretend otherwise. The window holds only three clean primary months, and at neighborhood grain the cells are still thin enough that the strictest robustness gate — dropping the top three employers behind any signal — legitimately removes the employer-concentrated skill detections along with the composition artifacts. Run at that integrity setting the backtest detects zero of five shifts as shipped (a looser, auditable counterfactual recovers two of five), and the signals that do survive are all volume, not skill.

So the drift surface stays non-public. It is not published, linked, or pitched until both gates pass on the accrued archive — not on this three-month window. A monthly re-run is planned; each additional clean month thickens the cells and lets a genuinely broad signal clear the bar without loosening anything. Until then the detector is an internal integrity instrument: it ships the exact checks that will decide when it is publishable, and it reports, here, that it is not yet. Saying so is the point.

Revisions

v0.1 — 2026-07-12. Initial publication, concurrent with R1 ("The Summer Restaurants Didn't Hire"). Ten checks above correspond to the methodology checklist every article on this site is required to pass before publication.

v0.2 — 2026-07-12. Added §11 (Source scope), from the first real-data run's single-source-corpus finding (93–99% DirectEmployers monthly; the non-DirectEmployers panel trends opposite in direction). Eleven checks now correspond to the methodology checklist.

v0.3 — 2026-07-13. Added §12 (Occupational coding), from the 500-title crosswalk audit: measured 6-digit precision (68.4% strict / 79.4% resolved overall; 80.8% knn_high), the ≈84%-resolved minor-group grain, and the 95.0%-resolved major-group-35 food-service binary (small n). States the honest-grain policy (exposure_join_level soc6 vs minor-group mean) and that R1's food-service universe triangulates three independent definitions rather than trusting the crosswalk alone. Twelve sections now correspond to the checklist.

v0.4 — 2026-07-13. Rewrote §1 and added §13 (Window policy), from the adversarial-panel fix round that refuted the v1 R1 headline as constructed. §1 now specifies the flow unit precisely as the distinct vacancy-location — key (COALESCE(clusterKey, dedupContentHash, _id), zip5-or-city), month = earliest postedAt in the key — after the panel showed the v1 shares were computed on the pipeline canonical view, whose content-only collapse is differential by employer and month (one chain 66×) and silently flattened the series; identical content at the same location now collapses to one vacancy while identical content at different locations counts separately (Lightcast/BG convention). §13 states the primary window policy: claims are made only on Apr 1–Jun 28 2026, with February (87% one ingest day, batch-fragile) and March (ingest burn-in dump) computed but greyed and never load-bearing, the seasonal baseline anchored in April, and the batch-day jackknife mandatory on the primary window. Thirteen sections now correspond to the checklist.

v0.5 — 2026-07-13. Added The drift program, and why it is not published — a disclosure section (not a fourteenth checklist item) sourced from the Tier-2 drift methodology's gate-status. States plainly that the skill-demand drift surface is non-public, gated on two population-scale signal-validity bars (≥16/20 ≈ ≥11/13 artifacts defensible; ≥3/5 backtest shifts detected) that are not met on the three-clean-month window, and that it stays non-public until both pass on the accrued archive. The thirteen numbered checklist sections are unchanged.