Methodology

Classification First

Each catalog record is classified before risk ranking. Active datasets are the only records promoted into the operational high-risk queue. Archives, event-specific records, Socrata measures, and story/reference assets remain visible but do not receive active freshness expectations.

• active_dataset: ongoing machine-readable data or records with clear cadence evidence.
• needs_manual_review: dated records without enough cadence or asset evidence for automatic classification.
• archive_snapshot: month, quarter, year, or bounded-year snapshots.
• event_specific: records tied to a specific incident or event.
• measure: Socrata measure or indicator assets.
• story_reference: Socrata stories, files, links, and other reference assets.

Evidence Codes

Classification evidence is stored on every row and exported to JSON, CSV, detail pages, and MCP tools.

• known_cadence: accrualPeriodicity is present.
• machine_readable_distribution: a distribution exposes downloadURL or accessURL.
• socrata_story_asset, socrata_measure_asset, socrata_reference_asset: Socrata view metadata identifies a non-table asset.
• month_quarter_snapshot: title or description names a dated month, quarter, or month range.
• bounded_year_range: title or description names a single year or bounded year range with snapshot/statistics language.
• event_keyword: title or description names an incident such as a flood, storm, hurricane, or pandemic.

Scoring

Scores start at 100. Missing modified dates, weak descriptions, missing owner/contact metadata, and missing license/category/tags reduce the score. Known-cadence datasets get a full stale penalty when modified dates are past 1.5x the expected period. Unknown-cadence active records get only a low-confidence freshness issue.

Active-like records without a distribution or machine-readable URL are hard-labelled high risk. Archive, event, measure, and reference records keep those issues visible but are not promoted into the active high-risk queue.

Category Suggestions

Records with missing categories receive a trained suggestion when the current catalog has enough labeled examples. The model uses title, description, tags, publisher/contact text, and Socrata asset type. Suggestions are review hints only and do not overwrite City of Austin catalog metadata.

Manual Overrides

Manual corrections live in classification_overrides.json. Each override must name a Socrata dataset id and one allowed classification group. Override evidence is exported with manual_override so demo reviewers can see which calls were human-reviewed.

Run 84, fetched 2026-07-08T03:48:19Z. Uses public State of Texas and City of Austin open data. Independently operated; not affiliated with or endorsed by the State of Texas or City of Austin.