BASPI5 round-trip MVP harness + diagnostic exemplar regression

CI execution amendment, 2026-09-03. The three validation layers remain mandatory when ontology inputs change, but they now run once inside the deployment-gating workflow through make ci-ontology. The existing parametrised test suite still names all 15 exemplar failures individually; a 15-runner matrix and separate duplicate byte-identity workflow added cost, not coverage. This amendment changes orchestration only, not the harness, fixtures, expected reports or acceptance criteria.

Context and Problem Statement

This ADR is the MVP gate for the OPDA ontology implementation programme. It ratifies the BASPI5 round-trip harness that closes ODR-0003 §“Programme retirement criterion” condition (i): the MVP round-trip closes when pdtf-transaction.json → loaded SHACL profile → rendered BASPI5 form → validated transaction with full dct:source traceability all succeed end-to-end.

Per ODR-0010 §Q7, this is the operational pressure-test for the entire ratified ODR stack: if BASPI5 round-trips, the seven kind: pattern ODRs (ODR-0005/0006/0007/0008/0009/0015 + the cross-cutting ODR-0010/0011/0012/0013/0017/0018) are coherent end-to-end.

The harness also closes ODR-0004 §8a diagnostic-exemplar pairing: each of the 15 ratified exemplars in source/03-standards/ontology/exemplars/ pairs with an expected-report.ttl recording the SHACL validation report it should produce. Pyshacl-generated reports must match the expected reports byte-for-byte; CI regression catches any TBox/shape change that breaks an exemplar’s validation outcome.

Together: this ADR transforms ODR-0003 + ODR-0010 + ODR-0004 from ratified rules into a working demonstration. Programme retirement triggers on completion.

Decision Drivers

• End-to-end coherence proof. The harness must exercise every ratified ODR’s emission — class graph + shapes graph + annotation graph + SKOS substrate + per-overlay profile + DASH UI predicates + PROV-O Plan-vs-Activity + DPV co-annotations + SHACL-AF rules.
• dct:source traceability (ODR-0010 §Q3) — every rendered form question links back to a data-dictionary leaf URI; every minted term traces to its glossary or regulator source.
• Regression-test discipline (ODR-0004 §8a) — exemplar + expected-report.ttl pairing means SHACL drift surfaces immediately, not after consumer breakage.
• Reproducibility — CI runs the same harness on every PR; round-trip success is the green-light criterion for shipping any TBox/shape change.
• Operational pressure-test (Davis termination signal 1 from Scope-Check 1 Q8) — “BASPI5 round-trip closes” is the MVP gate Davis named in Scope-Check 1. Closes Davis’s termination test.

Considered Options

• A — Manual end-to-end testing. Pro: simplest to start. Con: not CI-friendly; doesn’t catch regressions; requires human in the loop for every change.
• B — Automated round-trip harness with per-exemplar expected-report (chosen). Pro: CI-integrated; deterministic; catches regressions at PR time. Con: substantial test fixture authoring up front.
• C — Property-based testing only (Hypothesis / QuickCheck over the schemas). Pro: covers more cases. Con: per-property test design is research-grade work, not a fit for the MVP gate.

Decision Outcome

Chosen option: B — Automated BASPI5 round-trip harness with per-exemplar expected-report.ttl pairing, integrated into CI. The harness consists of three layers:

1. Round-trip layer: Python script in tests/baspi5-round-trip/ that loads a BASPI5 JSON document, parses it into RDF via the ratified ontology, validates against the BASPI5 profile, and regenerates a BASPI5 JSON document. Round-trip equivalence (input JSON ≡ output JSON after normalisation) is the MVP gate.
2. Exemplar regression layer: pytest-based suite that validates each of the 15 exemplars against the shapes graph and compares the resulting sh:ValidationReport to its committed expected-report.ttl.
3. dct:source traceability layer: SPARQL queries verifying every minted form question + every shape’s sh:path resolves to a data-dictionary leaf URI.

Round-trip layer

# tests/baspi5-round-trip/test_round_trip.py
import json
import pytest
from pyshacl import validate
from rdflib import Graph, ConjunctiveGraph

@pytest.fixture
def opda_ontology():
    """Load the full ratified ontology corpus."""
    g = ConjunctiveGraph()
    for ttl in [
        "source/03-standards/ontology/foundation.ttl",
        "source/03-standards/ontology/opda-vocabularies.ttl",
        "source/03-standards/ontology/opda-property.ttl",
        "source/03-standards/ontology/opda-agent.ttl",
        "source/03-standards/ontology/opda-transaction.ttl",
        "source/03-standards/ontology/opda-claim.ttl",
        "source/03-standards/ontology/opda-governance.ttl",
        "source/03-standards/ontology/opda-descriptive.ttl",
        "source/03-standards/ontology/profiles/baspi5.ttl",
    ]:
        g.parse(ttl, format="turtle")
    return g

def test_baspi5_round_trip(opda_ontology, baspi5_sample_json):
    # 1. JSON → RDF
    transaction_rdf = json_to_rdf(baspi5_sample_json, opda_ontology)

    # 2. Validate against BASPI5 profile
    conforms, results_graph, results_text = validate(
        transaction_rdf,
        shacl_graph=opda_ontology,
        ont_graph=opda_ontology,
        inference="rdfs",
        advanced=True,
    )
    assert conforms, f"BASPI5 profile violations: {results_text}"

    # 3. Verify dct:source traceability
    for shape in opda_ontology.subjects(RDF.type, SH.NodeShape):
        for path in opda_ontology.objects(shape, SH.path):
            sources = list(opda_ontology.objects(shape, DCT.source))
            assert sources, f"Shape {shape} missing dct:source"

    # 4. RDF → JSON (regenerate the form)
    regenerated_json = rdf_to_baspi5_json(transaction_rdf, opda_ontology)

    # 5. Round-trip equivalence (after normalisation)
    assert normalise(regenerated_json) == normalise(baspi5_sample_json), \
        "BASPI5 round-trip lost information"

Round-trip equivalence after normalisation accounts for: ordering of arrays (sorted); whitespace; default-value insertion (filled in JSON; absent in RDF if xsd default).

Exemplar regression layer

For each of the 15 exemplars, pair with <exemplar>-expected-report.ttl:

source/03-standards/ontology/exemplars/
├── registered-freehold-house.ttl                          # already exists
├── registered-freehold-house-expected-report.ttl          # ADR-0014 deliverable
├── unregistered-pre-first-registration-house.ttl
├── unregistered-pre-first-registration-house-expected-report.ttl
├── flat-with-split-uprn.ttl
├── flat-with-split-uprn-expected-report.ttl
├── ... (12 more pairings)

Generator emits each expected-report.ttl once (per ADR-0007); subsequent runs compare against committed report. Drift → CI failure.

# registered-freehold-house-expected-report.ttl
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix opda: <https://w3id.org/opda/#> .
@prefix dct: <http://purl.org/dc/terms/> .

[]
    a sh:ValidationReport ;
    sh:conforms true ;          # Baseline easy case — no violations expected
    sh:result () ;              # Empty result set
    dct:source <https://w3id.org/opda/exemplars/registered-freehold-house> ;
    opda:pairedWith <source/03-standards/ontology/exemplars/registered-freehold-house.ttl> ;
    .

For exemplars exercising IC-bearing surfaces (e.g. unregistered-pre-first-registration-house.ttl — LegalEstate-without-RegisteredTitle case), the expected report includes specific sh:Info SHACL-AF rule materialisations:

# unregistered-pre-first-registration-house-expected-report.ttl
[]
    a sh:ValidationReport ;
    sh:conforms true ;
    sh:result (
        [
            a sh:ValidationResult ;
            sh:resultSeverity sh:Info ;
            sh:focusNode opda-x:estate ;
            sh:sourceShape opda:UPRNSuccessionRule ;
            sh:resultMessage "Estate has no RegisteredTitle yet; lifecycle event prov:wasGeneratedBy first-registration activity expected."@en ;
        ]
    ) ;
    .

dct:source traceability layer

# tests/baspi5-round-trip/test_traceability.py
def test_every_shape_traces_to_dictionary(opda_ontology):
    """Every shape's sh:path predicate MUST have dct:source resolving to a data-dictionary leaf."""
    query = """
    PREFIX sh: <http://www.w3.org/ns/shacl#>
    PREFIX dct: <http://purl.org/dc/terms/>
    SELECT ?shape ?path WHERE {
      ?shape sh:property [ sh:path ?path ] .
      FILTER NOT EXISTS { ?shape dct:source ?src }
    }
    """
    results = list(opda_ontology.query(query))
    assert not results, f"Shapes missing dct:source: {results}"

def test_every_minted_class_traces_to_odr(opda_ontology):
    """Every emitted owl:Class MUST have dct:source resolving to a ratified ODR."""
    # Similar SPARQL; assertion on ODR-NNNN regex in source URI

CI integration

.github/workflows/baspi5-round-trip.yml:

name: BASPI5 round-trip MVP gate
on:
  push: { branches: [main] }
  pull_request: { branches: [main] }

jobs:
  round-trip:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.11' }
      - run: pip install -e tools/opda-gen
      - run: pip install pyshacl==0.25.0 pytest
      - run: pytest tests/baspi5-round-trip/

  exemplar-regression:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        exemplar:
          - registered-freehold-house
          - unregistered-pre-first-registration-house
          - flat-with-split-uprn
          - flat-no-uprn-newly-converted
          - rural-plot-inspire-no-uprn
          - listed-building-divergent-addresses
          - person-with-name-change
          - organisation-with-merger
          - proprietorship-relator-multi-proprietor
          - simple-transaction-with-milestones
          - lease-extension-transaction
          - chain-of-transactions
          - claim-with-document-evidence
          - claim-with-electronic-record-evidence
          - claim-with-vouch-evidence
    steps:
      - uses: actions/checkout@v4
      - run: pip install pyshacl==0.25.0
      - run: |
          pyshacl --advanced \
            -s source/03-standards/ontology/opda-shapes.ttl \
            -d source/03-standards/ontology/exemplars/${{ matrix.exemplar }}.ttl \
            -f turtle \
            > /tmp/${{ matrix.exemplar }}-actual-report.ttl
          # rdflib-based diff (semantic equivalence, not byte-identity, because
          # SHACL reports can vary blank-node IDs but be semantically equivalent)
          python tests/baspi5-round-trip/compare_reports.py \
            /tmp/${{ matrix.exemplar }}-actual-report.ttl \
            source/03-standards/ontology/exemplars/${{ matrix.exemplar }}-expected-report.ttl

Consequences

• Good, because the MVP gate is now mechanically verifiable — pytest tests/baspi5-round-trip/ returns green or red.
• Good, because exemplar regression catches TBox/shape drift at PR time — Cagle’s “distinctions earn their keep when SHACL treats them differently” operationalised as CI test.
• Good, because dct:source traceability is enforced — provenance is not honour-system.
• Good, because round-trip equivalence is the canonical end-to-end pressure-test — closes Davis’s Scope-Check 1 Q8 termination signal 1.
• Good, because programme retirement triggers on this ADR’s confirmation — ODR-0003 retirement criterion (i) closes; criterion (ii) “every linked ODR is accepted” already met (17 ODRs accepted).
• Bad, because expected-report.ttl emission requires SHACL report semantic comparison (blank-node renames; ordering); the comparison utility is non-trivial.
• Bad, because BASPI5 sample JSON document needs careful construction — synthetic data that exercises every ratified pattern. Mitigation: start with one real BASPI5 submission anonymised by Council member.
• Bad, because round-trip equivalence under normalisation can mask information loss in edge cases. Mitigation: explicit canary tests for known lossy cases (per-form variant rendering; profile composition collapse).
• Neutral, because non-BASPI5 overlays (TA6, NTS, etc.) are post-MVP — TA6 round-trip harness etc. land as follow-up ADRs in the Phase-7 deferred-overlay queue.

Confirmation

The ADR is honoured when all eight hold (this is the programme retirement gate):

1. Round-trip harness implemented. tests/baspi5-round-trip/test_round_trip.py exists with the three-layer test suite (round-trip + exemplar regression + dct:source traceability).
2. All 15 exemplars have expected-report.ttl pairings. Generator emits via opda-gen emit-exemplar-reports.
3. CI green on BASPI5 round-trip. pytest tests/baspi5-round-trip/test_round_trip.py::test_baspi5_round_trip passes.
4. CI green on all 15 exemplar regressions. Matrix job in baspi5-round-trip.yml is fully green.
5. dct:source traceability tests pass. No shape or class lacks dct:source; every dct:source URI resolves.
6. Round-trip preserves information. No silent data loss between input BASPI5 JSON and regenerated BASPI5 JSON.
7. Real BASPI5 form rendering works. A DASH-compatible viewer (e.g. TopBraid Composer; or pyshacl-rendered DASH form preview) produces a recognisable BASPI5 form from profiles/baspi5.ttl + a sample transaction RDF.
8. ODR-0003 retirement criterion (i) closes. The programme retires when the gate passes — subsequent ontology work lands as fresh ODRs/ADRs.

Manual test: pytest tests/baspi5-round-trip/ -v → all green; opda-gen emit-exemplar-reports && git diff → empty diff; pyshacl --advanced -s opda-shapes.ttl -d exemplars/registered-freehold-house.ttl → sh:conforms true.

Programme-wide validation gate (per ADR programme plan §9 — Validation discipline). In addition to the ADR-specific criteria above, this ADR moves proposed → accepted only when all four of the following hold (independent of the worker that implemented this ADR):

• (a) Soundness check PASS — every emitted artefact traces to a cited ODR/ADR ## Rules or ## Operational specifications clause via dct:source (for Turtle) or code-comment provenance header (for Python). The validation agent extracts emitted-artefact provenance and verifies each resolves to a ratified section.
• (b) Completeness check PASS — every cited ODR’s ## Rules and ## Operational specifications subsection is realised by an emitted artefact OR explicitly deferred with a named follow-up trigger. The validation agent enumerates cited subsections and checks coverage.
• (c) Cross-ADR consistency check PASS — every downstream ADR’s confirmation criteria can be met given this ADR’s emission (e.g. classes emitted here are referenceable by downstream shapes; shapes here are composable by downstream profiles). The validation agent simulates the downstream contract against this ADR’s output.
• (d) Validation report committed at docs/adr/validation/ADR-0014-validation-report.md, produced by an independent validation-agent spawn (NOT the implementing worker; mirrors the Council Devil’s Advocate independence per ODR-0001 §Roles for every session; see ADR programme plan §8 swarm orchestration topology).

A FAIL on any of (a)–(d) blocks accepted status; the implementing worker amends and validation re-runs. Two consecutive validation failures on the same ADR escalate to a Council mini-session per ODR-0001 §Self-amendment process — engineering does not re-deliberate; surfaced ## Rules ambiguity routes to Council ratification.

More Information

• Ratified ODR foundations: ODR-0010 §Q7 MVP gate; ODR-0004 §8a diagnostic exemplar harness; ODR-0003 §Programme retirement criterion; Davis Scope-Check 1 Q8 termination signal 1.
• Predecessor ADRs: All of ADR-0006 through ADR-0013 must be status: accepted (or their confirmation criteria met) before this ADR can close.
• No successor ADRs in this programme — retirement on close.
• Subsequent OPDA ontology work: Post-MVP overlay additions (TA6, NTS, LPE1, etc.) per ADR-0013 Phase-2/3; module amendments via fresh ODR ratification + corresponding ADR realisation; consumer-profile additions per ad-hoc need.
• BASPI5 sample data: A real anonymised BASPI5 submission is preferable to synthetic data. OPDA Council member or member-firm may contribute under CC0 license. If unavailable, hand-construct a representative synthetic transaction exercising: built-form + EPC + utilities + occupiers (PII) + Survey provenance + chain dependency + capacity/authority + listed-building variation.
• Out of scope for this ADR:
  • TA6 / NTS / other-overlay round-trip harnesses (Phase-2 follow-up ADRs).
  • VC/DID / wallet integration (ODR-0016 deferred-until-trigger).
  • Generator pipeline performance optimisation (in-scope only if MVP gate regresses on emission time).

Amendments

• 2026-05-27 — Implementation landed (commit a4ff1c7). Three-layer round-trip harness at tests/baspi5_round_trip/ (8 files / ~765 LOC): JSON↔RDF translators; semantic-equivalence report comparator; per-exemplar regression suite; dct:source traceability tests; conftest fixtures for the merged 24-TTL corpus + synthetic BASPI5 sample. 15 paired <exemplar>-expected-report.ttl files emitted at source/03-standards/ontology/exemplars/ via opda-gen emit-exemplar-reports. .github/workflows/baspi5-round-trip.yml with matrix job over the 15 exemplars. Bundled six queued follow-ups closed inline: G14 (opda:hasSpecialCategoryData declared as foundation DatatypeProperty; Council route preserved via scope-note for S012 Q3 canonical predicate naming); G16 (ADR-0013 report DP count 19→20); G17 (“five foundation classes” → “six” in foundation.py + opda-classes.ttl header); G18 (opda:role declared as DatatypeProperty in opda-agent.ttl for DASH ergonomics); G19 (4 BASPI5 anchor URL mismatches corrected against actual baspi5Ref values; 0 mismatches verified); G20 (ADR-0013 report anchor count 36→28). Generator bumped 0.4.0 → 1.0.0 (MVP-gate release marker); foundation owl:versionIRI bumped 0.4.0 → 1.0.0. Test suite grew 118 → 154 (+36). Implementation report at docs/adr/implementation-reports/ADR-0014-implementation.md.
• 2026-05-27 — Cat 4 SHACL shape over-firing fixed (commit df5a165). ADR-0014 worker discovered the Cat 4 SHACL shape (SpecialCategoryPIIWithoutLawfulBasisShape from ADR-0012) fired sh:Violation on 7 of 15 exemplars — any exemplar with a Person instance. The ADR-0012 template used sh:property + sh:hasValue true + sh:not [sh:minCount 1] which forced every Person to carry opda:hasSpecialCategoryData=true. Engineering bug (not Council route — ODR-0013 §Q1 Cat 4 intent was correct; SHACL implementation was wrong). Fixed by switching to SHACL-AF sh:sparql constraint that fires the Violation only on the conditional intersection (Person has PII flag true AND no lawful basis). Regenerated all 23 TTLs (no change beyond opda-agent-shapes.ttl) + 15 expected-report.ttl files (7 now conform cleanly; 8 unchanged).
• 2026-05-28 — Independent validation PASS — PROGRAMME RETIRED (commit 28dd92a). All 8 §Confirmation + 4/4 programme-wide gates PASS. Cat 4 fix independently verified correct (15 exemplars now produce honest validation outcomes: 12 conform cleanly + 3 surface legitimate Cat 2 prov:wasDerivedFrom MinCount violations on claim evidence exemplars that were previously masked by Cat 4 over-firing — recorded as expected baseline in their paired expected-report.ttl files). All 6 G-closures (G14, G16, G17, G18, G19, G20) verified. ODR-0003 §“Programme retirement criterion” closes: criterion (i) MVP round-trip cleared by this PASS verdict; criterion (ii) every linked ODR is accepted already met (17/17). The OPDA ontology implementation programme RETIRES at commit 28dd92a. Subsequent ontology work lands as fresh ADRs without revisiting this programme’s sequencing. Validation report at docs/adr/validation/ADR-0014-validation-report.md.
• 2026-06-01 — pyshacl→Jena validator migration PENDING (ADR-0036/0037; NOT yet implemented). This harness still validates with pyshacl (conftest.py, test_exemplar_regression.py, baspi5_roundtrip_test.py). ADR-0036 (SHACL 1.2 via Apache Jena jena-shacl 6.1.0) and ADR-0037 (Jena as the sole RDF/SHACL toolchain; rdflib/pyshacl prohibited) are accepted as direction, but the migration is gated on a parity check (ADR-0036 §Confirmation — Jena and pyshacl must agree across the 15-exemplar matrix before pyshacl is removed) and that work is not done. Until then pyshacl remains the in-tree validator. The evidence-shape changes (Council session-035/036) were authored SHACL-Core specifically so they validate identically under either engine, keeping the eventual swap clean. Follow-up: install Jena 6.1.0, build the parity harness, then retire the pyshacl path.