Open Property Data Association
Smart Property Data Trust Framework
Become a member Join a working group

    Test a rule with ordinary and difficult cases

    Use small, explicit cases to establish when a requirement applies, what meets it and where the problem lies. Missing evidence is not automatically success or failure.

    Editorial watercolour cutaway of a surveyor considering an enclosed-room access limit and a separate obstructed loft hatch as two distinct test cases.

    Test the intended meaning, not just a convenient example

    A requirement can be written clearly and still ask for the wrong thing. “Every report must identify exactly one limitation” is easy to count, but it assumes that every report has a limitation and that there can only be one. Before asking a tool to enforce it, bring cases that challenge those assumptions.

    Review has two distinct questions. Does this record meet the stated requirement? And does the requirement express what the agreed specification ought to demand? A failed record may reveal missing information. It may instead expose an overreaching rule. Passing an inappropriate rule is not a modelling success.

    We will use a deliberately fictional teaching profile. Its conditions are supplied here, not inferred from legislation, professional standards or the current Property Pack candidate. The exercise evaluates descriptions; it cannot prove what happened at a property.

    Read the condition before looking for the required information

    Fictional requirement R — report the access limitation

    When a report states that inspection access was limited, it must identify at least one limitation being reported for that inspection.

    Subject
    The identified report version and the inspection it describes.
    Trigger
    The report states that inspection access was limited.
    What is required
    A description identifying at least one reported access limitation; “limited” alone does not identify it.
    Applicability convention for this exercise
    Use explicit evidence about the trigger. When that evidence is absent, say “applicability not established”. Do not infer a pass, failure or formal validation result.
    Check applicability before checking the requirement The fictional rule requires a named limitation when a report explicitly states limited access. If the antecedent is missing, applicability is not established. Fictional rule: limited access → identify the reported limitation Applies · met Access: limited Limitation: enclosed area locked Applies · unmet Access: limited Limitation: missing Not established Access statement: absent Do not infer a pass or a failed requirement The three cases use the same rule. Missing evidence is not a made-up outcome.
    1. The stated teaching convention: when a report explicitly says access was limited, it must identify the reported limitation.
    2. Limited access plus a named locked enclosed area: the rule applies and the requirement is met.
    3. Limited access with no named limitation: the rule applies and the requirement is unmet.
    4. No access statement: applicability is not established. Do not invent a pass, a failed requirement or non-applicability from the missing antecedent.

    For a fully supplied record, we can inspect what it definitely contains or omits. For an excerpt, a missing passage may simply be outside the excerpt. That difference is part of the test premises. Without it, a reviewer could call a complete source defective because a short quotation did not include every section.

    Four cases, four explanations

    Teaching outcomes under requirement R, not results from a candidate validator
    Supplied caseOutcomeWhy
    A · Report says access was limited and identifies the locked enclosed area.Requirement met.The trigger is established and a limitation is identified.
    B · Complete report says access was limited; it definitely contains no limitation description.Requirement unmet.The trigger is established; the required information is absent from the complete record.
    C · Evidence explicitly establishes that the report records unrestricted access for its stated scope.This conditional requirement does not apply.The limited-access trigger is explicitly not met. Other requirements may still apply.
    D · An excerpt supplies neither an access statement nor evidence about whether one exists elsewhere.Applicability not established.The supplied evidence does not establish the trigger. Do not turn that gap into a pass or failure.

    Harbour Court report version 2 supplies an instance of case A: it says access was limited and identifies the locked enclosed area. This establishes only that the fictional description meets R. It does not establish that the enclosed area was actually locked, that every limitation was reported or that the inspection was professionally adequate.

    Case C does not claim that no limitation existed in the world. It describes the explicit access evidence stipulated for the exercise. A contradictory source could raise a separate question about accuracy. Keep that evidence question visible instead of expecting a completeness rule to decide it.

    Try it: change one premise at a time

    Start with case A. First remove the limitation description from the complete report while leaving the limited-access statement. Then, separately, restore the description and add a second limitation: an inaccessible loft, in a new fictional inspection scope. Predict the outcome each time.

    A reasoned response

    The first change produces case B: R applies and is unmet. The second still meets R because it requires at least one identified limitation, not exactly one. A proposed maximum of one would reject a report containing two relevant limitations without any supplied reason for that restriction.

    Now repeat the same “locked enclosed area” description twice. Repetition does not demonstrate two different limitations. If a rule is meant to count distinct limitations rather than entries, that distinction needs to be expressed. R asks only for at least one; it does not settle how duplicates should be represented.

    A harder variation

    Suppose the report says “See appendix”, but the appendix was not supplied. You cannot assert that the complete report lacks a limitation. Ask for permitted evidence of the referenced content and clarify whether the profile accepts a reference that can be followed. The problem may be an incomplete review bundle, not a defective report.

    Use a counterexample to locate the design problem

    A reviewer proposes strengthening R to “Every inspection report must identify a limitation”. Bring a report that explicitly records physical access to every agreed area and does not report any limitation. The proposed rule would require the author to invent a limitation or misuse the field to say “none”. That is evidence that the rule's applicability condition has been lost, not evidence that the report is incomplete under R.

    Another reviewer proposes “Any nonempty text is sufficient”. The record “limited” then passes, although it does not identify what was limited. This exposes a gap between the intended requirement and a shallow text-presence check. Technical specialists can explain which parts can be checked reliably and which need human judgement; the specification should not disguise the gap.

    A worked review note

    The proposed maximum of one limitation excludes the supplied case with a locked enclosed area and an inaccessible loft. Both concern the same identified inspection. Please explain the business reason for the maximum or allow several limitations. This challenges the rule's design, not the accuracy of either observation.

    Keep the rule's version, the smallest relevant case and the expected outcome with your note. Say explicitly whether you are challenging the rule, identifying a record that fails it or asking for missing evidence. For the underlying distinctions, see rules, requirements and exceptions.

    Comments

    Loading comments…