Data validation rules for inspection records

A technician on a scaffold reads 0.312 inches on the gauge and types 0.321. Nobody notices. The short-term corrosion rate at that CML drops, the remaining life stretches, and the next inspection is scheduled two years later than it should be. Inspection records are unusual among business data in that a small error does not stay small. It propagates through a chain of calculations that ends in a date on a work list. Inspection data quality rules are the controls that break that chain early, and the difficulty is applying them in a way that catches the error without driving the technician back to paper.

Why do inspection data quality rules matter more than in other systems?

Three properties of inspection data make validation more consequential than in most enterprise systems.

The values feed calculations that set schedules. A thickness reading is an input to a corrosion rate, which is an input to a remaining life, which is an input to an inspection interval under API 510 or API 570. An error in the reading is an error in the interval. In most systems a bad value is a reporting problem; here it is a scheduling problem with safety consequences.

Errors are hard to detect later. A thickness reading is a measurement of a physical state at a moment. It cannot be re-taken for that date. If the 2019 reading is wrong, the only evidence is its inconsistency with 2016 and 2022, and that inconsistency may not be examined until someone questions the interval.

The data is captured under the worst conditions. Gloves, glare, height, heat, confined space, no network. The capture environment is where errors originate and where controls are hardest to apply, which is why so many inspection systems apply them at the office instead, after the technician has left the site and cannot re-measure.

The consequence is that validation cannot be a report run quarterly. It has to be built into the record at the moment of capture, tightened at review, and enforced again when a calculation runs, with each layer catching what the previous one could not.

What are the classes of validation rule?

Rules fall into five classes. Naming them helps because each has a natural home in the record's life and a natural response when it fails.

1. Identity rules. The reading belongs to a real CML, on a real component, on a real equipment item in the hierarchy. The technician is a real user with a qualification for the method. The instrument is a registered device. Identity failures are the only class that should block, because a value with no home cannot be stored correctly.
2. Completeness rules. The fields the procedure requires are present: date, value, unit, technician, instrument, and for calculations the governing inputs (nominal thickness, tMin and its method, design pressure, material). Completeness failures at capture are warnings; at calculation they are blocks, because a remaining life without tMin is not a number.
3. Range rules. The value is plausible. A thickness is positive, below nominal plus tolerance, above zero, in the expected unit. A DFT reading is within the coating specification's plausible band. A date is not in the future and not before the equipment existed. Range failures are warnings with a required reason.
4. Consistency rules. The value agrees with what else is known. A thickness has not increased beyond gauge repeatability since the last reading. A short-term rate is not an order of magnitude off the long-term rate. A reading marked as a specific CML matches that CML's grid position. Consistency failures are flags for review, because the resolution needs judgement.
5. Provenance rules. The value carries where it came from: gauge over Bluetooth, keyboard entry, workbook import, archive extraction. Retyped and imported values are distinguishable from instrument readings. Every value has an actor and a timestamp. Provenance failures are defects in the system, not the technician, and should be impossible by design.

A useful check on any rule set is to classify each rule and ask whether it is enforced where its class says it should be. Most programs discover that their consistency rules run only in a spreadsheet the engineer maintains, and that their provenance rules do not exist.

Which rules belong at capture time?

Capture is the only moment the technician is standing at the CML and can re-measure. Rules that fire here have a chance to fix the data; rules that fire later can only flag it. The tension is that capture is also the moment when friction drives people to paper.

The rules that belong at capture:

• Identity, enforced. The record is opened from the CML, by QR code, barcode, a drawing point or a work pack, so the reading cannot land on the wrong point. This is prevention rather than validation and it eliminates the largest class of error.
• Instrument capture, preferred. A reading that arrives from the gauge over Bluetooth or a keyboard wedge has no transcription error. Where the platform supports instrument profiles, the form should prefer them and mark keyboard entry as such.
• Unit and format, enforced. The field accepts a number in the configured unit. Mixed units in one column are a migration problem that starts at capture.
• Range, as a warning with a reason. A reading above nominal, or below the previous reading by more than a plausible loss, prompts: re-measure, or confirm and give a reason. The reason is stored on the reading.
• Consistency with the last reading, as a warning. An increase beyond the gauge's repeatability triggers a re-measure prompt. If the technician confirms, the reading is saved with the flag and goes to review.
• Completeness of procedure-required fields, as a warning before submit. The technician sees what is missing and can fill it while still on the scaffold.

What does not belong at capture: rules that require the whole survey, rules that require engineering judgement, and any rule whose failure blocks saving a value the technician has already measured. A blocked save produces a note in a pocket and a retyped number that evening, with none of the capture context.

All of this has to work offline. Rules that live on a server are not rules in the field. The rule vocabulary, the reference data and the last reading at each CML must be on the device.

Which rules belong at review and calculation time?

The office has what the field does not: the whole survey, the history, and time. Rules that need those belong here.

At review, before readings are accepted into the governing history:

• Every capture-time flag is resolved by a reviewer, with the resolution recorded: accepted with reason, re-measured, reassigned to the correct CML, or rejected.
• Survey completeness is checked against the plan: every CML in the work pack has a reading or a documented reason it does not (obstructed, insulation not removed, scaffold not available).
• Consistency across CMLs on the same component is examined: a single outlier in a grid of forty is a flag, a shift across the whole grid is a finding.
• Provenance is checked: readings that arrived by keyboard rather than instrument, or from an import, are visible and can be sampled.

At calculation, when rates, remaining life and intervals are derived:

• Governing inputs are required. A calculation with no tMin, no nominal, or no rate basis does not return a number; it returns the missing input.
• The method is recorded with the result. Minimum thickness returns the design code it routed to and the formula, so a reviewer can check it. Fitness-for-service screening results are provisional until an engineer verifies them.
• Nothing is auto-derated. A calculation informs a decision; it does not make one.
• Recomputation is idempotent. The same inputs produce the same result, every time, and the result carries the version of the calculator that produced it.

The segregation between review and calculation matters. A reviewer who accepts a flagged reading should not be the same person who approves the calculator that consumes it, and the platform should enforce that rather than rely on procedure.

How should a validation failure be handled without losing the reading?

The default in many systems is that invalid data is rejected. In inspection the default should be that invalid data is kept and flagged, because the reading is evidence even when it is wrong, and because rejection in the field produces unrecorded workarounds.

A handling pattern that works:

• Store the value as entered. Never modify a technician's reading. If it is wrong, the correction is a new value with its own provenance, and the original remains visible.
• Store the flag with the value. The rule that fired, its severity, the technician's response at capture, and later the reviewer's resolution. The flag is part of the record.
• Route by severity. Identity failures block. Completeness and range failures require a reason and go to review. Consistency failures go to review with the history shown beside them. Provenance failures alert the administrator.
• Show the history at the point of resolution. A reviewer deciding whether 0.321 is right needs the 2016, 2019 and 2022 readings on the same screen, and the gauge tolerance.
• Record the resolution as an audited event. Who resolved, when, what they chose, and why. In a hash-chained audit trail this becomes part of the permanent record and is never auto-deleted.
• Handle conflicts the same way. When two technicians touch the same record offline and sync disagrees, keep both values in a conflict record with who resolved it and when. Last write wins is a data loss policy.

The reading the technician took is a fact about that moment. The system's job is to keep it, question it and record the answer, not to make it disappear.

How do you measure data quality over time?

Validation rules that fire and are resolved generate data about the program. That data is the quality measure, and it is more useful as a trend than as an audit.

Metrics worth tracking per survey and per unit:

• Flag rate by rule class. Proportion of readings that triggered identity, completeness, range, consistency and provenance rules. A rising range flag rate on one crew is a training signal; a rising consistency flag rate on one unit may be a real mechanism.
• Resolution time. Days from flag to resolution. A growing backlog means reviewers are overloaded or the rules are too noisy.
• Resolution mix. Proportion resolved as re-measured, accepted with reason, reassigned, or rejected. A high reassignment rate means identity capture is weak.
• Instrument capture rate. Proportion of readings that arrived from a gauge rather than a keyboard. This is the single best leading indicator of transcription error.
• Governing input completeness. Count of CMLs missing tMin, nominal, or rate basis. Every one is a CML whose interval cannot be computed.
• Offline conflict rate. Frequency of sync conflicts and time to resolve them.

Report these as a scheduled report to the inspection lead and the integrity engineer. Data quality that is measured improves; data quality that is audited annually is discovered annually.

How intelliSPEC handles it

Validation in intelliSPEC starts with identity. Records open from the CML via QR and barcode, from drawing points detected automatically on uploaded drawings, or from the work pack, so a reading lands on the point it was taken at. Seven instrument profiles (DeFelsko PosiTector 6000 and UTG, Elcometer 224T and MTG, Evident 38DL Plus, and the TSL 1153 RFID reader) bring readings in over Bluetooth or keyboard wedge without retyping. Range, requiredness and cross-field rules are configured per form through the form designer rather than in code, and the field application runs them fully offline with no time limit on the local store, so the rules are present on the scaffold and not only on the server.

Nothing is silently overwritten. Field-level conflict records hold the local value, the remote value, who resolved it and when. Every create, update, import and export is written to a hash-chained audit trail with actor, tenant, request identifier and device details, never auto-deleted. Workflow transitions (submit, approve, reject, hold, resume, reopen, owner sign-off and others) give reviewers a place to resolve flags before readings govern.

At calculation time, intelliINTEGRITY computes short-term, long-term and maximum corrosion rates with the maximum governing, remaining life as (t minus tmin) divided by rate, and API 510 and API 570 intervals capped at half the remaining life; minimum thickness returns the design code and the formula alongside the number. Fitness-for-service screening results are provisional until an engineer verifies them and equipment is never auto-derated. A segregation-of-duties rule prevents a calculator's author from approving it. Field capture runs through intelliINSPECT and intelliNDT, and scheduled reports carry the quality metrics above to the people who own them. Terms used here are defined in the glossary.