From spreadsheets and PDFs to a queryable record

Every plant has one. A workbook called something like , maintained by one inspector for eleven years, with a tab per equipment item, a column per survey, conditional formatting that turns a cell red at some threshold only the author remembers, and a formula for corrosion rate that references a hidden sheet. Beside it, a network folder of signed PDF reports, one per inspection, named by date and contractor. Together they are the plant's inspection history. Apart from the author, nobody can use them. An Excel to IDMS migration is the process of turning that pair into a record that anyone with the right permission can query, and doing it without losing the eleven years.

Why does the spreadsheet win for so long?

The spreadsheet is not a failure of discipline. It wins because it is good at the job it was asked to do, and understanding why is the first step to replacing it.

• It is immediate. A reading goes in the moment it is taken. No form to open, no workflow to submit, no sync.
• It is flexible. A new column appears when a new question does. The author never waits for a release.
• It computes. The rate formula is right there, visible and editable. The engineer trusts it because the engineer wrote it.
• It reports. The workbook is the report. Filter, sort, print, send.
• It belongs to someone. One person owns it and keeps it right.

Each of these strengths is also the reason it fails at scale. Immediacy means no validation: a reading against the wrong CML is accepted silently. Flexibility means every workbook has a different shape, so nothing can be queried across them. Visible formulas mean editable formulas, and no history of edits. The workbook being the report means the data cannot produce any other report. Belonging to someone means it leaves when they do.

The migration must preserve the strengths, or the workbook comes back. That is why the target is not "a database" but a queryable record with a field application as fast as the spreadsheet was.

What does a queryable record mean in practice?

A record is queryable when a question about the plant can be answered by asking the system rather than by opening files. Concretely, the following should each be a single view or a single API call, not a research project.

• Every CML in the unit whose remaining life is under four years.
• Every reading taken by a given technician in a given month, with the instrument used.
• Every equipment item with an open CUI finding and an inspection due within twelve months.
• The full thickness history at one CML, with the design inputs and the rate calculation shown.
• Every record changed in the last week, by whom.

Four properties make this possible, and they are what the migration has to build.

Identity. Every reading is attached to a real CML, which is attached to a real component on a real equipment item in a real hierarchy. The workbook approximates this with tab names and row labels; the record enforces it.

Structure. Findings are fields (mechanism, severity, location, status), not sentences. A red cell becomes a status value with a definition.

Provenance. Every value carries who entered it, when, from what device, and every change afterwards. The workbook's "last modified by" covers the whole file; the record's audit trail covers the cell.

Derivation. Rates, remaining life and intervals are computed by the system from the history, the same way every time, with the formula and code reference returned alongside the number. The engineer can check the arithmetic without owning it.

How do you go from Excel to IDMS: a column by column method

The workbook is the primary source and it is more structured than it looks. The method is to make that structure explicit before anything is loaded.

1. Inventory. List every workbook, every tab, and every column, with a sample of values. Note hidden sheets and external references. The hidden sheet is often where the design data lives.
2. Classify every column into one of five kinds:
   • Identity: equipment tag, component, CML number, location description.
   • Design input: nominal thickness, minimum thickness (and how it was set), design pressure, material, diameter and schedule, allowable stress, joint efficiency, corrosion allowance.
   • Reading: a thickness or other measurement, with a date, and if available a technician and an instrument.
   • Derived value: corrosion rate, remaining life, next due date, anything with a formula.
   • Annotation: comments, colours, notes in the margin.
3. Map identity and design inputs to target fields with the same engineering meaning. Where the workbook has one "tmin" column and the target has tMin plus the method that produced it, decide the method and record the decision.
4. Unpivot the readings. A workbook stores surveys as columns: one column per year. A record stores readings as rows: one row per CML per date. This unpivot is where most of the loading effort goes and where most errors hide, because a column header like "2019 (Q3)" has to become a date and a blank cell has to become "no reading" rather than zero.
5. Recompute the derived values in the target and compare them to the workbook's own formulas. Differences are either precision, definition (which reading was "initial"), or errors in the load. Resolve each.
6. Convert annotations. A comment that says "obstructed, reading from adjacent" becomes a reading qualifier. A red cell becomes a status. A note that fits no field is attached as a note on the record, never discarded.
7. Keep the workbook reference. Every migrated reading records which workbook, tab and cell it came from. When someone doubts a number, that reference settles it in seconds.

Where the target offers workbook import with assisted column mapping, let it draft the mapping and review every design input by hand. The assistant will match column names well; it cannot know that the "Nominal" column was overwritten with actual readings in 2014.

What do you do with the PDFs?

The PDF archive is larger than the workbooks and less structured. Resist two temptations: treating the PDFs as the record, and trying to extract everything from every one.

The PDFs are evidence. The signed report proves that an inspection happened, who did it and what they concluded. That proof is valuable and should be kept forever. But the readings and findings inside it belong in the history as data, where they can be queried, and the PDF should be attached to that data as its source.

A workable triage:

• Index everything. Every report gets a record with equipment covered, inspection type, date, contractor and author as fields, and the PDF attached. This is fast, and it turns a folder into a searchable archive even before any extraction.
• Extract selectively. Reports that carry thickness readings or findings not already in a workbook are extracted: readings become dated history at the right CML, findings become mechanism records with status. Reports whose content is already in the workbook are indexed only, with a cross-reference.
• Preserve the discrepancy. When a PDF and a workbook disagree about the same reading, load both with their sources and flag the CML. An engineer decides. Silent reconciliation is how history gets rewritten.
• Do not trust OCR on numbers without review. A misread decimal point becomes a corrosion rate. Every extracted thickness should be checked against the source page before it enters the history.

The extraction of thickness from scanned tables is tedious and it is where retrieval-based assistance helps most, provided it cites the exact source record so a reviewer can check. A system whose AI layer answers only from the tenant's own documents and cites the source record for every answer, as Spectra does, can locate the report that mentions a CML across thousands of PDFs; the transcription still needs a human check.

What breaks in the first ninety days after the switch?

The migration is complete when the field starts entering readings into the new system. The next ninety days determine whether it stays there. Expect these failures and plan for them.

• The field is slower. If the technician could type a reading in Excel in five seconds, a form that takes thirty loses. Instrument profiles that take the gauge reading over Bluetooth without retyping, forms that open to the right CML from a QR code, and full offline capture are what close that gap.
• The engineer cannot find the pivot table. The overdue-CML view, the rate outliers, the unit summary: whatever the workbook's owner used to produce weekly needs to exist as a saved view or scheduled report on day one.
• Someone finds a wrong number. They will, and it will usually trace to the unpivot step or a units mismatch. The workbook cell reference on the reading is what turns a crisis into a correction.
• The workbook keeps getting updated. Set a date after which the workbook is read-only, announce it, and enforce it. A parallel run without an end is two systems forever.
• A form is missing a field. The workbook had a column the form does not. If the platform is configurable, add the field in an afternoon. If it is not, the column comes back in a new workbook.

The measure of success is simple. Ninety days after cutover, the most skeptical inspector on the unit should be entering readings in the field application and not opening Excel.

How do you keep the spreadsheet from coming back?

The spreadsheet returns whenever the system fails to answer a question or slows down a task. Prevention is therefore about coverage and speed, not policy.

• Answer the workbook's questions natively. Before cutover, list every question the workbook's owner used it for. Each becomes a view, a report or a query.
• Make export easy. People will still want a spreadsheet for analysis. Give them a workbook export from the record so the spreadsheet is a copy of the truth, not the truth.
• Keep the field fast. Count the taps from opening the work pack to logging a routine reading. Every added tap is an argument for paper.
• Let the forms change. When a procedure adds a field, the form must add it in days, not releases. Otherwise the new field lives in a new column in a new workbook.
• Show the history. When an engineer opens a CML and sees eleven years of readings with the rate computed and the formula shown, the workbook has nothing left to offer.

How intelliSPEC handles it

intelliSPEC was founded in 2012 to replace paper reports and spreadsheets, and the loading surface reflects that. Workbook import with assisted column mapping drafts the mapping from a source workbook to the record; CSV import and an open API with 514 documented endpoints cover the rest, and every import is written to a hash-chained audit trail that carries actor, tenant, request identifier and device details and is never auto-deleted. Readings land on CML records that carry 64 fields, including nominal thickness, tMin and its method, design inputs, grid geometry and three corrosion rates, so a workbook column maps to a field with the same meaning rather than a comments box.

Once loaded, the derived values are computed by intelliINTEGRITY rather than copied: short-term, long-term and maximum corrosion rates with the maximum governing, remaining life as (t minus tmin) divided by rate, minimum thickness routed by equipment type to the relevant code with the formula returned, and API 510 and API 570 intervals capped at half the remaining life. In the field, intelliINSPECT captures readings fully offline with no time limit on the local store, takes gauge readings from seven instrument profiles over Bluetooth or keyboard wedge without retyping, opens records from QR and barcode, and records field-level conflicts with the local value, the remote value, who resolved it and when. Report definitions, scheduled reports and workbook export give engineers the views the master workbook used to provide, with the spreadsheet now a copy of the record instead of the other way round. Per-user pricing for the field and engineering seats is on the pricing page.