M1.5 — Data Quality, Integration & Traceability

M1.5 — Data Quality, Integration & Traceability

1. Practical outcome

By the end of this section, the learner should be able to establish and operate a project-controls data environment in which important information is complete enough, accurate enough, timely enough, consistent enough and traceable enough to support management decisions. They should be able to identify authoritative sources, define common identifiers and mappings, align reporting cut-offs, reconcile data between systems, retain evidence of transformations and preserve the history needed for trend, audit and learning.

The objective is not to create a technically impressive data architecture for its own sake. It is to make sure that a manager can rely on the project position being presented and, when challenged, that the practitioner can show where the information came from, how it was transformed and why it is credible.

A useful question throughout this Knowledge Area is: “If a decision-maker challenges this number, date or status, can we trace it back to the underlying project evidence and explain every material transformation on the way?”

2. Core concepts

Project controls is only as reliable as the data beneath it

Schedules, forecasts, dashboards and performance indicators can look sophisticated while being built on incomplete, stale, duplicated or inconsistently coded information. Poor data does not become good information merely because it has been automated or visualised. In fact, automation can make weak data more dangerous by distributing it faster and with greater apparent authority.

Data quality therefore belongs inside project controls, not outside it as an information-technology issue. The project-controls practitioner needs enough understanding of sources, definitions, interfaces and transformations to judge whether the information used for analysis is fit for the decision being made.

Data quality has several dimensions

No single test proves that data is “good”. Different dimensions matter for different decisions. Useful dimensions include:

·       Completeness — are all required records, fields, periods and scope elements present?

·       Accuracy — does the value correctly represent the underlying fact or approved judgement?

·       Timeliness — is the information current enough for the decision being made?

·       Consistency — are the same definitions, codes, calendars, status rules and units being applied across the project?

·       Validity — does the value conform to the agreed format, range, logic or business rule?

·       Uniqueness — are duplicate records being prevented or identified?

·       Traceability — can the reported value be followed back to its source evidence and forward through the transformations that produced the management output?

A dataset does not need to be perfect in every dimension before it can be used. The important professional judgement is whether its limitations are known and whether it is sufficiently reliable for the decision at hand. A minor coding gap may be tolerable for an internal trend chart but unacceptable for a contractual claim, baseline change or board decision.

Authoritative source does not necessarily mean one system contains everything

Projects commonly use different systems for different purposes. The planning system may be authoritative for activity dates and logic; the finance system for posted actual cost; procurement for commitments; the risk system for risk records; a document system for approvals; and a reporting platform for management views. Trying to force every type of information into one application can be less effective than defining clear source authority and controlled interfaces between systems.

The important point is to define, for each material data item, which source is authoritative, who owns it, when it is refreshed and how conflicts are resolved. A dashboard is rarely the authoritative source simply because senior managers see it; it is usually a presentation layer built from other records.

Common identifiers create the joins between control disciplines

Integration depends on stable identifiers. WBS codes, control-account IDs, milestone IDs, contract/package IDs, cost codes, organisation codes and risk/change references allow records created in different systems to be related without relying on free-text descriptions. M1.2 established the control structures; M1.5 uses those structures to connect data reliably.

A description such as “IT installation” may appear in several systems with small wording differences. A controlled code such as WBS 1.4.3 or milestone MS-OPEN-010 gives the project a far more dependable integration key. Where systems use different coding schemes, the mapping between them should be explicit, versioned and governed.

Data lineage and provenance explain where information came from

Data lineage is the chain from source record through extraction, mapping, transformation and calculation to the final report or indicator. Provenance adds context about origin, ownership, timing and any adjustments. Together they allow a practitioner to answer questions such as: Which schedule snapshot supplied this milestone date? Which finance extract supplied this actual cost? Which accrual file was added? Which formula produced this KPI? Who approved the manual correction?

Lineage is especially important when management information is assembled through spreadsheets, integration platforms, data warehouses or business-intelligence tools, because the final number may be several steps removed from the original evidence.

Integration is about coherent meaning, not merely moving data

Two systems can be technically connected while still producing misleading information. For example, cost data may use a calendar month cut-off while schedule data uses a status date five days earlier; one system may include approved change while another still reports the previous baseline; or one may classify a package by WBS while another uses a supplier code with no controlled mapping.

True integration requires semantic alignment: the records should refer to the same scope, time period, units, coding definitions and authorised project position. Technical interfaces help, but project-controls logic determines whether the joined data actually means what the report implies.

Reporting cut-off and temporal alignment matter

Project information changes continuously. A meaningful reporting position therefore needs a defined data date and clear cut-off rules. Schedule status, physical progress, actual cost, accruals, commitments, risk status and change status should be aligned as far as practical to the same reporting period. Where they cannot be aligned, the timing difference should be explicit.

A common source of apparent inconsistency is simply that one dataset is newer than another. The correct response is not always to “fix” the numbers; it may be to explain the timing difference and design the reporting process so that the difference is controlled.

Reconciliation tests whether related data tells the same underlying story

Reconciliation compares totals, mappings or states across related sources. Examples include reconciling project actual cost to the finance ledger, schedule activity counts to the approved WBS, procurement commitments to the cost forecast, progress evidence to schedule status, or baseline totals to the approved budget.

Test record counts, unique keys, unmatched items and allocation totals as well as the grand total. Omitting one 100 CU item and duplicating another 100 CU item leaves the total unchanged while misreporting scope and ownership. A successful total reconciliation therefore does not, by itself, prove that a join or mapping is correct.

A reconciliation difference is not automatically an error. It may reflect legitimate timing, scope or accounting treatment. The practitioner’s task is to identify the difference, explain its cause and determine whether an adjustment or disclosure is required.

Transformations and calculations need controls too

Management information often contains calculated values that do not exist directly in any source system: forecast variance, milestone slippage, earned-value indices, trend measures, risk exposure totals or RAG status. The formulas, thresholds and transformation rules used to create these outputs should be documented and stable enough to reproduce.

Manual adjustments are sometimes necessary, particularly around accruals, corrections or temporary system limitations. The issue is not that manual adjustment is always poor practice; it is that it should be controlled. The original value, adjusted value, reason, owner and approval should remain visible.

Exceptions should remain visible until understood

Good data processes do not silently discard records that fail validation. They identify exceptions and route them for resolution. An unmapped cost transaction, an activity with an invalid WBS code, a risk with no owner or a milestone missing from the current extract should create a visible exception rather than disappearing from the report.

The practitioner should distinguish between a data-quality defect, a genuine project-control exception and a harmless technical formatting issue. That distinction determines who needs to act and how urgently.

Historical snapshots are part of the control record

Current-state reporting is not enough. Trend analysis, assurance, lessons learned, estimating feedback and forensic analysis all rely on knowing what the project believed at earlier points in time. The project should retain sufficient period snapshots and source evidence to reconstruct material changes in schedule, cost, risk, progress and forecast.

This does not mean every working file must be retained forever. Retention should be proportionate and governed. Formal configuration control of authorised baselines is addressed in M6.3; M1.5 is concerned with preserving enough project-control data history and lineage to understand how information changed.

Ownership and stewardship make data quality operable

Every material dataset should have an accountable owner or steward who understands its definition, source, update cycle, validation and downstream uses. Without ownership, data-quality problems become everybody’s problem and therefore nobody’s responsibility.

Project-controls teams should also define who can change reference data, who approves mappings, who resolves reconciliation breaks and who signs off the reporting extract. This is governance at the data level rather than merely technical administration.

Automation and AI can support control but do not replace source authority

Automated validation, anomaly detection, matching and narrative generation can reduce effort and highlight unusual patterns. They are particularly useful for repeated checks such as duplicate records, missing codes, impossible dates or unexpected period movements. However, an automated tool cannot decide that an unofficial source should become authoritative simply because it is easier to access, and it cannot repair unclear scope ownership or inconsistent baseline definitions without professional judgement.

The stronger principle is to automate controlled processes and retain traceability over the rules and source data used. Where AI contributes to interpretation or commentary, the underlying project facts should remain independently identifiable and verifiable.

3. Step-by-step method — how to establish and operate trusted project-control data

1. Start from the decisions and outputs that the data must support.

List the management questions, reports, forecasts, controls and assurance needs that depend on project data. This prevents the project from collecting fields simply because a system makes them available. Define which information is decision-critical and therefore deserves the strongest control.

2. Inventory the material data sources.

Identify the systems, registers, spreadsheets, interfaces and source evidence used for schedule, cost, commitment, progress, risk, change, resources, commercial information and management reporting. Record the owner, update frequency, format and key limitations of each source.

3. Define authoritative source ownership for each material data item.

For activity dates and logic, actual costs, commitments, progress evidence, risk status, change status and other key data, state which source is authoritative. Name the owner or steward and define how disagreements between sources are resolved.

4. Confirm the control structures and common identifiers.

Use the WBS, CBS, OBS, control accounts, contract/package codes, milestone IDs and other structures established in M1.2. Check that identifiers are unique, stable and available in the systems that need to exchange data.

5. Build and govern the mapping rules.

Where different systems use different structures, create explicit mapping tables. Define one-to-one, one-to-many and many-to-one relationships carefully. Identify unmapped and invalid combinations and assign responsibility for maintaining the mapping.

6. Define data definitions and business rules.

Create a practical data dictionary for decision-critical fields. Define units, formats, permitted values, status meanings, milestone types, date conventions, currency, treatment of null values and any calculation logic. Make sure different teams use the same meaning for apparently simple terms such as “actual”, “forecast”, “complete” or “approved”.

7. Align the reporting calendar, data date and cut-offs.

Define when each source closes, when extracts are taken and the common reporting date. Where a source cannot align exactly, document the lag and how it is handled. Prevent late data from moving between reporting periods without a controlled rule.

8. Design validation checks close to the source.

Check mandatory fields, code validity, duplicates, date logic, numerical ranges, ownership, status consistency and other rules before data reaches the final report. Where possible, prevent invalid data at entry rather than detecting it only after reporting.

9. Define extraction, transformation and calculation logic.

Document how source records are selected, filtered, converted, mapped and calculated. Record formulas for management indicators and the rules behind thresholds or RAG status. Keep complex logic reviewable rather than burying it in undocumented spreadsheet formulas or opaque scripts.

10. Integrate the datasets using controlled joins.

Combine data using approved identifiers and mappings. Test whether the integrated records still refer to the same scope, period and authorised position. Do not assume that a successful technical join proves semantic consistency.

11. Reconcile totals and key states.

Compare project-control data with its authoritative sources and related systems. Reconcile actual cost to finance, commitments to procurement, schedule milestones to reporting outputs, progress to schedule status and baseline totals to approvals. Record material differences and their explanations.

12. Manage exceptions explicitly.

Create an exception process for missing, invalid, duplicate, unmapped or contradictory records. Assign each material exception an owner and resolution date. Distinguish between items that block reporting, items that require disclosure and items that can be corrected in the next cycle.

13. Establish end-to-end traceability for key management outputs.

Select important indicators, dates and values and prove that they can be traced from the report back to the source record. Retain the extract version, transformation logic, mapping version and any approved manual adjustment needed to reproduce the result.

14. Preserve period snapshots and audit history.

Retain the reporting datasets, extracts or controlled snapshots needed for trend and later review. Preserve enough history to understand when a value changed, why it changed and what management saw at the time. Avoid overwriting prior periods with the latest view.

15. Monitor data quality as a project-control measure.

Track material exceptions such as unmapped costs, missing milestone commentary, invalid codes, unreconciled totals, late data submissions or duplicate records. Use trends to identify systemic weaknesses rather than repeatedly correcting symptoms.

16. Improve the process under change control.

When structures, systems, interfaces or definitions change, assess the effect on historical comparability and downstream reports. Update mappings, dictionaries, validation and training deliberately. Do not let a system migration or new dashboard silently change the meaning of project information.

4. Worked Reference Project example — Community & Learning Centre

The Community & Learning Centre provides a realistic multi-system environment. The project is 24 months long, has a 50 million CU approved budget and is controlled through six Level-1 WBS elements. The examples below show how data quality becomes a practical project-controls issue rather than an abstract technology topic.

Reference Project source landscape

·       Planning system — authoritative for activity IDs, logic, calendars, status dates, baseline dates and current forecast dates.

·       Finance ledger — authoritative for posted actual expenditure.

·       Procurement system — authoritative for purchase orders, contractual commitments and supplier/package references.

·       Progress evidence register — controlled evidence for physical accomplishment under the rules established in M1.4.

·       Risk register — authoritative for risk descriptions, owners, qualitative assessments, response actions and status.

·       Change register — authoritative for proposed, approved, rejected and implemented changes.

·       Management reporting / BI layer — integrates approved extracts from the above sources but is not itself the primary source for most underlying facts.

The common WBS, control-account, package and milestone identifiers established in M1.2 are used to connect these sources. Each reporting cycle uses a defined data date and an extract log identifying which version of each source fed the report.

Example A — a milestone appears “on time” because the report uses stale data

At the Month 11 data date, the planning team updates milestone MS-DIG-030, “Digital systems ready for integrated testing”, from Day 330 to Day 344 because installation productivity has deteriorated. The schedule extract is issued correctly. However, the BI report refreshes from the previous month’s cached schedule file and continues to display Day 330 with a green status.

Nothing is wrong with the milestone logic itself; the data lineage is wrong. The practitioner should be able to see which schedule snapshot supplied the dashboard, identify that the extract date precedes the current status cycle, correct the pipeline and disclose the reporting error. A visually attractive green dashboard is less valuable than a traceable amber one.

Example B — reconciling the reported actual cost

Suppose the finance ledger contains 12.10 million CU of posted actual cost at the reporting cut-off. The project also has 0.30 million CU of approved accruals for work performed but not yet invoiced, giving a project-control actual position of 12.40 million CU for the period. The dashboard initially shows 12.25 million CU because one 0.15 million CU accrual line was omitted during transformation.

The reconciliation should therefore show: finance posted actuals 12.10; approved accrual adjustment +0.30; control actual 12.40. The 12.25 dashboard value is not “close enough” simply because the difference is small relative to 50 million CU. The difference must be understood because the same transformation fault may affect other packages and later forecasts.

Example C — one supplier package, three different descriptions

The specialist furniture supplier appears as “Learning Furniture”, “Furniture Package” and “Supplier 07” in three systems. Free-text matching initially causes some commitments to map to WBS 1.3 Furniture & Specialist Equipment and others to remain unallocated. The remedy is not better manual searching each month. The project establishes a stable package identifier PKG-1.3-07 and controlled mappings from each system to that identifier.

After the mapping is introduced, the same supplier commitment can be aggregated by WBS, CBS, contract/package and responsible manager without changing the source transaction. This is the practical link between M1.2 coding and M1.5 integration.

Example D — tracing an executive indicator back to evidence

The executive report states: “Public opening readiness is forecast 14 days late.” To demonstrate traceability, the practitioner should be able to identify the opening-readiness milestone ID, the current schedule version, the data date, the controlling predecessor path, the approved opening milestone, the formula used to calculate variance and any manual override applied to the displayed status.

The same principle applies to cost, risk and performance indicators. A manager does not need to see the full lineage every month, but the project-controls team should be able to produce it when confidence is challenged.

Example E — a legitimate difference that should not be “corrected”

Procurement shows a 4.0 million CU commitment for a technology contract, while the cost forecast shows 3.6 million CU remaining commitment exposure after agreed scope reduction. The figures are different for a legitimate reason: the procurement system still records the contractual ceiling, while the forecast reflects the latest expected outturn pending a formal contract amendment.

The practitioner should not force the systems to equal each other merely to make reconciliation pass. The correct control is to explain the basis, identify the timing and governance required to close the difference, and ensure management understands which figure is appropriate for which decision.

5. Interpretation and professional judgement

The strongest data-control environments are not those with the most technology. They are those where definitions, ownership, cut-offs and interfaces are understood well enough that management information can be trusted and challenged intelligently.

How good does the data need to be?

The answer depends on materiality and use. A forecast used for a board funding decision may require stronger reconciliation and evidence than a local working dashboard. The practitioner should define tolerances deliberately rather than pretending every field can be perfectly accurate at all times.

Is the problem a data error or a legitimate difference in definition?

Two values can differ while both are correct in their own context. Actual cost, cash paid, commitment and forecast are different concepts. Baseline finish, current forecast finish and contractual required finish are different dates. Good reconciliation identifies the basis of each number before labelling one “wrong”.

Is the source truly authoritative, or merely convenient?

Teams often begin treating a spreadsheet or dashboard as the truth because it is easier to access than the source system. Convenience should not silently redefine source authority. If an intermediate dataset becomes operationally important, its governance and lineage should be explicit.

Does integration preserve meaning?

Joining schedule and cost records by a common code is useful only if the code refers to the same scope boundary and reporting period in both sources. A technically successful join can still produce a false management story if scope, baseline version or cut-off differs.

When is a manual adjustment acceptable?

Manual adjustments may be appropriate for accruals, known source errors, temporary mapping gaps or one-off corrections. They become poor practice when they are undocumented, repeated indefinitely, overwrite the source, or are used to make the report look more favourable. A controlled adjustment should record the original value, adjusted value, rationale, owner, approval and period.

Should an exception block the report?

Not every exception has the same consequence. An unmapped 500 CU transaction on a 50 million CU project may be disclosed and corrected later. An unmapped 2 million CU package, a missing critical milestone or a stale baseline could make the whole report misleading. The practitioner needs a materiality and decision-impact test, not a purely technical pass/fail rule.

Is the reporting process hiding uncertainty behind precision?

Data systems often display dates to the day and costs to the unit even when the underlying forecast is uncertain. Traceability should not be confused with certainty. A perfectly traceable forecast can still be uncertain; good reporting distinguishes data quality from forecast confidence.

Are we preserving enough history to learn?

If each month simply overwrites the previous forecast, the project loses the ability to understand forecast accuracy, recurring bias and the progression of problems. Period snapshots should support trend analysis and future estimating without creating unnecessary storage or administration.

Are automated controls complementing or replacing judgement?

Automated checks can identify duplicates, missing codes and unusual movements very effectively. They cannot always determine whether a late milestone is reasonable, whether a mapping is conceptually correct or whether a reconciliation difference is commercially legitimate. Use automation to focus professional attention, not to remove it.

A useful practitioner test is: “Can another competent person reproduce the management position from the controlled source data without relying on private knowledge held by the person who built the report?” If not, the data environment is too dependent on individuals.

6. Practice diagnostic

Use this table to challenge the quality of the approach or deliverable. The rows consolidate the most important recurring red flags and checks; they are prompts for professional judgement rather than a substitute for the method above.

7. Try-it-yourself practical exercise

Using the Community & Learning Centre Reference Project, design and test a small integrated project-controls data environment rather than simply drawing a system diagram:

1. Define at least six management outputs that the project needs each month, including a milestone forecast, actual cost, commitment position, physical progress, top risk status and approved-change position.

2. For each output, identify the authoritative source, owner, reporting cut-off and the key identifier needed to connect it to the wider project model.

3. Create a simple data dictionary for at least twelve decision-critical fields. Include field name, definition, unit/format, authoritative source, owner and validation rule.

4. Map the relevant WBS, CBS, organisation, package and milestone identifiers for a selected part of the Reference Project. Include at least one case where two systems use different codes and require an explicit mapping.

5. Define the monthly reporting calendar showing when schedule, finance, procurement, progress, risk and change extracts are frozen. Explain any unavoidable timing differences.

6. Design at least ten validation checks, including missing codes, duplicate records, impossible dates, invalid status combinations and numerical reasonableness tests.

7. Introduce three deliberate data defects: a duplicated cost transaction, an activity with an invalid WBS code and a stale milestone extract. Show which checks detect each defect and how it should be resolved.

8. Reconcile a fictional finance ledger of 12.10 million CU posted actuals plus 0.30 million CU approved accruals to a management-control actual of 12.40 million CU. Then introduce a dashboard value of 12.25 million CU and identify the omitted transformation.

9. Create one controlled manual adjustment. Record the original value, adjusted value, reason, owner, approval and reporting period so that the adjustment is transparent and reversible.

10. Take one executive indicator — for example, “Opening readiness forecast 14 days late” — and create its lineage from dashboard back to milestone ID, schedule version, data date, baseline/target and variance calculation.

11. Identify one legitimate cross-system difference that should be explained rather than forced to reconcile, such as a contractual ceiling versus expected forecast exposure.

12. Define an exception log with severity, owner, due date and reporting treatment. Decide which exceptions should block publication and which can be disclosed and corrected later.

13. Define the period snapshots you would retain for 24 months and explain what later trend, assurance or lessons-learned questions those snapshots would support.

14. Produce a one-page Data Quality & Integration Control Note summarising source authority, identifiers, cut-offs, validation, reconciliation, lineage, exception management and history retention.

What good performance looks like

The exercise is successful when the learner can demonstrate that the same project fact can move from source evidence to management reporting without losing its identity or meaning; that differences between systems are reconciled or explained; that errors remain visible until resolved; and that another competent practitioner could reproduce the reported position from the retained records.

8. Bridge to capability

The How-to teaches how to create trusted, integrated and traceable control data. At APPLIED and above, capability assessment should test whether the practitioner can resolve realistic source conflicts, late records and mapping changes, and reproduce a material reported value. Source-authority records, validation results, reconciliations and exception trails should support the assessment. Supporting work products are illustrative and should be assessed with the practitioner’s reasoning. Sample learning questions illustrate the judgement at each recognition level; they are discussion prompts rather than validated assessment items.

Learning question: Do you understand what makes project-control data reliable and traceable?

Capability question for APPLIED and above: Can you demonstrate that a material management output can be reconciled, reproduced and traced through its transformations to authoritative evidence?