Reproducible datasets: Definitions, transforms, QA checks

Build fair-lending analysis on controlled definitions and a repeatable data preparation process that reviewers can follow.

Agree on the meaning of each field

Application date, action date, approved amount, offered price, and final terms can have different meanings across lending platforms. Create a data dictionary that names the authoritative source, business definition, format, permissible values, and responsible owner. Describe the treatment of corrections and late-arriving information. Resolve ambiguity with the business owner before using a convenient field as a substitute for the intended concept.

Version the transformation pipeline

Keep the extraction logic, joins, derived fields, mapping tables, and exclusion rules under change control. Record the source snapshot or reproducible retrieval boundary used for each run. Reconcile records before and after major transformations and retain exceptions for review. If an upstream schema or policy changes, assess the effect before allowing the change to flow into a recurring analytical report.

Design checks around analytical risk

Check uniqueness, completeness, valid ranges, time ordering, relationship integrity, and consistency between related fields. Include control totals and selected reconciliations against authoritative operational reports. Define what happens when a check fails: Stop processing, quarantine affected records, or continue with a disclosed limitation after approval. A quality check has little value if no owner is accountable for responding to it.

Package the run for reuse

Save the run identifier, versions, parameters, processing counts, failed checks, approved exceptions, and output location. Apply access and retention controls suitable for the records involved. A later reviewer should be able to reconstruct the analytical population without guessing which spreadsheet or query was used. Cicrim helps institutions organize this workflow so recurring analysis can be maintained by named data and control owners.

Working-session checklist

• Source and business dictionary
• Versioned extraction and transformation logic
• Join and population reconciliations
• Quality checks and exception decisions
• Run record and controlled output location