Truflation applies stringent quality control procedures daily to ensure the highest standards of data integrity. The following processes are implemented to evaluate the accuracy and reliability of the data sources we use.
Key Quality Control Processes
1. Historical Data Comparison
- Compares the most recently received data with previous entries from the same source.
- Assesses consistency across time.
- Action: If discrepancies are detected, a red flag is raised for review.
2. Data Presence Check
- Verifies that at least 13 consecutive months of data are available for year-over-year comparisons.
- Process for Missing Data:
- If monthly data is unavailable, the previous month's data is carried forward daily until new data arrives.
- Example: September data is used in October until October data is received (usually in early November).
- This applies similarly to weekly and daily datasets.
- Action: If data is missing, a red flag 🚩 is raised.
3. Current Data Comparison
- Evaluates the most recent data input for:
- Missing data entries.
- Duplicate data values (same as the previously logged data).
- Action: If issues are identified, a red flag 🚩 is raised for further investigation.
4. Data Variation Analysis
- Checks if the most recent data differs by more than 5% (positive or negative) compared to the previous data from the same source.
- Action: If the variation exceeds 5%, a red flag 🚩 is raised for verification.
- If no issues are found, the data is cleared for the next stage.
Final Step: Red Flag Handling
- Data flagged during any of the above steps undergoes a rigorous review by the Truflation team.
- Only after successful validation, flagged data is approved for use.
These automated processes ensure Truflation delivers consistent, accurate, and reliable data to our users every day.