GLOBAL INGESTION & NORMALIZATION SPECIFICATION

Data Methodology & Quality Rules

Transparent specification of our global ingestion architecture, statutory reconciliation rules, scoring mechanics, and foundational African pilot boundaries.

1. Pipeline Stages (As Built & Running)

Corridor Sweeps & Trade Gazette Extraction

Headless Python connectors crawl scheduled public transport manifests, border post carrier declarations, and public gazettes across key trade corridors, starting with primary African freight arteries (North-South, Northern Corridor, Trans-Kalahari, Central, and West African corridors) and scaling to global routes. Connectors adhere strictly to robots.txt and rate limits.

Statutory Registry Re-Identification

Raw corporate records are matched against official national company registries, including initial pilot reconciliation with South Africa (CIPC), Zimbabwe (DCIP), Botswana (CIPA), Kenya (BRS), Nigeria (CAC), and expanding continental and international registries. Registration numbers are validated for proper country prefix and statutory year format.

GLEIF ISO 17442 Validation (No Invented LEIs)

When an LEI is claimed or found in statutory filings, it is validated via ISO 7064 MOD 97-10 check digits against the Global Legal Entity Identifier Foundation (GLEIF) open directory. If unindexed, the LEI field is returned as null. CurrentDebrief never synthesizes or invents identifiers.

Company-Level Channels & Personal Contacts Out of Scope

Following industry-standard enterprise data governance (such as Veridion), personal contacts and direct personal cell numbers for named executives are strictly out of scope. We adhere strictly to robots.txt, index only company-level channels (generic business emails such as info@, operations@, and corporate desks), and exclude private cell phones to eliminate regulatory risk under POPIA, NDPR, and GDPR.

Published 4-Component Score Computation

Every record receives a deterministic completeness score computed from confirmed fields using our published formula. A 100% score is awarded only if every single component is verified against primary sources.

2. Published Quality Score Formula (25% Each)

Deterministic Formula

25%

Statutory Registration

Reg number + named statutory authority (+15% if reg number only)

25%

Physical Depot & Hub

Street address line + recognized city (+15% if city only)

25%

Operational Contact

Validated phone/switchboard (+15% if corporate email only)

25%

LEI or Web Domain

Valid ISO 17442 LEI (+25%) or verified corporate web domain (+20%)

Status Classification Rules:
• Source-confirmed: Requires verified statutory registration, named authority, and either valid LEI or corporate web domain.
• Public record: Established corporate filing with validated registration number.
• Unverified: Public directory listing pending registry reconciliation.

3. Algorithmic Validation Rules

LEI ISO 7064 MOD 97-10

20 alphanumeric characters. First 4 characters identify the LOU. Last 2 characters are mathematical checksum digits. Failing codes are rejected.

E.164 Prefix Parsing

Leading trunk zeros stripped and country codes prepended (+27, +263, +267, +254, +234). Valid length checks applied per national telecom numbering plan.

Plausibility Bands

Estimated turnover brackets are scaled to employee headcount ranges and verified sector averages. Values are explicitly labeled as Modelled estimate.

Geographic Boundary Check

Depot and headquarters municipalities are mapped against known administrative city dictionaries for the stated country code.

4. Known Gaps & Current Limitations

In keeping with our transparency commitment, we openly state the current operational boundaries of our pilot warehouse:

Global Scope & Foundational African Pilot: CurrentDebrief is architected as a global commercial intelligence platform. Africa serves as our initial regional deployment and testbed under AfCFTA principles. Our active baseline pilot dataset is concentrated across 5 anchor commercial hubs (Harare, Johannesburg, Gaborone, Nairobi, Lagos), while ingestion connectors and trade corridor feeds continuously expand across continental Africa and planned international trade markets.
Commercial Turnover: Private company financial filings in emerging markets are often not publicly indexed. Commercial turnover figures are modelled estimates based on headcount and corridor volume.
Personal Contacts Strictly Out of Scope (Veridion Compliance Standard): In compliance with global data protection laws (POPIA, NDPR, GDPR) and ethical web crawling (strict robots.txt adherence), CurrentDebrief puts personal contacts out of scope. We do not sell, license, or index personal mobile numbers of private executives. Only verified company-level channels (generic business emails) are indexed.
Omission over Guessing: When a data attribute is unverified, it is returned as null rather than populated with synthetic placeholder values.

Inspect Live Warehouse Coverage

Review entity counts by territory, fill rates, and audit benchmarks.

View Coverage & Gaps Data Provenance