FBI CJIS Data Mesh RFI: What Its Traceability Requirement Does Not Specify

Date: August 26, 2026 · Author: Dmitrii Zatona 

In October 2024, the FBI posted a market-research notice for a data mesh at its Criminal Justice Information Services Division — the part of the Bureau that operates the country’s central law-enforcement data systems. The notice is explicitly preliminary: it outlines requirements envisioned for a contemplated acquisition, not a specification. One of its sentences reaches further than the rest: after stating that the architecture should empower each domain’s data scientists to build and deploy models, the document adds that “traceability of data used in model development” and a clear audit trail will promote transparency and compliance. Read as engineering, that sentence contemplates a property while leaving five operational choices unmade — what object is audited, at what granularity, which versions are retained, what event produces the record, and who can verify it. This article maps the five choices, examines what the notice’s own architectural vocabulary does and does not settle, and surveys — specification by specification — what existing open forms would express of each choice. A document that calls itself preliminary is entitled to leave choices open; the engineering interest is in what each choice turns out to involve.

1. What was published

The notice is RFI-05292024 , “CJIS Data Modernization and Data Mesh,” posted on SAM.gov on October 25, 2024 by the FBI’s Finance and Facilities Division, Procurement Section, with responses due November 18, 2024. SAM.gov classifies it as a Sources Sought notice; the attached seven-page document titles itself a Request for Information under FAR 15.201. Its point of contact is Contracting Specialist Tyler Cutright. The archived SAM record contains one PDF attachment and no published Q&A attachment or amendment — alongside the document’s own schedule, which invited questions by November 1 and stated that answers would follow by November 8. Two dates sit side by side without explanation in the record: the PDF’s title page reads 05/13/2024, the notice posted October 25, 2024; the PDF’s submission deadline reads November 15, the notice’s structured deadline November 18. The notice archived on December 3, 2024.

What followed is a bounded statement about public records: no related document — no solicitation, award, or later notice referencing this RFI, its Data Modernization Initiative, or its Data Services Value Stream — was identified in the reviewed sources as of August 25, 2026, across SAM.gov searches by number, phrase, office stream, and point of contact, and USAspending searches by keyword and agency (the full method is in the sources). The caveats are structural: the RFI itself asks respondents for GSA Schedule, GWAC, and NASA SEWP vehicles, and orders competed on such vehicles need not appear as public notices; the Department of Justice’s procurement forecast sits behind a dashboard that could not be read. Closed record, then — with the usual limits on what “closed” can mean in public data.

Method note: this is the fourth teardown in a series, following the FBI TSC , INL , and N3C  analyses. The TSC teardown concerned record-level evidence for a particular predictive output; this article concerns the traceability of data used before model development. Neither property supplies the other, and nothing below reuses that article’s analysis.

2. What the notice contemplates

The document’s introduction states its own status: responses may be used to formulate final requirements, and the description outlines preliminary requirements envisioned for the initiative. Its background describes a Data Mesh architectural approach fostering a decentralized environment where individual domains own, manage, and curate their data, and names the CJIS Division’s data systems — NCIC, NTOC, NGI, UCR, NICS, N-DEx — as the division’s context. (The document lists those systems as background; it does not assign any of them to mesh domains, and neither does this article.) The contemplated requirement, under its own heading, organizes the interest into five categories:

“This Section outlines the technical requirements for implementing a Data Mesh architecture within the FBI/CJIS. This approach prioritizes domain ownership, self-service data access, and decentralized governance. To achieve this vision, solutions should address five key categories: 1) Data Ingestion & Integration, ensuring seamless data movement between domains and sources. 2) ETL & Data Pipelines, enabling efficient data transformation and preparation for analysis. 3) Data Storage & Management, providing secure and scalable storage options for domain-owned data assets. 4) Data Governance & Quality, establishing best practices for data management, access control, and quality assurance across the mesh. 5) Self-Service Analytics, empowering users with tools to explore and analyze data products exposed by domains, fostering a data-driven culture within FBI/CJIS.”

Each of the five is an established product category — ingestion, pipelines, storage, governance, analytics — and the market the notice addresses ships all of them. The sentence this article is about comes two paragraphs later:

“The architecture should empower data scientists within each domain to leverage their domain expertise and readily available data to build and deploy models into production. Traceability of data used in model development and a clear audit trail will promote transparency and compliance.”

The modality matters and is a fact of the text: the document says envisioned, contemplated, should, and will promote — not “shall” or “must.” A market-research notice is entitled to that register; nothing in this article reads the sentence as a requirement. What can be read is what the sentence does not specify. “Traceability of data used in model development” names a property without naming its object: whether the traceable thing is a data product, the specific extract drawn from it, or a snapshot fixed at training time. “A clear audit trail” names a record without naming the event that produces an entry or the party who can check one. Five operational choices — object, granularity, retained versions, record-producing event, verifier — sit inside those two phrases, unchosen. In a document that also prioritizes decentralized governance and speaks of data products exposed by domains, those choices become operationally consequential to the extent data products are exchanged between domains; the notice does not state that any particular model would consume products from more than one domain, and this article asserts no such flow.

3. Five axes, six elements

To price the unmade choices, this article uses a decomposition of its own — the following is the author’s analytical model, not something the notice defines. The five unchosen choices act as axes over six artifact elements:

One calibration from the literature the notice’s vocabulary evokes. In the two defining data-mesh articles (2019 , 2020 ), the choices above are also unmade: lineage and provenance appear four times in total, as catalog metadata — “data provenance and data lineage as the metadata associated with each data product” — and as unelaborated platform-capability bullets; “data product versioning” appears once, undefined; the words “audit” and “trail” do not occur in either text (established by full-text scan of those two articles; a comparison, not a claim of dependence). Governance in that literature is federated and computational — “automated execution of decisions by the platform” — which names an executor for governance decisions and, in the two texts scanned, no record semantics for them.

4. Three problems

4.1 “Data used in model development” does not define its object

Three readings of the phrase produce three different engineering artifacts. If the object is the data product, traceability means a record that model M consumed product P — a record silent about what P contained when M was trained, since products update. If the object is the extract, traceability means recording what was drawn — and what that record survives depends on its form: an extract kept as a query plus identifiers can be re-run against data that has since changed, while an extract kept with its values or their fixity preserves what was actually read. If the object is a snapshot — the training body fixed at a point in time — traceability means content that can be re-identified later, which requires a fixity mechanism (a hash, a frozen copy, a version pin) that the product reading does not need. The engineering consequence runs downstream: an audit trail over products is a different artifact, with different storage and different failure modes, than an audit trail over snapshots, and a reviewer who holds one cannot answer questions posed against the other. The notice’s phrase is compatible with all three readings. Nothing in the document — or in the five categories, which name governance and quality but not versioning — selects one.

4.2 “A clear audit trail” does not define the event or the verifier

An audit trail is a sequence of records, and two definitions make it an engineering object: which event produces a record, and who can check the result. Candidate events multiply in a mesh vocabulary: a product’s publication, a consumer’s subscription, an extraction, a training-run start, a model’s deployment. The notice names none. Neither does it name a reader: a domain’s own engineers, a central governance function, or a party outside the operator entirely — each implies a different record design.

What a shipped ML stack documents illustrates the difference between having records and having chosen those two definitions. MLflow — a widely deployed open-source tracking and registry platform — records training inputs with a digest, links registered model versions to their producing runs (“You can trace back exactly how a model was trained”), and covers elements 2 through 4 of the model above inside one deployment. The same product’s REST API  documents runs/delete, runs/update, and model-versions/delete. That is a documented illustration of a mutable, operator-controlled provenance record; no reviewed MLflow documentation defines a verifier-facing assertion or a retention rule. The point is not a defect in a tool that never promised those things — it is that the difference between such a record and an audit trail with chosen event and verifier semantics is precisely the two definitions the contemplated sentence has not made.

The notice adopts none of the following, and none of it defines the notice’s phrase; it shows what chosen audit semantics look like elsewhere. NIST SP 800-53’s audit family frames the choices as explicit organization-defined assignments — AU-2 has the organization identify the “organization-defined event types that the system is capable of logging,” AU-12 assigns which components generate records, AU-9 requires protecting audit information from modification and deletion. The FBI’s own CJIS Security Policy (version 6.1, June 25, 2026), which governs systems handling criminal-justice information and is not referenced by this RFI, fills those assignments in: an enumerated event list down to attempts to access, modify, or destroy the audit log itself, record content requirements, and retention of audit records for “a minimum of one (1) year.” NIST’s AI Risk Management Framework — a voluntary framework — names the motivation at the training-data level: “Maintaining the provenance of training data … can assist with both transparency and accountability.” Selected audit semantics exist in the world, down to the retention figure; selecting is the part the contemplated sentence has not yet done.

4.3 The conditional handoff

The third problem exists to the extent data products are exchanged between domains — the condition is the notice’s own vocabulary of domains owning their data and data products exposed by domains, and this article adds no assertion about actual flows. When an exchange happens, whatever crosses carries the traceability or it does not: the consuming side’s records can only reference what arrived. Three things would need to travel — identity and version of the product, some binding to content, and whatever ownership or approval assertion the governance model wants downstream — and the survey question is which open, cross-system forms exist to carry them.

The specification landscape is younger here than in any previous article of this series, and the survey found real coverage with specific holes. The Linux-Foundation-stewarded Open Data Product Standard  (Bitol, v1.0.0) is the closest data-product interchange form: a YAML document with product identity, status, and ports whose inputs are pinned by data-contract id and version. Per its own field tables, it carries no approval or attestation fields, no content binding (no hash of data or schema), and no runtime records — declared design-time dependencies only. Its companion Open Data Contract Standard  (v3.1.0) versions the contract document — “Current version of the data contract.” — and defines no field for the version of the data served under it; a grep of its specification pages finds no lineage, provenance, audit, or approval fields at all. One documented regression marks the direction of travel: the Data Contract Specification (v1.2.1), the one contract format with a field-level lineage object, was deprecated by its authors in favor of ODCS — which carries none.

Vendor implementations document more than the specifications do, each inside its own boundary. AWS DataZone documents data products as first-class objects with API revisions that pin member-asset revisions, a subscription workflow whose requests are approved, rejected, or revoked, and lineage that the platform “versions … with each event” — with the durable audit copy landing in an operator-owned bucket via CloudTrail. Collibra documents a product lifecycle in which “Designated stakeholders must approve it” and contract manifests imported and exported in ODCS form — the approval being a platform workflow state that the interchanged document, per ODCS’s schema, cannot carry. Databricks documents domain-per-catalog data-mesh guidance with lineage aggregated centrally in the metastore — a documented configuration worth noting precisely because it cuts against any assumption that decentralized ownership implies decentralized record-keeping: a mesh can centralize its lineage collection, and a warehouse can distribute its owners. Starburst documents export and import of product definitions as YAML in its own schema. Across the four: approval semantics exist without interchange (DataZone, Collibra), interchange exists without approval (Bitol ODPS, Starburst), and no surveyed form carries identity, content binding, and approval together across a system boundary — a negative bounded by the documents listed in the sources.

5. The scale

The division whose data operations form this notice’s background publishes its own magnitudes, quoted here as context with their dates and no further use. NCIC holds more than 18 million active records across 22 files (FBI page, as of 2023); the CJIS Division’s Acting Assistant Director put its traffic at “an average of 10 million daily transactions” in December 2024 testimony. NICS processed 26,123,215 background checks in 2025 per the FBI’s own statistics — figures the same document says “do not represent the number of firearms sold.” The NGI fingerprint system’s July 2026 fact sheet lists a criminal repository of 88.4 million fingerprint composites and roughly 25,000 submitting partners that month. The UCR program received 2025 data from more than 17,000 agencies covering 96.2 percent of the population. These are the published magnitudes of the division’s existing systems, named in the notice as background; the notice does not connect them to the contemplated mesh, and neither does this article.

6. The procurement record

The record around the notice is thin and datable. The RFI’s background names an organizational anchor: the division established an agile Value Stream, the Data Services Value Stream, dedicated to this Data Modernization effort. The division’s own 2024 Year in Review records the same event — CJIS IT staff “established the Data Services Value Stream in support of the 2030 vision” — retrieved from an Internet Archive snapshot of June 23, 2025; the live file returned no document as of this check. It is the one public FBI document located that uses the term. The notice itself was posted once, amended never, and archived after 39 days; the section-1 search protocol found no related document since. Whether market research this old still feeds an active acquisition plan, the public record does not say — the RFI’s own introduction reserved exactly that possibility, to formulate acquisition strategies for competitive solicitations, and reserved is where the record leaves it.

7. Who would build it

The notice describes its intended respondents concretely: vendors of commercial off-the-shelf software deployable in government cloud environments — FBI/CJIS prioritizes COTS that reside in a Government Cloud — holding GSA Schedules, GWACs, or NASA SEWP vehicles, able to offer test licenses for up to 120 days in a non-operational environment, and able to describe licensing and subscription pricing for budget assessment. That is what the notice requests: commercially available platforms and software for the five categories of Section 2, deployable in a government cloud, with pricing and trial terms. Which vendors responded, and whether any acquisition strategy followed, is not visible in public records (Section 1). What the bounded survey adds: for the five categories, shipped products exist; for the object, event, and verifier of the model-development traceability sentence, the reviewed specifications and vendor documents define no defaults — those definitions are not among what the surveyed forms carry.

8. What does not exist as a standard

The per-element survey, led by what comes closest. CycloneDX  — the OWASP bill-of-materials standard, whose 1.5 release added a machine-learning BOM — covers the most at once: components typed as models and data with versions and content hashes, a model card structure whose modelParameters.datasets are “The datasets used to train and evaluate the model,” formulation workflows describing how a component came together, machine-readable data governance with owners, custodians, and stewards, signable documents, attestations with signatories, and cross-document references that let one BOM cite an element of another. A portable, signed container for declared model-and-data association exists — that much should be said plainly. What its schemas (1.6 and 1.7, inspected field by field) do not define: an immutable-snapshot semantic — the inline dataset form carries no version or hash fields at all; an observed-run identity — formulation is an author’s declaration, with no contract that its timestamps reflect an execution; an approval predicate — attestations assert conformance to named standards, and “owner approved dataset version V for training model M” has no defined form; and any retention rule for the records themselves.

No open specification was identified, as of August 25, 2026, that defines all six elements as interoperable record semantics across system boundaries; the survey list, with each specification’s version and its specific missing fields, is in the sources. The elements individually are in better shape than this series has found in any prior domain — MLMD’s schema answers “which training run led to this model” inside one store; DVC pins training bytes inside one tool; CycloneDX signs and ships the declared association. What the five axes of Section 3 need, and no surveyed form supplies together, is the connective set: a snapshot that stays what it was, a run that identifies itself, an approval that travels, and a record checkable by whichever verifier the fifth axis turns out to name — a choice the surveyed forms leave as open as the notice does.

A disclosure: I work on a specification in this area — verifiable provenance for derived outputs — and that work is separate from, and not linked in, this analysis.

The contemplated sentence is one line of a preliminary document, and the document says so itself. What the five choices inside it have in common is engineering weight: each one changes what the resulting records can answer, and the last of them — the verifier — changes whom the records are for. A future solicitation may make those choices, and this market research may inform it. The record, as it stands, closed with them open.

If you are building a response to this document and the provenance or attestation piece has to be designed and built, that is contract work I take on.