The synthetic company: build me a fake retail chain

Ask a frontier model to invent a retail chain. It hands you a convincing one in 30 seconds. Product names that scan, customers with plausible complaints, a tidy table of Tuesday transactions. The demo is smooth enough that a reasonable engineer concludes the rest is prompt engineering. The measurements say otherwise. Asked to draw 1,000 samples from 15 named distributions, 10 of 11 frontier models failed every one of them under independent sampling. Batch prompting passed a median of 7% of statistical tests. Asked to discover the logic between columns, single-prompt baselines top out at F1 0.55 where the validated loop reaches 0.97. Asked for variety, aligned models collapse toward the most typical answer, a bias that traces back to the preference data itself. The model that writes a believable customer cannot roll the dice behind her.

This is the opener of a series of five long reads on building the synthetic company, a simulated business that produces data real enough to develop and test systems against. The running example throughout is a retail chain: stores, SKUs, registers, staff, customers, books. The architecture question has settled over the past 18 months, in my read of both the research and the market. What remains open is a sheet of engineering decisions. This article walks the architecture first, the decision sheet second.

The two layers, and who owns what

Every pipeline that works splits into two layers with opposite temperaments.

The deterministic layer, the scaffold, is everything that computes or refuses. Statistical samplers that draw counts and amounts from declared distributions. Schema types, primary keys, foreign keys. Accounting identities. Row validators, test suites, execution checks. Its defining property is that the same input always produces the same verdict. It can tell you no.

The semantic layer is everything that invents plausible content. Product names, complaint text, support tickets, personas, dialogue. It samples from a learned distribution, which makes it superb at texture and unreliable at arithmetic, at sampling, at anything that must hold across 10,000 rows at once. The dice study above says it plainly for engineers who need statistical guarantees. Current LLMs lack a functional internal sampler. The sampler has to live outside the model. An earlier Virginia Tech analysis traced the table-generation half of the problem to the architecture itself. Autoregressive generation struggles to hold functional dependencies and conditional mixtures of distributions.

The whole division of labor in one breath. The sampler decides that Tuesday had 214 transactions. The model writes what the customer in transaction 141 complained about.

The part most teams miss is that the scaffold is not exotic infrastructure. A SQL database refuses a text string in an integer column and refuses an order row pointing at a customer who does not exist. An accounting package in the QuickBooks class refuses a journal entry whose debits and credits disagree by a cent. Neither consults a probability before saying no. Your company already runs deterministic verifiers at production scale. Building the synthetic company mostly means promoting them from record-keepers to test oracles. The gap they cover is real: when Modern Treasury graded frontier models on generating charts of accounts and balanced ledger transactions across four scenarios in January 2026, one model passed everything. Their conclusion reads like this series’ thesis, correctness in the flow of funds still depends on systems that enforce invariants.

There is direct evidence the split beats either half alone. Amalgam pairs an LLM with probabilistic graphical models precisely because each fails where the other holds: PGMs capture distributions but choke on complex schemas, LLMs handle the schemas but skew the distributions. The hybrid hits a 91% average chi-square p-value with a realism score of 3.8 against 3.3 for the prior state of the art, with real data at 4.7.

18 months of consolidation agree

The market ran this experiment. Gretel, one of the category’s founding vendors, was reported acquired by NVIDIA in March 2025 at a price above its $320M valuation. Its technology resurfaced as NeMo Data Designer, open-sourced in late 2025 under Apache 2.0. Per the launch discussion and an ex-Gretel founder, the same framework built training datasets for the Nemotron model family. Fabricate, built by the creator of Mockaroo, the old workhorse of deterministic fake data, was acquired by Tonic in April 2025. The press release names the two halves outright, an AI-powered engine plus a rules engine. Mostly AI, one of the earliest and best-funded pure players, wound down in March 2026 after more than €30M raised, with the remains picked up by Syntho that June.

My read of that ledger: the survivors all sell the same shape, a deterministic frame with a model working inside it.

Data Designer’s column menu is the cleanest census of the division of labor I have seen. Samplers for Category, Gaussian, Bernoulli, Poisson, DateTime, UUID. Conditional parameters that shift a distribution based on another column’s value. Expression columns computed from other fields. Then, and only then, LLM text and code columns with the scaffold’s values injected into their prompts. After generation, Python, SQL, and remote validators, with LLM-as-judge scoring optional. Every stage of the architecture below ships in that one menu.

The architecture, stage by stage

One row, end to end, with illustrative values. The samplers assign transaction 4,417 to store 12 at 16:41 on a Thursday, then draw a 3-item basket and a $47.13 total from the declared distributions. The rule graph attaches a loyalty ID that provably exists in the customer table. The LLM column, handed all of that as context, writes one line of complaint text about the slow self-checkout. The grammar mask guarantees the output parses as a record. The SQL validator confirms the foreign keys resolve. A test asserts the total matches price times quantity within rounding. Only then does row 4,417 join the dataset. Multiply by a million and nothing about the loop changes except the bill.

Entry: spec or seed. Two ways in. Spec-first writes the world down, store counts, opening hours, category mix, margin rules, with no production data touched. Seed-grounded fits the scaffold from a real sample, which buys realism and imports a privacy obligation. Everything downstream is the same pipeline.

The scaffold sets structure. Samplers own every count, amount, and timestamp. Keys own identity, this order belongs to that customer at this store. Declared rules own the dependencies you already know, no refund without a matching sale, ship date after order date, the P&L consistent with the transaction log.

Rule discovery, for the rules nobody wrote down. A retail schema carries tribal logic that lives in nobody’s documentation. TabKG is the strongest verified recipe for extracting it. An ensemble of LLMs each proposes a graph of column relationships from the metadata. An edge survives only if a majority of models propose it. Type-specific validators then check every surviving edge against real data, functional dependencies for hierarchies, formula accuracy for mathematical links, ordering for temporal ones, and prune anything scoring under 0.90. Dependent columns are afterwards rebuilt by code, lookup tables, closed-form formulas, time offsets. Consistency holds by construction. This pipeline lifts relationship discovery from a single-prompt F1 of 0.27-0.55 to 0.897 on a 172,765-row retail table and 0.973 on a purchasing table, with one instructive wrinkle. The cross-model ensemble scored 0.604 on the harder retail set where repeated sampling of one model scored 0.897. Model diversity is a tool, off the shelf it is a coin flip.

Semantic fill. With structure frozen, LLM columns write the content, a product description here, a complaint referencing the delayed order there, always conditioned on scaffold values injected into the prompt. The conditioning runs in one direction only.

The decoding engine. Grammar-masked decoding makes format valid by construction. The mask precomputes which tokens can legally follow. A malformed record cannot be emitted at all. The XGrammar family now ships inside mainstream serving stacks. vLLM, for one, auto-selects a structured-output backend per request. Two verified caveats keep this layer humble. JSONSchemaBench, from the team behind Guidance, ran 10,000 real-world schemas through six enforcement engines and found declared schema support routinely exceeds what engines actually handle, which vLLM’s own docs corroborate by listing the JSON Schema keywords its backends skip. And AdapTrack, at ICSE 2026, showed greedy masking can satisfy the constraint while distorting what the model was trying to say, with a backtracking scheme that provably preserves the conditional distribution. A mask can force a legal date format. Nothing about the mask keeps the refund dated after the purchase.

The verifier gate. The strongest recipe here came out of code data, where deterministic oracles are cheap. KodCode accepts a synthetic coding sample only after its unit tests execute and pass. OmniSQL executed its 2.5M generated queries against 16,000+ synthetic databases before keeping them. The business translation is direct, run every generated batch through the checks the company already trusts, foreign keys, type constraints, the trial balance, plus purpose-written row validators. One calibration finding matters before you tighten everything to maximum. Cohere Labs measured two things. Richer test suites lift downstream performance by about 3 pass@1 while merely adding more tests plateaus. A rigid 100%-pass gate deletes useful diversity, with relaxed or soft thresholds recovering 2-4 points. Their conclusion is that the gate stays, with recalibrated thresholds.

The agent layer, strictly optional. Rows answer questions about records. Some questions are about behaviour, whether a promotion shifts baskets, how a rumour about a recall travels. That is when the semantic layer graduates from filling cells to playing customers, the memory-stream architecture this blog took apart in August. The transaction-level version is PersonaLedger: 24M real card transactions from 2,000 users compressed into 27 summary statistics per user, made differentially private with AIM at epsilon 1 to 10, then handed to an LLM that emits 12-field transaction records persona by persona. A fraud classifier trained on roughly 5,000 synthetic rows reached AUC 0.70 on real holdout data at epsilon 1, evidence the route works. The warning travelled further than the result. Generated activity clustered into 9am-to-2pm business hours and the personas skewed older and retired, which the authors attribute to LLM priors overriding the statistical distributions provided as input. One labelled caveat from me: their ground truth is itself a public synthetic population. Read the drift finding as a specification-adherence failure, which is the reading that matters for a scaffold anyway.

The decision sheet

Eight calls remain after the architecture is fixed. Defaults are mine, from the sources above plus the scars of building against generated data.

1. Platform or assembly. Default platform scaffold, Data Designer if you want OSS and code, Fabricate if you want relational databases described in English. Assemble custom only when compliance or schema depth forces it.
2. Spec-first or seed-grounded. Greenfield, spec-first. With production data and governance clearance, fit the scaffold from the seed. The quadrant sets the budget.
3. Who draws the numbers. Samplers, always. After the dice result, letting the model produce a count, an amount, or a timestamp is an unforced error.
4. Authored or discovered rules. Author the invariants you can name, accounting identities, key relationships. Run TabKG-style discovery with validation for the tribal logic you cannot.
5. Decoding engine. Masked decoding for format, after checking your engine’s actual coverage against the schema features you use. Where content is statistical and the schema tight, remember the mask can bend the distribution.
6. Hard or soft gates. Hard gates on invariants that must never break, balance, referential integrity. Calibrated thresholds on quality checks, per the Cohere finding.
7. Privacy stance. Spec-first sidesteps the issue. Seed-grounded wants privacy applied at the statistics layer, the PersonaLedger route, with the epsilon priced into the utility target.
8. The evaluation contract, written first. Schema validity, statistical distance, downstream utility trained-on-synthetic tested-on-real, and behavioural pattern preservation. The last one is the killer, this blog’s July benchmark review covered generators acing every fidelity metric while erasing the fraud patterns detection systems actually read. TabKG’s own utility numbers carry the same message, 98% dependency consistency and still a TSTR gap of 71.90 versus 88.17 AUC against real data. Consistency is the entry ticket. Utility still has to be earned on top.

The sheet looks bureaucratic. It is the cheapest part of the project. Every line on it is a failure I have either watched or paid for.

A workable first month. Week 1, schema and samplers for 5 core tables, stores, products, customers, orders, payments. Week 2, wire the verifiers you already trust, key checks, type checks, the balance identity, into automated gates. Week 3, add LLM columns under a schema mask and watch the reject rate. Week 4, write the evaluation contract and run the first train-on-synthetic test-on-real baseline. If week 3’s reject rate stays high after prompt fixes, the scaffold is missing a rule the model keeps guessing at. That signal alone pays for the month.

Where it still breaks

Four failure modes survive good architecture, all verified above. Priors fight the spec, the store whose every customer shops before 2pm will reappear in any persona-driven layer left ungated. The mask bends distributions silently, format-valid rows that no longer say what the model meant. The verifier ceiling means your data can only be as good as what the checker checks. A 100%-pass gate quietly deletes the tail your fraud team needed. And mode collapse, alignment’s typicality bias pulls 2,000 customers toward one median persona unless diversity is engineered back in, the same defect that made Smallville’s agents too agreeable to say no. All four are invisible in a demo. Month three finds them. Budget for the gate, in money and in tokens, the way this blog priced agent memory in June, where energy per correct answer swung 47x across implementations.

Rubin proposed this in 1993

The foundational source for this series is 8 pages long and 33 years old. In the Journal of Official Statistics, Donald Rubin proposed to “release no actual microdata but only synthetic microdata, constructed using multiple imputation so that they can be validly analyzed using standard statistical software.” He called the idea rather radical, but possibly stimulating. The proposal is a statistics problem end to end, a defined imputation procedure at the center, validity of standard downstream analysis as the acceptance test. Synthetic data began as a statistics problem with a deterministic procedure at its core. 33 years later the procedure is back, wrapped around a language model. The LLM wave swapped the imputation engine for a generator with world knowledge. That added the semantic layer Rubin never had. For one demo-drunk cycle it also forgot that the statistics still have to come from somewhere.

The bottom line

The synthetic company is buildable in 2026, at retail-chain scale, on parts you mostly do not have to write. The scaffold ships as open source or a platform. The decoding engine ships inside your serving stack. Some of the strongest verifiers are already running your business under the names SQL and trial balance. The model supplies the one thing the deterministic world never could, believable words attached to every record.

What you cannot rent is the gate. The verifier suite and the evaluation contract encode what your company means by valid. Every failure mode catalogued above is invisible until something deterministic tests for it. The rest of this series builds the layers one at a time: static tables on Codd’s constraints, event streams against process logic, agents behind the registers, then the validity contract that decides what ships.

The synthetic company splits cleanly in 2026. A deterministic scaffold owns every number and relationship. A language model owns every word inside that frame. A verifier gate decides what survives. Rent the scaffold and the engine. Own the gate and the evaluation contract behind it, because they are where valid gets defined.