Fixtures → ExMachina Migration — Program Design

Date: 2026-06-07 Status: In progress. Step 1 shipped (PR #4464). This is a multi-PR program. Goal: Replace the YAML-dump test-fixture system (test/db_fixtures/) with ExMachina factories, so tests build their own data per-test, run async, and the brittle yaml2csv/COPY toolchain can be deleted.

Backed by analysis workflow wismsg66l (5 investigators + adversarial scope verification). This is finishing a strangler, not a greenfield rewrite — the factory end-state already exists and is proven (EuropaWeb.LiveViewCase: 65 files, 59 async, zero YAML dependence).

Decisions (agreed)

  • End-state: factory-drive the transactional core (~80%); keep a small, documented committed seed for long-tail config tables (theme_files ~2472 rows, shipping_zone_territories ~1689, sync_products, themes) rather than over-investing in factories for high-row, low-churn data. Delete the yaml2csv/convert.py/load_fixtures toolchain regardless.
  • Async is the goal: migrated core tests end up async: true; the CQRS/Oban cohort that can’t see the sandbox stays shared-mode.

Scope reality (verified)

  • 75 YAML files → committed Postgres rows (via bin/convert_db_fixtures.sh → load_fixtures.exs, ~375 statements) and an in-memory :persistent_term map (data_case.ex:44-60), exposed to 142 DataCase modules.
  • ~119 files bind the fixtures: map; 135 of 142 DataCase files also use SQLCase (sandbox-isolated but read the committed baseline).
  • Mongo JSON snapshots (32 files) are in scope — they denormalise the same platform/store UUIDs; 9 tests depend on the SQL↔Mongo UUID match.
  • Implicit global dependency: one seeded musicglue/iron-maiden platform+store, referenced literally in 39 test files (SQL + Mongo + hardcoded UUIDs).
  • ⚠️ grep undercounts coupling — tests read fixtures by raw UUID and handle, not just the documented map key (proven: customer_support_providers has 0 map refs but a test hardcodes its fixture UUID). Per-table retirement must use empirical detection, not grep.

Decomposition (each PR green on its own)

  1. ✅ Step 1 (PR #4464): decouple the 5 DataCase-without-sandbox files → ExUnit.Case, async: true. They’re pure/stubbed unit tests that never touched the DB; no factories needed. Retires zero YAML.
  2. Detection harness: a CI mode that empties (TRUNCATE, skip COPY) a target table and runs the suite to discover real consumers. Unblocks safe per-table retirement.
  3. Fill factory gaps + product_graph builder: add missing factories (customer via the KMS/UpsertCustomer path, stockable + a stock-tally helper) and a composable platform→store→vendor→product→variant→shippable→stockable→shipping_profile builder.
  4. Per-table retirement, smallest-fanout-first: convert consumers to factories, then delete the YAML + db_fixtures_mapping.json entries. Order by verified fanout: customer_support_providers/store_credits/voucher_codes/tickets → customer_groups → domains → purchases → consignments → fulfillers → vendors → products/variants/stores/platforms (entangled core) last.
  5. Mongo snapshots + cross-system UUID tests: lockstep with the SQL core (regenerate from factory graphs, or pin the shared UUIDs as named constants during transition).
  6. Decommission toolchain: delete the fixtures map from data_case.ex, load_fixtures.exs, bin/convert_db_fixtures.sh, yaml2csv-*, convert.py, and the test.setup alias steps (mix.exs:201-207).

Using the detection harness (step 2, PR pending)

FIXTURE_SKIP_TABLES (comma-separated table names) makes load_fixtures.exs skip loading those tables’ rows and data_case.ex omit them from the in-memory fixtures map — emptying both consumption paths so a full run surfaces every real consumer (not just the grep-visible map users). No-op when unset.

Per-table retirement loop:

  1. Detect: FIXTURE_SKIP_TABLES=<table> <run the full suite via dockerstack>. Every failure is a real consumer of that table’s committed rows (including the grep-invisible raw-UUID/handle readers).
  2. Cross-check the map path: grep -rn '<table>' test for fixtures[...] / handle / UUID references the run might not have exercised.
  3. Migrate each consumer to build its data via factories inline.
  4. Confirm: re-run with the table skipped → green.
  5. Retire: delete test/db_fixtures/<table>.yaml and its db_fixtures_mapping.json entries.

Skip multiple tables at once to retire a connected cluster together (e.g. FIXTURE_SKIP_TABLES=products,variants,shippables).

Open questions for later phases

  • Mongo snapshot strategy (regenerate-from-factories vs. pin shared UUIDs as constants).
  • Cross-system UUID strategy for the 9 SQL+Mongo tests + hardcoded-UUID files (named constants vs. full randomization + assertion rewrites).
  • Whether the amd64-only yaml2csv binary is painful enough on arm64 dev machines to bring toolchain deletion forward.

Key files

test/db_fixtures/ (75 YAML), test/db_fixtures_mapping.json, test/support/load_fixtures.exs, test/support/data_case.ex:34-60, test/support/sql_case.ex, test/support/live_view_case.ex (the proven factory end-state), test/support/factory.ex + test/factories/ (20 modules), test/support/shovel/snapshot_case.ex, bin/convert_db_fixtures.sh, mix.exs:201-207.

This site uses Just the Docs, a documentation theme for Jekyll.