CSV Importer

The CSV importer lets sellers and collectors bulk-add inventory (or bulk-list inventory for sale) from any spreadsheet. It is the largest piece of client-side business logic in the frontend — a three-step wizard that parses, maps, validates, fixes and submits anywhere from a single row to 500 000 rows without leaving the browser.

This page is the developer reference: architecture, data flow, atom graph, error model, performance budget and extension points.

For the user-facing guide, see Help Center → Selling → Importing your inventory from a CSV.

Where it lives

apps/frontend/app/[locale]/(fullscreen)/csv-importer/
├── page.tsx                 # Next.js entry, resolves importId/listId/gameSlug/listing
├── csv-importer-ui.tsx      # Providers + JotaiQueryProvider + AddProductsProvider
├── types.ts                 # CardNexusColumn enum, ColumnMappingState, ErrorGroup, …
├── _atoms/
│   ├── ui.ts                # Step state, csvContent, parsed rows, mapping state
│   ├── validation.ts        # Backend validation status / pagination
│   ├── error-groups.ts      # Derived ErrorGroup[] used by the Check Results step
│   ├── mutations.ts         # importInventory mutation + analytics
│   └── effects.ts           # Auto-advance / reset side effects
├── components/
│   ├── upload-csv.tsx       # Step 1
│   ├── map-columns*/        # Step 2
│   └── check-results/       # Step 3
├── hooks/
│   ├── use-incremental-validation.ts  # Chunked backend validation
│   ├── use-check-results.ts           # Submit + KPI orchestration
│   └── use-map-columns-logic.ts       # Auto-detect + per-column logic
└── utils/
    ├── column-detection.ts            # Regex-based header auto-match
    ├── condition-matcher.ts           # "NM" → CardCondition.NearMint, etc.
    ├── language-matcher.ts            # "en", "english", "anglais" → "English"
    ├── finish-matcher.ts              # "foil", "rev holo" → CardFinish
    ├── price-parser.ts                # "12,50 €" → 12.5 (or cents in listing mode)
    ├── quantity.ts                    # Strict integer parsing
    ├── entry-transforms.ts            # Row → ValidateInventoryInputDTO entry
    ├── apply-suggestion.ts            # Mutates a ResultRow with a fix
    └── deduplication.ts               # Merge / keep-first / keep-last

The backend side lives in packages/backend/features/import-export/src/csv-import.service.ts and is exposed via the oRPC handler packages/backend/api/src/lib/orpc/handlers/inventory/validate-input.handler.ts.

Mental model

The importer is a purely client-side parser + a stateless validator. The frontend owns the entire row lifecycle (parse → map → validate → fix → submit). The backend only ever sees normalized entries via inventory.validateInventoryInput (read-only) and inventory.importInventory (write).

┌──────────────┐    ┌───────────────┐    ┌──────────────────┐    ┌──────────────────┐
│  Upload CSV  │ ─► │  Map columns  │ ─► │  Check results   │ ─► │  importInventory │
│  papaparse   │    │  + value map  │    │  validate batch  │    │  (mutation)      │
└──────────────┘    └───────────────┘    └──────────────────┘    └──────────────────┘
   Step 1               Step 2                  Step 3                  Submit

Each step exposes a submitAction via submitActionAtom that the global footer (CSVImporterStepFooter) invokes when the user clicks the primary CTA. Steps never call each other directly; they communicate through atoms.

Step 1 — Upload CSV (components/upload-csv.tsx)

Papaparse is used in worker + chunked mode so the main thread stays responsive on huge files:

parse<string[]>(file, {
  worker: true,
  skipEmptyLines: "greedy",
  chunk: (results, parser) => { /* abort early if fatal */ },
  complete: () => resolve({ columns, rows }),
})

Hard limits enforced in this step:

The five terminal parse errors (EMPTY_FILE, EMPTY_HEADERS, NO_DATA_ROWS, TOO_MANY_ROWS, PARSE_FAILED) are surfaced as toasts via csvImporter.uploadCSV.errors.* i18n keys. Anything else (RFC 4180 minor errors like ragged columns) is tolerated — rows are kept and surface as validation errors later.

On success, the parsed CSVContent is written to csvContentAtom, parse wall-clock time is stored in parseTimeMsAtom for the File parsed in {time} KPI, and the wizard advances to Map Columns.

CSV template download

downloadCsvTemplate(game) builds a per-game template from GameConfiguration.csvTemplateExample. Headers are the 9 stable CardNexus columns; rows are one per supported vendor ID type. The body is prefixed with a BOM () so Excel opens it as UTF-8 without prompting.

Step 2 — Map Columns (components/map-columns/)

The user maps each CardNexusColumn to one of:

• a CSV column index (regular mapping)
• DEFAULT_OPTION_VALUE (-1) + a overrides[column] string → apply that value to every row
• SKIP_OPTION_VALUE (-2) → omit the column entirely

State lives in a single atom:

columnMappingStateAtom: {
  selections: { CardName: 0, Expansion: 2, Quantity: 5, ... },
  overrides:  { Finish: "Non-foil", Language: "English" },
}

This atom is read/written through two atom families (columnSelectionAtomFamily, additionalColumnAtomFamily) so each MappingCard re-renders independently.

Auto-detection

utils/column-detection.ts matches CSV headers against the regexes in types.ts’s columnMatches map. The matcher is greedy on synonyms ("qty", "copies", "in stock" → Quantity) but stops at the first hit per CardNexus column. Users can always override.

Identifier rules — the most important invariant

The importer needs at least one identifier to match a CSV row to a CardNexus product:

This priority is computed in identifierStatusAtom (in _atoms/ui.ts). The “Validate & continue” CTA is gated on anyIdentifierMapped === true.

Value mappings

For columns like Condition / Language / Finish, the matcher utilities (condition-matcher.ts, language-matcher.ts, finish-matcher.ts) auto-resolve common variants ("NM" → NearMint, "foil" → Foil). Values they can’t resolve are surfaced to the user, who can pick the canonical CardNexus value once — and the choice is stored in valueMappingsAtom (Map<column, Map<rawValue, canonicalValue>>) and applied to every row that contained that raw value.

Listing mode (csvListingModeAtom)

When the importer is opened from the inventory’s “Sell via CSV” CTA, listing is true:

• A Price column becomes required.
• Prices are parsed by price-parser.ts (handles 12,50 €, $12.50, 12.5) and converted to integer cents.
• Each submitted entry carries a listing: { price, currency } block where currency is the seller’s profile currency.

The same wizard renders in either mode — only this atom + a couple of conditional fields change.

Step 3 — Check Results (components/check-results/)

Parse phase (client-only)

Once columns are mapped, parsedRowsAtom derives ValidResult[] + a ParseSkippedRow[] of rows that couldn’t even be turned into an entry (no_identifier, invalid_quantity, or missing_price in listing mode). The result is memoized on a signature of the mapping state so editing one default value doesn’t re-parse 500 000 rows. There’s also an parseAllRowsAsyncAtom that yields between 5 000-row chunks for the progress UI.

Validate phase (use-incremental-validation.ts)

Parsed entries are sent to inventory.validateInventoryInput in chunks with an async pool. Each chunk:

• looks up products by vendorId, then printNumber, then name (priority cascade in csv-import.service.ts)
• returns each entry tagged either valid (with productId) or invalid with a ValidateInventoryInputError code

Validation runs incrementally as users fix errors. Fixed rows are re-validated in a new chunk; everything else stays cached. There’s no full re-run.

The error code → group → suggestion pipeline

There are 7 backend error codes (packages/shared/primitives/src/inventory.ts):

errorGroupsAtom (in _atoms/error-groups.ts) buckets invalid rows by (code, invalidValue) so duplicates collapse into a single “Invalid expansion: ‘Alfa’” entry the user can fix once. Group priority is defined in GROUP_PRIORITY and drives the display order.

Fixing strategies

Each error group exposes one or more actions in error-group.tsx:

• Accept suggestion — apply-suggestion.ts mutates the row in place, drops the error, and re-queues it for validation.
• Fix all rows — same as above but for every row in the bucket.
• Merge duplicates — utils/deduplication.ts runs one of three strategies (merge, keepFirst, keepLast) and rewrites the row set.
• Edit manually — opens a quick-search dialog (use-quick-search-dialog.ts) for the row.
• Exclude from import — moves the row to excludedRowsAtom; it counts in the “Excluded” KPI and is never re-validated.

Fixed rows are tracked in fixedRowOriginalsAtom so users can undo a fix (see “X rows fixed — undo my fix” footer).

KPIs (kpi-grid.tsx)

Four tiles, all derived atoms:

1. Total rows — csvContent.rows.length
2. Valid & ready — results.validRows.length (fixed and skipped surfaced as footer text)
3. Errors remaining — sum over errorGroupsAtom where count > 0
4. Excluded — excludedRowsAtom.size

Submitting (importInventoryMutationAtom)

hooks/use-check-results.ts exposes onSubmit. It:

1. Confirms via a dialog summarising rows imported / excluded / skipped.
2. Calls inventory.importInventory with valid entries only (excluded rows and unparseable rows never reach the backend).
3. Emits ampli.csvImported(...) analytics.
4. Routes to the success screen (ImportSuccess) or the inventory page.

The submit is all-or-nothing per call but the call is non-atomic on the backend — partial inserts are possible if Mongo blips. The service returns per-row results so the success screen reflects what actually landed.

Performance budget

The big perf milestones in this branch (chunked async parse, virtualized error groups, lazy-mounted error groups) all land in commits prefixed perf(csv-importer).

Adding a new game

The importer is data-driven. To make it work for a new game:

1. Register the game in packages/integrations/games/game-configuration/src/lib/games/<game>/<game>.game.ts, including:
   • vendorIds (each entry becomes an option in the VendorIdType dropdown)
   • attributes.printNumber, attributes.variant (drives column visibility)
   • csvTemplateExample (drives the downloadable template)
2. No frontend change is needed if the matchers (condition / language / finish) already cover the game’s wire vocabulary. Most games inherit the defaults.
3. Add fixtures under apps/frontend/app/[locale]/(fullscreen)/csv-importer/__tests__/game-scenarios.spec.ts for the happy path.

Adding a new validation error

1. Add the code to ValidateInventoryInputError in packages/shared/primitives/src/inventory.ts.
2. Have csv-import.service.ts set error: <new code> on failing entries and (optionally) attach a suggestion payload to the row.
3. Map the code to a label/help string in apps/frontend/app/[locale]/(fullscreen)/csv-importer/_atoms/error-groups.ts (ERROR_LABEL_KEYS, GROUP_PRIORITY, canBulkFor, mergeSuggestionsFromRow).
4. Add i18n keys under csvImporter.checkResults.errorGroup.label.<code> and csvImporter.checkResults.errorGroup.help.<code>.
5. Add a unit test scenario in __tests__/game-scenarios.spec.ts.

Test coverage

Run frontend tests with pnpm vitest run inside apps/frontend. The whole suite is ~250 tests and finishes in ~1.5 s locally.

Common gotchas