TCGCSV-First Architecture

Context and Problem Statement

The original SWU importer used the SWU API as the primary data source and TCGCSV as a secondary source to attach TCGPlayer product IDs. This SWU-API-first approach iterated over SWU API cards, then searched TCGCSV groups for matching products.

This architecture caused several classes of missed products:

1. Promo cards in different TCGCSV groups — Weekly Play Promos, Star Wars Celebration Promos, and Judge Promos exist in dedicated TCGCSV groups that have no corresponding SWU API expansion. The importer’s cross-group search hack (findTcgPlayerProduct) was fragile and incomplete.
2. Accent mismatches — SWU API card names may include accented characters (e.g., “Chewbacca” vs “Chewbacca”) that don’t match the TCGCSV product name, causing failed matches.
3. Products only on TCGPlayer — Some TCGPlayer products have no SWU API equivalent at all. The two-pass system (see superseded Pass 2 ADR) attempted to catch these, but added complexity and still produced lower-quality entries.

TCGPlayer is the marketplace. Every product on TCGPlayer must exist in our system for pricing, inventory tracking, and marketplace operations. If a product exists on TCGPlayer but not in our database, users cannot track or sell it.

Decision Drivers

• Every TCGPlayer product must have a corresponding product in our database with a guaranteed productId
• The SWU API is valuable for enrichment (translations, card attributes, art) but is not exhaustive
• The two-pass architecture added complexity without fully solving the coverage problem
• Name normalization (accent stripping, case folding) should work in one direction: normalize both sides and compare

Considered Options

1. TCGCSV-first: Let TCGCSV products drive the import, use SWU API only for enrichment
2. Improved SWU-API-first: Keep the SWU API as primary, fix the cross-group search and accent matching
3. Dual-primary merge: Iterate both sources independently and merge results

Decision Outcome

Chosen option: TCGCSV-first, because it guarantees 100% TCGPlayer coverage by construction and eliminates the need for pass 2, cross-group search, and reconciliation.

Import Flow

Key implementation details

• getCards() iterates all TCGCSV products in a single pass. No pass 2, no matchedTcgCsvProductIds tracking, no createCardsFromUnmatchedTcgCsv().
• getExpansions() is also TCGCSV-first. TCGCSV groups define what expansions exist, with SWU API data enriching expansion metadata (release dates, set codes).
• findApiCardMatch() replaces the old findTcgPlayerProduct(). Instead of searching TCGCSV for a match, it searches the SWU API enrichment index by normalized name and filters by variant type.
• normalizeCardName() strips accents via NFD decomposition, lowercases, and trims. Applied to both TCGCSV base names and SWU API names so comparisons are accent-insensitive.
• Products that skip enrichment (no API match) still get full TCGPlayer IDs, TCGCSV attributes (extRarity, extCardType, extNumber), and TCGPlayer images.

Impacted Files

Consequences

Positive

• 100% TCGPlayer coverage — every TCGCSV product gets a product record with a guaranteed TCGPlayer productId. No products are missed.
• Simpler code — 774 lines vs. the previous 1139 lines. Single-pass iteration, no reconciliation, no cross-group search hack.
• No pass 2 needed — supersedes the Pass 2 ADR. All products are handled in one pass.
• Accent-insensitive matching — normalizeCardName() with NFD decomposition handles accented characters transparently.
• Promo groups handled naturally — TCGCSV groups like “Weekly Play Promos” or “Star Wars Celebration” are iterated just like any other group. No special-case code.

Negative

• SWU API cards with no TCGCSV match are not imported — If a card exists in the SWU API but has no TCGPlayer product, it won’t appear in our system. This is acceptable: if it’s not on TCGPlayer, we don’t need it for marketplace operations. These cards may be future releases, regional exclusives, or data-only entries in the SWU API.
• Enrichment is best-effort — Products that don’t match any SWU API card get limited attributes (no translations, no card-specific metadata beyond what TCGCSV provides). In practice, most products match because the SWU API covers the vast majority of released cards.