Product Enrichment
Turns a distributor part number into a sellable product page — description, grouped specifications, image gallery and manufacturer PDFs — sourced from Icecat and then served from Solidlio's own storage.
How Solidlio helps
Solidlio holds a local copy of Icecat’s product index — around 26 million entries — and matches your catalog against it on part number, alternate part number, GTIN and brand. Matched products are enriched from Icecat’s Level-4 API: the marketing description, the full specification set grouped the way the manufacturer groups it, every gallery image, and every manual, datasheet and safety sheet. The images and PDFs are then copied out of Icecat and into Solidlio’s own object storage, so a quote sent today still renders in two years. Products Icecat will not release under your subscription are recorded rather than dropped, and become a prioritized CSV you can send Icecat to have them opened up.
Capabilities
| Capability | What it does |
|---|---|
| Local product index | Streams Icecat’s full index — ~26M entries — to disk, then parses it in constant memory and upserts in 1,000-row transactions. |
| Five-strategy matching | GTIN first, then brand-scoped MPN, alternate MPN and a punctuation-normalized MPN, then brand-name lookup, then unscoped MPN at 6+ characters, then substitute part. |
| Level-4 enrichment | One call returns descriptions, specifications, feature groups, images and documents; all of it is written to queryable tables in one pass. |
| Grouped specifications | Features are attached to Icecat’s own feature groups, so a spec sheet renders as Processor / Memory / Connectivity rather than one flat list. |
| Manufacturer documents | Manuals, datasheets, safety data sheets, energy labels, warranty documents and quick-start guides, typed and language-tagged. |
| Media brought in-house | Every image and PDF is fetched behind an SSRF guard, one-way hash deduplicated across products, and stored in a private container. |
| CDN delivery | Stored media is served from an anonymous, immutably cacheable URL — one year, ETag, Range — so storefront pages and emailed quotes load without a token. |
| Staff-preview enrichment | An administrator opening an unenriched product on the storefront enriches it before the page renders, cached for every buyer after. Buyers never trigger a paid call. |
| Subscription gap tracking | Every refused lookup is logged with its part number and brand, prioritized in a review queue, and exported as a ready-to-send Icecat access request. |
| Reference data sync | Seven Icecat master lists — suppliers, categories, features, measures, feature groups, languages and category-feature mappings — synced in dependency order. |
| Serialized job queue | Index, match, enrich and media jobs run one at a time with live progress; a duplicate request returns a conflict instead of a second run. |
How it works
-
Sync the reference data — Suppliers, categories, features and measures come down as Icecat’s master lists. Everything downstream references them.
-
Build the index — The full product file streams to disk, then through gunzip and a SAX parser into the database. Progress reports 0–49% for the download and 50–100% for the parse.
-
Match — Every catalog product without an Icecat link is tried against five strategies in confidence order. A match records the Icecat ID, the data quality —
ICECATfor editorially standardized,SUPPLIERfor as-supplied — and marks the product pending enrichment. -
Enrich — Matched products are fetched from Icecat’s Level-4 API at five requests per second. Each outcome is recorded as enriched, forbidden, not found or errored, so a re-run never repeats work that already succeeded.
-
Bring the media home — Images and PDFs download ten products at a time, deduplicate on content hash, and land in private storage. The product’s URLs switch from Icecat’s host to Solidlio’s the moment the bytes arrive.
-
Work the refusals — Products outside the subscription are already in the access queue. Set a priority, export the CSV, send it to Icecat, re-enrich when access lands.
Operations and audit
- Every enrichment call is logged. Part number, brand, GTIN, Icecat ID, HTTP status, error message, source and timestamp — granted and refused alike. Refusals become the subscription worklist; a
404is recorded distinctly from a403, because “does not exist” and “not in your plan” are different problems. - Raw responses are retained. The complete Icecat payload is stored per product per language alongside content-availability flags, so what was received can be inspected after the fact.
- Every scheduled run is recorded. The daily index sync and weekly reference sync each write a run-history row — running, then completed, failed or timed out — and a run that fails every retry notifies platform administrators.
- Media is never public at the storage layer. The container is private; the anonymous serve route is the only way in, and it will only release keys carrying a
catalogpath segment. - Outbound fetches are constrained. Media downloads reject non-HTTP schemes and every private, loopback, link-local and carrier-grade-NAT address range.
Integrations
Icecat — Full Icecat (level 4). Two channels: the OCI file repositories at data.icecat.biz for the bulk index and reference XML, and the live per-product JSON API at live.icecat.biz for on-demand detail.
Plans and access
Availability depends on your plan, role and account permissions. Capacity limits and service terms apply. Compare current plans and integration requirements, or ask us about your setup.
A part number is not a product. This is the difference.