Marketplace Data API: Architecture, Schema, and Use Cases

The engineering decision this guide supports

A marketplace data API is not simply an endpoint that returns products. Engineering teams must decide where source-specific collection ends, where normalization begins, and how consumers obtain repeatable snapshots without losing provenance. The target is an auditable data product that survives retries, late arrivals, source changes, and ambiguous product matches.

Architecture criteria before choosing a provider

• Request semantics: source, entity, locale, location, device, and observation time are explicit.
• Async at scale: bulk jobs have stable IDs, status, pagination, callbacks, and replay.
• Identity: source IDs remain separate from canonical product and offer IDs.
• History: observations are append-only; corrected records do not silently rewrite the past.
• Quality: validation, quarantine, lineage, and metrics are first-class outputs.
• Evolution: additive versus breaking changes and schema versions are defined.

A production marketplace data flow

scope registry -> scheduler -> source adapters -> raw immutable store
                                      |                 |
                                      v                 v
                              parse + validate -> quarantine
                                      |
                                      v
                         identity + normalization
                                      |
                       observation/event warehouse
                           |         |         |
                         API      exports    alerts

Raw storage is a debugging and replay boundary, not the consumer interface. The normalization layer should produce a consistent observation while retaining source payload references. Bright Data’s documentation is a concrete example of upstream modes: it distinguishes synchronous, asynchronous, and discovery requests, then supports API download, webhooks, cloud delivery, and streaming; see its official scraper overview. Apify similarly separates an Actor run from the dataset that stores structured results in its API guide.

Canonical schema: product, offer, observation, and provenance

{
  "schema_version": "1.0",
  "observation_id": "obs_01J...",
  "observed_at": "2026-08-19T12:00:00Z",
  "source": {"marketplace": "amazon", "country": "US"},
  "context": {"postal_code": "10001", "locale": "en-US", "device": "desktop"},
  "source_entity": {"id": "B0EXAMPLE", "url": "https://…", "page_type": "product"},
  "product": {
    "canonical_id": null,
    "title": "Example detergent, 2 x 1 L",
    "brand": "Acme",
    "identifiers": [{"type": "asin", "value": "B0EXAMPLE"}],
    "pack": {"count": 2, "unit_size": 1, "unit": "L"}
  },
  "offer": {
    "seller_id": "seller-42", "currency": "USD",
    "price": 18.99, "list_price": 21.99, "availability": "in_stock"
  },
  "placement": {"query": null, "organic_rank": null, "sponsored": null},
  "provenance": {"collector": "adapter-v23", "raw_ref": "s3://…", "request_id": "req_…"},
  "quality": {"state": "accepted", "warnings": ["canonical_match_pending"]}
}

Keep price, availability, seller, and placement in the observation—not on a timeless product row. A product may have many offers, and the same offer may vary by location. Oxylabs explicitly documents Amazon delivery location as a request parameter and warns that returned pages are often location-sensitive in its product reference. That context belongs in the record, not only in request logs.

Identifier strategy

API patterns for real-time and batch consumers

POST /v1/jobs
{"marketplace":"amazon","entity_ids":["B0EXAMPLE"],"locations":["10001"]}

202 {"job_id":"job_123","status":"queued"}
GET /v1/jobs/job_123
GET /v1/observations?marketplace=amazon&since=2026-08-19T00:00:00Z&cursor=…

Return 202 for work that cannot meet a bounded synchronous latency. Support idempotency keys, cursor pagination, partial failure detail, and a webhook signature. Zyte’s extraction reference demonstrates a synchronous single-URL contract with documented product types and request constraints; it is a useful reminder to design batch orchestration separately when the upstream endpoint blocks until completion. See the Zyte API reference.

Use cases and the fields they actually require

• Competitive pricing: product match, offer, seller, promotion, stock, location, and observation time. Start with pricing data.
• Share of shelf: query, organic rank, sponsored flag, result page, and location.
• Assortment gaps: category path, canonical product, variant, seller, first/last seen.
• Investment research: durable historical coverage, documented source changes, and reproducible aggregates for the investor workflow.

Begin with a defined source such as Amazon, then add a second marketplace only after the canonical record and match evaluation are stable.

Failure modes and POC validation

• A successful request contains a consent wall, empty shell, wrong locale, or wrong product variant.
• A parser maps list price into sale price without raising an error.
• Retries create duplicate observations or overwrite a valid earlier record.
• Matching merges different pack sizes; downstream unit economics become false but plausible.
• A source adds a field or changes type and breaks consumers.

The POC must include golden records, missing records, variants, multiple sellers and locations, injected retries, partial batch failure, and a schema-change exercise. Measure semantic accuracy separately from request success. Validate rate limits, late-data behavior, raw retention, replay, and legal constraints.

Request a marketplace API contract sample

Ask PLOTT DATA for a sample built around one marketplace, two locations, a fixed identifier list, and the canonical schema above. Request both accepted and rejected records, provenance fields, and a field-completeness report so engineering can test ingestion, idempotency, and quality gates before committing to production.