Affiliate product feeds or APIs: which integration should you build?
Affiliate networks usually offer two ways to get product data: downloadable feed files and programmatic APIs. Both can work; the right choice depends on what the network actually authorises for your account, how fresh the data must be and how much you are willing to maintain. These notes come from integrating three networks into Cumpără cu AI.
Start with what is authorised
The first question is not technical. Read the network’s documentation and terms for your account type, and ask the network when in doubt. An endpoint that happens to respond is not necessarily a supported or permitted one. For one network we read catalogues exclusively from its official feed files, and a guard in code refuses any other address on that network’s hosts, checked again on every redirect, so a configuration mistake cannot send a forbidden request.
Lesson: build each source behind an adapter, so you can switch from an API to a feed (or back) without touching the rest of the application.
How the two approaches compare
- Freshness. APIs can be queried on demand; feeds are snapshots, refreshed on the network’s schedule. For a nightly catalogue, a feed is usually enough.
- Speed. One large file can be much faster than thousands of paged requests: for one shop, our import went from about 40 minutes through an API to about 3 minutes from the file.
- Completeness. Feeds sometimes carry fields an API omits (or the reverse): one network’s export gives real stock, extra pictures and barcodes.
- Formats. XML and CSV variants differ per network and per shop: escaping, category separators, empty brands.
- Caching. Some networks cache a format for hours; another format of the same feed may be built on every request.
- Limits. APIs have rate limits and tokens to rotate; feeds have size and download-time limits.
Design for imperfect data
Whatever the source, plan for files that change shape. One shop’s file changed overnight: brands became “Black Friday”, barcodes became department names and category separators stopped being escaped. Without guards, more than 130,000 products would have been recreated under new identities. We now treat a barcode as real only if it is 8 to 14 digits, keep a known listing on its product when its title is unchanged, and restored the affected products from that morning’s backup the same evening.
Other useful guards: a growth limit that holds back an unusual surge of new products until someone approves it, a minimum share of a shop’s recent offers before a partial file replaces the old data, and freshness rules that hide offers no source has confirmed for a few days.
A practical recommendation
For a catalogue refreshed daily, prefer the network’s official feed files where they exist, read them once per run, split them by programme and validate them before use. Use APIs for what files cannot give you, when the network documents and permits it. Either way, log what each run did and alert on anything unusual.