Overview
Collections allow you to schedule Requests to run en masse. You can add up to 15,000 Requests (a Request is simply a call to the Product Data API) per Collection (100 per Collection if using the include_html=true request parameter) and you can have up to 10,000 Collections on your account at any one time.
Collections can be scheduled to run monthly, weekly, daily, hourly or run manually on-demand and you can get Result Sets in JSON, JSON Lines or CSV formats. Rainforest API can also call a webhook on your side when Collections complete.
Collection & Result Set Expiry Collections will be deleted from your account after 2 months of inactivity - i.e. you must start a Collection at least once every two months for it to remain active.
Result Sets are available for download for 14 days so your app should download and process the Result Set data within this 14 day window.
The Rainforest API Collections API allows you to create, update and delete Collections. You can also use the Collections API to add Requests to a Collection and retrieve Result Sets.
The Collections API is Free of Charge There is no charge for usage of the Collections API and requests to it will not decrement your plans monthly requests quota. When you start a Collection the requests within the Collection will be charged as per requests made via the Product Data API. So a Collection with 200 requests in it, run once, will cost 200 credits.
The Rainforest API Dashboard also provides a visual editor for Collections and is a great way to get familiar with the key concepts in Collections before creating them programmatically through the Collections API.
To help you get started here's some of the terminology you'll encounter whilst interacting with Collections on Rainforest API:
You can have 10,000 Collections active on your account at any one time. A Collection holds (up to 15,000) Requests. A Collection can have a schedule and can be set to start automatically or manually. When a Collection is started, its Requests are run through the Rainforest API Product Data API and a Result Set is generated. Collections can notify you when a new Result Set is available via email or webhook POST. Rainforest API will automatically delete Collections that have not been run for 2 months. .
If you start multiple Collections they are queued and processed sequentially. To control the order of processing you can assign each Collection a priority . Higher priority Collections are started before lower priority ones. Learn more about priorities.
A Collection can contain up to 15,000 Requests (100 when adding requests with include_html=true ). A Request is simply a collection of parameters relating to an individual request to the Product Data API.
When Collections complete (that is, when all of their Requests have been evaluated) a Result Set is generated. You can use the Collections API to retrieve Result Sets and ingest them into your app. Rainforest API retains Result Sets for 14 days so you should download your Result Set data within this 14 day window.
Rainforest API provides API's to retrieve Result Sets in JSON and CSV formats.