ISLA-bot
ISLA-bot/1.0 is the product indexer for ISLA, a UK price comparison service operated by Indexa Technologies Ltd (company number 17581242).
What it indexes
ISLA-bot indexes publicly listed product prices in GBP and stock availability, so that UK consumers can compare where to buy. It reads public product pages only, using the structured product data a page already publishes.
How it crawls
- User agent:
ISLA-bot/1.0 (+https://isla.indexatechnologies.com/bot) - Respects robots.txt and Crawl-delay directives.
- Low request rates: at most one request per second to a site, and slower where a Crawl-delay is set.
- Visits a product page only when a consumer looks for that product, at most once every six hours, using conditional requests.
Verifying ISLA-bot
Every request is signed with Web Bot Auth (HTTP Message Signatures). Our public key directory is at /.well-known/http-message-signatures-directory.
Controlling access
ISLA-bot follows the robots.txt rules for its user agent token. To stop it indexing your site:
User-agent: ISLA-bot Disallow: /
Feed partnerships
We welcome direct product feeds from UK merchants. To discuss a feed partnership, or anything about ISLA-bot, contact hello@indexatechnologies.com.