MarketcheckBot
Version: 2.0
Bot Type: Good (Identifies itself)
Category: Analytics
Obeys Robots.txt: Yes
Obeys Crawl Delay: Yes
User-Agent String: MarketcheckBot/2.0 (https://marketcheck.com/MarketcheckBot)
IP Address: 35.208.195.36
Reverse DNS suffix: bot.marketcheck.com
What is MarketcheckBot?
MarketcheckBot is Marketcheck’s web crawling bot that crawls and extracts millions of live adverts from thousands of websites for the industries listed on our website. The adverts collected are distributed in the industry to various stakeholders across the value chain for use cases like search, syndication and analytics. MarketcheckBot helps the host website (the advert publisher) take the adverts deep into the industry and increase the advert footprint and publisher’s visibility without the publisher having to build the distribution tier themselves. To date, MarketcheckBot has enabled dozens of web & mobile apps and inventory syndication partners to feature these adverts in their applications or partner directories that drive traffic back to live advert pages on the publisher’s website.
How to Whitelist MarketcheckBot
You can whitelist MarketcheckBot using any one or a combination of the following methods:
1. By User-Agent String
Allow requests matching the following User-Agent in your WAF, CDN, or bot management platform (e.g. Cloudflare, Akamai, Imperva):
MarketcheckBot/2.0 (https://marketcheck.com/MarketcheckBot)
Note: If you had previously allowlisted the older version, also ensure the legacy User-Agent is still permitted:
Autobot (https://marketcheck.com/)
2. By Single IP Address (Primary)
If your firewall or platform supports allowlisting only one IP, use the primary egress IP:
| Type | IP Address |
|---|---|
| Primary IP | 35.208.195.36 |
3. By Multiple IP Addresses (All Egress IPs)
For complete coverage, allowlist all of MarketcheckBot’s fixed egress IPs:
| # | IP Address |
|---|---|
| 1 | 35.209.28.127 |
| 2 | 35.208.69.97 |
| 3 | 35.209.228.167 |
| 4 | 35.208.126.38 |
| 5 | 35.208.83.130 |
| 6 | 35.208.223.13 |
| 7 | 35.209.64.221 |
| 8 | 35.208.201.253 |
| 9 | 35.209.244.59 |
| 10 | 35.35.209.145.85 |
4. Via robots.txt
To explicitly permit MarketcheckBot on your site, add the following to your robots.txt:
User-agent: MarketcheckBot
Allow: /
Crawl-delay: 3
How MarketcheckBot accesses your site
Unlike other web spiders, MarketcheckBot is a focussed crawler that only crawls a certain set of web pages from the target website on a daily basis. MarketcheckBot uses an algorithmic process to fetch only the modified set of web pages from the target website to keep bandwidth usage at a minimum. For most sites, MarketcheckBot shouldn’t access your site more than once every few seconds on average, with a minimum delay of at least 3 seconds between successive web hits. MarketcheckBot also adheres to the Crawl-Delay directive in the robots.txt rules configured on the target website.
Blocking MarketcheckBot from accessing content on your website
If you wish to block MarketcheckBot from accessing all or partial content on your website, you could block it by placing Disallow directive in your robots.txt as follows – To keep it off a particular directory or path on your website –
User-agent: MarketcheckBot (https://marketcheck.com/MarketcheckBot) Disallow: //
To keep it off from your entire website –
User-agent: MarketcheckBot (https://marketcheck.com/MarketcheckBot) Disallow: /
Reporting observations/issues with MarketcheckBot
MarketcheckBot respects the directives in robots.txt, but if you still do spot instances where it’s not playing by the rules you can always reach out to us for getting it to behave. Please report spam/observations/suspicious activity to business@marketcheck.com We take bad behaviour by our bot very seriously and are very prompt in resolving any reported issues on MarketcheckBot behaviour.
