SiteSeer crawls its customer's websites with the following user-agent:
Mozilla/5.0 (compatible; SiteSeerBot/1.0; +https://siteseer.io/bot/)
You may customize the user agent we use within your Site Settings.
- SiteSeer crawls your site once a day, starting at 0:00 UTC.
- SiteSeer does not crawl external pages (pages that do not belong to our customers).
- SiteSeer's IP address is not static, and will be part of Cloudflare's IP ranges.
- SiteSeer honors your site's robots.txt. If you would like to block certain subpaths from being crawled, add them to your robots.txt.
- SiteSeer will slow down its request rate when your site begins to return 429 status codes, down to 1 request / 10 seconds. We retry these errors up to 10 times before giving up.
If you are a site administrator and believe SiteSeer's crawlers are abusing your site, please contact us.
Previously "zyppybot"
SiteSeer customers who signed up prior to October 2022 will continue to be crawled using our previous user-agent: zyppybot.
WPEngine Customers
Websites hosted on WPEngine's Advanced Network do not allow SiteSeerBot by default. You will need to file a support ticket via my.wpengine.com asking for SiteSeerBot to be whitelisted in their Advanced Network configuration. Access Rules created from the WPEngine dashboard will have no effect because SiteSeerBot is being blocked within their Advanced Network product, before reaching your WPEngine instance.