SiteSeer crawls it's customer's websites with the user-agent zyppybot by default. However, you may customize the user agent we use within your Site Settings.

  • SiteSeer crawls your site once a day, starting at 0:00 UTC.
  • SiteSeer does not crawl external pages (pages that do not belong to our customers).
  • SiteSeer's IP address is not static, and will be part of Cloudflare's IP ranges.
  • SiteSeer honors your site's robots.txt. If you would like to block certain subpaths from being crawled, add them to your robots.txt.
  • SiteSeer will slow down its request rate when your site begins to return 429 status codes, down to 1 request / 10 seconds. We retry these errors up to 10 times before giving up.

If you are a site administrator and believe SiteSeer's crawlers are abusing your site, please contact us.

WPEngine Customers

Websites hosted on WPEngine's Advanced Network do not allow zyppybot by default. You will need to file a support ticket via my.wpengine.com asking for zyppybot to be whitelisted in their Advanced Network configuration. Access Rules created from the WPEngine dashboard will have no effect because zyppybot is being blocked within their Advanced Network product, before reaching your WPEngine instance.