Seobility Bot

The Seobility Bot crawls sites to gather SEO-Information.

Blocking the bot completely

In order to stop requests from the Seobility Bot add the following Line to your Robots.txt:

User-agent: Seobility
Disallow: /

If you see the bot misbehaving severly, please contact us.

General vs. specific directives

The Seobility Bot will follow all * directives as long as no Seobility directive is provided in the file. For Example:

User-agent: * 
Disallow: /secret.html

secret.html will not be crawled

User-agent: * 
Disallow: /secret.html

User-agent: Seobility
Disallow: /login.html

secret.html will be crawled, login.html will not be crawled.

Only allow Seobility to crawl your site

If you want to grant access only to Seobility and deny all other bots then you can use the following robots.txt entry. This technique can be used to test SEO changes on a subdomain (e.g. http://test.example.com/) before releasing the changes.

User-agent: *
Disallow: /

User-agent: Seobility
Disallow: