The Seobility Bot crawls sites to gather SEO-Information.
Blocking the bot completely
In order to stop requests from the Seobility Bot add the following Line to your Robots.txt:
User-agent: SeobilityBot (SEO Tool; https://www.seobility.net/sites/bot.html) Disallow: /
If you see the bot misbehaving severly, please contact us.
General vs. specific directives
The Seobility Bot will follow all * directives as long as no Seobility directive is provided in the file. For Example:
User-agent: * Disallow: /secret.html
secret.html will not be crawled
User-agent: * Disallow: /secret.html User-agent: Seobility Disallow: /login.html
secret.html will be crawled, login.html will not be crawled.
Only allow Seobility to crawl your site
If you want to grant access only to Seobility and deny all other bots then you can use the following robots.txt entry. This technique can be used to test SEO changes on a subdomain (e.g. http://test.example.com/) before releasing the changes.
User-agent: * Disallow: / User-agent: Seobility Disallow: