Log In Sign Up

fbsearch.ru
robots.txt

Robots Exclusion Standard data for fbsearch.ru

Resource Scan

Scan Details

Site Domain fbsearch.ru
Base Domain fbsearch.ru
Scan Status Ok
Last Scan2026-01-07T11:41:00+00:00
Next Scan 2026-01-14T11:41:00+00:00

Last Scan

Scanned2026-01-07T11:41:00+00:00
URL https://fbsearch.ru/robots.txt
Domain IPs 91.227.16.11
Response IP 91.227.16.11
Found Yes
Hash f608818776e484f34f33e1e66c29c2d60852313db0e1554e435486d1136e6e66
SimHash 20001dc38b73

Groups

*

Rule Path
Disallow /logs/
Disallow /data/
Disallow /opds.php
Disallow /cache.php
Disallow /query.php
Disallow /opds/
Disallow /old.php
Disallow /fail.php
Disallow /search.php

Other Records

Field Value
crawl-delay 5

mediapartners-google

Rule Path
Allow /search.php

ahrefsbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

petalbot

Rule Path
Disallow /
Back to top

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
  • Those bots do not contribute anything for the community and should not be
  • allowed to index anything on the net and waste resources.
  • Chinese bot
Back to top

Warnings

  • `host` is not a known field.
Back to top