shop.flixbus.it
robots.txt

Robots Exclusion Standard data for shop.flixbus.it

Resource Scan

Scan Details

Site Domain shop.flixbus.it
Base Domain flixbus.it
Scan Status Ok
Last Scan2024-09-19T08:00:39+00:00
Next Scan 2024-10-03T08:00:39+00:00

Last Scan

Scanned2024-09-19T08:00:39+00:00
URL https://shop.flixbus.it/robots.txt
Domain IPs 13.226.2.123, 13.226.2.27, 13.226.2.29, 13.226.2.74
Response IP 13.226.2.123
Found Yes
Hash a7b0c8159404667d2c0f9c8fa2e37f766d6d4b7ddb94d58d4861423a0ef669fe
SimHash 4004c4428233

Groups

bytespider

Rule Path
Disallow /

*

Rule Path
Allow /$
Disallow /

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
  • Explicitly target tiktok bots