shop.flixbus.de
robots.txt

Robots Exclusion Standard data for shop.flixbus.de

Resource Scan

Scan Details

Site Domain shop.flixbus.de
Base Domain flixbus.de
Scan Status Ok
Last Scan2024-06-07T04:34:16+00:00
Next Scan 2024-06-21T04:34:16+00:00

Last Scan

Scanned2024-06-07T04:34:16+00:00
URL https://shop.flixbus.de/robots.txt
Domain IPs 18.239.199.41, 18.239.199.45, 18.239.199.51, 18.239.199.61
Response IP 18.165.171.129
Found Yes
Hash a7b0c8159404667d2c0f9c8fa2e37f766d6d4b7ddb94d58d4861423a0ef669fe
SimHash 4004c4428233

Groups

bytespider

Rule Path
Disallow /

*

Rule Path
Allow /$
Disallow /

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
  • Explicitly target tiktok bots