Log In Sign Up

bitbox.swiss
robots.txt

Robots Exclusion Standard data for bitbox.swiss

Resource Scan

Scan Details

Site Domain bitbox.swiss
Base Domain bitbox.swiss
Scan Status Ok
Last Scan2025-07-15T11:49:59+00:00
Next Scan 2025-08-14T11:49:59+00:00

Last Scan

Scanned2025-07-15T11:49:59+00:00
URL https://bitbox.swiss/robots.txt
Domain IPs 104.26.0.40, 104.26.1.40, 172.67.68.208, 2606:4700:20::681a:128, 2606:4700:20::681a:28, 2606:4700:20::ac43:44d0
Response IP 104.26.1.40
Found Yes
Hash 16b729ed78028aa38b553e59f6dc9d9c9733612014f36166b0304ebb7f802559
SimHash 0d80bb82c93d

Groups

*

Rule Path
Disallow /blog/teaser-*

adsbot-google
adsbot-google-mobile
adsbot-google-mobile-apps
mediapartners-google

Rule Path
Disallow /blog/teaser-*

yandex

No rules defined. All paths allowed.

Back to top

Comments

  • This applies to all but google's ads bots, unfortunately:
  • https://developers.google.com/search/docs/advanced/robots/create-robots-txt
  • Duplicated entries from above specifically for google's ads bots.
  • https://developers.google.com/search/docs/advanced/crawling/overview-google-crawlers
  • Duplicate pages with GET parameters were found
Back to top

Warnings

  • `clean-param` is not a known field.
Back to top