Log In Sign Up

tobtr.com
robots.txt

Robots Exclusion Standard data for tobtr.com

Resource Scan

Scan Details

Site Domain tobtr.com
Base Domain tobtr.com
Scan Status Ok
Last Scan2024-11-10T22:32:16+00:00
Next Scan 2024-11-17T22:32:16+00:00

Last Scan

Scanned2024-11-10T22:32:16+00:00
URL http://tobtr.com/robots.txt
Domain IPs 3.211.2.89, 3.215.198.242, 35.172.89.242
Response IP 3.211.2.89
Found Yes
Hash ab1aee0eb0bda2da21d2ea0ed7c9c3e9b4985583ecb4f3d74f292e9d2b69e13d
SimHash 535c5023fd35

Groups

twitterbot

Rule Path
Allow /

googlebot

Rule Path
Disallow /
Allow /content/*.css
Allow /scripts/dist/*.js
Allow /content/fonts/*.woff
Allow /content/fonts/*.ttf
Allow /content/fonts/*.eot
Allow *.gif
Allow *.png
Allow *.jpeg
Allow *.jpg
Allow *.svg

googlebot-images

Rule Path
Disallow /
Allow *.gif
Allow *.png
Allow *.jpeg
Allow *.jpg
Allow *.svg

*

Rule Path
Disallow /
Back to top