phillwebb.net
robots.txt

Robots Exclusion Standard data for phillwebb.net

Resource Scan

Scan Details

Site Domain phillwebb.net
Base Domain phillwebb.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-06-10T16:47:15+00:00
Next Scan 2025-09-08T16:47:15+00:00

Last Successful Scan

Scanned2023-10-27T01:57:59+00:00
URL https://phillwebb.net/robots.txt
Domain IPs 104.21.7.209, 172.67.188.6, 2606:4700:3033::6815:7d1, 2606:4700:3035::ac43:bc06
Response IP 172.67.188.6
Found Yes
Hash 8feeb8af73bd08e119f86177339a64f1f48284fa669c49e4a8e053ec410b28f8
SimHash 4c50c7235517

Groups

*

Rule Path
Allow /

googlebot
googlebot-image
mediapartners-google
googlebot-news
google-amphtml
googlebot-mobile
bingbot
duckduckbot
yandexbot
yandex
twitterbot
slurp
msnbot
bingpreview
applebot

Rule Path
Allow /

googlebot

Rule Path
Allow *.js
Allow *.css
Allow *.woff
Allow *.ttf
Allow *.jpg
Allow *.png
Allow *.webp
Allow *.avif
Allow *.svg
Allow *.gif
Allow *.woff2