paulwesterberg.net
robots.txt

Robots Exclusion Standard data for paulwesterberg.net

Resource Scan

Scan Details

Site Domain paulwesterberg.net
Base Domain paulwesterberg.net
Scan Status Ok
Last Scan2025-11-06T18:36:41+00:00
Next Scan 2025-11-13T18:36:41+00:00

Last Scan

Scanned2025-11-06T18:36:41+00:00
URL https://paulwesterberg.net/robots.txt
Domain IPs 69.174.52.66
Response IP 69.174.52.66
Found Yes
Hash b15105bd88c13eee9e05949a870a1f355d2ebc3f9b9f89d03c73814962d73053
SimHash 8a30506a8036

Groups

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /