proboards57.com
robots.txt

Robots Exclusion Standard data for proboards57.com

Resource Scan

Scan Details

Site Domain proboards57.com
Base Domain proboards57.com
Scan Status Ok
Last Scan2024-05-17T18:00:29+00:00
Next Scan 2024-05-24T18:00:29+00:00

Last Scan

Scanned2024-05-17T18:00:29+00:00
URL https://proboards57.com/robots.txt
Domain IPs 2a02:4780:38:8102:9dd8:836a:e3d1:ad2f, 91.108.100.119
Response IP 77.37.48.117
Found Yes
Hash bd20e23828af163706c1a63748556a02aa0020386df0d73fca7fa2d8b0f81e19
SimHash 4c205b50e0a4

Groups

mediapartners-google

Rule Path
Disallow

grapeshot

Rule Path
Disallow

*

Rule Path
Disallow /search
Disallow /comments/

mediapartners-google*

Rule Path
Disallow

anthropic-ai
awariorssbot
awariosmartbot
bytespider
ccbot
chatgpt-user
claudebot
claude-web
cohere-ai
dataforseobot
facebookbot
friendlycrawler
google-extended
googleother
gptbot
img2dataset
imagesiftbot
magpie-crawler
meltwater
omgili
omgilibot
perplexitybot
piplbot
seekr
truliabot
youbot

Rule Path
Disallow /