sea.museum
robots.txt

Robots Exclusion Standard data for sea.museum

Resource Scan

Scan Details

Site Domain sea.museum
Base Domain sea.museum
Scan Status Ok
Last Scan2024-09-01T15:46:12+00:00
Next Scan 2024-10-01T15:46:12+00:00

Last Scan

Scanned2024-09-01T15:46:12+00:00
URL https://sea.museum/robots.txt
Redirect https://www.sea.museum/robots.txt
Redirect Domain www.sea.museum
Redirect Base sea.museum
Domain IPs 52.84.229.26, 52.84.229.30, 52.84.229.79, 52.84.229.8
Redirect IPs 13.237.224.152, 13.238.2.197
Response IP 13.238.2.197
Found Yes
Hash 88205978a52a0157a08affbd077cac291010ff7c7d0761981856e2a371823c75
SimHash d0d0d964a1b1

Groups

*

Rule Path
Disallow /explore/blog

*

Rule Path
Disallow /explore/blog/

*

Rule Path
Disallow /Explore/Blog

*

Rule Path
Disallow /blog

*

Rule Path
Disallow /Blog

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

Warnings

  • 2 invalid lines.