shiplocation.com
robots.txt

Robots Exclusion Standard data for shiplocation.com

Resource Scan

Scan Details

Site Domain shiplocation.com
Base Domain shiplocation.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-06-18T23:13:32+00:00
Next Scan 2025-06-19T23:13:32+00:00

Last Successful Scan

Scanned2025-06-11T23:13:23+00:00
URL https://shiplocation.com/robots.txt
Redirect https://www.shiplocation.com/robots.txt
Redirect Domain www.shiplocation.com
Redirect Base shiplocation.com
Domain IPs 104.26.8.57, 104.26.9.57, 172.67.72.8, 2606:4700:20::681a:839, 2606:4700:20::681a:939, 2606:4700:20::ac43:4808
Redirect IPs 104.26.8.57, 104.26.9.57, 172.67.72.8, 2606:4700:20::681a:839, 2606:4700:20::681a:939, 2606:4700:20::ac43:4808
Response IP 104.26.9.57
Found Yes
Hash 8dcc8c2332fd9663da6b5c01b72bb6329e103ccf143d741f6765436290c236c3
SimHash 6858d940e314

Groups

*

Rule Path
Allow

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

seekr

Rule Path
Disallow /

Comments

  • Block Open AI
  • Block Google (Gemini)
  • Block Claude
  • Block CommonCrawl
  • Block Diffbot
  • Block Meta (Facebook)
  • Block ByteDance
  • Block Webz.io
  • Block ImagesiftBot
  • Block Meltwater
  • Block Seekr