celestial-seas.com
robots.txt
Robots Exclusion Standard data for celestial-seas.com
Resource Scan
Scan Details
| Site Domain | celestial-seas.com |
| Base Domain | celestial-seas.com |
| Scan Status | Ok |
| Last Scan | 2025-10-27T13:07:02+00:00 |
| Next Scan | 2025-11-26T13:07:02+00:00 |
Last Scan
| Scanned | 2025-10-27T13:07:02+00:00 |
| URL | https://celestial-seas.com/robots.txt |
| Domain IPs | 104.21.90.145, 172.67.157.194, 2606:4700:3030::ac43:9dc2, 2606:4700:3035::6815:5a91 |
| Response IP | 104.21.90.145 |
| Found | Yes |
| Hash | f0221b058b7d891837ccd7bfad0bd003610705ecea9eaf01e390fec77790f6f9 |
| SimHash | 7015d111a290 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /user/ |
| Disallow | /masterlist/ |
| Disallow | /sublist/ |
| Disallow | /character/ |
| Disallow | /myo/ |
| Disallow | /gallery/ |
| Disallow | /submissions/ |
| Disallow | /claims/ |
| Disallow | /reports/ |
| Disallow | /trades/ |
| Disallow | /comment/ |
| Disallow | /images/ |
| Disallow | /files/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
amazonbot
anthropic-ai
applebot-extended
bytespider
ccbot
chatgpt-user
claudebot
claude-web
cohere-ai
diffbot
facebookbot
facebookexternalhit
friendlycrawler
google-extended
gptbot
icc-crawler
imagesiftbot
img2dataset
meta-externalagent
oai-searchbot
omgili
omgilibot
perplexitybot
petalbot
scrapy
timpibot
velenpublicwebcrawler
youbot
semrushbot
| Rule | Path |
|---|---|
| Disallow | / |