dungeon-coursers.com
robots.txt
Robots Exclusion Standard data for dungeon-coursers.com
Resource Scan
Scan Details
| Site Domain | dungeon-coursers.com |
| Base Domain | dungeon-coursers.com |
| Scan Status | Ok |
| Last Scan | 2025-08-27T02:35:54+00:00 |
| Next Scan | 2025-09-26T02:35:54+00:00 |
Last Scan
| Scanned | 2025-08-27T02:35:54+00:00 |
| URL | https://dungeon-coursers.com/robots.txt |
| Domain IPs | 104.21.53.166, 172.67.215.169, 2606:4700:3031::ac43:d7a9, 2606:4700:3032::6815:35a6 |
| Response IP | 172.67.215.169 |
| Found | Yes |
| Hash | c0800ca44b20bf79ca3b59b2b70fdcd49c368493a67eafe77cf60119d0a96fc5 |
| SimHash | 7011d101a290 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /user/ |
| Disallow | /masterlist/ |
| Disallow | /sublist/ |
| Disallow | /character/ |
| Disallow | /myo/ |
| Disallow | /gallery/ |
| Disallow | /submissions/ |
| Disallow | /claims/ |
| Disallow | /reports/ |
| Disallow | /trades/ |
| Disallow | /comment/ |
| Disallow | /images/ |
| Disallow | /files/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
amazonbot
anthropic-ai
applebot-extended
bytespider
bingbot
ccbot
chatgpt-user
claudebot
claude-web
cohere-ai
diffbot
mj12bot
duckduckbot
facebookbot
facebookexternalhit
facebookexternalhit
friendlycrawler
gptbot
icc-crawler
imagesiftbot
img2dataset
meta-externalagent
oai-searchbot
omgili
omgilibot
perplexitybot
petalbot
scrapy
slurp
timpibot
velenpublicwebcrawler
youbot
semrushbot
yandex
| Rule | Path |
|---|---|
| Disallow | / |