mythologie.ca
robots.txt

Robots Exclusion Standard data for mythologie.ca

Resource Scan

Scan Details

Site Domain mythologie.ca
Base Domain mythologie.ca
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-05-12T04:34:12+00:00
Next Scan 2025-06-11T04:34:12+00:00

Last Successful Scan

Scanned2025-04-13T04:13:24+00:00
URL https://mythologie.ca/robots.txt
Redirect https://www.mythologie.ca/robots.txt
Redirect Domain www.mythologie.ca
Redirect Base mythologie.ca
Domain IPs 192.99.199.128
Redirect IPs 192.99.199.128
Response IP 192.99.199.128
Found Yes
Hash df826a956ecfcecc87a8ff29a19487a2eb6ac58931a68f3093791cbeda4f9030
SimHash 7a1ed920aa13

Groups

*

Rule Path
Disallow /stuff/

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

httrack

Rule Path
Disallow /

httrack

Rule Path
Disallow /

wget

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /