llanj.org
robots.txt

Robots Exclusion Standard data for llanj.org

Resource Scan

Scan Details

Site Domain llanj.org
Base Domain llanj.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-08T09:25:31+00:00
Next Scan 2025-12-07T09:25:31+00:00

Last Successful Scan

Scanned2025-07-17T23:19:29+00:00
URL https://llanj.org/robots.txt
Domain IPs 65.49.60.171
Response IP 65.49.60.171
Found Yes
Hash 6a79d9ee9f3ed7ee362df10af0eb4cf2bcc381dba1b1f4ecedae9dfa865f9082
SimHash 630d0802c113

Groups

*

Rule Path
Allow /

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 18

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

extecontextcrawl

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claudebot/1.0

Rule Path
Disallow /

meta-externalagent/1.1

Rule Path
Disallow /

gptbot/1.2

Rule Path
Disallow /

amazonbot/0.1

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

dotbot/1.2

Rule Path
Disallow /

dataforseobot/1.0

Rule Path
Disallow /

Other Records

Field Value
sitemap https://llanj.org/wiki/sitemap_index.xml
sitemap https://llanj.org/wiki/sitemap_index.xml