asrclkrec.com
robots.txt

Robots Exclusion Standard data for asrclkrec.com

Resource Scan

Scan Details

Site Domain asrclkrec.com
Base Domain asrclkrec.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-23T06:09:14+00:00
Next Scan 2025-11-30T06:09:14+00:00

Last Successful Scan

Scanned2025-10-23T02:22:37+00:00
URL https://asrclkrec.com/robots.txt
Redirect https://www.rivcoacr.org/robots.txt
Redirect Domain www.rivcoacr.org
Redirect Base rivcoacr.org
Domain IPs 18.161.111.105, 18.161.111.112, 18.161.111.13, 18.161.111.60
Redirect IPs 18.66.218.100, 18.66.218.37, 18.66.218.85, 18.66.218.88
Response IP 65.9.112.96
Found Yes
Hash 3da87ef911552211dc76cc035d7ae1ea3fd05be57b33cfc4107438fc930c03d9
SimHash 61905941c2a7

Groups

*

Rule Path
Disallow /calendar
Disallow /meetings
Disallow /meetingdashboard
Disallow /media
Disallow /portal
Disallow /311
Disallow /newsletters

Other Records

Field Value
crawl-delay 5

anthropic-ai

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

seekr

Rule Path
Disallow /

youbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

weborama

Rule Path
Disallow /

garlik

Rule Path
Disallow /

hypefactors

Rule Path
Disallow /

seekport

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.rivcoacr.org/sitemap/

Comments

  • AI-Related Bots
  • General Crawlers and Indexing Bots