Log In Sign Up

arthritis.ca
robots.txt

Robots Exclusion Standard data for arthritis.ca

Resource Scan

Scan Details

Site Domain arthritis.ca
Base Domain arthritis.ca
Scan Status Ok
Last Scan2025-04-19T11:40:55+00:00
Next Scan 2025-05-19T11:40:55+00:00

Last Scan

Scanned2025-04-19T11:40:55+00:00
URL https://arthritis.ca/robots.txt
Domain IPs 52.233.38.143
Response IP 52.233.38.143
Found Yes
Hash d6634a96de271df54e0e9b700c5d38d8be4d38bb949e81b370981a056cade33f
SimHash 7e1d9142eba1

Groups

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

*

No rules defined. All paths allowed.

Back to top

Other Records

Field Value
sitemap https://www.arthritis.ca/googlesitemap.xml
Back to top

Comments

  • SemrushBot is allowed again as per user request task 18333808
  • User-agent: SemrushBot
  • Disallow: /
  • User-agent: SemrushBot-SA
  • Disallow: /
Back to top