aurorabaycare.com
robots.txt

Robots Exclusion Standard data for aurorabaycare.com

Resource Scan

Scan Details

Site Domain aurorabaycare.com
Base Domain aurorabaycare.com
Scan Status Ok
Last Scan2024-05-03T14:23:03+00:00
Next Scan 2024-06-02T14:23:03+00:00

Last Scan

Scanned2024-05-03T14:23:03+00:00
URL https://aurorabaycare.com/robots.txt
Redirect https://www.aurorabaycare.com/robots.txt
Redirect Domain www.aurorabaycare.com
Redirect Base aurorabaycare.com
Domain IPs 3.219.78.25, 3.220.212.11, 3.220.214.71
Redirect IPs 13.226.120.10, 13.226.120.103, 13.226.120.109, 13.226.120.15, 2600:9000:2755:3800:4:2473:fa80:93a1, 2600:9000:2755:5400:4:2473:fa80:93a1, 2600:9000:2755:7000:4:2473:fa80:93a1, 2600:9000:2755:a800:4:2473:fa80:93a1, 2600:9000:2755:b000:4:2473:fa80:93a1, 2600:9000:2755:be00:4:2473:fa80:93a1, 2600:9000:2755:cc00:4:2473:fa80:93a1, 2600:9000:2755:ea00:4:2473:fa80:93a1
Response IP 108.156.133.50
Found Yes
Hash 6361f17550ca478e8a5b884700641c1908fa2e2b10b684240e6e9cb4dba3f87c
SimHash 4e1dd951d6b1

Groups

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

yandex

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

crownpeaksearchg2crawler

Rule Path
Allow /

*

Rule Path
Allow /
Disallow /_master
Disallow /_site-configuration
Disallow /_folder-configuration
Disallow /news/_in-the-news/
Disallow /_assets/documents/misc

googlebot

Rule Path
Allow /
Disallow /_master
Disallow /_site-configuration
Disallow /_folder-configuration
Disallow /news/_in-the-news/
Disallow /_assets/documents/misc

bingbot

Rule Path
Allow /
Disallow /_master
Disallow /_site-configuration
Disallow /_folder-configuration
Disallow /news/_in-the-news/
Disallow /_assets/documents/misc

twitterbot

Rule Path
Allow /
Disallow /news/_in-the-news/
Disallow /_assets/documents/misc

Other Records

Field Value
sitemap https://www.aurorabaycare.com/sitemap-index.xml

Comments

  • Core: Disallow
  • Core: Allow, Delay
  • Core: Allow
  • Custom

Warnings

  • 2 invalid lines.