chrome-stats.com
robots.txt

Robots Exclusion Standard data for chrome-stats.com

Resource Scan

Scan Details

Site Domain chrome-stats.com
Base Domain chrome-stats.com
Scan Status Ok
Last Scan2024-09-15T17:50:59+00:00
Next Scan 2024-09-22T17:50:59+00:00

Last Scan

Scanned2024-09-15T17:50:59+00:00
URL https://chrome-stats.com/robots.txt
Domain IPs 104.26.6.201, 104.26.7.201, 172.67.70.71, 2606:4700:20::681a:6c9, 2606:4700:20::681a:7c9, 2606:4700:20::ac43:4647
Response IP 172.67.70.71
Found Yes
Hash b50c8d0403982cdf17a05329b373549464f7087d9ad830f5f6d198d3ea9f289f
SimHash dd1fd492de31

Groups

semrushbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

barkrowler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

ezoicbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

yandex

Rule Path
Disallow

*

Rule Path
Disallow /api/
Disallow /reviewer/

Other Records

Field Value
sitemap https://chrome-stats.com/sitemaps/sitemap.xml

Warnings

  • `clean-param` is not a known field.