firefox-stats.com
robots.txt

Robots Exclusion Standard data for firefox-stats.com

Resource Scan

Scan Details

Site Domain firefox-stats.com
Base Domain firefox-stats.com
Scan Status Ok
Last Scan2025-10-02T19:48:15+00:00
Next Scan 2025-10-09T19:48:15+00:00

Last Scan

Scanned2025-10-02T19:48:15+00:00
URL https://firefox-stats.com/robots.txt
Redirect https://chrome-stats.com/robots.txt
Redirect Domain chrome-stats.com
Redirect Base chrome-stats.com
Domain IPs 104.21.1.110, 172.67.129.45, 2606:4700:3032::6815:16e, 2606:4700:3034::ac43:812d
Redirect IPs 104.26.6.201, 104.26.7.201, 172.67.70.71, 2606:4700:20::681a:6c9, 2606:4700:20::681a:7c9, 2606:4700:20::ac43:4647
Response IP 172.67.70.71
Found Yes
Hash a7cf0213c707ba3cd56cf8b3f95e5312d59547a710724d0ce042bc51b74d5df3
SimHash b7174572ac31

Groups

velenpublicwebcrawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

applebot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

yandex
*

Rule Path
Disallow /api/
Disallow /images/
Disallow /assets/
Disallow /icons/
Disallow /res/
Disallow /reviewer/
Disallow /a/
Disallow /docs/embed?id=
Disallow /pricing?c=

Other Records

Field Value
sitemap https://chrome-stats.com/sitemaps/sitemap.xml

Warnings

  • `clean-param` is not a known field.