moz.com
robots.txt

Robots Exclusion Standard data for moz.com

Resource Scan

Scan Details

Site Domain moz.com
Base Domain moz.com
Scan Status Ok
Last Scan2024-06-29T04:19:06+00:00
Next Scan 2024-07-06T04:19:06+00:00

Last Scan

Scanned2024-06-29T04:19:06+00:00
URL https://moz.com/robots.txt
Domain IPs 104.16.127.24, 104.16.64.25, 2606:4700::6810:4019, 2606:4700::6810:7f18
Response IP 104.16.127.24
Found Yes
Hash 569f87f68971c1a14f93bfdc547c0fc6689e3c90167d5bf0b0e0cfa353c69895
SimHash 6849bc62e631

Groups

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

gptbot

Rule Path
Disallow /blog/
Disallow /learn/seo/

*

Rule Path
Allow /researchtools/ose/$
Allow /researchtools/ose/dotbot$
Allow /researchtools/ose/links$
Allow /researchtools/ose/just-discovered$
Allow /researchtools/ose/pages$
Allow /researchtools/ose/domains$
Allow /researchtools/ose/anchors$
Allow /products/
Allow /local/
Allow /learn/
Allow /researchtools/ose/
Allow /researchtools/ose/dotbot$
Disallow /products/content/
Disallow /local/enterprise/confirm
Disallow /researchtools/ose/
Disallow /page-strength/*
Disallow /thumbs/*
Disallow /api/user?*
Disallow /checkout/*
Disallow /local/search/
Disallow /local/details/
Disallow /messages/
Disallow /content/audit/*
Disallow /content/search/*
Disallow /marketplace/
Disallow /cpresources/
Disallow /vendor/
Disallow /community/q/questions/*/view_counts
Disallow /admin-preview/*

Other Records

Field Value
sitemap https://moz.com/sitemaps-1-sitemap.xml
sitemap https://moz.com/community/q/sitemap.xml