Log In Sign Up

seomoz.org
robots.txt

Robots Exclusion Standard data for seomoz.org

Resource Scan

Scan Details

Site Domain seomoz.org
Base Domain seomoz.org
Scan Status Ok
Last Scan2024-10-04T19:11:01+00:00
Next Scan 2024-10-11T19:11:01+00:00

Last Scan

Scanned2024-10-04T19:11:01+00:00
URL https://seomoz.org/robots.txt
Redirect https://moz.com/robots.txt
Redirect Domain moz.com
Redirect Base moz.com
Domain IPs 104.21.51.253, 172.67.192.133, 2606:4700:3031::6815:33fd, 2606:4700:3031::ac43:c085
Redirect IPs 104.16.127.24, 104.16.64.25, 2606:4700::6810:4019, 2606:4700::6810:7f18
Response IP 104.16.127.24
Found Yes
Hash b215a0655cb7144559c1cda21c80477eed02bfcde64592865d8696853a5bca47
SimHash 61497c427630

Groups

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

gptbot

Rule Path
Disallow /blog/
Disallow /learn/seo/

*

Rule Path
Allow /researchtools/ose/$
Allow /researchtools/ose/dotbot$
Allow /researchtools/ose/links$
Allow /researchtools/ose/just-discovered$
Allow /researchtools/ose/pages$
Allow /researchtools/ose/domains$
Allow /researchtools/ose/anchors$
Allow /products/
Allow /local/
Allow /learn/
Allow /researchtools/ose/
Allow /researchtools/ose/dotbot$
Disallow /products/content/
Disallow /local/enterprise/confirm
Disallow /researchtools/ose/
Disallow /page-strength/*
Disallow /thumbs/*
Disallow /api/user?*
Disallow /checkout/*
Disallow /local/search/
Disallow /local/details/
Disallow /messages/
Disallow /content/audit/*
Disallow /content/search/*
Disallow /marketplace/
Disallow /cpresources/
Disallow /vendor/
Disallow /community/q/questions/*/view_counts
Disallow /admin-preview/*
Disallow /community/q/socket.io/*
Back to top

Other Records

Field Value
sitemap https://moz.com/sitemaps-1-sitemap.xml
sitemap https://moz.com/community/q/sitemap.xml
Back to top

Comments

  • File last updated July 4, 2024
Back to top