artofproblemsolving.com
robots.txt

Robots Exclusion Standard data for artofproblemsolving.com

Resource Scan

Scan Details

Site Domain artofproblemsolving.com
Base Domain artofproblemsolving.com
Scan Status Ok
Last Scan2025-03-12T13:36:05+00:00
Next Scan 2025-04-11T13:36:05+00:00

Last Scan

Scanned2025-03-12T13:36:05+00:00
URL https://artofproblemsolving.com/robots.txt
Domain IPs 104.26.10.229, 104.26.11.229, 172.67.69.208, 2606:4700:20::681a:ae5, 2606:4700:20::681a:be5, 2606:4700:20::ac43:45d0
Response IP 104.26.11.229
Found Yes
Hash 64a2d8494d5f783ef2e80d847832dbde113fe8d6e1189df56a7418c3c33c2c4d
SimHash ee4de431ca8b

Groups

*

Rule Path
Disallow /private/
Disallow /m/maintenance/
Disallow /maintenance/
Disallow /m/oops/
Disallow /oops/
Disallow /*ajax
Disallow /m/store/*-information
Disallow /m/store/book-
Disallow /m/store/checkout
Disallow /m/store/class-license
Disallow /m/store/finalize
Disallow /m/store/review
Disallow /m/store/addtocart
Disallow /m/store/clear
Disallow /m/store/coppa-enrollment
Disallow /m/store/dispatcher
Disallow /m/store/coupon
Disallow /m/store/loading
Disallow /m/store/paypal
Disallow /store/*-information
Disallow /store/book-
Disallow /store/checkout
Disallow /store/class-license
Disallow /store/finalize
Disallow /store/review
Disallow /store/addtocart
Disallow /store/clear
Disallow /store/coppa-enrollment
Disallow /store/dispatcher
Disallow /store/coupon
Disallow /store/loading
Disallow /store/paypal

baiduspider

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

ahrefsbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://artofproblemsolving.com/aops-sitemap-index.xml

Comments

  • using SEM internally
  • User-agent: SemrushBot
  • Disallow: /