topteachingtasksmembers.com
robots.txt

Robots Exclusion Standard data for topteachingtasksmembers.com

Resource Scan

Scan Details

Site Domain topteachingtasksmembers.com
Base Domain topteachingtasksmembers.com
Scan Status Ok
Last Scan2025-09-05T14:14:56+00:00
Next Scan 2025-10-05T14:14:56+00:00

Last Scan

Scanned2025-09-05T14:14:56+00:00
URL https://topteachingtasksmembers.com/robots.txt
Domain IPs 104.21.11.151, 172.67.166.93, 2606:4700:3030::ac43:a65d, 2606:4700:3035::6815:b97
Response IP 172.67.166.93
Found Yes
Hash cb5ad4cfe6f03b75f515eb3f8c086533799611b5fa280a9e00a12b87fae8515b
SimHash 8819585327d2

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /cart/
Disallow /wishlist/
Disallow /checkout/
Disallow /my-account/
Disallow /*add-to-cart%3D*
Disallow /*?filter
Disallow /*?orderby=*
Disallow /*?add-to-wishlist=*

*

Rule Path
Disallow /search/
Disallow /*?s=*
Disallow /*%26p%3D*
Disallow /%26preview%3D*

Other Records

Field Value
sitemap https://topteachingtasksmembers.com/sitemap_index.xml

Comments

  • Block Woocommerce assets
  • Block Search Assets