archive2.solutiontree.com
robots.txt

Robots Exclusion Standard data for archive2.solutiontree.com

Resource Scan

Scan Details

Site Domain archive2.solutiontree.com
Base Domain solutiontree.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-08-19T13:26:53+00:00
Next Scan 2024-11-17T13:26:53+00:00

Last Successful Scan

Scanned2023-04-05T11:44:28+00:00
URL https://archive2.solutiontree.com/robots.txt
Domain IPs 104.26.4.171, 104.26.5.171, 172.67.70.244, 2606:4700:20::681a:4ab, 2606:4700:20::681a:5ab, 2606:4700:20::ac43:46f4
Response IP 104.26.4.171
Found Yes
Hash b7a33ac5be221c788c8a62ced23d1928316084154498c2344588b41d3983e89e
SimHash e926f80bcbb9

Groups

*

Rule Path
Disallow /lib/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /customer/
Disallow /sendfriend/
Disallow /review/
Disallow /*SID%3D
Disallow /blog/wp-login.php
Disallow /blog/wp-admin
Disallow /custom/
Disallow /misc/
Disallow /plc-navigator/
Disallow /scripts/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /checkout/
Disallow /control/
Disallow /contacts/
Disallow /customer/
Disallow /customize/
Disallow /newsletter/
Disallow /poll/
Disallow /review/
Disallow /sendfriend/
Disallow /tag/
Disallow /wishlist/
Disallow /search/
Disallow /search/?*
Disallow /customer/
Disallow /search/
Disallow /searchresults/

deepcrawl

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Comments

  • Default Instructions
  • Directories
  • Paths (clean URLs)
  • disabling indexing
  • disabling indexing
  • disable deepcrawl....
  • disable ahrefs bot
  • disable petalbot