mstaml.com
robots.txt

Robots Exclusion Standard data for mstaml.com

Resource Scan

Scan Details

Site Domain mstaml.com
Base Domain mstaml.com
Scan Status Ok
Last Scan2025-07-12T03:41:30+00:00
Next Scan 2025-07-19T03:41:30+00:00

Last Scan

Scanned2025-07-12T03:41:30+00:00
URL https://mstaml.com/robots.txt
Redirect https://www.mstaml.com/robots.txt
Redirect Domain www.mstaml.com
Redirect Base mstaml.com
Domain IPs 104.21.56.43, 172.67.177.94, 2606:4700:3035::ac43:b15e, 2606:4700:3037::6815:382b
Redirect IPs 104.21.56.43, 172.67.177.94, 2606:4700:3035::ac43:b15e, 2606:4700:3037::6815:382b
Response IP 104.21.56.43
Found Yes
Hash 9cc9f7a6622395d7dba1c08d11df4300d09e8f1d10a70fa02d2f5cc2dbdc584d
SimHash e0c0998c8831

Groups

mediapartners-google

Rule Path
Disallow

baiduspider

Rule Path
Disallow /

*

Rule Path
Disallow /imagesData/
Disallow /adm/
Disallow /out.php
Disallow /market*%3BisSavedSearch%3D
Disallow /market*?isSavedSearch=
Disallow /market*?*&isSavedSearch=
Disallow /payment
Disallow /products-feed*.xml
Disallow /huawei-cars-feed-*
Disallow /huawei-real-estates-feed-*

Other Records

Field Value
sitemap https://www.mstaml.com/site-map-index.xml

Comments

  • allow every thing for Google AdSense spider
  • disallow every thing for Baidu (chinese spider)
  • disallow adm and user private pages for other spiders and disable imagesData to prevent apache error (36)File name too long
  • Disallow: /*?showPwa=1
  • Disallow: /*?*&showPwa=1