shiekh.com
robots.txt

Robots Exclusion Standard data for shiekh.com

Resource Scan

Scan Details

Site Domain shiekh.com
Base Domain shiekh.com
Scan Status Ok
Last Scan2024-11-13T13:17:29+00:00
Next Scan 2024-11-27T13:17:29+00:00

Last Scan

Scanned2024-11-13T13:17:29+00:00
URL https://shiekh.com/robots.txt
Redirect https://www.shiekh.com:443/robots.txt
Redirect Domain www.shiekh.com
Redirect Base shiekh.com
Domain IPs 35.166.51.19, 54.203.199.77
Redirect IPs 151.101.1.146, 151.101.129.146, 151.101.193.146, 151.101.65.146
Response IP 199.232.45.146
Found Yes
Hash a49add1af235dfa2245d7064096ba9431345febe73977120051d89e540a4cb87
SimHash f81ed43384bd

Groups

*

Rule Path
Disallow /index.php/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/*
Disallow /catalog/product/view/*
Disallow /catalogsearch/
Disallow /checkout/*
Disallow /control/
Disallow /contacts/
Disallow /customer/*
Disallow /customize/
Disallow /newsletter/*
Disallow /sendfriend/*
Disallow /wishlist/*
Disallow /*?*price=
Disallow /*?*size=
Disallow /*?*size_filter=
Disallow /*?*color_primary=
Disallow /*?*brand=
Disallow /*?*cat=
Disallow /*?*gender=
Disallow /*?*p=
Disallow /*?*style=
Disallow /*?*Size=
Disallow /*?*Color=
Disallow /*?*Brand=
Disallow /*?*occasion=
Disallow /*?*Style=
Disallow /*?*Gender=
Disallow /*?*Category=
Disallow /*?*Occasion=
Disallow /*?*SID=
Disallow /*?*phqqbk=
Disallow /*?*product_list_order=
Disallow /*?*product_list_limit=
Disallow /*?*q=
Disallow /*.CVS
Disallow /*.Zip$
Disallow /*.Svn$
Disallow /*.Idea$
Disallow /*.Sql$
Disallow /*.Tgz$
Disallow /*.php$
Disallow /rep-list-ah2dz443a

Other Records

Field Value
sitemap https://www.shiekh.com/media/sitemaps/shiekh_sitemap.xml

Comments

  • Paths (clean URLs)
  • Query Strings
  • CVS, SVN directory and dump files
  • user-agent: pinterestbot
  • Crawl-delay: 0.5