prestashop.com
robots.txt

Robots Exclusion Standard data for prestashop.com

Resource Scan

Scan Details

Site Domain prestashop.com
Base Domain prestashop.com
Scan Status Ok
Last Scan2024-11-06T18:58:34+00:00
Next Scan 2024-11-13T18:58:34+00:00

Last Scan

Scanned2024-11-06T18:58:34+00:00
URL https://prestashop.com/robots.txt
Domain IPs 104.16.210.130, 104.16.211.130, 2606:4700::6810:d282, 2606:4700::6810:d382
Response IP 104.16.211.130
Found Yes
Hash 1ad3cb6f4272e1204a9f0c3eccd4c12142efb21134aaf14c6fc327427f9405dc
SimHash a86009978cc4

Groups

*

Rule Path
Disallow /wp-admin
Disallow /wp-includes
Disallow /wp-content/plugins
Disallow /wp-content/cache
Disallow /feed
Disallow */feed
Disallow /wp-login.php
Disallow */trackback
Disallow /*/comments
Disallow /cgi-bin
Disallow /*.php$
Disallow /*.inc$
Disallow /*.gz
Disallow /*.cgi
Disallow /blog?q=*
Disallow /blog/authors/
Disallow /forums/startTopic/
Disallow /forums/discover/unread/
Disallow /forums/markallread/
Disallow /forums/staff/
Disallow /forums/online/
Disallow /forums/discover/
Disallow /forums/leaderboard/
Disallow /forums/search/
Disallow /forums/tags/
Disallow /forums/*?advancedSearchForm=
Disallow /forums/register/
Disallow /forums/lostpassword/
Disallow /forums/login/
Disallow /forums/*?sortby=
Disallow /forums/*?filter=
Disallow /forums/*?tab=
Disallow /forums/*?do=
Disallow /forums/*ref%3D
Disallow /forums/*?forumId*
Disallow /forums/*content_id%3D
Disallow /forums/ignore/
Disallow /forums/settings/
Disallow /forums/logout/
Disallow /forums*/markallread/*
Disallow /forums/applications/*attachment.php?id=*
Disallow /forums/profile/

Other Records

Field Value
sitemap https://prestashop.com/sitemap.xml

Comments

  • Rules for Invision Community (https://invisioncommunity.com)
  • Block pages with no unique content
  • Block faceted pages and 301 redirect pages
  • Adding Test to reduce error in GSC
  • Block profile pages as these have little unique value, consume a lot of crawl time and contain hundreds of 301 links
  • Sitemap URL