Log In Sign Up

desertcart.pe
robots.txt

Robots Exclusion Standard data for desertcart.pe

Resource Scan

Scan Details

Site Domain desertcart.pe
Base Domain desertcart.pe
Scan Status Ok
Last Scan2024-06-09T18:42:18+00:00
Next Scan 2024-07-09T18:42:18+00:00

Last Scan

Scanned2024-06-09T18:42:18+00:00
URL https://desertcart.pe/robots.txt
Redirect https://www.desertcart.pe/robots.txt
Redirect Domain www.desertcart.pe
Redirect Base desertcart.pe
Domain IPs 104.21.86.235, 172.67.137.156, 2606:4700:3032::ac43:899c, 2606:4700:3035::6815:56eb
Redirect IPs 34.197.19.238, 52.204.181.104
Response IP 52.204.181.104
Found Yes
Hash 084c031bf17480624f9b7a54489c807c52a1d86141cc5c3ee78c8c7b54ee0839
SimHash 8915023dfec2

Groups

*

Rule Path
Disallow /searchv1
Disallow /searchv2
Disallow /a/
Disallow /invite/
Disallow /api/products/search

baiduspider

Rule Path
Disallow /products/
Disallow /tags/
Disallow /nodes/

yandexbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /tags/
Disallow /api/

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

fr_bot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.desertcart.pe/sitemaps/pe/sitemap-index.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
Back to top