shopcarp.com
robots.txt

Robots Exclusion Standard data for shopcarp.com

Resource Scan

Scan Details

Site Domain shopcarp.com
Base Domain shopcarp.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-04-15T10:47:32+00:00
Next Scan 2024-07-14T10:47:32+00:00

Last Successful Scan

Scanned2023-05-29T10:45:17+00:00
URL https://shopcarp.com/robots.txt
Redirect https://www.shopcarp.com/robots.txt
Redirect Domain www.shopcarp.com
Redirect Base shopcarp.com
Domain IPs 104.21.49.197, 172.67.166.204, 2606:4700:3030::6815:31c5, 2606:4700:3037::ac43:a6cc
Redirect IPs 104.21.49.197, 172.67.166.204, 2606:4700:3030::6815:31c5, 2606:4700:3037::ac43:a6cc
Response IP 104.21.49.197
Found Yes
Hash f3cfaca0fa797b2d5f956dc5778e4c8828965025b8f29b94e3f9d8cbad6f817f
SimHash 3510c7a8ce01

Groups

*

Rule Path
Disallow /cdn-cgi/
Disallow /busca*
Disallow /*%26sort%3D*
Disallow /*%26order%3D*
Disallow /*%26limit%3D*
Disallow /*%26q%3D*
Disallow /*?q=*
Disallow /*%26filter%3D*
Disallow /*?filter=*
Disallow /*%26size%3D*
Disallow /*?size=*
Disallow /login$
Disallow /carrinho$
Disallow /cadastro$
Disallow /checkout$
Disallow /vale-presente
Disallow /meus-pedidos
Disallow /minha-conta
Disallow /minha-conta-afiliado

domaincrawler/3.0

Rule Path
Disallow /

dirbuster-0.12

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

twengabot-discover

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

twengabot-2.0

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

addthis.com

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandex

Rule Path
Disallow /

slurp

Rule Path
Disallow /

spbot

Rule Path
Disallow /

everyonesocialbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://files.irroba.com.br/carpaaua/feeds/sitemap.xml