dkcwb.com
robots.txt

Robots Exclusion Standard data for dkcwb.com

Resource Scan

Scan Details

Site Domain dkcwb.com
Base Domain dkcwb.com
Scan Status Ok
Last Scan2024-11-12T04:48:51+00:00
Next Scan 2024-11-26T04:48:51+00:00

Last Scan

Scanned2024-11-12T04:48:51+00:00
URL https://www.dkcwb.com/robots.txt
Domain IPs 13.227.254.115, 13.227.254.129, 13.227.254.20, 13.227.254.92
Response IP 13.227.254.92
Found Yes
Hash c251fa203ac45e611b2894129525959adc5c6d5f183ee2d737e9c1bc4ccff03b
SimHash 7254db65f431

Groups

*
googlebot
googlebot-image

Rule Path
Allow /
Disallow /conta/*
Disallow /carrinho/*
Disallow /buscar
Disallow /documentacao
Disallow /api/*
Disallow /*fq%3D*
Disallow /*q%3D*
Disallow /*sort%3D*
Disallow /compre_junto/*
Disallow /_events/*
Disallow /tracking/convertion
Disallow /static/*
Disallow /store/*

Other Records

Field Value
crawl-delay 10

dirbuster-0.12
baiduspider
baiduspider-video
baiduspider-image
baiduspider+
twengabot-discover
twengabot
twengabot-2.0
mj12bot
bdcbot
spbot
linkpadbot
wbsearchbot
addthis.com robot tech.support@clearspring.com
addthis.com
exabot
yandeximages
yandex
slurp
spbot
ahrefsbot
everyonesocialbot
blexbot
seekport crawler
geedobot
gptbot
chatgpt-user
claude-web
claudebot
anthropic-ai
cohere-ai
bytespider
perplexitybot
diffbot
scrapy
magpie-crawler
ccbot
omgili
omgilibot
node/simplecrawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.dkcwb.com/sitemap.xml