deskidea.com
robots.txt

Robots Exclusion Standard data for deskidea.com

Resource Scan

Scan Details

Site Domain deskidea.com
Base Domain deskidea.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-10-03T15:53:23+00:00
Next Scan 2025-11-02T15:53:23+00:00

Last Successful Scan

Scanned2025-09-04T07:04:23+00:00
URL https://deskidea.com/robots.txt
Domain IPs 93.189.94.201
Response IP 93.189.94.201
Found Yes
Hash 7f167495d98af896031f7e90b2dce3530bc2cfdb483f13cd173895dc832d04e3
SimHash 29dfcad3cbdb

Groups

*

Rule Path
Disallow /dashboard/
Disallow /test/
Disallow /customer_dashboard/
Disallow /tracking/
Disallow /cart/
Disallow /checkout/
Disallow /wishlist/
Disallow /account/
Disallow /core/

Other Records

Field Value
crawl-delay 15

googlebot

Rule Path
Disallow /dashboard/
Disallow /test/
Disallow /customer_dashboard/
Disallow /tracking/
Disallow /cart/
Disallow /checkout/
Disallow /wishlist/
Disallow /account/
Disallow /core/

Other Records

Field Value
crawl-delay 8

bingbot
bing
shopwiki
twengabot
twengabot-discover

Rule Path
Disallow /dashboard/
Disallow /test/
Disallow /customer_dashboard/
Disallow /tracking/
Disallow /cart/
Disallow /checkout/
Disallow /wishlist/
Disallow /account/
Disallow /core/

Other Records

Field Value
crawl-delay 10

sbider

Rule Path
Disallow /

linguee

Rule Path
Disallow /

kalooga

Rule Path
Disallow /

spbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sbider

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webzip

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

copyrightcheck

Rule Path
Disallow /

httrack

Rule Path
Disallow /

linko

Rule Path
Disallow /

teleport

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

orthogaffe

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /