toygeek.com
robots.txt

Robots Exclusion Standard data for toygeek.com

Resource Scan

Scan Details

Site Domain toygeek.com
Base Domain toygeek.com
Scan Status Ok
Last Scan2025-12-10T15:01:14+00:00
Next Scan 2026-01-09T15:01:14+00:00

Last Scan

Scanned2025-12-10T15:01:14+00:00
URL https://toygeek.com/robots.txt
Domain IPs 104.21.85.130, 172.67.206.45, 2606:4700:3030::6815:5582, 2606:4700:3037::ac43:ce2d
Response IP 104.21.85.130
Found Yes
Hash ccd1bf734e2ed79ffd4d6ac79cd49facf474d66475380f9cb91c4fe8e11ab10f
SimHash 125ad662ce89

Groups

*

Rule Path
Allow /
Disallow /public/scripts/address-book.js
Disallow /public/scripts/category_scripts.js
Disallow /public/scripts/category_scripts_mobile.js
Disallow /public/scripts/checkout_mobile_scripts.js
Disallow /public/scripts/checkout_paypal_mobile_scripts.js
Disallow /public/scripts/checkout_paypal_scripts.js
Disallow /public/scripts/checkout_scripts.js
Disallow /public/scripts/customer.js
Disallow /public/scripts/jquery.lazy.js
Disallow /public/scripts/jquery-ui-autocomplete.min.js
Disallow /public/scripts/jquery-ui-autocomplete-pack.js
Disallow /public/scripts/main.js
Disallow /public/scripts/payment-options.js
Disallow /public/scripts/paypal_messages.js
Disallow /public/scripts/product_script_new.js
Disallow /public/scripts/product_scripts.js
Disallow /public/scripts/stack_checkout_desktop.js
Disallow /public/scripts/stack_checkout_mobile.js
Disallow /public/scripts/stack_paypal_checkout_desktop.js
Disallow /public/scripts/stack_paypal_checkout_mobile.js
Disallow /public/scripts/subscriptions.js
Disallow /public/scripts/subscriptions_new_modal.js
Disallow /public/scripts/ws-jquery-ui-autocomplete-pack.js
Disallow /public/scripts/sweet_alert.js
Disallow /public/scripts/wishlist.js

Other Records

Field Value
crawl-delay 60

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

bubing

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

exensa

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

simplepie

Rule Path
Disallow /

simplepie/1.1.1

Rule Path
Disallow /

spiderbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

webmail.tfsource.com

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

bananabot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://toygeek.com/sitemap.xml