hurtszewski.com
robots.txt

Robots Exclusion Standard data for hurtszewski.com

Resource Scan

Scan Details

Site Domain hurtszewski.com
Base Domain hurtszewski.com
Scan Status Ok
Last Scan2025-06-01T18:28:22+00:00
Next Scan 2025-07-01T18:28:22+00:00

Last Scan

Scanned2025-06-01T18:28:22+00:00
URL https://hurtszewski.com/robots.txt
Redirect https://www.hurtszewski.com/robots.txt
Redirect Domain www.hurtszewski.com
Redirect Base hurtszewski.com
Domain IPs 185.123.161.96
Redirect IPs 185.123.161.96
Response IP 185.123.161.96
Found Yes
Hash e74e9ba999a22e9c8f3bb386b06e8650748684ccdd3dff920942b7e076c49799
SimHash 034c1400c5b1

Groups

*

Rule Path
Disallow /admin
Disallow /download
Disallow /includes
Disallow /temp
Disallow /tmp
Disallow /img
Disallow /account.php
Disallow /advanced_search.php
Disallow /checkout_shipping.php
Disallow /checkout_payment.php
Disallow /checkout_process.php
Disallow /create_account.php
Disallow /login.php
Disallow /password_forgotten.php
Disallow /shopping_cart.php
Disallow /patch-1-02-b.php
Disallow /product_reviews_write.php
Disallow /account.html
Disallow /advanced_search.html
Disallow /checkout_shipping.html
Disallow /checkout_payment.html
Disallow /checkout_shipping.html
Disallow /checkout_process.html
Disallow /create_account.html
Disallow /login.html
Disallow /password_forgotten.html
Disallow /popup_image.html
Disallow /shopping_cart.html
Disallow /product_reviews_write.html
Disallow /images/
Disallow /uploaded/

twiceler

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

speedy

Rule Path
Disallow /

gulliver

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

fast

Rule Path
Disallow /

scooter

Rule Path
Disallow /

vscooter

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

nutch

Rule Path
Disallow /

fast-webcrawler

Rule Path
Disallow /

teoma_agent1

Rule Path
Disallow /

lycos_spider_(t-rex)

Rule Path
Disallow /

architextspider

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

yahoobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

Comments

  • Disallow: /private