joejob.it
robots.txt

Robots Exclusion Standard data for joejob.it

Resource Scan

Scan Details

Site Domain joejob.it
Base Domain joejob.it
Scan Status Ok
Last Scan2025-11-24T00:54:50+00:00
Next Scan 2025-12-24T00:54:50+00:00

Last Scan

Scanned2025-11-24T00:54:50+00:00
URL https://joejob.it/robots.txt
Redirect https://www.joejob.it/en_en/robots.txt?___store=en_en&___from_store=it_it
Redirect Domain www.joejob.it
Redirect Base joejob.it
Domain IPs 192.124.249.88
Redirect IPs 192.124.249.88
Response IP 192.124.249.88
Found Yes
Hash 3c2f6ab509c9f0b445339d8cecad9ce87c69cf3ab3f689743079bb55d15e9c11
SimHash 9541405fa3b4

Groups

*

Rule Path
Disallow /*?___from_store=*
Disallow /*?SID=
Disallow /*?price*
Disallow /*?*product_list_order=
Disallow /*?*product_list_dir=
Disallow /*?*product_list_mode=
Disallow /wishlist/
Disallow /it/catalogsearch/
Disallow /it/catalogsearch/*
Disallow /it/catalog/product_compare/
Disallow /it/catalog/category/view/
Disallow /it/catalog/product/view/
Disallow /en/catalogsearch/
Disallow /en/catalogsearch/*
Disallow /en/catalog/product_compare/
Disallow /en/catalog/category/view/
Disallow /en/catalog/product/view/
Disallow /it/checkout/
Disallow /it/checkout/cart/
Disallow /it/onestepcheckout/
Disallow /it/customer/
Disallow /it/customer/account/
Disallow /it/customer/account/login/
Disallow /en/checkout/
Disallow /en/checkout/cart/
Disallow /en/onestepcheckout/
Disallow /en/customer/
Disallow /en/customer/account/
Disallow /en/customer/account/login/
Disallow /it/blog?blog_category=*
Disallow /en/blog?blog_category=*
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /nginx.conf.sample
Disallow /package.json
Disallow /php.ini.sample
Disallow /RELEASE_NOTES.txt
Disallow /*?SID=
Disallow /*?
Disallow /*.php$
Disallow /*.CVS
Disallow /*.Zip$
Disallow /*.Svn$
Disallow /*.Idea$
Disallow /*.Sql$
Disallow /*.Tgz$

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

msnbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Comments

  • Do not index session ID
  • CVS, SVN directory and dump files