tavolashop.com
robots.txt

Robots Exclusion Standard data for tavolashop.com

Resource Scan

Scan Details

Site Domain tavolashop.com
Base Domain tavolashop.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-04-01T13:02:31+00:00
Next Scan 2025-04-15T13:02:31+00:00

Last Successful Scan

Scanned2025-02-22T12:54:13+00:00
URL https://tavolashop.com/robots.txt
Domain IPs 172.66.40.140, 172.66.43.116, 2606:4700:3108::ac42:288c, 2606:4700:3108::ac42:2b74
Response IP 172.66.43.116
Found Yes
Hash a8c85dc4a38a7569dfb22fe0114c3bc595d9f9f8900d608f3017a82734a69a16
SimHash b141635fc6b8

Groups

gptbot

Rule Path
Disallow /

*

Rule Path
Allow */*/*.css*
Allow */*/*.js*
Allow *.jpg?
Allow *.png?
Allow *.gif?
Allow /*?cat=*
Disallow */Data/EmarsysData/*
Disallow */app/
Disallow */bin/
Disallow */dev/
Disallow */lib/
Disallow */phpserver/
Disallow */pkginfo/
Disallow */report/
Disallow */setup/
Disallow */update/
Disallow */var/
Disallow */vendor/
Disallow */index.php/
Disallow */catalog/product_compare/*
Disallow */catalog/category/view/*
Allow */catalogsearch/result/?q=
Allow */catalogsearch/
Disallow */checkout/*
Disallow */control/*
Disallow */contacts/*
Disallow */customer/*
Disallow */customize/*
Disallow */newsletter/*
Disallow */review/*
Disallow */sendfriend/*
Disallow */wishlist/*
Disallow */composer.json
Disallow */composer.lock
Disallow */CONTRIBUTING.md
Disallow */CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow */COPYING.txt
Disallow */Gruntfile.js
Disallow */LICENSE.txt
Disallow */LICENSE_AFL.txt
Disallow */nginx.conf.sample
Disallow */package.json
Disallow */php.ini.sample
Disallow */RELEASE_NOTES.txt
Disallow /*?*product_list_mode=
Disallow /*?*product_list_order=
Disallow /*?*product_list_limit=
Disallow /*?*product_list_dir=
Disallow /*?SID=
Disallow /*?
Allow /*?p=*
Disallow /*.php$
Disallow /*.CVS
Disallow /*.Zip$
Disallow /*.Svn$
Disallow /*.Idea$
Disallow /*.Sql$
Disallow /*.Tgz$

Comments

  • Directories
  • Paths (clean URLs)
  • Files
  • Do not index pages that are sorted or filtered.
  • Do not index session ID
  • CVS, SVN directory and dump files