mobilitysmart.co.uk
robots.txt

Robots Exclusion Standard data for mobilitysmart.co.uk

Resource Scan

Scan Details

Site Domain mobilitysmart.co.uk
Base Domain mobilitysmart.co.uk
Scan Status Ok
Last Scan2025-10-29T19:16:16+00:00
Next Scan 2025-11-28T19:16:16+00:00

Last Scan

Scanned2025-10-29T19:16:16+00:00
URL https://mobilitysmart.co.uk/robots.txt
Redirect https://www.mobilitysmart.co.uk/robots.txt
Redirect Domain www.mobilitysmart.co.uk
Redirect Base mobilitysmart.co.uk
Domain IPs 104.26.4.175, 104.26.5.175, 172.67.70.43, 2606:4700:20::681a:4af, 2606:4700:20::681a:5af, 2606:4700:20::ac43:462b
Redirect IPs 104.26.4.175, 104.26.5.175, 172.67.70.43, 2606:4700:20::681a:4af, 2606:4700:20::681a:5af, 2606:4700:20::ac43:462b
Response IP 172.67.70.43
Found Yes
Hash ceddfe2e39728c56c75bc27320ce1d464792fc3491d51e67fb33dc33ff581764
SimHash 8505717a80b8

Groups

*

Rule Path Comment
Allow / -
Disallow /app/ -
Disallow /bin/ -
Disallow /dev/ -
Disallow /lib/ -
Disallow /phpserver/ -
Disallow /pkginfo/ -
Disallow /report/ -
Disallow /setup/ -
Disallow /update/ -
Disallow /var/ -
Disallow /vendor/ -
Disallow /index.php/ -
Disallow /catalog/product_compare/ -
Disallow /catalog/category/view/ -
Disallow /catalog/product/view/ -
Disallow /catalogsearch/ -
Disallow /checkout/ -
Disallow /control/ -
Disallow /contacts/ -
Disallow /customer/ -
Disallow /customize/ -
Disallow /newsletter/ -
Disallow /review/ -
Disallow /sendfriend/ -
Disallow /wishlist/ -
Disallow /composer.json -
Disallow /composer.lock -
Disallow /CONTRIBUTING.md -
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html -
Disallow /COPYING.txt -
Disallow /Gruntfile.js -
Disallow /LICENSE.txt -
Disallow /LICENSE_AFL.txt -
Disallow /nginx.conf.sample -
Disallow /package.json -
Disallow /php.ini.sample -
Disallow /RELEASE_NOTES.txt -
Disallow /*?*product_list_mode= -
Disallow /*?*product_list_order= -
Disallow /*?*product_list_limit= -
Disallow /*?*product_list_dir= -
Disallow /*?*brand= -
Disallow /*?*price= -
Disallow /*?*cat= -
Disallow /*?*manufacturer= -
Disallow /*?*size= -
Disallow /*?*color= -
Disallow /*?*inStock= -
Disallow /*?*p= pagination with params
Disallow /*?SID= -
Disallow /*.php$ -
Disallow /*.CVS -
Disallow /*.Zip$ -
Disallow /*.Svn$ -
Disallow /*.Idea$ -
Disallow /*.Sql$ -
Disallow /*.Tgz$ -

Other Records

Field Value
sitemap https://www.mobilitysmart.co.uk/sitemap/sitemap.xml

Comments

  • Directories
  • Paths (clean URLs)
  • Files
  • Do not index pages that are sorted or filtered.
  • Do not index session ID
  • CVS, SVN directory and dump files