Log In Sign Up

blockandcompany.com
robots.txt

Robots Exclusion Standard data for blockandcompany.com

Resource Scan

Scan Details

Site Domain blockandcompany.com
Base Domain blockandcompany.com
Scan Status Ok
Last Scan2024-11-13T10:04:43+00:00
Next Scan 2024-12-13T10:04:43+00:00

Last Scan

Scanned2024-11-13T10:04:43+00:00
URL https://blockandcompany.com/robots.txt
Domain IPs 192.240.162.199
Response IP 192.240.162.199
Found Yes
Hash dd2813300fa9c8d8db407b567f8453c1062ce196e644f7ddb4e18efa07d537d5
SimHash 9c2cba41c7f3

Groups

*

Rule Path Comment
Disallow /404/ -
Disallow /app/ -
Disallow /cgi-bin/ -
Disallow /downloader/ -
Disallow /errors/ -
Disallow /includes/ -
Disallow /lib/ -
Disallow /magento/ -
Disallow /media/captcha/ -
Disallow /media/customer/ -
Disallow /media/dhl/ -
Disallow /media/downloadable/ -
Disallow /media/import/ -
Disallow /media/sales/ -
Disallow /media/tmp/ -
Disallow /media/wysiwyg/ -
Disallow /media/xmlconnect/ -
Disallow /pkginfo/ -
Disallow /report/ -
Disallow /scripts/ -
Disallow /shell/ -
Disallow /stats/ -
Disallow /var/ -
Disallow /index.php/ -
Disallow /catalog/product_compare/ -
Disallow /catalog/category/view/ -
Disallow /catalog/product/view/ -
Disallow /catalog/product/gallery/ -
Disallow /catalogsearch/ -
Disallow /checkout/ -
Disallow /control/ -
Disallow /contacts/ -
Disallow /customer/ -
Disallow /customize/ -
Disallow /newsletter/ -
Disallow /poll/ -
Disallow /review/ -
Disallow /tag/ -
Disallow /wishlist/ -
Disallow /quickview/ -
Disallow /cron.php -
Disallow /cron.sh -
Disallow /error_log -
Disallow /install.php -
Disallow /LICENSE.html -
Disallow /LICENSE.txt -
Disallow /LICENSE_AFL.txt -
Disallow /STATUS.txt -
Disallow /get.php Magento 1.5+
Disallow /*.php$ -
Disallow /rss* -
Disallow /*PHPSESSID -
Disallow /upload -
Disallow /facebook -
Disallow /report -
Disallow /sitewideimages -
Disallow /onlinecatalogs -
Disallow /sendfriend -

sistrix

Rule Path
Disallow /

mj12bot

Rule Path
Disallow

sogou pic spider

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

indy library

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://banksupplies.com/sitemaps/sitemap.xml
Back to top

Comments

  • BankSupplies Robots.txt last modified 12/13/2016 by Meg;
  • Website Sitemap
  • Directories - HEI
  • Disallow: /media/ - HEI
  • Disallow: /media/catalog/
  • Disallow: /skin/
  • Paths (clean URLs) - HEI
  • Files - HEI
  • Paths (no clean URLs) - HEI
  • Disallow: /*?SID=
  • Other Disallows - Meg
  • Disallow assorted bots we don't need - Meg
  • Note: Scot Stier from AlevaStores.com has requested rogerbot be allowed 7/21/2020 MT
  • User-Agent: rogerbot
  • Disallow: /
  • Allow Majestic bot per Meg -
Back to top

Warnings

  • 2 invalid lines.
Back to top