laboratory-equipment.com
robots.txt

Robots Exclusion Standard data for laboratory-equipment.com

Resource Scan

Scan Details

Site Domain laboratory-equipment.com
Base Domain laboratory-equipment.com
Scan Status Ok
Last Scan2024-06-19T12:37:25+00:00
Next Scan 2024-07-19T12:37:25+00:00

Last Scan

Scanned2024-06-19T12:37:25+00:00
URL https://laboratory-equipment.com/robots.txt
Redirect https://www.laboratory-equipment.com/robots.txt
Redirect Domain www.laboratory-equipment.com
Redirect Base laboratory-equipment.com
Domain IPs 104.26.4.105, 104.26.5.105, 172.67.68.141, 2606:4700:20::681a:469, 2606:4700:20::681a:569, 2606:4700:20::ac43:448d
Redirect IPs 104.26.4.105, 104.26.5.105, 172.67.68.141, 2606:4700:20::681a:469, 2606:4700:20::681a:569, 2606:4700:20::ac43:448d
Response IP 104.26.5.105
Found Yes
Hash 2c554ab1fba9df350c99a2cb347160052dbb4fd93426553346e69bb3edb07494
SimHash ad356a21cbbb

Groups

yandex
ichiro
moget
naverbot
baiduspider
baidu
sogou
youdao
mj12bot
goodzer
istellabot
gigabot
dotbot
seznambot
ltx71
spiderbot
opensiteexplorer
openlinkprofiler
majestic12
datagnion
sogou
ahrefs
scoutjet
changedetection
istellabot
napoveda.seznam
linkfluence
smarter
mj12bot
mojeek
naver
deusu
slack
aihitdata
wotbox
kazbt
mediatoolkit
safedns
aboundex
webmeup-crawler
orangebot
baiduspider-video
baiduspider-image

Rule Path
Disallow /

serpstatbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Allow /customer/design/index
Allow /customer/design/index/
Disallow /404/
Disallow /app/
Disallow /cgi-bin/
Disallow /downloader/
Disallow /errors/
Disallow /includes/
Disallow /lib/
Disallow /magento/
Disallow /pkginfo/
Disallow /report/
Disallow /scripts/
Disallow /shell/
Disallow /stats/
Disallow /var/
Disallow /*search?*
Disallow /*?*
Disallow /index.php/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalogsearch/
Disallow /checkout/
Disallow /quotation/
Disallow /contacts/
Disallow /customer/
Disallow /newsletter/
Disallow /poll/
Disallow /review/
Disallow /sendfriend/
Disallow /tag/
Disallow /wishlist/
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /STATUS.txt

Other Records

Field Value
sitemap https://www.terrauniversal.com/terra_sitemap.xml
sitemap https://www.laboratory-equipment.com/lab_sitemap.xml
sitemap https://www.terrauniversal.com/blog_terra_sitemap.xml
sitemap https://www.laboratory-equipment.com/blog_lab_sitemap.xml

Comments

  • Directories
  • Paths (clean URLs)
  • Files