mymicrolab.com
robots.txt

Robots Exclusion Standard data for mymicrolab.com

Resource Scan

Scan Details

Site Domain mymicrolab.com
Base Domain mymicrolab.com
Scan Status Ok
Last Scan2025-06-10T01:30:48+00:00
Next Scan 2025-07-10T01:30:48+00:00

Last Scan

Scanned2025-06-10T01:30:48+00:00
URL https://mymicrolab.com/robots.txt
Domain IPs 179.61.189.145, 2a02:4780:16:a25d:8f0b:faf6:87d3:bbf4
Response IP 77.37.115.175
Found Yes
Hash 476d42360d364ad74bcffc9099516a740cda16bb30482d9f198c1adcc09a106c
SimHash 6941dc866693

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

msnbot

Rule Path
Disallow

slurp

Rule Path
Disallow

teoma

Rule Path
Disallow

gigabot

Rule Path
Disallow

robozilla

Rule Path
Disallow

nutch

Rule Path
Disallow

ia_archiver

Rule Path
Disallow

baiduspider

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

psbot

Rule Path
Disallow

yahoo-blogs/v3.9

Rule Path
Disallow

*

Rule Path
Disallow
Disallow /customer/account/login/*
Disallow /customer/account/create/*
Disallow /admin_1m29ex
Disallow /customer/account/login/referer/*
Disallow /home
Disallow /staging2/
Disallow /staging/

Other Records

Field Value
crawl-delay 5

httrack
httrack

Rule Path
Disallow /

netcaptor
netcaptor

Rule Path
Disallow /

offline explorer
offline explorer

Rule Path
Disallow /

spiderku/0.9
spiderku/0.9

Rule Path
Disallow /

steeler
steeler

Rule Path
Disallow /

webcopier v3.3
webcopier v3.3

Rule Path
Disallow /

webcopier v3.2a
webcopier v3.2a

Rule Path
Disallow /

webcopier
webcopier

Rule Path
Disallow /

webcrawler

Rule Path
Disallow /

web downloader/4.9
web downloader/4.9

Rule Path
Disallow /

web downloader/5.8
web downloader/5.8

Rule Path
Disallow /

webgather 3.0
webgather 3.0

Rule Path
Disallow /

webstripper/2.56
webstripper/2.56

Rule Path
Disallow /

webzip/3.65
webzip/3.65

Rule Path
Disallow /

webzip
webzip

Rule Path
Disallow /

wget
wget

Rule Path
Disallow /

zao
zao

Rule Path
Disallow /

zeus 2.6
zeus 2.6

Rule Path
Disallow /

Other Records

Field Value
sitemap https://mymicrolab.com/sitemap.xml

Warnings

  • 17 invalid lines.