dotmed.com
robots.txt

Robots Exclusion Standard data for dotmed.com

Resource Scan

Scan Details

Site Domain dotmed.com
Base Domain dotmed.com
Scan Status Ok
Last Scan2024-11-16T00:28:04+00:00
Next Scan 2024-11-23T00:28:04+00:00

Last Scan

Scanned2024-11-16T00:28:04+00:00
URL https://dotmed.com/robots.txt
Redirect https://www.dotmed.com/robots.txt
Redirect Domain www.dotmed.com
Redirect Base dotmed.com
Domain IPs 64.94.160.2
Redirect IPs 64.94.160.2
Response IP 64.94.160.2
Found Yes
Hash 292fa0cdbbd9339196299f1cd12fc82901ccdb44839b6a95bbca23c4d03a7ca5
SimHash 6b8794c1f280

Groups

*

Rule Path
Disallow /redirect.html
Disallow /users/my/
Disallow /admin2/
Disallow /admin/
Disallow /images/
Disallow /tracking/
Disallow /listings/details.html
Disallow /listing/print/
Disallow /listings/print/
Disallow /listings/detail.html
Disallow /listings/search_equipment.html
Disallow /listings/search_parts.html
Disallow /auctions/index.html
Disallow /virtual-trade-show/emailbooth.html
Disallow /sl/shipping/createrequest.html
Disallow /users/leads/index.html
Disallow /ads/addisplay.html
Disallow /ads/adhtm.html
Disallow /ads/displayhtmlad.html
Disallow /virtual-trade-show/category/flag_model.html
Disallow /ajax/components/
Disallow /images/auctionpics/*/*-*.jpg
Disallow *chap%3D*
Disallow *?amount=*
Disallow /login.html?*
Disallow /listing/*?mobile=0
Disallow /listing/*?lang=en
Disallow /browse/*?*
Disallow /features/auctions_self.html?*
Disallow /features/auctions/private_sale.html?*

paracrawl

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

blp_bbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

voyager

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

vocusbot

Rule Path
Allow /news/
Disallow /

grapeshot

Rule Path
Disallow /

fairshare.cc

Rule Path
Allow /news/
Disallow /

petalbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sputnikbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seokicks-robot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

gnowitnewsbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

Warnings

  • 4 invalid lines.