midocean.com
robots.txt

Robots Exclusion Standard data for midocean.com

Resource Scan

Scan Details

Site Domain midocean.com
Base Domain midocean.com
Scan Status Ok
Last Scan2024-06-25T18:23:51+00:00
Next Scan 2024-07-25T18:23:51+00:00

Last Scan

Scanned2024-06-25T18:23:51+00:00
URL https://midocean.com/robots.txt
Redirect https://www.midocean.com//robots.txt
Redirect Domain www.midocean.com
Redirect Base midocean.com
Domain IPs 20.50.151.10
Redirect IPs 152.199.39.108, 2606:2800:247:1cb7:261b:1f9c:2074:3c
Response IP 152.199.39.108
Found Yes
Hash 2ce0617e71c7914499bdb49b2a62d949cdeb8d60dec6694785662703565c22b6
SimHash 7394c915e699

Groups

*

Rule Path
Disallow /INTERSHOP/
Disallow /ViewCart-View
Disallow /ViewCheckout
Disallow /viewdata
Disallow /my-account
Disallow /Favourites
Disallow /saved-addresses
Disallow /profile-settings
Disallow /my-logo
Disallow /my-quotes
Disallow /quotes-list
Disallow /*?gclid=
Disallow /*?SearchParameter=

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

trendictionbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

kaszonibot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

Comments

  • dynamic url's
  • no hammering

Warnings

  • 2 invalid lines.