Log In Sign Up

xmirex.com
robots.txt

Robots Exclusion Standard data for xmirex.com

Resource Scan

Scan Details

Site Domain xmirex.com
Base Domain xmirex.com
Scan Status Ok
Last Scan2024-11-08T13:28:27+00:00
Next Scan 2024-12-08T13:28:27+00:00

Last Scan

Scanned2024-11-08T13:28:27+00:00
URL https://xmirex.com/robots.txt
Domain IPs 51.195.176.123
Response IP 51.195.176.123
Found Yes
Hash bbe94ddeb9e62453b3ed8d2e7370e1be1df83d8478d53cfdd955d5e049b5e5c2
SimHash db7ed451ae50

Groups

*

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

blexbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

linguee

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mail.ru_bot/2.0

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

mail.ru_bot/2.0; +http://go.mail.ru/help/robots

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

semrush

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

crowdtanglebot

Rule Path
Disallow /

go-resty

Rule Path
Disallow /

researchscan

Rule Path
Disallow /

seekport

Rule Path
Disallow /

checkmarknetwork

Rule Path
Disallow /

indeedbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

senutobot

Rule Path
Disallow /
Back to top

Comments

  • Block MegaIndex.ru
  • MJ12bot
  • AhrefsBot
  • Mail.RU_Bot/2.0 Russia
  • Mail.RU Russia
  • Mail.Ru Russia
  • TurnitinBot
  • ZoomInfoBot
  • MauiBot
  • TweetmemeBot
  • magpie-crawler
  • YisouSpider
  • bytespider
  • semrush
  • BLEXbot
  • serpstatbot
  • CrowdTanglebot
  • go-resty
  • Researchscan
  • Seekport
  • CheckMarkNetwork
  • IndeedBot
  • 360Spider
Back to top

Warnings

  • 2 invalid lines.
Back to top