Log In Sign Up

zeromarkets.com
robots.txt

Robots Exclusion Standard data for zeromarkets.com

Resource Scan

Scan Details

Site Domain zeromarkets.com
Base Domain zeromarkets.com
Scan Status Ok
Last Scan2025-07-27T15:12:46+00:00
Next Scan 2025-08-26T15:12:46+00:00

Last Scan

Scanned2025-07-27T15:12:46+00:00
URL https://zeromarkets.com/robots.txt
Redirect https://www.zeromarkets.com/robots.txt
Redirect Domain www.zeromarkets.com
Redirect Base zeromarkets.com
Domain IPs 104.21.37.151, 172.67.209.187, 2606:4700:3031::6815:2597, 2606:4700:3037::ac43:d1bb
Redirect IPs 104.21.37.151, 172.67.209.187, 2606:4700:3031::6815:2597, 2606:4700:3037::ac43:d1bb
Response IP 104.21.37.151
Found Yes
Hash 2aff8fb9f3be78c9e70d87751612fb6efedcd6f0bc9b3ec70e8e3003ee243ae3
SimHash d638d159d7bc

Groups

*

Rule Path
Disallow /wp-admin
Disallow /search/page
Disallow /tools/virtual-private-server-vps-for-forex-trading/
Disallow /es-int/tools/virtual-private-server-vps-for-forex-trading/
Disallow /ko/tools/virtual-private-server-vps-for-forex-trading/
Disallow /ru/tools/virtual-private-server-vps-for-forex-trading/
Disallow /fr/tools/virtual-private-server-vps-for-forex-trading/
Disallow /tr/tools/virtual-private-server-vps-for-forex-trading/
Disallow /de/tools/virtual-private-server-vps-for-forex-trading/
Disallow /it/tools/virtual-private-server-vps-for-forex-trading/
Disallow /ar/tools/virtual-private-server-vps-for-forex-trading/
Disallow /cn/tools/virtual-private-server-vps-for-forex-trading/
Disallow /mn/tools/virtual-private-server-vps-for-forex-trading/
Disallow /pk/tools/virtual-private-server-vps-for-forex-trading/
Disallow /ae/tools/virtual-private-server-vps-for-forex-trading/
Disallow /pt-int/tools/virtual-private-server-vps-for-forex-trading/
Disallow /pt-br/tools/virtual-private-server-vps-for-forex-trading/

mj12bot
seokicks-robot
ia_archiver
linkchecker
archive.org_bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /wp-admin
Disallow /search/page
Disallow /platforms/*
Disallow /fr/*
Disallow /tr/*
Disallow /de/*
Disallow /it/*
Disallow /es/*
Disallow /ru/*
Disallow /br/*
Disallow /id/*
Disallow /my/*
Disallow /ar/*
Disallow /cn/*
Disallow /th/*
Disallow /vn/*

Other Records

Field Value
crawl-delay 10

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

asterias

Rule Path
Disallow /

backdoorbot/1.0

Rule Path
Disallow /

black hole

Rule Path
Disallow /

blowfish/1.0

Rule Path
Disallow /

botalot

Rule Path
Disallow /

builtbottough

Rule Path
Disallow /

bullseye/1.0

Rule Path
Disallow /

bunnyslippers

Rule Path
Disallow /

cegbfeieh

Rule Path
Disallow /

cheesebot

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

cherrypickerelite/1.0

Rule Path
Disallow /

cherrypickerse/1.0

Rule Path
Disallow /

copyrightcheck

Rule Path
Disallow /

cosmos

Rule Path
Disallow /

crescent

Rule Path
Disallow /

crescent internet toolpak http ole control v.1.0

Rule Path
Disallow /

dittospyder

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

erocrawler

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

foobot

Rule Path
Disallow /

harvest/1.5

Rule Path
Disallow /

hloader

Rule Path
Disallow /

httplib

Rule Path
Disallow /

fast

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /
Back to top

Comments

  • User-agent: AhrefsBot
  • Disallow: /
  • User-agent: msnbot
  • User-agent: bingbot
  • User-Agent: Mail.Ru
  • User-agent: Yandex
  • Crawl-delay: 10
  • Some bots are known to be trouble, particularly those designed to copy
  • entire sites. Please obey robots.txt.
  • Misbehaving: requests much too fast:
  • Hits many times per second, not acceptable
  • http://www.nameprotect.com/botinfo.html
  • A capture bot, downloads gazillions of pages with no public benefit
  • http://www.webreaper.net/
Back to top