Log In Sign Up

base.mcx666.com
robots.txt

Robots Exclusion Standard data for base.mcx666.com

Resource Scan

Scan Details

Site Domain base.mcx666.com
Base Domain mcx666.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-05T00:10:09+00:00
Next Scan 2026-01-03T00:10:09+00:00

Last Successful Scan

Scanned2024-11-16T15:15:44+00:00
URL http://base.mcx666.com/robots.txt
Domain IPs 119.3.214.180
Response IP 119.3.214.180
Found Yes
Hash e1689d422d5ab82849a8f3925c8ac89554cb1da6f4c819db053de5060bd56740
SimHash c31e43aa4832

Groups

abonti

Rule Path
Disallow /

admantx

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

check_http

Rule Path
Disallow /

checks.panopta.com

Rule Path
Disallow /

curl

Rule Path
Disallow /

dle_spider.exe

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

email exractor

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

gbplugin

Rule Path
Disallow /

gooblog

Rule Path
Disallow /

gsa-crawler

Rule Path
Disallow /

holmesbot

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

infoseek sidewinder

Rule Path
Disallow /

infoseek sidewinder/2.0b (linux 2.4 i686)

Rule Path
Disallow /

ip-web-crawler.com

Rule Path
Disallow /

jakarta commons-httpclient

Rule Path
Disallow /

java

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

moget

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

pear http_request class

Rule Path
Disallow /

php

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

pycurl

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

rmnbot

Rule Path
Disallow /

ruby

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seoengworldbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

squider

Rule Path
Disallow /

ssearch_bot

Rule Path
Disallow /

suchmaschinenoptimierung.de

Rule Path
Disallow /

synapse

Rule Path
Disallow /

wget

Rule Path
Disallow /

woopingbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

xenu link sleuth

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yeti

Rule Path
Disallow /

seoprofiler

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15
Back to top