6tie.net
robots.txt

Robots Exclusion Standard data for 6tie.net

Resource Scan

Scan Details

Site Domain 6tie.net
Base Domain 6tie.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-03-18T09:27:21+00:00
Next Scan 2025-05-17T09:27:21+00:00

Last Successful Scan

Scanned2025-01-18T09:25:51+00:00
URL https://6tie.net/robots.txt
Redirect https://blog.aiisen.com/robots.txt
Redirect Domain blog.aiisen.com
Redirect Base aiisen.com
Domain IPs 104.21.2.10, 172.67.128.126, 2606:4700:3035::6815:20a, 2606:4700:3037::ac43:807e
Redirect IPs 104.21.10.218, 172.67.146.203, 2606:4700:3031::6815:ada, 2606:4700:3033::ac43:92cb
Response IP 104.21.10.218
Found Yes
Hash 5fb25794b91cd3dd2a9b45f4dd5b28d79907af0e50b828a3a76e2b4108c51db1
SimHash 4116431e4332

Groups

baiduspider

Rule Path
Allow /

googlebot

Rule Path
Allow /

yisouspider

Rule Path
Allow /

bytespider

Rule Path
Allow /

dotbot

Rule Path
Disallow /

abonti

Rule Path
Disallow /

admantx

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

check_http

Rule Path
Disallow /

checks.panopta.com

Rule Path
Disallow /

curl

Rule Path
Disallow /

dle_spider.exe

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

email exractor

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

gbplugin

Rule Path
Disallow /

gooblog

Rule Path
Disallow /

gsa-crawler

Rule Path
Disallow /

holmesbot

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

infoseek sidewinder

Rule Path
Disallow /

infoseek sidewinder/2.0b (linux 2.4 i686)

Rule Path
Disallow /

ip-web-crawler.com

Rule Path
Disallow /

jakarta commons-httpclient

Rule Path
Disallow /

java

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

moget

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

pear http_request class

Rule Path
Disallow /

php

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

pycurl

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

rmnbot

Rule Path
Disallow /

ruby

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seoengworldbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

squider

Rule Path
Disallow /

ssearch_bot

Rule Path
Disallow /

suchmaschinenoptimierung.de

Rule Path
Disallow /

synapse

Rule Path
Disallow /

wget

Rule Path
Disallow /

woopingbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

xenu link sleuth

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yeti

Rule Path
Disallow /

seoprofiler

Rule Path
Disallow /

liebaofast

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

Warnings

  • 2 invalid lines.