Log In Sign Up

19xny.com
robots.txt

Robots Exclusion Standard data for 19xny.com

Resource Scan

Scan Details

Site Domain 19xny.com
Base Domain 19xny.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-08-06T20:41:42+00:00
Next Scan 2024-10-05T20:41:42+00:00

Last Successful Scan

Scanned2024-05-16T20:39:32+00:00
URL http://19xny.com/robots.txt
Domain IPs 156.245.104.80
Response IP 156.245.104.80
Found Yes
Hash 8eac61ef7d02414ce1cbe02b2cb974bf6b315df4d364db0151083f6aeb9364c4
SimHash 5132d5c8f4b2

Groups

baiduspider

Rule Path
Allow /

sogou spider

Rule Path
Allow /

sosospider

Rule Path
Disallow /

googlebot

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

zmeu phpmyadmin

Rule Path
Disallow /

winhttp

Rule Path
Disallow /

httpclient

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

yyspider

Rule Path
Disallow /

jaunty

Rule Path
Disallow /

obot

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

indy library

Rule Path
Disallow /

flightdeckreports bot

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ezooms.bot

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

crawldaddy

Rule Path
Disallow /

asktbfxtv

Rule Path
Disallow /

alexa toolbar

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

coolpadwebkit

Rule Path
Disallow /

java

Rule Path
Disallow /

feedly

Rule Path
Disallow /

universalfeedparser

Rule Path
Disallow /

apachebench

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

zmeu

Rule Path
Disallow /

lightdeckreports bot

Rule Path
Disallow /

digext

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

headlesschrome

Rule Path
Disallow /

feeddemon

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

opensiteexplorer

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

hubspot

Rule Path
Disallow /

opensiteexplorer

Rule Path
Disallow /

python

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /
Back to top

Warnings

  • 2 invalid lines.
Back to top