Log In Sign Up

curtiswatkins.com
robots.txt

Robots Exclusion Standard data for curtiswatkins.com

Resource Scan

Scan Details

Site Domain curtiswatkins.com
Base Domain curtiswatkins.com
Scan Status Ok
Last Scan2025-08-31T15:11:29+00:00
Next Scan 2025-09-07T15:11:29+00:00

Last Scan

Scanned2025-08-31T15:11:29+00:00
URL https://curtiswatkins.com/robots.txt
Domain IPs 104.21.73.92, 172.67.189.87, 2606:4700:3033::6815:495c, 2606:4700:3034::ac43:bd57
Response IP 172.67.189.87
Found Yes
Hash 661c86d34014caf52bd0225c229d5b795cad7b3331d380caee1073ee50c952c1
SimHash 5935b16bd12a

Groups

semrushbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

acoonbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alexa

Rule Path
Disallow /

toolbar

Rule Path
Disallow /

apachebench

Rule Path
Disallow /

applebot

Rule Path
Disallow /

asktbfxtv

Rule Path
Disallow /

chinasospider

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

coolpadwebkit

Rule Path
Disallow /

crawldaddy

Rule Path
Disallow /

curl

Rule Path
Disallow /

digext

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

edisterbot

Rule Path
Disallow /

elefent

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

feeddemon

Rule Path
Disallow /

feedly

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

httpclient

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

indy

Rule Path
Disallow /

library

Rule Path
Disallow /

jaunty

Rule Path
Disallow /

java

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

jorgee

Rule Path
Disallow /

lightdeckreports

Rule Path
Disallow /

bot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

microsoft

Rule Path
Disallow /

url

Rule Path
Disallow /

control

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

obot

Rule Path
Disallow /

perl

Rule Path
Disallow /

psbot

Rule Path
Disallow /

purebot

Rule Path
Disallow /

python

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

spbot

Rule Path
Disallow /

spiderman

Rule Path
Disallow /

swebot

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

turnitinbot-agent

Rule Path
Disallow /

universalfeedparser

Rule Path
Disallow /

wangidspider

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

webdup

Rule Path
Disallow /

wget

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wsanalyzer

Rule Path
Disallow /

xbfmozilla

Rule Path
Disallow /

xenu

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yottaa

Rule Path
Disallow /

yunguance

Rule Path
Disallow /

yyspider

Rule Path
Disallow /

zmeu

Rule Path
Disallow /

googlebot

Rule Path
Disallow /

yahoo! slurp china

Rule Path
Disallow /

yahoo! slurp

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ezooms.bot

Rule Path
Disallow /
Back to top