nstools.fr
robots.txt
Robots Exclusion Standard data for nstools.fr
Resource Scan
Scan Details
| Site Domain | nstools.fr |
| Base Domain | nstools.fr |
| Scan Status | Ok |
| Last Scan | 2024-09-28T17:53:56+00:00 |
| Next Scan | 2024-10-05T17:53:56+00:00 |
Last Scan
| Scanned | 2024-09-28T17:53:56+00:00 |
| URL | https://nstools.fr/robots.txt |
| Domain IPs | 178.170.40.38, 2a00:c70:1:178:170:40:38:1 |
| Response IP | 178.170.40.38 |
| Found | Yes |
| Hash | 504ffe595b6221e31ebf19755a5d41e3c30bb2ac062c20bfd0407b80e7e7521a |
| SimHash | 1bfe00d5540a |
Groups
abonti
ahrefsbot
alexibot
alphaseobot
blexbot
bubing
backlinkcrawler
barkrowler
ccbot
changedetection
checkmarknetwork
cipacrawler
cliqzbot
cloudservermarketspider
compspybot
dbot
dl2bot
dataprovider
daum
domain re-animator bot
domaincrawler
domaintunocrawler
dotbot
downloadbot
exb
exabot
extlinksbot
ezooms
findxbot
g-i-g-a-b-o-t
garlikcrawler
gigabot
gobyus
grapeshotcrawler
haosouspider
iccrawler
implisensebot
infohelfer
istellabot
jobboersebot
komodiabot
kraken
lssrocketcrawler
linkstats
linkpadbot
lipperhey
mj12bot
mlbot
mxt
macinroy
mail.ru_bot
mappy
mauibot
mediabot
medialbot
megaindex
metajobbot
mixrankbot
mojeekbot
monobot
ncbot
nerdybot
netseer
netcraftsurveyagent
nimbostratus-bot
nutch
openhosebot
pagesinventory
pixray*
plista
pocketparser
r6_commentreader
red
researchbot
seodiver
seokicks-robot
smtbot
safednsbot
screaming frog seo spider
searchmetricsbot
semrushbot
semrushbot-sa
seobility
seobilitybot
seznambot
siteexplorer
sitedomain-bot
slurp
sogou web spider
spiderbot
spiderlytics
surdotlybot
surveybot
testcrawler
thumbsniper
turnitinbot
unisterbot
uptimebot
wbsearchbot
xenu’s
xenu’s link sleuth 1.1c
xovibot
youdaobot
aihitbot
alexa
archive.org_bot
betabot
ca-crawler
careerbot
coccoc
crawler4j
discobot
domaintuno
dubaiindex
findlinks
fr-crawler
http://www.nominet.org.uk/privacypolicy
icjobs
ia_archiver
ips-agent
it2media-domain-crawler
it2media-domain-crawler
jobs.de-robot
linkdexbot
ltx71
magpie-crawler
meanpathbot
memorybot
mindupbot
netestate
netestate
obot
pagespeedbot
plukkie
proximic
psbot
publiclibraryarchive
rogerbot
ssearch
seoscanners
sg-orbiter
sistrix
sitecheck.internetseer.com
sogou
sogou spider
spbot
ssearch_bot
tracemyfile
trendictionbot
um-ic
vebidoobot
waybackarchive.org
wonderbot
woriobot
wotbox
xxx
yoozbot
| Rule | Path |
|---|---|
| Disallow | / |
bingbot
msnbot
msnbot-media
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
*
| Rule | Path |
|---|---|
| Disallow |
Warnings
- 3 invalid lines.
Comments