puroresu.com
robots.txt
Robots Exclusion Standard data for puroresu.com
Resource Scan
Scan Details
| Site Domain | puroresu.com |
| Base Domain | puroresu.com |
| Scan Status | Ok |
| Last Scan | 2024-06-28T12:05:28+00:00 |
| Next Scan | 2024-07-05T12:05:28+00:00 |
Last Scan
| Scanned | 2024-06-28T12:05:28+00:00 |
| URL | https://puroresu.com/robots.txt |
| Redirect | https://www.puroresu.com/robots.txt |
| Redirect Domain | www.puroresu.com |
| Redirect Base | puroresu.com |
| Domain IPs | 104.21.21.38, 172.67.196.94, 2606:4700:3032::ac43:c45e, 2606:4700:3033::6815:1526 |
| Redirect IPs | 104.21.21.38, 172.67.196.94, 2606:4700:3032::ac43:c45e, 2606:4700:3033::6815:1526 |
| Response IP | 172.67.196.94 |
| Found | Yes |
| Hash | 95048a3df8b37427cefb7741994d579c95d7c195ed15187498c5a2047075d144 |
| SimHash | 1a486160f462 |
Groups
ahrefsbot
alphabot
alphaseobot
baiduspider
bubing
easouspider
ezooms
yandexbot
mauibot
mj12bot
sitesucker
httrack
sistrix
sistrix crawler
sistrix
seokicks-robot
jobs.de-robot
unisterbot
dotbot
searchmetricsbot
surveybot
seodiver
spbot
wotbox
dotbot
meanpathbot
backlinkcrawler
magpie-crawler
obot
fr-crawler
blexbot
megaindex
megaindex.ru
megaindex.com
cloudservermarketspider
trendictionbot
exabot
careerbot
lipperhey-kaus-australis
seoscanners.net
metajobbot
spiderbot
linkstats
jobboersebot
iccrawler
plista
domain re-animator bot
lipperhey-kaus-australis
turnitinbot
coccoc
um-ic
mindupbot
sg-orbiter
ccbot
qwantify
kraken
plukkie
safednsbot
haosouspider
rogerbot
openhosebot
screaming frog seo spider
thumbsniper
r6_commentreader
implisensebot
cliqzbot
aihitbot
trendictionbot
wbsearchbot
istellabot
edisterbot
discobot
huaweisymantecspider
jikespider
npbot
sogou spider
sosospider
yandex
nyu internet census
dnyzbot
semrushbot
psbot
the knowledge ai
sogou web spider
sogou
crawler.feedback
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /scgi-bin/ |
| Disallow | /images/ |
| Disallow | /banners/ |
| Disallow | /images/links/ |
| Disallow | /flags/ |
| Disallow | /history/today/ |
| Disallow | /wrestlers/photo/ |
| Disallow | /titles/ |
Warnings
- 1 invalid line.