hometheaterforum.com
robots.txt
Robots Exclusion Standard data for hometheaterforum.com
Resource Scan
Scan Details
| Site Domain | hometheaterforum.com |
| Base Domain | hometheaterforum.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-08-06T19:09:02+00:00 |
| Next Scan | 2024-11-04T19:09:02+00:00 |
Last Successful Scan
| Scanned | 2023-10-12T16:14:35+00:00 |
| URL | https://hometheaterforum.com/robots.txt |
| Redirect | https://www.hometheaterforum.com/robots.txt |
| Redirect Domain | www.hometheaterforum.com |
| Redirect Base | hometheaterforum.com |
| Domain IPs | 104.26.6.148, 104.26.7.148, 172.67.69.4 |
| Redirect IPs | 104.26.6.148, 104.26.7.148, 172.67.69.4 |
| Response IP | 172.67.69.4 |
| Found | Yes |
| Hash | 40ab7f8f736b836911bbe705ca1386477bdd9a6c14a2edc5019e27765955d41f |
| SimHash | fb1fc651655d |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /community/find-new/ |
| Disallow | /community/whats-new/ |
| Disallow | /community/goto/ |
| Disallow | /community/posts/ |
| Disallow | /community/uix/ |
| Disallow | /community/forums/-/mark-read* |
| Disallow | /community/account/ |
| Disallow | /community/threads/*?view=reaction_score$ |
| Disallow | */unread$ |
| Disallow | */latest$ |
| Allow | /search/$ |
| Disallow | /search/* |
| Allow | */threads/post-* |
| Disallow | */post-* |
| Disallow | /testxf/ |
| Disallow | */login/ |
| Disallow | */admin.php |
| Allow | / |
slurp
yahoo! slurp
msnbot
msnbot-media
twitterbot
baiduspider
tweetedtimes bot
tweetmemebot
paperlibot
bomborabot
wf search
heritrix
expo9
netseer
addthis.com
qwantify
feed
sogou
zumbot
feedspot
acebookexternalhit
facebookexternalhit
storygizebot
scrapy
germcrawler
netvibes
vagabondo
genieo
um-ic
typhoeus
newspaper
weborama-fetcher
scooperbot
byo-2
simplepie
grouphigh
feedparser
universalfeedparser
pulsepoint
steeler
garlikcrawler
flipboard
flipboardrss
moreover
mercury
feedburner
yahoo
omgili
bloglovin
yahoocachesystem
mtalk
everyonesocialbot
ezinearticleslinkscanner
dlvr.it
pfblogs
aol
toutiaospider
photon
picofeed
um-fc
digg
synaptive
yisouspider
byo-4
feedbin
twingly
tiny
www-mechanize
commafeed
tkbot
tineye-bot
qwantify
exabot
coccoc
blexbot
umbot-ln
queryseekerspider
yandex
yandexbot
yandexmobilebot
yandeximageresizer
yandeximages
sogou spider
sogou web spider
baiduspider
youdaobot
mj12bot
spbot
seznambot
ahrefsbot
linkdex
linkdexbot
semrushbot
semrushbot-sa
cfnetwork
sputnikbot
turnitinbot
proximic
grapeshot
viglink
clickagy
yak
deusu
taboolabot
pinterestbot
psbot
plukkie
getintent
facebot
linkdexbot
vegi
istellabot
leikibot
claritybot
piplbot
feedspotbot
moatbot
yeti
contxbot
seokicks-robot
rogerbot
discordbot
tapatalk
mail.ru_bot
coccocbot
hybridbot
slack-imgproxy
twitterbot
adbeat_bot
kraken
mojeekbot
calculon
mcsbot
avkzarabotok
searchmetricsbot
uptimebot
yacybot
applebot
gigabot
g2reader-bot
seoterritory
ahrefsbot
feedbot
findxbot
wotbox
msnbot
smtbot
ocarinabot
dragonbot
zombiebot
metabot.ru
rssmicro
_zbot
uptimerobot
uipbot
zoominfobot
flamingo_searchengine
bot-pge.chlooe.com
girafabot
multiviewbot
safednsbot
gnowitnewsbot
treato-bot
pvblcbot
laserlikebot
trendiction
mediatoolkitbot
charliebot
bubing
superfeedr
focusbot
sottopop
maxpointcrawler
nutch-1.10
wonderbot
zarabotok--doma
treato-bot
pvblcbot
laserlikebot
trendiction
mediatoolkitbot
charliebot
bubing
superfeedr
focusbot
sottopop
maxpointcrawler
nutch-1.10
wonderbot
zarabotok--doma
uptimerobot
uipbot
zoominfobot
flamingo_searchengine
bot-pge.chlooe.com
girafabot
multiviewbot
safednsbot
gnowitnewsbot
treato-bot
pvblcbot
laserlikebot
trendiction
mediatoolkitbot
charliebot
bubing
superfeedr
focusbot
sottopop
maxpointcrawler
nutch-1.10
wonderbot
zarabotok--doma
treato-bot
pvblcbot
laserlikebot
trendiction
mediatoolkitbot
charliebot
bubing
superfeedr
focusbot
sottopop
maxpointcrawler
nutch-1.10
wonderbot
zarabotok--doma
viglink
clickagy
yak
deusu
taboolabot
pinterestbot
psbot
plukkie
getintent
facebot
linkdexbot
vegi
istellabot
leikibot
claritybot
piplbot
feedspotbot
moatbot
yeti
contxbot
seokicks-robot
rogerbot
discordbot
tapatalk
mail.ru_bot
coccocbot
hybridbot
slack-imgproxy
twitterbot
adbeat_bot
kraken
mojeekbot
calculon
mcsbot
avkzarabotok
searchmetricsbot
uptimebot
yacybot
applebot
gigabot
g2reader-bot
seoterritory
ahrefsbot
feedbot
findxbot
wotbox
msnbot
smtbot
ocarinabot
dragonbot
zombiebot
metabot.ru
rssmicro
_zbot
uptimerobot
uipbot
zoominfobot
flamingo_searchengine
bot-pge.chlooe.com
girafabot
multiviewbot
safednsbot
gnowitnewsbot
treato-bot
pvblcbot
laserlikebot
trendiction
mediatoolkitbot
charliebot
bubing
superfeedr
focusbot
sottopop
maxpointcrawler
wonderbot
zarabotok--doma
cliqzbot
dotbot
exabot
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 6 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.hometheaterforum.com/community/sitemap.php |
| sitemap | https://www.hometheaterforum.com/sitemap_index.xml |
Warnings
- 1 invalid line.