hellhorror.com
robots.txt
Robots Exclusion Standard data for hellhorror.com
Resource Scan
Scan Details
| Site Domain | hellhorror.com |
| Base Domain | hellhorror.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-07-01T16:39:33+00:00 |
| Next Scan | 2024-07-08T16:39:33+00:00 |
Last Successful Scan
| Scanned | 2023-02-04T00:08:22+00:00 |
| URL | https://hellhorror.com/robots.txt |
| Domain IPs | 172.66.40.224, 172.66.43.32, 2606:4700:3108::ac42:28e0, 2606:4700:3108::ac42:2b20 |
| Response IP | 172.66.43.32 |
| Found | Yes |
| Hash | f98737ea5ebb380f816c8aea6367e0c5aa33ffba105e514c0324642cbcaf589d |
| SimHash | 87327106e020 |
Groups
netestate ne crawler
netestate foaf crawler
netestate rss crawler
searchmetericsbot
siteliner
spbot
turnitinbot
pixray-seeker
seokicks
seokicks-robot
wbsearchbot
aranhabot
backlinkcrawler
ezooms
gsitecrawler
mail.ru
mail.ru_bot
jorgee
piplbot
ccbot
scoutjet
semrushbot
semrushbot-sa
siteauditbot
semrushbot-si
semrushbot-swa
semrushbot-ct
semrushbot-coub
infotigerbot
dotbot
dotbot
velenpublicwebcrawler
megaindex.ru
dotbotzoominfobot
moget
ichiro
naverbot
yeti
youdaobot
mtrobot
blexbot
seznambot
yandex
petalbot
obot
outbrain
magpie-crawler
baiduspider
baiduspider-video
baiduspider-image
sogou spider
barkrowler
| Rule | Path |
|---|---|
| Disallow | / |
pinterestbot
msnbot
bingbot
| Rule | Path |
|---|---|
| Disallow | /ajax |
| Disallow | /ups |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
*
| Rule | Path |
|---|---|
| Disallow | /cdn-cgi |
| Disallow | /recommends |
| Disallow | /banner.json |
| Disallow | /user/ajax |
| Disallow | /*_cp.jp*g* |
| Disallow | /imgs/videos |
| Disallow | /mrf4u |
| Disallow | /.well-known |
| Disallow | /share |
| Disallow | /ajax |
| Disallow | /widget |
| Disallow | /widgets |
| Disallow | /adcentric |
| Disallow | /adinterax |
| Disallow | /amobee |
| Disallow | /atlas |
| Disallow | /doubleclick |
| Disallow | /eyereturn |
| Disallow | /eyewonder |
| Disallow | /liquidus |
| Disallow | /klipmart |
| Disallow | /pointroll |
| Disallow | /smartadserver |
| Disallow | /unicast |
| Disallow | /viewpoint |
| Disallow | /addineyeV2.html |
| Disallow | /canvas.html |
| Disallow | /DARTIframe.html |
| Disallow | /interim.html |
| Disallow | /oggiPlayerLoader.htm |
| Disallow | /videoeggbackup.html |
| Disallow | /jivox |
| Disallow | /revjet |
| Disallow | /exponential |
| Disallow | /ipinyou |
| Disallow | /spongecell |
| Disallow | /bonzai |
| Disallow | /facebook_xd_receiver.html |
| Disallow | /readme.html |
| Disallow | /rpc_relay.html |
| Disallow | /twitterlists |
| Disallow | /admin |
| Disallow | /errors |
| Disallow | /homepage |
| Disallow | /null |
| Disallow | /admin |
| Disallow | /components |
| Disallow | /wp |
| Disallow | /mike |
| Disallow | /test |
| Disallow | /dashboard |
| Disallow | /auth/activate |
| Disallow | /auth/reset_password |
| Disallow | /login |
| Disallow | /logout |
| Disallow | /contributors |
| Disallow | /phpinfo |
| Disallow | /gmc_ |
| Disallow | /upfilees |
| Disallow | /cdn-fpw |
| Disallow | /*ezoic* |
| Disallow | /google_matched_content_* |
| Disallow | /passback-*.html |
| Disallow | /images/*%20*.jp*g* |
| Disallow | /*jp*gPage/*/Page/* |
| Disallow | /*.htmlPage/* |
| Disallow | /thumbsup_2.2 |
| Disallow | /thumbsup |
| Disallow | /*.php$ |
| Disallow | /*mraid.js$ |
| Disallow | /mod-banner.json |
| Disallow | /user/ajax/is_valid_user |
| Disallow | /user/ajax/topnav_user |
| Disallow | /ups |
| Disallow | /user/profile |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://hellhorror.com/sitemaps_articles_1.xml |
| sitemap | https://hellhorror.com/sitemaps_articles_1.xml.gz |
| sitemap | https://hellhorror.com/sitemaps_demons_1.xml |
| sitemap | https://hellhorror.com/sitemaps_killers_1.xml |
| sitemap | https://hellhorror.com/sitemaps_movies_1.xml |
| sitemap | https://hellhorror.com/sitemaps_movies_1.xml.gz |
| sitemap | https://hellhorror.com/sitemaps_theaters_1.xml |
| sitemap | https://hellhorror.com/sitemaps_theaters_1.xml.gz |
| sitemap | https://hellhorror.com/sitemaps_theaters_2.xml |
| sitemap | https://hellhorror.com/sitemaps_anime_1.xml |
| sitemap | https://hellhorror.com/sitemaps_features.xml |
| sitemap | https://hellhorror.com/sitemaps_toplists.xml |
| sitemap | https://hellhorror.com/sitemaps_ap.xml |
| sitemap | https://hellhorror.com/sitemaps_years.xml |
| sitemap | https://hellhorror.com/sitemaps_casts_1.xml |
| sitemap | https://hellhorror.com/sitemaps_directors_1.xml |
| sitemap | https://hellhorror.com/sitemaps_demons_alpha.xml |
| sitemap | https://hellhorror.com/sitemaps_misc_category_pages.xml |
| sitemap | https://hellhorror.com/sitemaps_movies_alpha.xml |
| sitemap | https://hellhorror.com/sitemaps_movie_genres.xml |
| sitemap | https://hellhorror.com/sitemaps_tv.xml |
| sitemap | https://hellhorror.com/sitemaps_movies_like.xml |
| sitemap | https://hellhorror.com/sitemaps_movies_like.xml.gz |
| sitemap | https://hellhorror.com/sitemap_index.xml |
Comments