darkhorizons.com
robots.txt

Robots Exclusion Standard data for darkhorizons.com

Resource Scan

Scan Details

Site Domain darkhorizons.com
Base Domain darkhorizons.com
Scan Status Ok
Last Scan2024-06-08T03:42:19+00:00
Next Scan 2024-06-15T03:42:19+00:00

Last Scan

Scanned2024-06-08T03:42:19+00:00
URL https://darkhorizons.com/robots.txt
Redirect https://www.darkhorizons.com/robots.txt
Redirect Domain www.darkhorizons.com
Redirect Base darkhorizons.com
Domain IPs 103.242.248.67
Redirect IPs 103.242.248.67
Response IP 103.242.248.67
Found Yes
Hash 3bd1f6bf9e0056649f9f5ac0c79e0538b89db75b033082baf1ea0b100262c7d2
SimHash 251df667ee59

Groups

mediapartners-google

Rule Path
Disallow

feedly/1.0.

Rule Path
Disallow

yandex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bloglovin

Rule Path
Disallow /

blp_bbot/0.1

Rule Path
Disallow /

blp_bbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

doximity-diffbot

Rule Path
Disallow /

echoboxbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mrqe agent

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

pvblcbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

ru_bot

Rule Path
Disallow /

scooperbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

veooz

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexmobilebot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

clickagy intelligence bot v2

Rule Path
Disallow /

*

Rule Path
Disallow /gnads/
Disallow /search.php
Disallow /*bingbingplay/
Disallow /*?hop
Disallow /404/
Disallow /1997/
Disallow /1998/
Disallow /1999/
Disallow /2000/
Disallow /2001/
Disallow /2002/
Disallow /2003/
Disallow /2004/
Disallow /2005/
Disallow /2006/
Disallow /2007/
Disallow /2008/
Disallow /2009/
Disallow /admin/
Disallow /all-stories/page/
Disallow /assets/
Disallow /boxoffice/
Disallow /features/
Disallow /films/
Disallow /gallery/
Disallow /ghost/
Disallow /GhostShip
Disallow /interviews/
Disallow /news03/
Disallow /news04/
Disallow /news05/
Disallow /news06/
Disallow /news07/
Disallow /news08/
Disallow /news09/
Disallow /news4/
Disallow /news5/
Disallow /news6/
Disallow /news7/
Disallow /news8/
Disallow /news9/
Disallow /redirect/trailer/
Disallow /trailer_links/
Disallow /trailers/
Disallow /trailers/embed/
Disallow /send_to_friend/
Disallow /admin/
Disallow /images/
Disallow /includes/
Disallow /wp-content/themes/
Disallow /wp-content/uploads/
Disallow /blocks/
Disallow /modules/
Disallow /language/
Disallow /comment.php
Disallow /profiles.php
Disallow /forum/createtopic.php
Disallow /search.php
Disallow /*.php/*/print$
Disallow /calendar.php
Disallow /submit.php
Disallow /stats.php
Disallow /users.php
Disallow /*/print/
Disallow /feed/
Disallow /comments/feed
Disallow /feed/$
Disallow /*.php/
Disallow /*utm_source$
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$
Disallow /searchgoogle
Disallow /wp-admin
Disallow /wp-app.php
Disallow /wp-atom.php
Disallow /wp-blog-header.php
Disallow /wp-comments-post.php
Disallow /wp-config-sample.php
Disallow /wp-config.php
Disallow /wp-cron.php
Disallow /wp-feed.php
Disallow /wp-login.php
Disallow /wp-links-opml.php
Disallow /wp-mail.php
Disallow /wp-pass.php
Disallow /wp-rdf.php
Disallow /wp-register.php
Disallow /wp-rss.php
Disallow /wp-rss2.php
Disallow /wp-settings.php
Disallow /wp-trackback.php
Disallow /xmlrpc.php
Disallow /advertise-n.htm
Disallow /boxoffice.php
Disallow /boxoffice/
Disallow /chat-n.htm
Disallow /contact.php
Disallow /contact_us/
Disallow /exorcist/
Disallow /GhostShip.htm
Disallow /ScoobyDoo2.htm
Disallow /dvdreleases2.php
Disallow /inter/index.htm
Disallow /index2.html
Disallow /index2n.html
Disallow /interviews.php
Disallow /links/
Disallow /links.php
Disallow /news.htm
Disallow /news-n.htm
Disallow /news.htm%20
Disallow /news.php
Disallow /news/archive
Disallow /news/
Disallow /news.xml
Disallow /readers.php
Disallow /rel-us06.php
Disallow /rel-us09.php
Disallow /releasedates.php
Disallow /release.htm
Disallow /release-n.htm
Disallow /reviews-a.php
Disallow /reviews-archive.php
Disallow /scripts.php
Disallow /search-n.htm
Disallow /sendmail.php
Disallow /sony.htm
Disallow /thisweekau-n.htm
Disallow /thisweekuk-n.htm
Disallow /thisweekus-n.htm
Disallow /trailers.php
Disallow /trailers/index-n.htm
Disallow /trailers-archive.php
Disallow /tv-archive.php
Disallow /tv.php
Disallow /tv.php/
Disallow /update-n.htm
Disallow /waroftheworlds.php
Disallow /dvds/*.htm$
Disallow /dvds/*.php$
Disallow /news/*.gif$
Disallow /news/*.htm$
Disallow /news/*.jpg$
Disallow /news/*
Disallow /news10/*.htm$
Disallow /reviews/*.htm$
Disallow /reviews/*.jpg$
Disallow /reviews/*.php$
Disallow /TV/*.htm$
Disallow /tv/*.htm$
Disallow /tv/*.php$
Disallow /features/
Disallow /news3/
Disallow /shows/
Disallow /dune-prophecy-series-skips-the-volume/
Disallow /dead-reckoning-sets-home-digital-release/
Disallow /batwing-revealed-in-the-flash-toy-photo/
Disallow /review-friends-with-benefits*/
Disallow /news/502/farenheit-review-lis-explanation
Disallow /features/1618/infographic-prometheus-species-origin
Disallow /gallery/1077/?image=4
Disallow /news/3101/folger-his-balls-of-fury
Disallow /news/51/jess-s-walk-on-role-josh-takes-helm
Disallow /features/1618/infographic-prometheus-species-origin
Disallow /admin.php

ia_archiver

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.darkhorizons.com/sitemap_index.xml