new.hulu.com
robots.txt

Robots Exclusion Standard data for new.hulu.com

Resource Scan

Scan Details

Site Domain new.hulu.com
Base Domain hulu.com
Scan Status Ok
Last Scan2024-05-09T00:26:20+00:00
Next Scan 2024-05-16T00:26:20+00:00

Last Scan

Scanned2024-05-09T00:26:20+00:00
URL https://new.hulu.com/robots.txt
Redirect https://www.hulu.com/robots.txt
Redirect Domain www.hulu.com
Redirect Base hulu.com
Domain IPs 23.52.171.216, 23.59.168.178
Redirect IPs 23.52.171.216, 23.59.168.178
Response IP 23.52.171.216
Found Yes
Hash ef31783a598121de578ecd673e17082f690323ee197e77bc07f746c73f1e0b09
SimHash a037282d71c1

Groups

blinkx

Rule Path
Disallow /

blinkx_ff_spider

Rule Path
Disallow /

bingbot

Rule Path
Disallow /manifest.json

googlebot

Rule Path
Allow /api/2.0/static/page_content
Disallow /beta
Disallow /_qos/
Disallow /account/authenticate
Disallow /activity_feeds/
Disallow /activities/
Disallow /api/2.0/
Disallow /channels/
Disallow /discussions/search
Disallow /email$
Disallow /email?
Disallow /exstack/
Disallow /facebook_canvas/
Disallow /feed/activity/
Disallow /feed/history/
Disallow /feed/queue/
Disallow /feed/search
Disallow /feed/search/
Disallow /feed/review/
Disallow /highest_rated
Disallow /highest_rated/
Disallow /labs/caption-search
Disallow /masthead$
Disallow /masthead?
Disallow /masthead/
Disallow /playlists/flag/
Disallow /plus/check_hold
Disallow /plus/check_psn
Disallow /plus/invite_request
Disallow /plus/signup
Disallow /plus/signup/
Disallow /plus/vip
Disallow /popular$
Disallow /popular?
Disallow /popular/
Disallow /posts/flag/
Disallow /posts/list/
Disallow /posts/item/
Disallow /profiles/
Disallow /r/
Disallow /recent$
Disallow /recent?
Disallow /recent/
Disallow /released$
Disallow /released?
Disallow /released/
Disallow /reviews/update_page
Disallow /rules$
Disallow /rules?
Disallow /search?
Disallow /search/
Disallow /social/
Disallow /support/geofilter
Disallow /support/search
Disallow /support/streaming
Disallow /support/vote
Disallow /users/
Disallow /feed/
Disallow /videos/expander
Disallow /videos/search
Disallow /videos/search/
Disallow /videos/show_trailer
Disallow /videos/slider
Disallow /videos/transcripts/
Disallow /widget/player/
Disallow /browse/alphabetical/feature_films
Disallow /*.swf$
Disallow /unsubscribe_email/

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

*

Rule Path
Allow /api/2.0/static/page_content
Disallow /_qos/
Disallow /account/authenticate
Disallow /activity_feeds/
Disallow /activities/
Disallow /api/2.0/
Disallow /channels/
Disallow /discussions/search
Disallow /email$
Disallow /email?
Disallow /exstack/
Disallow /facebook_canvas/
Disallow /feed/activity/
Disallow /feed/history/
Disallow /feed/queue/
Disallow /feed/search
Disallow /feed/search/
Disallow /feed/review/
Disallow /highest_rated
Disallow /highest_rated/
Disallow /labs/caption-search
Disallow /masthead$
Disallow /masthead?
Disallow /masthead/
Disallow /mozart/
Disallow /playlists/flag/
Disallow /plus/check_hold
Disallow /plus/check_psn
Disallow /plus/invite_request
Disallow /plus/signup
Disallow /plus/signup/
Disallow /plus/vip
Disallow /popular$
Disallow /popular?
Disallow /popular/
Disallow /posts/flag/
Disallow /posts/list/
Disallow /posts/item/
Disallow /profiles/
Disallow /r/
Disallow /recent$
Disallow /recent?
Disallow /recent/
Disallow /released$
Disallow /released?
Disallow /released/
Disallow /reviews/update_page
Disallow /rules$
Disallow /rules?
Disallow /search?
Disallow /search/
Disallow /social/
Disallow /support/geofilter
Disallow /support/search
Disallow /support/streaming
Disallow /support/vote
Disallow /tempo/
Disallow /tempo-fallback/
Disallow /users/
Disallow /feed/
Disallow /videos/expander
Disallow /videos/search
Disallow /videos/search/
Disallow /videos/show_trailer
Disallow /videos/slider
Disallow /videos/transcripts/
Disallow /widget/player/
Disallow /browse/alphabetical/feature_films
Disallow /*.swf$
Disallow /unsubscribe_email

Other Records

Field Value
sitemap https://www.hulu.com/sitemap_index.xml

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-Agent: *
  • Disallow: /