Log In Sign Up

bollywoodhungama.in
robots.txt

Robots Exclusion Standard data for bollywoodhungama.in

Resource Scan

Scan Details

Site Domain bollywoodhungama.in
Base Domain bollywoodhungama.in
Scan Status Ok
Last Scan2025-04-07T15:25:36+00:00
Next Scan 2025-05-07T15:25:36+00:00

Last Scan

Scanned2025-04-07T15:25:36+00:00
URL http://bollywoodhungama.in/robots.txt
Redirect https://www.bollywoodhungama.com/robots.txt
Redirect Domain www.bollywoodhungama.com
Redirect Base bollywoodhungama.com
Domain IPs 35.200.201.113
Redirect IPs 23.209.46.75, 23.209.46.87, 2600:1413:b000:1e::17d1:2e4b, 2600:1413:b000:1e::17d1:2e57
Response IP 23.45.207.172
Found Yes
Hash 985c2d422a25e46e16eb70f627dcd38201d84890aece41561e4b03d92900a34a
SimHash 6c6c185aa43b

Groups

*
ahrefsbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

yandex

Rule Path
Allow /

googlebot

Rule Path
Allow /
Disallow /private/

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

majestic

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

alphabot

Rule Path
Disallow /

nutch

Rule Path
Disallow /
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-content/mu-plugins/
Disallow /readme.html
Disallow /license.txt
Disallow /wp-config.php
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /xmlrpc.php
Disallow /?p=*
Disallow /*?*
Disallow /*?s=*
Disallow /*?search=*
Allow /wp-content/uploads/
Allow /wp-admin/admin-ajax.php
Allow /wp-includes/*.css
Allow /wp-includes/*.js
Allow /wp-content/themes/*.css
Allow /wp-content/themes/*.js
Allow /wp-content/plugins/*.css
Allow /wp-content/plugins/*.js

*

Rule Path
Disallow /movie//songs/
Disallow /movie//user-reviews/
Back to top

Other Records

Field Value
sitemap https://www.bollywoodhungama.com/sitemap_index.xml
sitemap https://www.bollywoodhungama.com/critic_sitemap/critic_review_sitemap.xml
sitemap https://www.bollywoodhungama.com/post-sitemap2.xml
sitemap https://www.bollywoodhungama.com/post-sitemap101.xml
Back to top

Comments

  • Global rules for all user agents
  • Block specific malicious or unwanted bots
  • Block access to sensitive WordPress directories
  • Block access to sensitive files
  • Block query parameters that may be associated with XSS attacks or content that shouldn't be indexed
  • Allow indexing of media uploads (e.g., images)
  • Allow crawling of CSS and JavaScript files for proper page rendering
  • Block incorrect URLs with double slashes
Back to top