huluwatchparty.com
robots.txt

Robots Exclusion Standard data for huluwatchparty.com

Resource Scan

Scan Details

Site Domain huluwatchparty.com
Base Domain huluwatchparty.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan4/28/2025, 8:58:59 AM
Next Scan 6/27/2025, 8:58:59 AM

Last Successful Scan

Scanned2/5/2025, 8:57:46 AM
URL https://huluwatchparty.com/robots.txt
Redirect https://www.huluwatchparty.com/robots.txt
Redirect Domain www.huluwatchparty.com
Redirect Base huluwatchparty.com
Domain IPs 104.21.37.140, 172.67.209.40, 2606:4700:3033::6815:258c, 2606:4700:3036::ac43:d128
Redirect IPs 104.21.37.140, 172.67.209.40, 2606:4700:3033::6815:258c, 2606:4700:3036::ac43:d128
Response IP 104.21.37.140
Found Yes
Hash e052eb5817e738a28eb959ffc7f419b2030bc65b95da307a8dc08c7357328bfa
SimHash 44709d7a3753

Groups

*

Rule Path
Allow /

googlebot

Rule Path
Allow /gogooglebot/

Comments

  • Allow all user agents.
  • User agent "Googlebot" should not crawl the `http://example.com/nogooglebot/` folder or any of its subdirectories.