blackoak.tv
robots.txt

Robots Exclusion Standard data for blackoak.tv

Resource Scan

Scan Details

Site Domain blackoak.tv
Base Domain blackoak.tv
Scan Status Ok
Last Scan2024-11-11T22:16:54+00:00
Next Scan 2024-11-18T22:16:54+00:00

Last Scan

Scanned2024-11-11T22:16:54+00:00
URL https://blackoak.tv/robots.txt
Redirect https://www.blackoak.tv/robots.txt
Redirect Domain www.blackoak.tv
Redirect Base blackoak.tv
Domain IPs 18.155.68.27, 18.155.68.45, 18.155.68.49, 18.155.68.79
Redirect IPs 76.76.21.123, 76.76.21.142
Response IP 76.76.21.98
Found Yes
Hash c7cca4d8f6f11124a64aaa8aadf1ab2d4ea91586170444f602a3b909aedca83a
SimHash 0840cb80c5f3

Groups

*

Rule Path
Disallow /login

*

Rule Path
Disallow /subscribe

*

Rule Path
Disallow /order

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.example.com/sitemap.xml

Comments

  • Block all crawlers for /login
  • Block all crawlers for /subscribe
  • Block all crawlers for /order
  • Allow all crawlers