Log In Sign Up

client.okezone.com
robots.txt

Robots Exclusion Standard data for client.okezone.com

Resource Scan

Scan Details

Site Domain client.okezone.com
Base Domain okezone.com
Scan Status Ok
Last Scan2024-04-25T02:51:35+00:00
Next Scan 2024-05-25T02:51:35+00:00

Last Scan

Scanned2024-04-25T02:51:35+00:00
URL https://client.okezone.com/robots.txt
Domain IPs 104.18.30.212, 104.18.31.212, 2606:4700::6812:1ed4, 2606:4700::6812:1fd4
Response IP 104.18.30.212
Found Yes
Hash 2aaf84519924fade1286b718e0e0392981aae6e46b0070089d279620d1a4cb35
SimHash b28448856650

Groups

*

Rule Path
Disallow /
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /kemenaker/
  • Disallow: /kemenkes/
  • Disallow: /mockup/
  • Disallow: /BRI/
  • Disallow: /batan/
  • Disallow: /bkkbn/
  • Disallow: /kemenag/
  • Disallow: /btn/
  • Disallow: /bumn/
Back to top