Log In Sign Up

haikasoru.com
robots.txt

Robots Exclusion Standard data for haikasoru.com

Resource Scan

Scan Details

Site Domain haikasoru.com
Base Domain haikasoru.com
Scan Status Ok
Last Scan2024-09-17T23:28:15+00:00
Next Scan 2024-09-24T23:28:15+00:00

Last Scan

Scanned2024-09-17T23:28:15+00:00
URL https://haikasoru.com/robots.txt
Redirect https://www.viz.com/robots.txt
Redirect Domain www.viz.com
Redirect Base viz.com
Domain IPs 34.236.124.80, 52.54.220.55, 54.164.187.56, 54.175.104.242, 54.198.141.174
Redirect IPs 108.157.254.117, 108.157.254.24, 108.157.254.38, 108.157.254.40
Response IP 108.157.254.24
Found Yes
Hash 9d86358d27d97fe445fa55b770f17e9af7c874a72f7e3dfb1f16242c262249a7
SimHash b687fdaf776c

Groups

*

Rule Path
Disallow /search
Disallow /comments
Disallow /votes
Disallow /load_dynamic_info
Disallow /products
Disallow /profiles
Disallow *.php
Disallow /manga
Disallow /account
Disallow /news
Disallow /node
Disallow /info
Disallow /moderate

Other Records

Field Value
crawl-delay 2
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top