Log In Sign Up

smcb.jp
robots.txt

Robots Exclusion Standard data for smcb.jp

Resource Scan

Scan Details

Site Domain smcb.jp
Base Domain smcb.jp
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-29T14:49:43+00:00
Next Scan 2024-12-28T14:49:43+00:00

Last Successful Scan

Scanned2024-06-02T14:47:42+00:00
URL https://smcb.jp/robots.txt
Domain IPs 108.157.254.14, 108.157.254.30, 108.157.254.56, 108.157.254.74
Response IP 108.157.254.30
Found Yes
Hash 96d0ddb5599c2377766427caea34c4c993d897a26b9b361fd608b0977a8a1e71
SimHash b2850d856550

Groups

*

No rules defined. All paths allowed.

Back to top

Other Records

Field Value
sitemap https://sitemap.smcb.jp/sitemaps/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top