Log In Sign Up

buysmartjapan.com
robots.txt

Robots Exclusion Standard data for buysmartjapan.com

Resource Scan

Scan Details

Site Domain buysmartjapan.com
Base Domain buysmartjapan.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-08-20T17:53:16+00:00
Next Scan 2024-11-18T17:53:16+00:00

Last Successful Scan

Scanned2023-04-06T14:08:03+00:00
URL https://buysmartjapan.com/robots.txt
Domain IPs 151.101.2.217
Response IP 151.101.2.217
Found Yes
Hash 0628beb07a1a408adffff9b6f1bc08e11b639a109aa840f3ebcd82d03b4a7a4d
SimHash b2a40fed5350

Groups

*

Rule Path
Disallow /mypage$
Disallow /mypage/
Disallow /mypage?
Disallow /ja/helps/calendar$
Disallow /ja/helps/calendar/
Disallow /ja/helps/calendar?
Disallow /en/helps/calendar$
Disallow /en/helps/calendar/
Disallow /en/helps/calendar?
Disallow /ko/helps/calendar$
Disallow /ko/helps/calendar/
Disallow /ko/helps/calendar?
Disallow /zh-TW/helps/calendar$
Disallow /zh-TW/helps/calendar/
Disallow /zh-TW/helps/calendar?
Disallow /zh-CN/helps/calendar$
Disallow /zh-CN/helps/calendar/
Disallow /zh-CN/helps/calendar?
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top