Log In Sign Up

job.tsunoru.jp
robots.txt

Robots Exclusion Standard data for job.tsunoru.jp

Resource Scan

Scan Details

Site Domain job.tsunoru.jp
Base Domain tsunoru.jp
Scan Status Ok
Last Scan2025-08-15T10:53:21+00:00
Next Scan 2025-09-14T10:53:21+00:00

Last Scan

Scanned2025-08-15T10:53:21+00:00
URL https://job.tsunoru.jp/robots.txt
Domain IPs 104.26.14.84, 104.26.15.84, 172.67.68.140, 2606:4700:20::681a:e54, 2606:4700:20::681a:f54, 2606:4700:20::ac43:448c
Response IP 104.26.14.84
Found Yes
Hash 05538e4acf61f90f522a1340503702173e360973cc126c4bc3d405fe190bc55d
SimHash d33ed840eaa8

Groups

ahrefsbot
amazonbot
awariorssbot
awariosmartbot
barkrowler
blexbot
bytespider
claudebot
dalvik
dataforseobot
dotbot
gptbot
imagesiftbot
mj12bot
petalbot
semrushbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120
Back to top

Other Records

Field Value
sitemap https://job.tsunoru.jp/sitemap.xml
Back to top

Comments

  • -----------------------------------------
  • Disallow・Allow系の指定をこの下に置く
  • -----------------------------------------
  • -----------------------------------------
  • Crawl-delayの指定をこの下に置く
  • このブロックにはDisallow系の指定は一切記載しない
  • -----------------------------------------
Back to top

Warnings

  • `user-agnet` is not a known field.
Back to top