vivivit.com
robots.txt

Robots Exclusion Standard data for vivivit.com

Resource Scan

Scan Details

Site Domain vivivit.com
Base Domain vivivit.com
Scan Status Ok
Last Scan2024-06-16T00:21:31+00:00
Next Scan 2024-07-16T00:21:31+00:00

Last Scan

Scanned2024-06-16T00:21:31+00:00
URL https://vivivit.com/robots.txt
Redirect https://www.vivivit.com/robots.txt
Redirect Domain www.vivivit.com
Redirect Base vivivit.com
Domain IPs 3.165.82.124, 3.165.82.47, 3.165.82.80, 3.165.82.99
Redirect IPs 13.33.88.28, 13.33.88.33, 13.33.88.43, 13.33.88.45
Response IP 13.33.88.33
Found Yes
Hash 65f3a6d2b5d25211474aacbe72bcff14f1aa2e4da77fc4081a966afede18cba8
SimHash 7d3332d4d2c9

Groups

twitterbot

Rule Path
Allow /profile/public/
Allow /portfolio/public/

facebookexternalhit

Rule Path
Allow /profile/public/
Allow /portfolio/public/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

*

Rule Path
Disallow /admin/
Disallow /api/
Disallow /vivivit_proposal.pdf
Disallow /password/
Disallow /confirmation/
Disallow /profile/public/
Disallow /portfolio/public/
Disallow /mediainfo/documents
Disallow /public/403_for_company.html
Disallow /public/403_for_company.png
Disallow /public/404.html
Disallow /public/404_bg.png
Disallow /public/422.html
Disallow /public/500.html
Disallow /public/500_bg.png
Disallow /public/hataraku_release.pdf
Disallow /public/humans.txt
Disallow /public/optout.html
Disallow /public/sample.csv
Disallow /public/vivivit_event_schedule.pdf
Disallow /public/vivivit_proposal.pdf
Disallow /public/public-assets/docs/
Disallow /public/public-assets/swf/
Allow /api/users/simple_info
Allow /api/v2/job_postings

Comments

  • アクセス頻度が高すぎるのでアクセス間隔を指定する
  • public
  • files
  • directories
  • public api