Log In Sign Up

zscaler.jp
robots.txt

Robots Exclusion Standard data for zscaler.jp

Resource Scan

Scan Details

Site Domain zscaler.jp
Base Domain zscaler.jp
Scan Status Ok
Last Scan2024-05-29T18:46:03+00:00
Next Scan 2024-06-28T18:46:03+00:00

Last Scan

Scanned2024-05-29T18:46:03+00:00
URL https://zscaler.jp/robots.txt
Redirect https://www.zscaler.jp/robots.txt
Redirect Domain www.zscaler.jp
Redirect Base zscaler.jp
Domain IPs 104.18.24.120, 104.18.25.120
Redirect IPs 104.18.24.120, 104.18.25.120, 2606:4700::6812:1878, 2606:4700::6812:1978
Response IP 104.18.24.120
Found Yes
Hash 1ed8ea223e29b8f107f4318f15e333a59f6bce719090aa6a13737d652dd39c87
SimHash 11843f59c6c2

Groups

*

Rule Path
Allow /resources?topic=*
Allow /blogs?type=*
Allow /blogs?topic=*
Allow /resources?type=*
Allow /zpedia?type=*
Allow /resources/webinars?type=*
Allow /resources/webinars?topic=*
Disallow *_archive.html
Disallow /community-api-webhook.php
Disallow /preview_*
Disallow /*?*
Allow /_next/image?url=*
Allow /core/*.css$
Allow /core/*.css?
Allow /core/*.js$
Allow /core/*.js?
Allow /core/*.gif
Allow /core/*.jpg
Allow /core/*.jpeg
Allow /core/*.png
Allow /core/*.svg
Allow /profiles/*.css$
Allow /profiles/*.css?
Allow /profiles/*.js$
Allow /profiles/*.js?
Allow /profiles/*.gif
Allow /profiles/*.jpg
Allow /profiles/*.jpeg
Allow /profiles/*.png
Allow /profiles/*.svg
Disallow /core/
Disallow /profiles/
Disallow /README.txt
Disallow /web.config
Disallow /admin/
Disallow /comment/reply/
Disallow /filter/tips
Disallow /node/add/
Disallow /search/
Disallow /user/register/
Disallow /user/password/
Disallow /user/login/
Disallow /user/logout/
Disallow /index.php/admin/
Disallow /index.php/comment/reply/
Disallow /index.php/filter/tips
Disallow /index.php/node/add/
Disallow /index.php/search/
Disallow /index.php/user/password/
Disallow /index.php/user/register/
Disallow /index.php/user/login/
Disallow /index.php/user/logout/
Allow /themes/*.css$
Allow /themes/*.css?
Allow /themes/*.js$
Allow /themes/*.js?
Allow /themes/*.gif
Allow /themes/*.jpg
Allow /themes/*.jpeg
Allow /themes/*.png
Allow /themes/*.svg
Allow /themes/*.woff2
Disallow /includes/
Disallow /misc/
Disallow /modules/
Disallow /profiles/
Disallow /scripts/
Disallow /themes/
Disallow /CHANGELOG.txt
Disallow /cron.php
Disallow /INSTALL.mysql.txt
Disallow /INSTALL.pgsql.txt
Disallow /INSTALL.sqlite.txt
Disallow /install.php
Disallow /INSTALL.txt
Disallow /LICENSE.txt
Disallow /MAINTAINERS.txt
Disallow /update.php
Disallow /UPGRADE.txt
Disallow /xmlrpc.php
Disallow /admin/
Disallow /comment/reply/
Disallow /filter/tips/
Disallow /node/add/
Disallow /search/
Disallow /user/register/
Disallow /user/password/
Disallow /user/login/
Disallow /user/logout/
Disallow /*?*
Disallow /cdn/*
Disallow /drupal
Disallow /drupal/*
Disallow *_archive.html
Disallow /community-api-webhook.php
Disallow /fr$
Disallow /fr/*
Disallow /de$
Disallow /de/*
Disallow /es$
Disallow /es/*
Disallow /it$
Disallow /it/*
Disallow /es-mx$
Disallow /es-mx/*
Disallow /ja$
Disallow /ja/*
Disallow /pt-br$
Disallow /pt-br/*
Disallow /en$
Disallow /en/*
Disallow /media/oembed
Disallow /*/media/oembed
Disallow /index.php/media/oembed
Disallow /index.php/*/media/oembed

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.zscaler.jp/sitemap_index.xml
Back to top

Comments

  • Zscaler: leftovers
  • Next settings
  • Query strings
  • Allow _next images
  • To be remove after proxy
  • CSS, JS, Images
  • Directories
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • Zscaler: CSS, JS, Images
  • Zscaler: Directories
  • Zscaler: Files
  • Zscaler: Paths (clean URLs)
  • Query strings
  • Zscaler: Disallow cdn URLs
  • Zscaler: GWT Error
  • Zscaler: leftovers
  • Disallow for proxy pages of m7 and revolutionaries
  • To be remove after proxy
  • CSS, JS, Images ends
Back to top