jai.com
robots.txt

Robots Exclusion Standard data for jai.com

Resource Scan

Scan Details

Site Domain jai.com
Base Domain jai.com
Scan Status Ok
Last Scan2025-09-05T12:25:49+00:00
Next Scan 2025-09-12T12:25:49+00:00

Last Scan

Scanned2025-09-05T12:25:49+00:00
URL https://jai.com/robots.txt
Redirect https://www.jai.com/robots.txt
Redirect Domain www.jai.com
Redirect Base jai.com
Domain IPs 185.173.20.33
Redirect IPs 185.173.20.33
Response IP 185.173.20.33
Found Yes
Hash 750c1e182e43a69f52f0453cdd7e10d9a5cb973367b1406182598a08e1149c87
SimHash a36a1933c51b

Groups

*

Rule Path Comment
Disallow /cpresources/ -
Disallow /vendor/ -
Disallow /.env -
Disallow /cache/ -
Disallow /partner/ Block the /partner.
Disallow *.pdf Block pdf files. Non-standard but works for major search engines.
Disallow /de/ -

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

siteauditbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.jai.com/sitemaps-1-sitemap.xml
sitemap https://www.jai.com/jp/sitemaps-1-sitemap.xml
sitemap https://www.jai.com/cn/sitemaps-1-sitemap.xml
sitemap https://www.jai.com/kr/sitemaps-1-sitemap.xml
sitemap https://www.jai.com/de/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://www.jai.com/
  • live - don't allow web crawlers to index cpresources/ or vendor/
  • SemrushBot
  • SiteAuditBot by Semrush