Log In Sign Up

adr-tool.com
robots.txt

Robots Exclusion Standard data for adr-tool.com

Resource Scan

Scan Details

Site Domain adr-tool.com
Base Domain adr-tool.com
Scan Status Ok
Last Scan2025-11-16T07:02:40+00:00
Next Scan 2025-11-23T07:02:40+00:00

Last Scan

Scanned2025-11-16T07:02:40+00:00
URL https://adr-tool.com/robots.txt
Domain IPs 195.78.67.52
Response IP 195.78.67.52
Found Yes
Hash 1d14e619c3d71237f8fa573d0c393a7f320163e7edbbbaa6251442275a5897cf
SimHash 000618440116

Groups

*

Rule Path
Disallow /apk/
Disallow /cache/
Disallow /smarty/
Disallow /classes/
Disallow /rest/
Disallow /rest-api/
Disallow /rest-api-new/
Disallow /css/
Disallow /js/
Disallow /images/favicons/
Disallow /cache/
Allow /images/adr-tool-logo*
Allow /css/adr-tool.css
Allow /android-chrome-192x192.png
Allow /favicons/android-icon-192x192.png

gptbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

chatgpt-user

Rule Path
Allow /

ccbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

claude-web

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

applebot

Rule Path
Allow /

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

slurp

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://adr-tool.com/sitemap.xml
sitemap https://adr-tool.com/sitemap-images.xml
Back to top

Comments

  • ADR Tool - Robots.txt Optimized for LLM Crawlers and Search Engines
  • Professional dangerous goods transportation compliance platform
  • General crawling guidelines
  • Allow essential resources for LLM understanding
  • Specific guidelines for AI/LLM crawlers
  • Search engine specific optimizations
  • Social media crawlers for better sharing
  • Sitemap locations
  • Website information for AI understanding
  • Primary content: ADR dangerous goods transportation compliance
  • Target audience: Transportation professionals, logistics managers, safety officers
  • Main features: UN number search, hazmat classification, consultant directory
  • Languages: English (primary), multilingual support
  • Update frequency: Daily for UN numbers, weekly for general content
  • Content type: Professional reference, regulatory compliance, safety information
Back to top