Log In Sign Up

quantumtools.me
robots.txt

Robots Exclusion Standard data for quantumtools.me

Resource Scan

Scan Details

Site Domain quantumtools.me
Base Domain quantumtools.me
Scan Status Ok
Last Scan2025-12-13T02:25:34+00:00
Next Scan 2025-12-20T02:25:34+00:00

Last Scan

Scanned2025-12-13T02:25:34+00:00
URL https://quantumtools.me/robots.txt
Domain IPs 162.0.212.2
Response IP 162.0.212.2
Found Yes
Hash 5803a67dc9f63f6ab58c693d536563015f597dfc9fb75ca6410172bb12cda6fd
SimHash 4d2d5e51e79d

Groups

*

Rule Path
Allow /
Disallow /tardisrocinante/
Disallow /parsonsmalze/
Disallow /detroitchicago/
Disallow /*?gcb=*
Disallow /*?cb=*
Disallow /*.js?gcb=*
Disallow /*.js?cb=*
Disallow /*?utm_*
Disallow /*?ref=*
Disallow /*?source=*
Disallow /*?campaign=*
Disallow /temp/
Disallow /cache/
Disallow /logs/
Disallow /url-removal-guide.md
Disallow /indexing-request-guide.md
Allow *.css
Allow *.js$
Allow *.png
Allow *.jpg
Allow *.jpeg
Allow *.gif
Allow *.webp
Allow *.svg
Allow *.ico

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

duckduckbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

baiduspider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

yandexbot

Rule Path
Allow /

semrushbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3

ahrefsbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://quantumtools.me/sitemap.xml
sitemap https://quantumtools.me/sitemap-images.xml
Back to top

Comments

  • Robots.txt for QUANTUM TOOLS
  • Updated on July 19, 2025
  • Default rules for all bots
  • Block dynamically generated ad/analytics files that shouldn't be indexed
  • Block other common dynamic/temporary content
  • Discourage indexing of index.html files (prefer directory URLs)
  • Discourage indexing of index.html files (prefer directory URLs)
  • Disallow: /index.html
  • Disallow: */index.html
  • Allow CSS and JS files for proper rendering (except blocked patterns above)
  • Allow all major search engine bots with specific directives
  • Disallow low-value pages or admin areas
  • Disallow: /admin/
  • Disallow: /private/
  • Sitemap location
  • Additional instructions
  • Cache: public, max-age=86400
  • Last-Modified: July 19, 2025
Back to top