Log In Sign Up

getdkan.org
robots.txt

Robots Exclusion Standard data for getdkan.org

Resource Scan

Scan Details

Site Domain getdkan.org
Base Domain getdkan.org
Scan Status Ok
Last Scan2025-09-21T12:37:23+00:00
Next Scan 2025-10-21T12:37:23+00:00

Last Scan

Scanned2025-09-21T12:37:23+00:00
URL https://getdkan.org/robots.txt
Domain IPs 104.21.92.61, 172.67.187.22, 2606:4700:3030::6815:5c3d, 2606:4700:3033::ac43:bb16
Response IP 104.21.92.61
Found Yes
Hash 735cdcc5aa803c1fa1f18e912a90b299fa195b637fd28f3a6c331048ee9097da
SimHash 2a56155f4cbb

Groups

*
drush

Rule Path
Disallow /admin/

featuresbot

Rule Path
Disallow /revert/

panelscrawler

Rule Path
Disallow /layout-chaos/

migratebot

Rule Path
Disallow /old-dkan/

jsonapi

Rule Path
Disallow /private/

halbot

Rule Path
Disallow /human-override/

viewsinfinitescroll

Rule Path
Disallow /bottomless/
Disallow /core/
Disallow /vendor/
Disallow /profiles/
Disallow /sites/default/files/private/
Back to top

Other Records

Field Value
sitemap https://getdkan.org/sitemap.xml
Back to top

Comments

  • robots.txt for GetDKAN.org — a Drupal distro walks into a website...
  • User Agents
  • --- General Access ---
  • Hello bots, welcome to the land of open data. Please don't DoS the Views.
  • --- Specific Agents ---
  • You're powerful. But stay out of the UI, command line hero.
  • Our features are already overridden. Don’t make it worse.
  • This layout is held together by hopes and ctools. Proceed with caution.
  • You already migrated it once. Let it rest.
  • You're a helpful spec, but some nodes are not for sharing.
  • Sorry HAL, no 2000s-era takeovers allowed.
  • This page never ends. Save your bandwidth.
  • Sitemaps
  • As generated by a very well-behaved cron(ish) job.
  • Web Files
  • Just pretend this is a full Drupal site. We're distro-pilled.
  • Final Reminders
  • Don't hack core. Seriously.
  • For more information, see: https://www.drupal.org/docs/robots-txt
  • Or just open an issue in the queue and hope someone responds. Classic.
  • ❤️ With love, from the DKAN maintainers and all our yml config files.
Back to top