eclkc.ohs.acf.hhs.gov
robots.txt

Robots Exclusion Standard data for eclkc.ohs.acf.hhs.gov

Resource Scan

Scan Details

Site Domain eclkc.ohs.acf.hhs.gov
Base Domain hhs.gov
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-03-12T12:34:55+00:00
Next Scan 2025-03-13T12:34:55+00:00

Last Successful Scan

Scanned2025-02-10T12:34:38+00:00
URL https://eclkc.ohs.acf.hhs.gov/robots.txt
Redirect https://headstart.gov/robots.txt?redirect=eclkc
Redirect Domain headstart.gov
Redirect Base headstart.gov
Domain IPs 2600:9000:28c2:1c00:13:a296:8200:93a1, 2600:9000:28c2:2400:13:a296:8200:93a1, 2600:9000:28c2:8c00:13:a296:8200:93a1, 2600:9000:28c2:a000:13:a296:8200:93a1, 2600:9000:28c2:ca00:13:a296:8200:93a1, 2600:9000:28c2:da00:13:a296:8200:93a1, 2600:9000:28c2:dc00:13:a296:8200:93a1, 2600:9000:28c2:f400:13:a296:8200:93a1, 3.171.198.101, 3.171.198.126, 3.171.198.17, 3.171.198.96
Redirect IPs 18.160.41.128, 18.160.41.40, 18.160.41.6, 18.160.41.90, 2600:9000:24f2:4c00:13:a296:8200:93a1, 2600:9000:24f2:600:13:a296:8200:93a1, 2600:9000:24f2:6600:13:a296:8200:93a1, 2600:9000:24f2:7000:13:a296:8200:93a1, 2600:9000:24f2:7200:13:a296:8200:93a1, 2600:9000:24f2:8000:13:a296:8200:93a1, 2600:9000:24f2:9a00:13:a296:8200:93a1, 2600:9000:24f2:ec00:13:a296:8200:93a1
Response IP 18.160.41.40
Found Yes
Hash b526ae6badf13de6008af79906788628be98d89238e8ecc364132feae5c6721d
SimHash 3152a9834e12

Groups

web-standards-spider

Rule Path
Allow /

*

Rule Path
Allow /core/*.css$
Allow /core/*.css?
Allow /core/*.js$
Allow /core/*.js?
Allow /core/*.gif
Allow /core/*.jpg
Allow /core/*.jpeg
Allow /core/*.png
Allow /core/*.svg
Allow /profiles/*.css$
Allow /profiles/*.css?
Allow /profiles/*.js$
Allow /profiles/*.js?
Allow /profiles/*.gif
Allow /profiles/*.jpg
Allow /profiles/*.jpeg
Allow /profiles/*.png
Allow /profiles/*.svg
Disallow /es/admin/
Disallow /admin/
Disallow /filter/tips
Disallow /search/
Disallow /user/register
Disallow /user/password
Disallow /user/login
Disallow /user/logout
Disallow /index.php/es/admin/
Disallow /index.php/admin/
Disallow /index.php/comment/reply/
Disallow /index.php/filter/tips
Disallow /index.php/node/add/
Disallow /index.php/search/
Disallow /index.php/user/password
Disallow /index.php/user/register
Disallow /index.php/user/login
Disallow /index.php/user/logout
Disallow /*%3B
Disallow /0x
Disallow /Customer%20Demo
Disallow /ECLKC
Disallow /eclkc
Disallow /HSIPC
Disallow /hslc
Disallow /Restricted%20Admin%20Access
Disallow /Review
Disallow /system
Disallow /Test
Disallow /Training
Disallow /User%20Documents
Disallow /web-standards
Disallow /Workflow
Disallow /sites/default/files/
Disallow /internal-use
Disallow /contributors
Disallow /archive
Disallow /archivo
Disallow /es/archive
Disallow /es/archivo
Disallow /event
Disallow /upcoming-events
Disallow /es/upcoming-events
Disallow /user
Disallow /es/user
Disallow /job-center/job
Disallow /sites/default/files/pdf/no-search
Disallow /sites/default/files/audio/transcripts
Disallow /sites/default/files/video/transcripts
Disallow /taxonomy/term
Disallow /es/taxonomy/term
Disallow /playlist
Disallow /es/playlist
Disallow /book/export
Disallow /pgor
Disallow /ods
Disallow /pdguide
Disallow /ipd/
Disallow /browse/*?

Other Records

Field Value
sitemap https://headstart.gov/sitemap.xml

Comments

  • only Drupal content for Beta search
  • CSS, JS, Images
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • Remove Hyperwave
  • Drupal excludes
  • External application excludes
  • Sitemap
  • IPD