Log In Sign Up

pathcare.co.za
robots.txt

Robots Exclusion Standard data for pathcare.co.za

Resource Scan

Scan Details

Site Domain pathcare.co.za
Base Domain pathcare.co.za
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-03-16T17:21:38+00:00
Next Scan 2024-06-14T17:21:38+00:00

Last Successful Scan

Scanned2021-08-14T18:11:55+00:00
URL https://pathcare.co.za/robots.txt
Found Yes
Hash 7f7fcc0d9436cf7551f88d1936f0b77b1c880f4a31e5e13c0f894fe0b55b7f8a
SimHash 015d8af3a275

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /wp-content/cache
Allow /wp-content/uploads

googlebot-image

Rule Path
Allow /*

mediapartners-google*

Rule Path
Allow /*

ia_archiver

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Back to top

Comments

  • Disallow: /wp-admin
  • Disallow: /wp-includes
  • Disallow: /wp-content/plugins
  • Disallow: /wp-content/themes
  • Disallow: /wp-content/plugins/
  • Disallow: /trackback
  • Disallow: /feed
  • Disallow: /comments
  • Disallow: /category/*/*
  • Disallow: */feed
  • Google Image
  • Google AdSense
  • Internet Archiver Wayback Machine
  • digg mirror
Back to top

Warnings

  • 4 invalid lines.
  • `crawl-delay` is not a known field.
Back to top