Log In Sign Up

cleancut.co.za
robots.txt

Robots Exclusion Standard data for cleancut.co.za

Resource Scan

Scan Details

Site Domain cleancut.co.za
Base Domain cleancut.co.za
Scan Status Ok
Last Scan2025-05-05T17:36:14+00:00
Next Scan 2025-06-04T17:36:14+00:00

Last Scan

Scanned2025-05-05T17:36:14+00:00
URL https://cleancut.co.za/robots.txt
Domain IPs 129.232.138.122
Response IP 129.232.138.122
Found Yes
Hash 74cbadaec86ccf62580624c53fd2f2d6bc766754529dcda10123619199487e78
SimHash 415dc2f3e275

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /wp-content/cache
Disallow */trackback
Disallow */my-account
Disallow */comments
Allow /wp-content/uploads

googlebot-image

Rule Path
Disallow
Allow /*

mediapartners-google*

Rule Path
Disallow
Allow /*

ia_archiver

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Comments

  • Disallow: /wp-admin
  • Disallow: /wp-includes
  • Disallow: /wp-content/plugins
  • Disallow: /wp-content/themes
  • Disallow: /wp-content/plugins/
  • Disallow: /trackback
  • Disallow: /feed
  • Disallow: /comments
  • Disallow: /category/*/*
  • Disallow: */feed
  • Google Image
  • Google AdSense
  • Internet Archiver Wayback Machine
  • digg mirror
Back to top