Log In Sign Up

poprush.co.za
robots.txt

Robots Exclusion Standard data for poprush.co.za

Resource Scan

Scan Details

Site Domain poprush.co.za
Base Domain poprush.co.za
Scan Status Ok
Last Scan2026-02-01T15:08:00+00:00
Next Scan 2026-03-03T15:08:00+00:00

Last Scan

Scanned2026-02-01T15:08:00+00:00
URL https://poprush.co.za/robots.txt
Domain IPs 102.222.124.22
Response IP 102.222.124.22
Found Yes
Hash 2b522bb06200e8b7c747ba4dfe5c46d27902f6dd40e618b8218e9ec689d8abc1
SimHash 6096c893e785

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /private/
Disallow /.git/

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://www.poprush.co.za/sitemap.xml
Back to top

Comments

  • Sitemap location
  • Block access to admin areas (if any)
  • Allow all search engines to crawl the site
  • Specific directives for major search engines
  • Crawl delay (optional - helps prevent server overload)
Back to top