myrecruit.co.za
robots.txt

Robots Exclusion Standard data for myrecruit.co.za

Resource Scan

Scan Details

Site Domain myrecruit.co.za
Base Domain myrecruit.co.za
Scan Status Ok
Last Scan2025-04-06T18:21:07+00:00
Next Scan 2025-05-06T18:21:07+00:00

Last Scan

Scanned2025-04-06T18:21:07+00:00
URL https://myrecruit.co.za/robots.txt
Redirect https://www.myrecruit.co.za/robots.txt
Redirect Domain www.myrecruit.co.za
Redirect Base myrecruit.co.za
Domain IPs 129.232.174.42
Redirect IPs 129.232.174.42
Response IP 129.232.174.42
Found Yes
Hash 74ea1bf95415df606f3ef410e684e77358a80d60cf500cb5e237ca4b6868cccb
SimHash 001cd0172593

Groups

*

Rule Path
Allow /

cursor

Rule Path
Disallow /

up.browser

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

httrack off-line browser

Rule Path
Disallow /

motor

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.myrecruit.co.za/sitemap.xml
sitemap https://www.myrecruit.co.za/sitemap.xml