Log In Sign Up

valueforest.co.za
robots.txt

Robots Exclusion Standard data for valueforest.co.za

Resource Scan

Scan Details

Site Domain valueforest.co.za
Base Domain valueforest.co.za
Scan Status Ok
Last Scan2025-10-17T15:22:41+00:00
Next Scan 2025-10-24T15:22:41+00:00

Last Scan

Scanned2025-10-17T15:22:41+00:00
URL https://valueforest.co.za/robots.txt
Redirect https://www.valueforest.co.za/robots.txt
Redirect Domain www.valueforest.co.za
Redirect Base valueforest.co.za
Domain IPs 206.189.246.73
Redirect IPs 206.189.246.73
Response IP 206.189.246.73
Found Yes
Hash b1b9e1c13453792397c69f839c25a6f928fa69ce4fd170f61a67cd988afe226d
SimHash 521cd050a608

Groups

*

Rule Path
Disallow

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

baiduspider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dataforseobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

meta-externalagent

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.valueforest.co.za/sitemaps/sitemap_index.xml
Back to top