africa.businessinsider.com
robots.txt

Robots Exclusion Standard data for africa.businessinsider.com

Resource Scan

Scan Details

Site Domain africa.businessinsider.com
Base Domain businessinsider.com
Scan Status Ok
Last Scan2024-05-08T01:43:31+00:00
Next Scan 2024-06-07T01:43:31+00:00

Last Scan

Scanned2024-05-08T01:43:31+00:00
URL https://africa.businessinsider.com/robots.txt
Domain IPs 18.67.181.115, 18.67.181.70, 18.67.181.9, 18.67.181.94
Response IP 13.35.18.78
Found Yes
Hash 3e014ea99dc9b57bb10f7a24be11d5c0233a027ffdeebbde3c09cf0796b4780e
SimHash e9041c442fb2

Groups

*

Rule Path
Disallow /widget/*

Other Records

Field Value
sitemap https://africa.businessinsider.com/sitemap-articles-index.xml.gz
sitemap https://africa.businessinsider.com/sitemap-stories-latest.xml.gz
sitemap https://africa.businessinsider.com/sitemap-categories.xml.gz
sitemap https://africa.businessinsider.com/sitemap-topics.xml.gz