scoop.co.za
robots.txt
Robots Exclusion Standard data for scoop.co.za
Resource Scan
Scan Details
| Site Domain | scoop.co.za |
| Base Domain | scoop.co.za |
| Scan Status | Ok |
| Last Scan | 2026-03-01T16:44:30+00:00 |
| Next Scan | 2026-03-31T16:44:30+00:00 |
Last Scan
| Scanned | 2026-03-01T16:44:30+00:00 |
| URL | https://scoop.co.za/robots.txt |
| Domain IPs | 172.66.41.15, 172.66.42.241, 2606:4700:3108::ac42:290f, 2606:4700:3108::ac42:2af1 |
| Response IP | 172.66.41.15 |
| Found | Yes |
| Hash | c8e13ecbbd2196e4c5ba02904e7b4fc9402d1bb31be4c1a4552f73328301c53d |
| SimHash | 591ebc41cff8 |
Groups
petalbot
mj12bot
semrush
semrushbot
dotbot
siteexplorer
obot
ahrefsbot
yandexbot
mail.ru_bot
aihitbot
barkrowler
blexbot
slackbot-linkexpanding
baiduspider
bytespider
claudebot
bingbot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /404/ |
| Disallow | /app/ |
| Disallow | /mailshot/ |
| Disallow | /cgi-bin/ |
| Disallow | /downloader/ |
| Disallow | /errors/ |
| Disallow | /includes/ |
| Disallow | /magento/ |
| Disallow | /pkginfo/ |
| Disallow | /report/ |
| Disallow | /scripts/ |
| Disallow | /shell/ |
| Disallow | /stats/ |
| Disallow | /var/ |
| Disallow | /index.php/ |
| Disallow | /catalog/product_compare/ |
| Disallow | /catalog/category/view/ |
| Disallow | /catalog/product/view/ |
| Disallow | /catalogsearch/ |
| Disallow | /media/catalog/en |
| Disallow | /control/ |
| Disallow | /contacts/ |
| Disallow | /customize/ |
| Disallow | /newsletter/ |
| Disallow | /poll/ |
| Disallow | /sendfriend/ |
| Disallow | /tag/ |
| Disallow | /wishlist/ |
| Disallow | /catalog/product/gallery/ |
| Disallow | /cron.php |
| Disallow | /name.php |
| Disallow | /cron.sh |
| Disallow | /error_log |
| Disallow | /install.php |
| Disallow | /LICENSE.html |
| Disallow | /LICENSE.txt |
| Disallow | /LICENSE_AFL.txt |
| Disallow | /STATUS.txt |
| Disallow | /*.php$ |
| Disallow | /*?SID= |
| Disallow | /catalog/category/view/ |
| Disallow | /catalog/category/view/id/154 |
| Disallow | /contact-us-page |
| Disallow | /hidden-no-fastway-delivery/ |
| Disallow | /catalog/category/view/id/167 |
| Disallow | /catalog/category/view/id/27 |
| Disallow | /rma-details |
| Disallow | /catalogsearch/result/ |
| Disallow | /pub/media/catalog/en |
| Disallow | /rma-details-submission |
| Disallow | /contact-us-page/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://scoop.co.za/sitemap.xml |
Comments