risingsunnewspapers.co.za
robots.txt

Robots Exclusion Standard data for risingsunnewspapers.co.za

Resource Scan

Scan Details

Site Domain risingsunnewspapers.co.za
Base Domain risingsunnewspapers.co.za
Scan Status Ok
Last Scan2024-11-14T07:09:15+00:00
Next Scan 2024-11-21T07:09:15+00:00

Last Scan

Scanned2024-11-14T07:09:15+00:00
URL https://risingsunnewspapers.co.za/robots.txt
Domain IPs 104.21.33.240, 172.67.151.177, 2606:4700:3033::ac43:97b1, 2606:4700:3037::6815:21f0
Response IP 104.21.33.240
Found Yes
Hash 8bec5f180135f90533503901fd0a0a58c715b4b8a1855c0d840b89c2b0d2da52
SimHash 0a5a4b6076b8

Groups

*

Rule Path
Allow /
Disallow /specials/
Disallow /shop/
Disallow /biz/
Disallow /looklocal/
Disallow /homes/
Disallow /love/
Disallow /jobs/
Disallow /guzzle/
Disallow /adops-test-page/
Disallow /builders/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /trackback/
Disallow /comments/
Disallow */trackback/
Disallow */comments/
Disallow wp-login.php
Disallow wp-signup.php
Disallow /wp-json/

googlebot-news

Rule Path
Allow /
Disallow /afp/
Disallow /lnn/
Disallow /category/news/mandela-day/
Disallow /motoring/
Disallow /category/galleries/
Disallow /galleries/
Disallow /events/event/
Disallow /classifieds-2/
Disallow /directory/
Disallow /specials/
Disallow /shop/
Disallow /biz/
Disallow /looklocal/
Disallow /homes/
Disallow /love/
Disallow /jobs/
Disallow /guzzle/
Disallow /adops-test-page/
Disallow /builders/

trendictionbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

linkscrawler

Rule Path
Disallow /

deusu

Rule Path
Disallow /

obot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

msie8.0

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

special_archiver

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

duckduckgo-favicons-bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

netestate

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /