ncuk.net.uk
robots.txt

Robots Exclusion Standard data for ncuk.net.uk

Resource Scan

Scan Details

Site Domain ncuk.net.uk
Base Domain ncuk.net.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2025-08-23T05:28:07+00:00
Next Scan 2025-11-21T05:28:07+00:00

Last Successful Scan

Scanned2022-10-05T22:16:58+00:00
URL http://ncuk.net.uk/robots.txt
Redirect https://www.ncuk.com/robots.txt
Redirect Domain www.ncuk.com
Redirect Base ncuk.com
Response IP 80.249.96.54
Found Yes
Hash 5d2758e08e330bc0619462fef88dcb195f258459a9b0aadeffeb72eca294f183
SimHash 581c8d42c2f0

Groups

*

Rule Path
Disallow /payment*
Disallow /terms*
Disallow /legal*

aboutusbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

panopta.com

Rule Path
Disallow /

searchmetrics

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

dataprovider.com

Rule Path
Disallow /

yandex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

doubt

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

xenu's

Rule Path
Disallow /

xenu's link sleuth 1.1c

Rule Path
Disallow /