nyt.org.uk
robots.txt

Robots Exclusion Standard data for nyt.org.uk

Resource Scan

Scan Details

Site Domain nyt.org.uk
Base Domain nyt.org.uk
Scan Status Ok
Last Scan2024-03-23T16:58:08+00:00
Next Scan 2024-04-22T16:58:08+00:00

Last Scan

Scanned2024-03-23T16:58:08+00:00
URL https://nyt.org.uk/robots.txt
Redirect https://www.nyt.org.uk:443/robots.txt
Redirect Domain www.nyt.org.uk
Redirect Base nyt.org.uk
Domain IPs 52.30.231.150
Redirect IPs 13.35.18.40, 13.35.18.61, 13.35.18.69, 13.35.18.85
Response IP 13.35.18.69
Found Yes
Hash fd4401ebe51485bca2b64226c20a0c4999258448d1f2ce2e785866c19b1586d2
SimHash f1145d40c991

Groups

*

Rule Path
Disallow /admin/
Disallow /dev/
Disallow /search/

Other Records

Field Value
sitemap https://www.nyt.org.uk/sitemap