glwengineering.co.uk
robots.txt

Robots Exclusion Standard data for glwengineering.co.uk

Resource Scan

Scan Details

Site Domain glwengineering.co.uk
Base Domain glwengineering.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-25T09:20:19+00:00
Next Scan 2025-12-25T09:20:19+00:00

Last Successful Scan

Scanned2025-10-03T03:35:24+00:00
URL https://glwengineering.co.uk/robots.txt
Redirect https://www.glwengineering.co.uk/robots.txt
Redirect Domain www.glwengineering.co.uk
Redirect Base glwengineering.co.uk
Domain IPs 185.151.30.158, 2a07:7800::158
Redirect IPs 185.151.30.158, 2a07:7800::158
Response IP 185.151.30.158
Found Yes
Hash cdef8493a5ec1e29ee8467309f3136aa6f4eafb229a95de54f92d1813dc84e8d
SimHash e9289c42eb93

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

semrushbot-sa

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.glwengineering.co.uk/sitemap_index.xml