goup.co.uk
robots.txt

Robots Exclusion Standard data for goup.co.uk

Resource Scan

Scan Details

Site Domain goup.co.uk
Base Domain goup.co.uk
Scan Status Ok
Last Scan2024-05-23T09:24:50+00:00
Next Scan 2024-06-22T09:24:50+00:00

Last Scan

Scanned2024-05-23T09:24:50+00:00
URL https://goup.co.uk/robots.txt
Redirect https://www.goup.co.uk/robots.txt
Redirect Domain www.goup.co.uk
Redirect Base goup.co.uk
Domain IPs 104.21.78.215, 172.67.137.162, 2606:4700:3034::6815:4ed7, 2606:4700:3037::ac43:89a2
Redirect IPs 104.21.78.215, 172.67.137.162, 2606:4700:3034::6815:4ed7, 2606:4700:3037::ac43:89a2
Response IP 172.67.137.162
Found Yes
Hash ef66e828b133fec773daa44081d58c7b5831cc086d405e63a793222d44133faf
SimHash 680018104fb3

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://www.goup.co.uk/sitemap_index.xml