gentside.co.uk
robots.txt

Robots Exclusion Standard data for gentside.co.uk

Resource Scan

Scan Details

Site Domain gentside.co.uk
Base Domain gentside.co.uk
Scan Status Ok
Last Scan2024-11-09T14:04:12+00:00
Next Scan 2024-11-16T14:04:12+00:00

Last Scan

Scanned2024-11-09T14:04:12+00:00
URL https://gentside.co.uk/robots.txt
Redirect https://www.gentside.co.uk/robots.txt
Redirect Domain www.gentside.co.uk
Redirect Base gentside.co.uk
Domain IPs 185.151.189.226
Redirect IPs 185.151.189.226, 2a0a:1580:2000:1a00::1f
Response IP 185.151.189.226
Found Yes
Hash ada3ca740bcfdcd4210793b5dad190615601796ed38cf09272b74345e1d95b81
SimHash 0b0c4474d503

Groups

*

Rule Path
Disallow /xhr/*
Disallow /partial/*
Disallow /landing
Disallow /offline
Disallow /passerelle_ta.php
Disallow *_pic*.html

mediapartners-google

Rule Path
Disallow

ia_archiver

Rule Path
Disallow /

spiderbot

Rule Path
Disallow /

spiderbot/nutch-1.7

Rule Path
Disallow /

*
googlebot-news

No rules defined. All paths allowed.

Other Records

Field Value
sitemap https://www.gentside.co.uk/sitemaps/sitemap.xml
sitemap https://www.gentside.co.uk/sitemaps/google_0.xml
sitemap https://www.gentside.co.uk/sitemaps/google_0.xml