gossip.fr
robots.txt

Robots Exclusion Standard data for gossip.fr

Resource Scan

Scan Details

Site Domain gossip.fr
Base Domain gossip.fr
Scan Status Ok
Last Scan2024-06-21T14:51:26+00:00
Next Scan 2024-06-28T14:51:26+00:00

Last Scan

Scanned2024-06-21T14:51:26+00:00
URL https://gossip.fr/robots.txt
Redirect https://www.gossip.fr/robots.txt
Redirect Domain www.gossip.fr
Redirect Base gossip.fr
Domain IPs 104.21.67.191, 172.67.179.197, 2606:4700:3034::6815:43bf, 2606:4700:3035::ac43:b3c5
Redirect IPs 104.21.67.191, 172.67.179.197, 2606:4700:3034::6815:43bf, 2606:4700:3035::ac43:b3c5
Response IP 104.21.67.191
Found Yes
Hash 4934307a2c1b6c7cb735569d77692675d2c1773c092cc8fec5bc402944a4e306
SimHash 2d05df545695

Groups

*

Rule Path
Disallow /*?d
Disallow /*?s

mediapartners-google

Rule Path
Disallow

adsbot-google-mobile

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot

Rule Path
Disallow

googlebot-news

Rule Path
Disallow

googlebot-video

Rule Path
Disallow

adsbot-google-mobile-apps

Rule Path
Disallow

Warnings

  • 1 invalid line.
  • `https` is not a known field.