wifi.free.fr
robots.txt

Robots Exclusion Standard data for wifi.free.fr

Resource Scan

Scan Details

Site Domain wifi.free.fr
Base Domain free.fr
Scan Status Ok
Last Scan2024-11-10T03:20:34+00:00
Next Scan 2024-11-24T03:20:34+00:00

Last Scan

Scanned2024-11-10T03:20:34+00:00
URL https://wifi.free.fr/robots.txt
Redirect https://www.free.fr/robots.txt
Redirect Domain www.free.fr
Redirect Base free.fr
Domain IPs 212.27.48.10
Redirect IPs 212.27.48.10, 2a01:e0c:1::1
Response IP 212.27.48.10
Found Yes
Hash f2c4b777426cd62d3f57e0bb39fe5943051f3f8599facd163e94c133031c1b10
SimHash 2d5572c26779

Groups

googlebot
googlebot-image
mediapartners-google
googlebot-news
googlebot-video
adsbot-google
adsbot-google-mobile
adsbot-google-mobile-apps
storebot-google
bingbot
adidxbot
duckduckbot
slurp
qwantify
pinterest

Rule Path
Allow /

*

Rule Path
Disallow /freebox/informations/avis-freebox/?page=
Allow /freebox/informations/avis-freebox/?page=1
Disallow /apps/
Disallow /*contentOnly*

buenibot

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

wget

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.free.fr/sitemap.xml

Warnings

  • 2 invalid lines.