techno-science.net
robots.txt

Robots Exclusion Standard data for techno-science.net

Resource Scan

Scan Details

Site Domain techno-science.net
Base Domain techno-science.net
Scan Status Ok
Last Scan2024-11-16T10:41:52+00:00
Next Scan 2024-11-23T10:41:52+00:00

Last Scan

Scanned2024-11-16T10:41:52+00:00
URL https://techno-science.net/robots.txt
Redirect https://www.techno-science.net/robots.txt
Redirect Domain www.techno-science.net
Redirect Base techno-science.net
Domain IPs 104.21.39.119, 172.67.145.66, 2606:4700:3030::6815:2777, 2606:4700:3033::ac43:9142
Redirect IPs 104.21.39.119, 172.67.145.66, 2606:4700:3030::6815:2777, 2606:4700:3033::ac43:9142
Response IP 172.67.145.66
Found Yes
Hash 5d6405933bce916fce2235840458b319880445cf8468b9e6138b911637d4ed2f
SimHash ecb0d3f26d13

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /outils/*
Disallow /archives/news*
Disallow /archives/articles*
Disallow /*/categorie*/*
Disallow /?onglet=archives-texte*
Disallow /?onglet=archives&date=*&
Disallow /?onglet=raz*
Disallow /?onglet=antispam*

Other Records

Field Value
crawl-delay 2

googlebot-news

Rule Path
Disallow /glossaire-definition/*
Disallow /definition/*
Disallow /boutique/*

aspiegelbot

Rule Path
Disallow /forum/*

Other Records

Field Value
crawl-delay 5