Log In Sign Up

mobile.free.fr
robots.txt

Robots Exclusion Standard data for mobile.free.fr

Resource Scan

Scan Details

Site Domain mobile.free.fr
Base Domain free.fr
Scan Status Ok
Last Scan2024-06-12T23:26:43+00:00
Next Scan 2024-07-12T23:26:43+00:00

Last Scan

Scanned2024-06-12T23:26:43+00:00
URL https://mobile.free.fr/robots.txt
Domain IPs 212.27.40.234
Response IP 212.27.40.234
Found Yes
Hash 58ce69c3cbb61b7a37c58dda1f2d06c439d84ac5ba2735b9cc722fbb3f1d990e
SimHash 5f1e5671ce43

Groups

*

Rule Path
Disallow /shop/products/?service=getCouponODR*
Disallow /*contentOnly*

googlebot-news

Rule Path
Disallow /

adidxbot disallow: /
yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

facebot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

wget

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://mobile.free.fr/sitemap.xml
Back to top

Comments

  • Instructions sur les badbots
  • Alexa Archive
  • crawler ibm watson
  • bot facebook
  • outil search metrics
  • Linux
  • Base de donnée étudiante
  • Aspirateur Web
Back to top