geekparadize.fr
robots.txt

Robots Exclusion Standard data for geekparadize.fr

Resource Scan

Scan Details

Site Domain geekparadize.fr
Base Domain geekparadize.fr
Scan Status Ok
Last Scan2024-09-22T17:15:50+00:00
Next Scan 2024-09-29T17:15:50+00:00

Last Scan

Scanned2024-09-22T17:15:50+00:00
URL https://geekparadize.fr/robots.txt
Domain IPs 51.83.25.5
Response IP 51.83.25.5
Found Yes
Hash 152bb54c7ef2cdef68ffeee25468c62932a90da7bf9bd8d65d094082ed39348f
SimHash 484250d0c9f2

Groups

*

Rule Path
Disallow /wp-*
Disallow /content/cache
Disallow /membres/*
Disallow /trackback
Disallow /*.php$
Disallow /*.inc$
Disallow /*.gz$
Disallow /wp-login.php
Allow /content/uploads/

Other Records

Field Value Comment
crawl-delay 30 10 seconds between page requests

googlebot-image

Rule Path
Allow /*

mediapartners-google*

Rule Path
Allow /*

mj12bot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

discobot

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

uptimerobot/2.0

Rule Path
Disallow /

ezooms robot

Rule Path
Disallow /

perl lwp

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

wiseguys robot

Rule Path
Disallow /

turnitin robot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

babya discoverer

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

trendkite-akashic-crawler

Rule Path
Disallow /

*

Rule Path
Disallow /CHANGELOG.txt
Disallow /cron.php
Disallow /INSTALL.mysql.txt
Disallow /INSTALL.pgsql.txt
Disallow /INSTALL.sqlite.txt
Disallow /install.php
Disallow /INSTALL.txt
Disallow /LICENSE.txt
Disallow /MAINTAINERS.txt
Disallow /update.php
Disallow /UPGRADE.txt
Disallow /xmlrpc.php

Other Records

Field Value
sitemap https://www.geekparadize.fr/sitemap_index.xml

Comments

  • Autoriser Google Image
  • Autoriser Google AdSense
  • Block MJ12bot as it is just noise
  • Block Sogou
  • Block SEOkicks
  • SEOkicks
  • Dicoveryengine.com
  • Blekkobot
  • Block BlexBot
  • Block SISTRIX
  • Block Uptime robot
  • Block Ezooms Robot
  • Block Perl LWP
  • Block netEstate NE Crawler
  • Block WiseGuys Robot
  • Block Turnitin Robot
  • Exabot
  • Babya Discoverer
  • Block CazoodleBot as it does not present correct accept content headers
  • Block MJ12bot as it is just noise
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot
  • Block trendkite-akashic-crawler
  • Files

Warnings

  • 2 invalid lines.
  • `visit-time` is not a known field.