gralon.net
robots.txt

Robots Exclusion Standard data for gralon.net

Resource Scan

Scan Details

Site Domain gralon.net
Base Domain gralon.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-10T14:26:31+00:00
Next Scan 2024-06-17T14:26:31+00:00

Last Successful Scan

Scanned2024-02-23T09:27:23+00:00
URL https://gralon.net/robots.txt
Redirect https://www.gralon.net/robots.txt
Redirect Domain www.gralon.net
Redirect Base gralon.net
Domain IPs 164.132.167.149, 2001:41d0:1008:1b95::1
Redirect IPs 104.26.2.242, 104.26.3.242, 172.67.74.92, 2606:4700:20::681a:2f2, 2606:4700:20::681a:3f2, 2606:4700:20::ac43:4a5c
Response IP 104.26.3.242
Found Yes
Hash eca7b62f8cd1a440f50f70da7ee5acdc7d835fb78fab7ad39e8b0ebe7f546236
SimHash f25914e0cf33

Groups

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

sogou spider

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

discobot

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

uptimerobot/2.0

Rule Path
Disallow /

ezooms robot

Rule Path
Disallow /

perl lwp

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

wiseguys robot

Rule Path
Disallow /

turnitin robot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

babya discoverer

Rule Path
Disallow /

*

Rule Path
Disallow /articles/selection-articles.html?*
Disallow /actualites/*
Disallow /actualites/actualites-tag.php?*
Disallow /articles/articles-tags.php?*
Disallow /partenaires.htm
Disallow /mentions-legales.htm
Disallow /politique-de-confidentialite.htm
Disallow /pop-carte.htm?*
Disallow /pop-carte-open.htm?*
Disallow /pop-carte-ads.htm?*
Disallow /plan-ville/zoom-plan.htm?*
Disallow /cdiscount.htm
Disallow /contact_mail.php
Disallow /contact_mail.v4.php
Disallow /aff_tel.php
Disallow /aff_tel.v4.php
Disallow /envoi_page_mail.php
Disallow /transfert.php
Disallow /mots-web/*
Disallow /location-vacances/disponibilite.htm
Disallow /evenements/redirect.htm
Disallow /ajax/
Disallow /charts/
Disallow /img-bandeau-pub/beauty-french-touch.swf
Allow /

Other Records

Field Value
sitemap https://www.gralon.net/sitemap.xml
sitemap https://www.gralon.net/sitemap-1.xml
sitemap https://www.gralon.net/sitemap-2.xml
sitemap https://www.gralon.net/sitemap-3.xml
sitemap https://www.gralon.net/sitemap-4.xml
sitemap https://www.gralon.net/sitemap-5.xml
sitemap https://www.gralon.net/sitemap-6.xml
sitemap https://www.gralon.net/sitemap-7.xml
sitemap https://www.gralon.net/sitemap-8.xml
sitemap https://www.gralon.net/sitemap-9.xml
sitemap https://www.gralon.net/sitemap-10.xml
sitemap https://www.gralon.net/sitemap-11.xml
sitemap https://www.gralon.net/sitemap-12.xml
sitemap https://www.gralon.net/sitemap-13.xml
sitemap https://www.gralon.net/sitemap-14.xml
sitemap https://www.gralon.net/sitemap-articles.xml
sitemap https://www.gralon.net/sitemap-tourisme.xml

Comments

  • Block MJ12bot as it is just noise
  • User-agent: MJ12bot
  • Disallow: /
  • Block Ahrefs
  • User-agent: AhrefsBot
  • Disallow: /
  • Block Sogou
  • Block SEOkicks
  • SEOkicks
  • Dicoveryengine.com
  • Blekkobot
  • Block BlexBot
  • Block SISTRIX
  • Block Uptime robot
  • Block Ezooms Robot
  • Block Perl LWP
  • Block netEstate NE Crawler
  • Block WiseGuys Robot
  • Block Turnitin Robot
  • Exabot
  • Trendiction Robot
  • Babya Discoverer

Warnings

  • 1 invalid line.