Log In Sign Up

maxitendance.com
robots.txt

Robots Exclusion Standard data for maxitendance.com

Resource Scan

Scan Details

Site Domain maxitendance.com
Base Domain maxitendance.com
Scan Status Ok
Last Scan2024-09-22T16:53:29+00:00
Next Scan 2024-09-29T16:53:29+00:00

Last Scan

Scanned2024-09-22T16:53:29+00:00
URL https://maxitendance.com/robots.txt
Domain IPs 104.21.68.85, 172.67.192.123
Response IP 104.21.68.85
Found Yes
Hash 380a47edeb329879206dcd3e807be885f7c7f8b75a850f22a6c02ca52ce91137
SimHash 235945605e9e

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /go/
Disallow /archives/
Disallow /author
Disallow *?replytocom
Disallow /comments/feed/
Disallow /wp-content/uploads/HLIC/
Disallow /trackback/
Disallow /plugins/feedback.php
Disallow /index.php
Disallow /feed/
Disallow /xmlrpc.php
Disallow *?wptheme
Disallow *?nomobile
Disallow /search?
Disallow /?p=*
Allow /*.js$
Allow /*.css$

mediapartners-google*

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

y!j-asr

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

orangebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

rankactivelinkbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

meltawer

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

libwww

Rule Path
Disallow /

wget

Rule Path
Disallow /

moreover

Rule Path
Disallow /

crystalsemantics

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

semantissimo

Rule Path
Disallow /

netseer

Rule Path
Disallow /

ltbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

xenu link sleuth

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

screaming frog

Rule Path
Disallow /

netseer

Rule Path
Disallow /

netseer.com

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

ning/1.0

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

genieo.com

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

wasalive

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

openoffice

Rule Path
Disallow /

openhosebot

Rule Path
Disallow /

metauri

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

inoreader.com-like

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

smtbot/1.0

Rule Path
Disallow /

chilkat/1.0.0

Rule Path
Disallow /

*

Rule Path
Disallow /*blackhole
Disallow /?blackhole
Back to top

Comments

  • disallow all files in these directories
  • slow down Search Engine
  • Robots exclus de toute indexation.
Back to top

Warnings

  • 1 invalid line.
Back to top