Log In Sign Up

maxiapple.com
robots.txt

Robots Exclusion Standard data for maxiapple.com

Resource Scan

Scan Details

Site Domain maxiapple.com
Base Domain maxiapple.com
Scan Status Ok
Last Scan2024-09-25T15:14:33+00:00
Next Scan 2024-10-02T15:14:33+00:00

Last Scan

Scanned2024-09-25T15:14:33+00:00
URL https://maxiapple.com/robots.txt
Domain IPs 104.21.25.48, 172.67.222.179
Response IP 172.67.222.179
Found Yes
Hash dc6b39d4838f9cf42c91354b384d14c3f7238623a1fe5bd41b92e94771f42992
SimHash 634041445e9c

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /go/
Disallow /archives/
Disallow /author
Disallow /comments/feed/
Disallow /index.php
Disallow /feed/
Disallow /plugins/feedback.php
Disallow /xmlrpc.php
Disallow *?wptheme
Disallow *?nomobile
Disallow /search?
Disallow /?p=*

*

Rule Path
Allow /*.js$
Allow /*.css$

Other Records

Field Value
crawl-delay 10

mediapartners-google*

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

feedburner

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

metauri

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

twitterbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

y!j-asr

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

orangebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

yandeximages

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

brandverity/1.0

Rule Path
Disallow /

wprocketbot

Rule Path
Disallow /

rankactivelinkbot

Rule Path
Disallow /

meltawer

Rule Path
Disallow /

iceweasel

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

readability.com

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

libwww

Rule Path
Disallow /

wget

Rule Path
Disallow /

moreover

Rule Path
Disallow /

fetch

Rule Path
Disallow /

crystalsemantics

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

semantissimo

Rule Path
Disallow /

netseer

Rule Path
Disallow /

ltbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

brandwatch

Rule Path
Disallow /

xenu link sleuth

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

screaming frog

Rule Path
Disallow /

netseer

Rule Path
Disallow /

netseer.com

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

ning/1.0

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

genieo.com

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

wasalive

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

openoffice

Rule Path
Disallow /

openhosebot

Rule Path
Disallow /

metauri

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

inoreader.com-like

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

rssing.com

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

maxpointinteractive

Rule Path
Disallow /

maxpoint.com

Rule Path
Disallow /

tiscali.it

Rule Path
Disallow /

nuzzel

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

photon/1.0

Rule Path
Disallow /

livelapbot/0.2

Rule Path
Disallow /

showyoubot

Rule Path
Disallow /

j12bot/v1.4.5

Rule Path
Disallow /

chilkat/1.0.0

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /
Back to top

Comments

  • disallow all files in these directories
  • slow down Search Engine
  • Robots exclus de toute indexation.
Back to top

Warnings

  • 1 invalid line.
Back to top