Log In Sign Up

minorplanetcenter.net
robots.txt

Robots Exclusion Standard data for minorplanetcenter.net

Resource Scan

Scan Details

Site Domain minorplanetcenter.net
Base Domain minorplanetcenter.net
Scan Status Ok
Last Scan2024-10-08T03:12:04+00:00
Next Scan 2024-11-07T03:12:04+00:00

Last Scan

Scanned2024-10-08T03:12:04+00:00
URL https://minorplanetcenter.net/robots.txt
Domain IPs 131.142.195.56
Response IP 131.142.195.56
Found Yes
Hash 788a551253a7bff65cce2d8c2e656f2a318ea9352a531a99d79dcd94a71db0cf
SimHash 029867f04648

Groups

*

Rule Path
Disallow /db_search/
Disallow /db_search_alt/
Disallow /db_search_dev/
Disallow /db_search/show_by_date/
Disallow /db_search/show_by_orbit_type/
Disallow /db_search/show_by_properties/
Disallow /db_search/show_object/
Disallow /db_search/show_orbit/
Disallow /iau/cbet/
Disallow /iau/css/
Disallow /iau/iauc/
Disallow /iau/iauc/05000/
Disallow /iau/iauc/06000/
Disallow /iau/iauc/08500/
Disallow /iau/iauc/09000/
Disallow /iau/icq/
Disallow /iau/mpec/K00/
Disallow /iau/mpec/K01/
Disallow /iau/mpec/K02/
Disallow /iau/mpec/K03/
Disallow /iau/mpec/K04/
Disallow /iau/mpec/K05/
Disallow /iau/mpec/K06/
Disallow /iau/mpec/K07/
Disallow /iau/mpec/K08/
Disallow /iau/mpec/K09/
Disallow /iau/mpec/K10/
Disallow /iau/mpec/K11/
Disallow /iau/mpec/K12/
Disallow /iau/mpec/K13/
Disallow /iau/mpec/K14/
Disallow /iau/mpec/K15/
Disallow /iau/mpec/K16/
Disallow /iau/nelpag/
Disallow /iau/rss/
Disallow /iau/rss/mpc/
Disallow /iau/rss/cbat/
Disallow /iau/unconf/
Disallow /iau/Ephemerides/Bright/
Disallow /iau/Ephemerides/Bright/2000/
Disallow /iau/Ephemerides/Bright/2001/
Disallow /iau/Ephemerides/Bright/2002/
Disallow /iau/Ephemerides/Bright/2003/
Disallow /iau/Ephemerides/Bright/2004/
Disallow /iau/Ephemerides/Bright/2005/
Disallow /iau/Ephemerides/Bright/2006/
Disallow /iau/Ephemerides/Bright/2007/
Disallow /iau/Ephemerides/Bright/2008/
Disallow /iau/Ephemerides/Bright/2009/
Disallow /iau/Ephemerides/Bright/2010/
Disallow /iau/Ephemerides/Bright/2011/
Disallow /iau/Ephemerides/Bright/2012/
Disallow /iau/Ephemerides/Bright/2013/
Disallow /iau/Ephemerides/Bright/2014/
Disallow /iau/Ephemerides/Bright/2015/
Disallow /iau/Ephemerides/Bright/2016/
Disallow /iau/Ephemerides/Comets/
Disallow /iau/Ephemerides/CritList/
Disallow /iau/Ephemerides/Distant/
Disallow /iau/Ephemerides/Unusual/
Disallow /images/
Disallow /js/
Disallow /stylesheets/
Disallow /tmp/
Disallow /NEOCPBlog/

Other Records

Field Value
crawl-delay 10

adidxbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

antbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

betabot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

bingpreview

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

clockwork data vault

Rule Path
Disallow /

exabot

Rule Path
Disallow /

filterdb.iss.net

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

gimme60bot

Rule Path
Disallow /

go.mail.ru/help/robots

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

guardcrwlr

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

icap-iod

Rule Path
Disallow /

kocmohabt

Rule Path
Disallow /

link checker

Rule Path
Disallow /

link sleuth

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

majestic12.co.uk/bot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

melvil rawi

Rule Path
Disallow /

microsoftpreview

Rule Path
Disallow /

mixbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

netcraft ssl

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

netscan

Rule Path
Disallow /

openlinkprofiler.org/bot

Rule Path
Disallow /

pingdom.com_bot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

rarebits

Rule Path
Disallow /

riddler

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

semanticscholarbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrush.com/bot

Rule Path
Disallow /

ssl checker

Rule Path
Disallow /

webmeup-crawler.com

Rule Path
Disallow /

slurp

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /cgi-bin/
Disallow /tmp/

mediapartners-google

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

pcore-http

Rule Path
Disallow /

pcore-http/v0.24.5

Rule Path
Disallow /

netshelter contentscan

Rule Path
Disallow /

bubing

Rule Path
Disallow /

linkdexbot/2.1

Rule Path
Disallow /

linkdexbot/2.2

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

a6-indexer

Rule Path
Disallow /
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • allowing this one temporarily for site analysis - djb 2019-04-29
  • User-Agent: W3C-checklink
  • Disallow:
Back to top

Warnings

  • `https` is not a known field.
Back to top