inspire.com
robots.txt

Robots Exclusion Standard data for inspire.com

Resource Scan

Scan Details

Site Domain inspire.com
Base Domain inspire.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-08T02:11:18+00:00
Next Scan 2024-06-22T02:11:18+00:00

Last Successful Scan

Scanned2023-01-22T11:54:43+00:00
URL https://inspire.com/robots.txt
Redirect https://www.inspire.com/robots.txt
Redirect Domain www.inspire.com
Redirect Base inspire.com
Domain IPs 44.198.80.30, 54.147.226.21
Redirect IPs 54.192.111.33, 54.192.111.34, 54.192.111.89, 54.192.111.99
Response IP 54.192.111.99
Found Yes
Hash 2b83edf582932f5e439a36e700c1830c1a4b57a191b71c6f565126a11fe02cc7
SimHash d13df1e6e324

Groups

accelobot

Rule Path
Disallow /

ah-crawler

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

fast+enterprise+crawler+6+used+by+lexisnexis+(lnbot@lexisnexis.com)

Rule Path
Disallow /

fast enterprise crawler 6 used by lexisnexis

Rule Path
Disallow /

fast+enterprise+crawler/6.7.0+(crawler@fast.no)

Rule Path
Disallow /

fast*

Rule Path
Disallow /

hotgrinds

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

newistic

Rule Path
Disallow /

nol gerbil

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

psbot

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

r6_feedfetcher

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

synthesio

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

whitevector

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

*

Rule Path
Disallow /groups/*/about/
Disallow /conditions
Disallow /api/
Disallow */sign-in/
Disallow */members/
Disallow */join/group/
Disallow */create/
Disallow */categories/*
Disallow */unfollow/
Disallow */follow/
Disallow /m/*
Disallow */reply/
Disallow /usr_res
Disallow /search*
Disallow /join/groups
Disallow /member
Disallow /resource/*
Disallow /explore/*
Disallow *NO_PERSIST%3D*
Disallow *a%3D*
Disallow *asat%3D*
Disallow *cmnt_id%3D*
Disallow *fb_page_id%3D*
Disallow *friend_uid%3D*
Disallow *group_id%3D*
Disallow *hole%3D*
Disallow *iframe%3D*
Disallow *msg_activity%3D*
Disallow *next%3D*
Disallow *nickname%3D*
Disallow *op%3D*
Disallow *order%3D*
Disallow *origin%3D*
Disallow *recipient_nickname%3D*
Disallow *recommend%3D*
Disallow *ref%3D*
Disallow *reply_id%3D*
Disallow *reply_parent_id%3D*
Disallow *reply_sort%3D*
Disallow *s_item%3D*
Disallow *s_type%3D*
Disallow *section%3D*
Disallow *slt%3D*
Disallow *sm_guid%3D*
Disallow *snapshot%3D*
Disallow *ux2%3D*
Disallow *ux3%3D*
Disallow *ux4%3D*
Disallow *ux%3D*
Disallow *uxconf%3D*
Disallow */static/*
Disallow *utm_campaign%3D*
Disallow *view_photo%3D*
Disallow *selection%3D*
Disallow *dd_query%3D*
Disallow *source%3D*
Disallow *utm_*
Disallow */journal/*
Disallow */groups/*.*
Allow /api/bc/conditions/
Allow /api/bc/search/
Allow /api/bc/discussions/*/similar/
Allow /api/popularposts
Allow /api/posts/
Allow /api/topic-content
Allow /api/treatment-data/
Allow /api/condition-data/
Allow /api/resource-data/
Allow /api/hp/treatments/
Allow /api/bc/groups/
Allow /m/Brian/*
Allow /m/TeamInspire/*

Other Records

Field Value
sitemap https://www.inspire.com/sitemap.xml

Comments

  • Sitemaps
  • Disallowed crawlers
  • All remaining user agents
  • Block query parameters for crawl efficiency
  • Whitelist