Log In Sign Up

pro.realtor.com
robots.txt

Robots Exclusion Standard data for pro.realtor.com

Resource Scan

Scan Details

Site Domain pro.realtor.com
Base Domain realtor.com
Scan Status Ok
Last Scan2024-11-12T02:54:47+00:00
Next Scan 2024-11-19T02:54:47+00:00

Last Scan

Scanned2024-11-12T02:54:47+00:00
URL https://pro.realtor.com/robots.txt
Domain IPs 13.35.210.44, 13.35.210.51, 13.35.210.75, 13.35.210.87
Response IP 13.35.210.51
Found Yes
Hash d93d639e1ab7cd9471c8ceae8659df6bd171d6603ce51d5a29e58630f35628d8
SimHash 2a02d65ae15d

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

*

Rule Path
Disallow /8058/
Disallow /addeliverymodule/
Disallow /advice/wp-content/themes/realtor/nobots/
Disallow /amenities?
Disallow /apartments/*?disp=2*
Disallow /apartments/*?view=map%2Fpg-*
Disallow /apartments/*/*-rentals
Disallow /apartments/*/cat-friendly
Disallow /apartments/*/dog-friendly
Disallow /apartments/*/*/pg-*
Disallow /apartments/near-me*
Disallow /content
Disallow /Content
Disallow /content/
Disallow /Content/
Disallow /diagnostics
Disallow /Diagnostics
Disallow /diagnostics/
Disallow /Diagnostics/
Disallow /foresee
Disallow /Foresee
Disallow /foresee/
Disallow /Foresee/
Disallow /home_advisor
Disallow /home_advisor/
Disallow /homes_around
Disallow /homemade/wp-content/plugins/
Disallow /homemade/wp-admin/
Disallow /homemade/readme.html
Disallow /homemade/refer/
Disallow /international/print/
Disallow /international/us/
Disallow /international/*?sort=distance
Disallow /international/*?sort=date
Disallow /*?lang=
Disallow /map
Disallow /Map
Disallow /map/
Disallow /Map/
Disallow /*?map_layer
Disallow /market-insights/
Disallow /Market-Insights/
Disallow /MARKET-INSIGHTS/
Disallow /median_prices
Disallow /median_prices?
Disallow /mortgage/home-loan/*?price=
Disallow /mortgage_payment
Disallow /mortgage/rates/*property_price%3D
Disallow /movies
Disallow /Movies
Disallow /movies/
Disallow /Movies/
Disallow /music
Disallow /Music
Disallow /music/
Disallow /Music/
Disallow /myhome/*_M
Disallow /myhome/M*
Disallow /*?ncs=
Disallow /network-analysis
Disallow /newhomesconstruction/*
Disallow /newhomecommunities/near-me*
Disallow /news/wp-content/themes/realtor/nobots/
Disallow /*not_found
Disallow /*not-found
Disallow /*not%26nbsp%3Bfound
Disallow /*Not_Found
Disallow /*Not-Found
Disallow /*Not%26nbsp%3BFound
Disallow /o/
Disallow /*pgsz%3D
Disallow /polaris-ldp/*
Disallow /polaris-ldp
Disallow /*?pos=
Disallow /print
Disallow /Print
Disallow /print/
Disallow /Print/
Disallow /programs
Disallow /Programs
Disallow /programs/
Disallow /Programs/
Disallow /properties/topleadform
Disallow /property-overview
Disallow /propertyrecord-search/*/beds-
Disallow /propertyrecord-search/*/type-
Disallow /propertyrecord-search/*/pg-*/pg-
Disallow /propertyrecord-search/*undefined
Disallow /public
Disallow /Public
Disallow /public/
Disallow /Public/
Disallow /rdc_user_check.js
Disallow /realestateagents/*agentname-
Disallow /realestateagents/58386a0cf70a250001177f2d
Disallow /realestateandhomes-detail/187-Harness-Downs-Rd_Port-Matilda_PA_16870_M47586-18534
Disallow /realestateandhomes-detail/4581-S-Valdai-Way_Aurora_CO_80015_M21324-97292
Disallow /realestateandhomes-detail/5845-Meadowlane-St_New-Port-Richey_FL_34652_M68583-60477
Disallow /realestateandhomes-detail/1100-Pinehurst-Dr_Chapel-Hill_NC_27517_M64159-81306
Disallow /realestateandhomes-detail/2027-Portsmouth-Dr_Richardson_TX_75082_M83039-98371
Disallow /realestateandhomes-detail/2137-W-Roosevelt-Dr_Milwaukee_WI_53209_M82248-14449
Disallow /realestateandhomes-detail/2137-W-Roosevelt-Dr_Milwaukee_WI_53209_M90548-95731
Disallow /realestateandhomes-detail/650-Church-Rd_Flourtown_PA_19031_M35761-59786
Disallow /realestateandhomes-detail/1132-Jason-Dr_Lompoc_CA_93436_M11599-85143
Disallow /realestateandhomes-detail/M1159985143
Disallow /realestateandhomes-detail/undefined
Disallow /realestateandhomes-print/
Disallow /realestateandhomes-search/*/*/pg-
Disallow /realestateandhomes-search/*?from_ldp=true
Disallow /*?cid=
Disallow /realestateandhomes-search/*/pg-*/show-recently-sold
Disallow /realestateandhomes-search/*/radius-
Disallow /realestateandhomes-search/undefined
Disallow /realestateteam/bright-birch-real-estate__mn_1639730
Disallow /rentals/*?view=map*
Disallow /rentals/api/optimizely.js
Disallow /restricted
Disallow /Restricted
Disallow /restricted/
Disallow /Restricted/
Disallow /*?row=
Disallow /similar_homes
Disallow /soldhomeprices/*/beds-
Disallow /soldhomeprices/*/type-
Disallow /srp_filter/
Disallow /*?view=nd
Disallow /wcf
Disallow /WCF
Disallow /wcf/
Disallow /WCF/
Disallow /webservice
Disallow /WebService
Disallow /webservice/
Disallow /WebService/
Disallow /webusercontrols
Disallow /WebUserControls
Disallow /webusercontrols/
Disallow /WebUserControls/
Disallow /widgets
Disallow /Widgets
Disallow /widgets/
Disallow /Widgets/
Disallow /YfcuVaVBkfnzl2j6UbEcK0Oiqkv88jx6w/

yandex

Rule Path
Disallow /api/v1/hulk

peer39_crawler

Rule Path
Disallow /

connatix

Rule Path
Disallow /

sogou

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow

googleother

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

facebookexternalhit

Rule Path
Allow /realestateandhomes-detail/*?cid=
Allow /realestateandhomes-search/*?cid=

twitterbot

Rule Path
Allow /realestateandhomes-detail/*?cid=
Allow /realestateandhomes-search/*?cid=

pinterestbot

Rule Path
Allow /realestateandhomes-detail/*?cid=
Allow /realestateandhomes-search/*?cid=
Back to top

Other Records

Field Value
sitemap https://www.realtor.com/mortgage/sitemap/mortgagesitemap.xml
sitemap https://www.realtor.com/realestateandhomes-detail/sitemap-rss-new/rss-new-index.xml
sitemap https://www.realtor.com/realestateandhomes-detail/sitemap-rss-open-house/rss-open-house-index.xml
sitemap https://www.realtor.com/realestateandhomes-detail/sitemap-rss-price/rss-price-index.xml
sitemap https://www.realtor.com/realestateandhomes-detail/sitemap-rss-sold/rss-sold-index.xml
sitemap https://www.realtor.com/realestateandhomes-detail/sitemap-rss-new-rental/rss-new-rental-index.xml
sitemap https://www.realtor.com/realestateandhomes-detail/sitemap-spec/sitemap-new-homes-ldp-spec-1-index.xml
sitemap https://www.realtor.com/local/sitemaps/district/sitemap-local-district-1-index.xml
sitemap https://www.realtor.com/local/sitemaps/school/sitemap-local-school-1-index.xml
sitemap https://www.realtor.com/sitemap_index.xml
Back to top

Comments

  • LEGAL NOTICE: Per https://www.realtor.com's Terms of Service, scraping data from this website is unauthorized without the express written permission from Move Sales, Inc., operator of https://www.realtor.com.
Back to top

Warnings

  • 4 invalid lines.
Back to top