homelight.com
robots.txt

Robots Exclusion Standard data for homelight.com

Resource Scan

Scan Details

Site Domain homelight.com
Base Domain homelight.com
Scan Status Ok
Last Scan2024-11-09T17:27:55+00:00
Next Scan 2024-12-09T17:27:55+00:00

Last Scan

Scanned2024-11-09T17:27:55+00:00
URL https://homelight.com/robots.txt
Redirect https://www.homelight.com/robots.txt
Redirect Domain www.homelight.com
Redirect Base homelight.com
Domain IPs 151.101.130.132, 151.101.194.132, 151.101.2.132, 151.101.66.132
Redirect IPs 151.101.130.132, 151.101.194.132, 151.101.2.132, 151.101.66.132
Response IP 199.232.46.132
Found Yes
Hash 793c8c2e08eea895992863dfb7ca77282a523b27209c848eb742b235a9a4568a
SimHash 6955f0d5e708

Groups

*

Rule Path
Disallow /reviews/*
Disallow /unavailable/*
Disallow /agents/sign-up
Disallow /agents/followup/*
Disallow /service-providers/new*
Disallow /client-sign-up
Disallow /serve/*
Disallow /assets/*
Disallow /agents-to-watch/*
Disallow /media/*
Disallow /sales/*
Disallow /admin/*
Disallow /email/*
Disallow /promo/*
Disallow /friends/*
Disallow /widget/*
Disallow /websocket
Disallow /geo/*
Disallow /u/*
Disallow /sign-up*
Disallow /agents/leads/manage
Disallow /agent-portal
Disallow /manage/*
Disallow /users/*
Disallow /insurance/*
Disallow /staging/*
Disallow /service-providers/*
Disallow /pest-inspection/*
Disallow /painter/*
Disallow /movers/*
Disallow /mortgage/*
Disallow /locksmith/*
Disallow /home-inspection/*
Disallow /escrow/*
Disallow /contractor/*
Disallow /sem/cities/*
Disallow /simple-sale/home-value/
Disallow /simple-sale/quiz/
Disallow /simple-sale/results/
Disallow /find-agent/quiz/*
Disallow /blog-visit/*
Disallow /blog-tracker*
Disallow /partner
Disallow /1000
Disallow /*attachment_id*
Disallow /ultra_listings_share_images/*

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

goo

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

baidu

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

siteauditbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 4

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

applebot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /real-listings*

Other Records

Field Value
sitemap https://www.homelight.com/blog/sitemap_index.xml
sitemap https://www.homelight.com/index_sell-house-fast-1.xml.gz
sitemap https://www.homelight.com/sitemap_top_real_estate_cities.xml
sitemap https://www.homelight.com/index_best-time-to-sell-1.xml.gz
sitemap https://www.homelight.com/index_agent-profile-1.xml.gz
sitemap https://www.homelight.com/index_states-directory-city-1.xml.gz
sitemap https://www.homelight.com/index_top-agents-zip-code-1.xml.gz
sitemap https://www.homelight.com/index_search-hub-zip-1.xml.gz
sitemap https://www.homelight.com/index_search-hub-state-property-type-1.xml.gz
sitemap https://www.homelight.com/index_search-hub-state-bedroom-1.xml.gz
sitemap https://www.homelight.com/index_search-hub