homestars.com
robots.txt

Robots Exclusion Standard data for homestars.com

Resource Scan

Scan Details

Site Domain homestars.com
Base Domain homestars.com
Scan Status Ok
Last Scan2024-06-04T16:39:36+00:00
Next Scan 2024-06-18T16:39:36+00:00

Last Scan

Scanned2024-06-04T16:39:36+00:00
URL https://homestars.com/robots.txt
Redirect https://homestars.com:443/robots.txt/
Domain IPs 3.212.242.53, 35.173.23.87, 52.23.121.143, 52.55.65.49, 52.71.140.118, 54.167.100.224
Response IP 52.23.121.143
Found Yes
Hash 2618ea50b8112fdad0ba1fd81754003c88167aeeab44bb7d63aa250b47af7536
SimHash 80b63b84a033

Groups

*

Rule Path
Disallow /hello
Disallow /plus
Disallow /member_upgrade
Disallow /search/
Disallow /private/
Disallow /members/*

turnitinbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

legs

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexwebmaster

Rule Path
Disallow /

yandex

Rule Path
Disallow /

speedy

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

spider

Rule Path
Disallow /

psbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

yeti

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

baiduspider-favo

Rule Path
Disallow /

baiduspider-cpro

Rule Path
Disallow /

baiduspider-ads

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

baidu

Rule Path
Disallow /

goo

Rule Path
Disallow /

majestic-seo

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

yandexwebmaster/2.0

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

java

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

voyager

Rule Path
Disallow /

scooter

Rule Path
Disallow /

robot

Rule Path
Disallow /

fido

Rule Path
Disallow /

motor

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

crawl

Rule Path
Disallow /

Other Records

Field Value
sitemap https://homestars.com/sitemap.xml.gz

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file

Warnings

  • 2 invalid lines.