homesandland.com
robots.txt

Robots Exclusion Standard data for homesandland.com

Resource Scan

Scan Details

Site Domain homesandland.com
Base Domain homesandland.com
Scan Status Ok
Last Scan2024-09-15T19:30:14+00:00
Next Scan 2024-09-22T19:30:14+00:00

Last Scan

Scanned2024-09-15T19:30:14+00:00
URL https://homesandland.com/robots.txt
Domain IPs 104.16.172.247, 104.16.173.247, 2606:4700::6810:acf7, 2606:4700::6810:adf7
Response IP 104.16.172.247
Found Yes
Hash 764364510d7233e3f5c16a9545b6749c0fe06724ceab1b179546bbe0cbf39674
SimHash e21699d0cb77

Groups

*

Rule Path
Disallow

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

amazonadbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

baiduspider
yisouspider
petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

owler@ows.eu/1

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

obot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.homesandland.com/sitemaps/dyna/sitemap_root_index.xml

Comments

  • Bard and Vertex AI generative APIs
  • Non Market

Warnings

  • 1 invalid line.