homeanddecor.com.sg
robots.txt

Robots Exclusion Standard data for homeanddecor.com.sg

Resource Scan

Scan Details

Site Domain homeanddecor.com.sg
Base Domain homeanddecor.com.sg
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-01T05:17:27+00:00
Next Scan 2024-12-01T05:17:27+00:00

Last Successful Scan

Scanned2024-10-03T04:41:54+00:00
URL https://homeanddecor.com.sg/robots.txt
Redirect https://www.homeanddecor.com.sg/robots.txt
Redirect Domain www.homeanddecor.com.sg
Redirect Base homeanddecor.com.sg
Domain IPs 13.35.238.108, 13.35.238.33, 13.35.238.76, 13.35.238.95
Redirect IPs 3.165.102.109, 3.165.102.31, 3.165.102.58, 3.165.102.85
Response IP 3.165.102.58
Found Yes
Hash 2eb7a0b7fe92f567cf61402f538ac33ae20e5a7672faa22f6e1edf9644fe8e8c
SimHash d0589914f933

Groups

*

Rule Path
Disallow /*/feed/$
Disallow /advanced-galleries/*
Disallow /search/*
Disallow /topics/

Other Records

Field Value
crawl-delay 10

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.homeanddecor.com.sg/_plat/api/sitemap.xml

Comments

  • For new training only
  • Not for training, only for user requests
  • Marker for disabling Bard and Vertex AI
  • Multi-purpose, commercial uses; including LLMs