cresstone.com
robots.txt

Robots Exclusion Standard data for cresstone.com

Resource Scan

Scan Details

Site Domain cresstone.com
Base Domain cresstone.com
Scan Status Ok
Last Scan2024-09-09T01:04:29+00:00
Next Scan 2024-10-09T01:04:29+00:00

Last Scan

Scanned2024-09-09T01:04:29+00:00
URL https://cresstone.com/robots.txt
Domain IPs 2607:f298:5:114b::60:584c, 67.205.26.38
Response IP 67.205.26.38
Found Yes
Hash fd71c5c654cca86afee4220e5f051b0c26c3fbaff23b4db34dc86379d5b16223
SimHash 2e14295045d7

Groups

googlebot
bingbot
slurp
baiduspider
duckduckbot
yandexbot
sogou
twitterbot
archive.org_bot
seznambot
exabot
mojeekbot
wotbox
yoozbot
tineye-bot
iskanie
y!j-asr
coccocbot-web
ia_archiver
qwantify
ccbot
cliqzbot
sogou web spider
roxieybot
msnbot
bingpreview

Rule Path
Disallow /i_am_a_bad_spider.html

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

googleother

Rule Path
Disallow /

*

Rule Path
Disallow /

Other Records

Field Value
sitemap http://cresstone.com/sitemap.xml

Warnings

  • `host` is not a known field.