gourmetcaree.jp
robots.txt

Robots Exclusion Standard data for gourmetcaree.jp

Resource Scan

Scan Details

Site Domain gourmetcaree.jp
Base Domain gourmetcaree.jp
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-18T10:43:59+00:00
Next Scan 2024-07-17T10:43:59+00:00

Last Successful Scan

Scanned2023-06-28T18:04:55+00:00
URL https://gourmetcaree.jp/robots.txt
Redirect https://www.gourmetcaree.jp:443/robots.txt
Redirect Domain www.gourmetcaree.jp
Redirect Base gourmetcaree.jp
Domain IPs 13.33.33.114, 13.33.33.40, 13.33.33.74, 13.33.33.90
Redirect IPs 13.33.33.114, 13.33.33.40, 13.33.33.74, 13.33.33.90
Response IP 52.222.144.37
Found Yes
Hash d148ff369f4e852e9f19249ce93d31f4cb50a2b7ac47723bc4fec4e974431c93
SimHash eb0f5400c2ba

Groups

*

Rule Path
Disallow /*/mypage/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

facebookexternalhit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

megaindex.ru

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

baidu

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mappy

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.gourmetcaree.jp/kanto/sitemap_kanto.xml
sitemap https://www.gourmetcaree.jp/tohoku/sitemap_tohoku.xml
sitemap https://www.gourmetcaree.jp/kansai/sitemap_kansai.xml
sitemap https://www.gourmetcaree.jp/matome/sitemap.xml
sitemap https://www.gourmetcaree.jp/column/sitemap.xml