Log In Sign Up

clearth-rent.co.jp
robots.txt

Robots Exclusion Standard data for clearth-rent.co.jp

Resource Scan

Scan Details

Site Domain clearth-rent.co.jp
Base Domain clearth-rent.co.jp
Scan Status Ok
Last Scan3/12/2025, 12:50:45 AM
Next Scan 4/11/2025, 12:50:45 AM

Last Scan

Scanned3/12/2025, 12:50:45 AM
URL https://clearth-rent.co.jp/robots.txt
Redirect https://www.clearth-rent.co.jp/robots.txt
Redirect Domain www.clearth-rent.co.jp
Redirect Base clearth-rent.co.jp
Domain IPs 160.16.124.211
Redirect IPs 160.16.124.211
Response IP 160.16.124.211
Found Yes
Hash ba2cf98111ad4f01348f320a884356916b5d4760950a68817353a610b7932b1c
SimHash e45510a0c181

Groups

*

Rule Path
Disallow /history
Disallow /my_list
Disallow /saved_condition

icc-crawler
meta-externalagent
bingbot
applebot
amazonbot
claudebot
gptbot
ahrefsbot
qualys
yandexbot
petalbot
seekportbot
dataforseobot
barkrowler

Rule Path
Disallow /rent_search/*/*/*/*
Disallow /rent_search/*/*/*-*-*
Disallow /rent_search/*/*-*-*
Disallow /rent_search/*-*-*
Disallow /contact_item/*

bingbot

Rule Path
Disallow /rent_search/word/*

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

icc-crawler
meta-externalagent
bingbot
slurp
claudebot
gptbot
amazonbot
applebot
seekportbot
ahrefsbot
dataforseobot
barkrowler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20
Back to top

Comments

  • disallow
  • Crawl-delay
Back to top