become.co.jp
robots.txt
Robots Exclusion Standard data for become.co.jp
Resource Scan
Scan Details
| Site Domain | become.co.jp |
| Base Domain | become.co.jp |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-09-07T13:22:14+00:00 |
| Next Scan | 2024-12-06T13:22:14+00:00 |
Last Successful Scan
| Scanned | 2024-05-11T12:28:14+00:00 |
| URL | http://become.co.jp/robots.txt |
| Domain IPs | 35.77.122.35, 54.199.177.15 |
| Response IP | 35.77.122.35 |
| Found | Yes |
| Hash | 12c22612cc4409e63799d966091711128718d24ed3ac5ce9d12c90d6655048b1 |
| SimHash | 513e5b74ab89 |
Groups
sogou
bidswitchbot
cincraw
applebot
dataforseobot
zoominfobot
amazonadbot
petalbot
seznambot
claudebot
yandexbot
freakoutadstxtbot
ahrefsbot
dotbot
crawler
serpstatbot
blexbot
amazonbot
semrushbot
gptbot
bytespider
adsbot
microadbot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /contents/static.ga |
| Disallow | /pa/ |
| Disallow | /ph/ |
| Disallow | /login/ |
| Disallow | /lss/ |
| Disallow | /bookmarks-pdp/ |
| Disallow | /bookmarks-pserp/ |
| Disallow | /adult/ |
| Disallow | /contents/static.tryads |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.become.co.jp/sitemap.xml |