twilog.togetter.com
robots.txt
Robots Exclusion Standard data for twilog.togetter.com
Resource Scan
Scan Details
| Site Domain | twilog.togetter.com |
| Base Domain | togetter.com |
| Scan Status | Ok |
| Last Scan | 2024-09-17T10:54:19+00:00 |
| Next Scan | 2024-10-17T10:54:19+00:00 |
Last Scan
| Scanned | 2024-09-17T10:54:19+00:00 |
| URL | https://twilog.togetter.com/robots.txt |
| Domain IPs | 3.115.27.7, 52.197.108.175, 52.197.11.50 |
| Response IP | 52.197.11.50 |
| Found | Yes |
| Hash | fa61740e14c2d9dd89a328315b92a94b97bb3b6a389735d2a8980b15a775a057 |
| SimHash | 507ac360d101 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*/hashtags-* |
| Disallow | /*/friends-* |
| Disallow | /*/*nomen* |
| Disallow | /*/*norep* |
| Disallow | /*/*rt* |
| Disallow | /*/*asc* |
| Disallow | /*/*desc* |
| Disallow | /*/stats* |
| Disallow | /switch-sp/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
bingbot
| Rule | Path |
|---|---|
| Disallow | /*/* |
| Disallow | /*/hashtags-* |
| Disallow | /*/friends-* |
| Disallow | /*/*nomen* |
| Disallow | /*/*norep* |
| Disallow | /*/*rt* |
| Disallow | /*/*asc* |
| Disallow | /*/*desc* |
| Disallow | /*/stats* |
| Disallow | /switch-sp/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |