afpbb.com
robots.txt
Robots Exclusion Standard data for afpbb.com
Resource Scan
Scan Details
| Site Domain | afpbb.com |
| Base Domain | afpbb.com |
| Scan Status | Ok |
| Last Scan | 2024-11-09T00:20:09+00:00 |
| Next Scan | 2024-11-16T00:20:09+00:00 |
Last Scan
| Scanned | 2024-11-09T00:20:09+00:00 |
| URL | https://www.afpbb.com/robots.txt |
| Domain IPs | 202.238.151.70 |
| Response IP | 202.238.151.70 |
| Found | Yes |
| Hash | b649c2ab549618dfae9a6950b28f4c43bf9f56c0830d91e388e631ce7d02f83d |
| SimHash | eb3944d40493 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /sys/ |
| Disallow | /articles/photo-slide/ |
| Disallow | /i/ |
| Disallow | /articles/biz/ |
| Disallow | /subcategory/*/http/ |
| Disallow | /search |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
discobot
| Rule | Path |
|---|---|
| Disallow | /sys/ |
| Disallow | /articles/photo-slide/ |
| Disallow | /i/ |
| Disallow | /articles/biz/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.afpbb.com/sdata/google_news_sitemap.xml |