dreamers.id
robots.txt
Robots Exclusion Standard data for dreamers.id
Resource Scan
Scan Details
| Site Domain | dreamers.id |
| Base Domain | dreamers.id |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Request timed out. |
| Last Scan | 2024-09-15T02:07:59+00:00 |
| Next Scan | 2024-09-29T02:07:59+00:00 |
Last Successful Scan
| Scanned | 2024-08-30T16:11:23+00:00 |
| URL | https://dreamers.id/robots.txt |
| Domain IPs | 103.227.252.251 |
| Response IP | 103.227.252.251 |
| Found | Yes |
| Hash | d686216e818a86a777dda5ab71a4a41e94684d5a35bf179624c71b4dd305d4ff |
| SimHash | 3c4b5d0a25d3 |
Groups
twitterbot
*
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /$ |
| Allow | /index.php |
| Allow | /m |
| Allow | /article/ |
| Allow | /img_*/ |
| Allow | /images/ |
| Allow | /list-album |
| Allow | /gallery |
| Allow | /entertainment |
| Allow | /news |
| Allow | /lifestyle |
| Allow | /sport |
| Allow | /forum |
| Allow | /fanfiction |
| Allow | /dreamerstv |
| Allow | /lirik |
| Allow | /dreamersads |
| Allow | /robots.txt |
| Disallow | /forum/user-* |
| Disallow | /user-* |
| Disallow | /neighbor-* |
| Disallow | /fanbase |
| Disallow | /cinema |
| Disallow | /dreamersmarket |
| Disallow | /blast |
| Disallow | /widgetplayer |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.dreamers.id/sitemap_index.xml |
| sitemap | http://www.dreamers.id/sitemap_index_*.xml |
| sitemap | http://www.dreamers.id/sitemap_ff.xml |
| sitemap | http://www.dreamers.id/sitemap_ff_*.xml |
| sitemap | http://cdn.attracta.com/sitemap/4757728.xml.gz |
Comments