en.ittefaq.com.bd
robots.txt
Robots Exclusion Standard data for en.ittefaq.com.bd
Resource Scan
Scan Details
| Site Domain | en.ittefaq.com.bd |
| Base Domain | ittefaq.com.bd |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-06-16T11:38:47+00:00 |
| Next Scan | 2024-08-15T11:38:47+00:00 |
Last Successful Scan
| Scanned | 2024-03-26T11:37:36+00:00 |
| URL | https://en.ittefaq.com.bd/robots.txt |
| Domain IPs | 104.21.1.92, 172.67.128.246, 2606:4700:3030::6815:15c, 2606:4700:3036::ac43:80f6 |
| Response IP | 172.67.128.246 |
| Found | Yes |
| Hash | bc8f8e360e43c88548ee77860d7a10bd3d6e37a6b5c6e2e9a3c9cea28c551e43 |
| SimHash | 444ac260be13 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /cgi-bin/ |
| Disallow | /cdn-cgi/ |
| Disallow | /register/ |
| Disallow | /login/ |
| Disallow | /register |
| Disallow | /login |
| Disallow | /api/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://en.ittefaq.com.bd/sitemap.xml |
| sitemap | https://en.ittefaq.com.bd/news-sitemap.xml |