bawaab.ml
robots.txt
Robots Exclusion Standard data for bawaab.ml
Resource Scan
Scan Details
| Site Domain | bawaab.ml |
| Base Domain | bawaab.ml |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2025-08-13T06:15:20+00:00 |
| Next Scan | 2025-11-11T06:15:20+00:00 |
Last Successful Scan
| Scanned | 2023-07-02T12:53:17+00:00 |
| URL | https://bawaab.ml/robots.txt |
| Redirect | https://www.bawaab.ml/robots.txt |
| Redirect Domain | www.bawaab.ml |
| Redirect Base | bawaab.ml |
| Domain IPs | 104.21.84.60, 172.67.187.30, 2606:4700:3031::6815:543c, 2606:4700:3033::ac43:bb1e |
| Redirect IPs | 104.21.84.60, 172.67.187.30, 2606:4700:3031::6815:543c, 2606:4700:3033::ac43:bb1e |
| Response IP | 172.67.187.30 |
| Found | Yes |
| Hash | 6355309966ce66cdedd19c9d44cbad2384302fab6b33615b62c5216d2a7460c5 |
| SimHash | cf009383423b |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*.php$ |
| Disallow | /error |
| Disallow | /ajax |
| Disallow | /search |
| Disallow | /redirect |
| Disallow | /img |
| Allow | /img/*.png$ |
| Allow | /img/*.PNG$ |
| Allow | /img/*.jpg$ |
| Allow | /img/*.jpeg$ |
| Allow | /img/*.svg$ |
| Allow | /img/*.gif$ |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.bawaab.ml/sitemap.xml |