crescent.ae
robots.txt
Robots Exclusion Standard data for crescent.ae
Resource Scan
Scan Details
| Site Domain | crescent.ae |
| Base Domain | crescent.ae |
| Scan Status | Ok |
| Last Scan | 2024-10-23T14:34:48+00:00 |
| Next Scan | 2024-11-22T14:34:48+00:00 |
Last Scan
| Scanned | 2024-10-23T14:34:48+00:00 |
| URL | https://www.crescent.ae/robots.txt |
| Domain IPs | 20.107.12.241 |
| Response IP | 20.107.12.241 |
| Found | Yes |
| Hash | a3f6058576bab6f5a959518dec8bd6ca36a7fb3fe8594c18bb74874f4facacdf |
| SimHash | af4b735cee03 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /Eassessment |
| Disallow | /downloads |
| Disallow | /Email-Signature |
| Disallow | |
| Disallow | /Vendor |
| Disallow | /Vendor2 |
| Allow | /fonts/* |
| Allow | /images/* |
| Allow | /plugins/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.crescent.ae/sitemap.xml |
Warnings
- 1 invalid line.