al-sharq.com
robots.txt
Robots Exclusion Standard data for al-sharq.com
Resource Scan
Scan Details
Site Domain | al-sharq.com |
Base Domain | al-sharq.com |
Scan Status | Ok |
Last Scan | 2024-09-26T01:46:22+00:00 |
Next Scan | 2024-10-03T01:46:22+00:00 |
Last Scan
Scanned | 2024-09-26T01:46:22+00:00 |
URL | https://al-sharq.com/robots.txt |
Domain IPs | 104.22.76.104, 104.22.77.104, 172.67.26.243, 2606:4700:10::6816:4c68, 2606:4700:10::6816:4d68, 2606:4700:10::ac43:1af3 |
Response IP | 104.22.76.104 |
Found | Yes |
Hash | c8ff3f95768b4e47872965c832c7d91639e1c1a93d14ad4777e5fcfa992e57f1 |
SimHash | 119e52e3cde0 |
Groups
*
No rules defined. All paths allowed.
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
*
Rule | Path |
---|---|
Disallow | /65.108.61.0 |
Warnings
- 1 invalid line.
Comments