chat2friend.com
robots.txt
Robots Exclusion Standard data for chat2friend.com
Resource Scan
Scan Details
Site Domain | chat2friend.com |
Base Domain | chat2friend.com |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Couldn't connect to server. |
Last Scan | 2024-10-15T23:16:19+00:00 |
Next Scan | 2024-10-22T23:16:19+00:00 |
Last Successful Scan
Scanned | 2024-09-14T23:15:21+00:00 |
URL | https://chat2friend.com/robots.txt |
Domain IPs | 66.85.156.3 |
Response IP | 66.85.156.3 |
Found | Yes |
Hash | 47979d5fec777cc289a3831173fe4d0fedf6736990ec9fcdaaebe8e8ac6cc60f |
SimHash | 64ca1062c3a2 |
Groups
*
Rule | Path |
---|---|
Disallow | /online.js |
mozilla/5.0(compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Rule | Path |
---|---|
Disallow | / |
*
No rules defined. All paths allowed.
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
*
Rule | Path |
---|---|
Disallow | /cgi-bin/ |
Disallow | /cgi-sys/ |
*
No rules defined. All paths allowed.
Other Records
Field | Value |
---|---|
sitemap | /sitemap.xml |
Warnings
- 8 invalid lines.
Comments