buildwith.net
robots.txt
Robots Exclusion Standard data for buildwith.net
Resource Scan
Scan Details
Site Domain | buildwith.net |
Base Domain | buildwith.net |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Server returned a server error. |
Last Scan | 2024-09-19T20:22:19+00:00 |
Next Scan | 2024-12-18T20:22:19+00:00 |
Last Successful Scan
Scanned | 2022-11-30T08:23:18+00:00 |
URL | https://buildwith.net/robots.txt |
Domain IPs | 104.21.89.6, 172.67.155.102 |
Response IP | 172.67.155.102 |
Found | Yes |
Hash | 410d0909f4b063355e837ad6e24139f4cd54e97c7deae976bbd5c06adaf36ef0 |
SimHash | ca0855c0e91b |
Groups
*
Rule | Path |
---|---|
Disallow | /cgi-bin/ |
Disallow | /wp-admin/ |
Disallow | /linkout/ |
Disallow | /recommended/ |
Disallow | /comments/feed/ |
Disallow | /trackback/ |
Disallow | /index.php |
Disallow | /xmlrpc.php |
Other Records
Field | Value |
---|---|
sitemap | https://www.buildwith.net/sitemap.xml |
Warnings
- 2 invalid lines.
Comments