differencebetween.com
robots.txt
Robots Exclusion Standard data for differencebetween.com
Resource Scan
Scan Details
| Site Domain | differencebetween.com |
| Base Domain | differencebetween.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-06-04T05:04:22+00:00 |
| Next Scan | 2025-09-02T05:04:22+00:00 |
Last Successful Scan
| Scanned | 2024-08-09T04:17:36+00:00 |
| URL | https://differencebetween.com/robots.txt |
| Domain IPs | 104.21.75.79, 172.67.217.144, 2606:4700:3034::ac43:d990, 2606:4700:3036::6815:4b4f |
| Response IP | 104.21.75.79 |
| Found | Yes |
| Hash | 44fdf78dabb4f086e50f9efba24791f7966da5cc835c516ce3f1274ed146abe2 |
| SimHash | 0b26dac4cc63 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /answers/ |
| Disallow | /ar/ |
| Disallow | /de/ |
| Disallow | /es/ |
| Disallow | /fr/ |
| Disallow | /ga/ |
| Disallow | /gl/ |
| Disallow | /hi/ |
| Disallow | /hu/ |
| Disallow | /it/ |
| Disallow | /nl/ |
| Disallow | /no/ |
| Disallow | /ru/ |
| Disallow | /stale/ |
| Disallow | /th/ |
| Disallow | /tr/ |
| Disallow | /zh-CN/ |
| Disallow | /tag/* |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.differencebetween.com/sitemap.xml.gz |