dictionary.reference.com
robots.txt
Robots Exclusion Standard data for dictionary.reference.com
Resource Scan
Scan Details
Site Domain | dictionary.reference.com |
Base Domain | reference.com |
Scan Status | Ok |
Last Scan | 2024-06-12T01:20:47+00:00 |
Next Scan | 2024-06-19T01:20:47+00:00 |
Last Scan
Scanned | 2024-06-12T01:20:47+00:00 |
URL | https://dictionary.reference.com/robots.txt |
Redirect | https://www.dictionary.com/robots.txt |
Redirect Domain | www.dictionary.com |
Redirect Base | dictionary.com |
Domain IPs | 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133 |
Redirect IPs | 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133, 2a04:4e42:200::645, 2a04:4e42:400::645, 2a04:4e42:600::645, 2a04:4e42::645 |
Response IP | 199.232.46.133 |
Found | Yes |
Hash | f670d0c6b7a3cce04414296230ec3c966e96a41962544b5c10b654141f4652f7 |
SimHash | 2a479860c332 |
Groups
*
Rule | Path |
---|---|
Disallow | /5480.iac. |
Disallow | /go/ |
Disallow | /audio.html/ |
Disallow | /houseads/ |
Disallow | /askhome/ |
Disallow | /23219321/iac. |
Disallow | /*%2C* |
Disallow | /browse-2/ |
Allow | / |
Other Records
Field | Value |
---|---|
sitemap | https://www.dictionary.com/sitemap-dict-index.xml |
sitemap | https://www.dictionary.com/e/sitemap_index.xml |
Comments