blog.dictionary.com
robots.txt

Robots Exclusion Standard data for blog.dictionary.com

Resource Scan

Scan Details

Site Domain blog.dictionary.com
Base Domain dictionary.com
Scan Status Ok
Last Scan2024-05-17T09:37:30+00:00
Next Scan 2024-06-16T09:37:30+00:00

Last Scan

Scanned2024-05-17T09:37:30+00:00
URL https://blog.dictionary.com/robots.txt
Redirect https://www.dictionary.com/robots.txt
Redirect Domain www.dictionary.com
Redirect Base dictionary.com
Domain IPs 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133, 2a04:4e42:200::645, 2a04:4e42:400::645, 2a04:4e42:600::645, 2a04:4e42::645
Redirect IPs 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133, 2a04:4e42:200::645, 2a04:4e42:400::645, 2a04:4e42:600::645, 2a04:4e42::645
Response IP 199.232.46.133
Found Yes
Hash f670d0c6b7a3cce04414296230ec3c966e96a41962544b5c10b654141f4652f7
SimHash 2a479860c332

Groups

gptbot

Rule Path
Disallow /

deepcrawl

Rule Path
Disallow /

seerbot/1.0

Rule Path
Disallow /

*

Rule Path
Disallow /5480.iac.
Disallow /go/
Disallow /audio.html/
Disallow /houseads/
Disallow /askhome/
Disallow /23219321/iac.
Disallow /*%2C*
Disallow /browse-2/
Allow /

Other Records

Field Value
sitemap https://www.dictionary.com/sitemap-dict-index.xml
sitemap https://www.dictionary.com/e/sitemap_index.xml

Comments

  • Production robots.txt