Log In Sign Up

dlford.io
robots.txt

Robots Exclusion Standard data for dlford.io

Resource Scan

Scan Details

Site Domain dlford.io
Base Domain dlford.io
Scan Status Ok
Last Scan2024-09-29T05:25:45+00:00
Next Scan 2024-10-06T05:25:45+00:00

Last Scan

Scanned2024-09-29T05:25:45+00:00
URL https://dlford.io/robots.txt
Redirect https://www.dlford.io/robots.txt
Redirect Domain www.dlford.io
Redirect Base dlford.io
Domain IPs 75.2.60.5
Redirect IPs 13.228.199.255, 2406:da18:880:3801::c8, 2406:da18:b3d:e200::64, 52.74.166.77
Response IP 13.215.144.61
Found Yes
Hash a7afad0576a5066e3d8fec2187c5d46bf839a8f63a9d2cc2c65fe46536500af3
SimHash 43142953a931

Groups

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /
Disallow /

claudebot

Rule Path
Disallow /

*

Rule Path
Disallow /*/generated/*
Disallow /privacy-policy
Disallow /contact-form-submitted
Disallow /subscription-complete
Allow /tags/*/
Disallow /tags
Allow /categories/*/
Disallow /categories
Disallow /wishlist
Back to top

Other Records

Field Value
sitemap https://www.dlford.io/sitemap.xml
Back to top

Warnings

  • `host` is not a known field.
  • `‍user-agent` is not a known field.
Back to top