Log In Sign Up

img.tfd.com
robots.txt

Robots Exclusion Standard data for img.tfd.com

Resource Scan

Scan Details

Site Domain img.tfd.com
Base Domain tfd.com
Scan Status Ok
Last Scan2025-03-12T23:31:33+00:00
Next Scan 2025-04-11T23:31:33+00:00

Last Scan

Scanned2025-03-12T23:31:33+00:00
URL https://img.tfd.com/robots.txt
Domain IPs 108.181.106.66
Response IP 108.181.106.66
Found Yes
Hash 545f12fa60146dacdd929b7f54e1781c95bc0334c8b82293e1e3d213ec950694
SimHash 0211440c0d63

Groups

npbot

Rule Path
Disallow /

msrbot

Rule Path
Disallow /

open*

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

scooter*

Rule Path
Disallow /

websearch*

Rule Path
Disallow /

zyborg*

Rule Path
Disallow /

wget*

Rule Path
Disallow /

turnitinbot*

Rule Path
Disallow /

slysearch*

Rule Path
Disallow /

seekbot*

Rule Path
Disallow /

netresearchserver*

Rule Path
Disallow /

seekbot*

Rule Path
Disallow /

mozdex*

Rule Path
Disallow /

link*

Rule Path
Disallow /

larbin*

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

charlotte

Rule Path
Disallow /
Back to top