Log In Sign Up

sex2k.org
robots.txt

Robots Exclusion Standard data for sex2k.org

Resource Scan

Scan Details

Site Domain sex2k.org
Base Domain sex2k.org
Scan Status Ok
Last Scan2025-03-09T07:33:34+00:00
Next Scan 2025-04-08T07:33:34+00:00

Last Scan

Scanned2025-03-09T07:33:34+00:00
URL https://sex2k.org/robots.txt
Domain IPs 104.21.48.241, 172.67.157.3, 2606:4700:3033::ac43:9d03, 2606:4700:3035::6815:30f1
Response IP 104.21.48.241
Found Yes
Hash 27cfa9877177f1503606d5c74c00dda7c104259eddfaa4cd6d1f153df133b472
SimHash d16d53044d5b

Groups

heritrix

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

proximic

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /
Back to top

Warnings

  • 1 invalid line.
Back to top