silktube.net
robots.txt

Robots Exclusion Standard data for silktube.net

Resource Scan

Scan Details

Site Domain silktube.net
Base Domain silktube.net
Scan Status Ok
Last Scan2025-05-25T08:58:03+00:00
Next Scan 2025-06-24T08:58:03+00:00

Last Scan

Scanned2025-05-25T08:58:03+00:00
URL https://silktube.net/robots.txt
Redirect http://www.silktube.net/robots.txt
Redirect Domain www.silktube.net
Redirect Base silktube.net
Domain IPs 104.21.39.54, 172.67.143.140, 2606:4700:3032::ac43:8f8c, 2606:4700:3035::6815:2736
Redirect IPs 104.21.39.54, 172.67.143.140, 2606:4700:3032::ac43:8f8c, 2606:4700:3035::6815:2736
Response IP 104.21.39.54
Found Yes
Hash 30944da81032b8f85f1bc5ab06bd9109a8542145f73444e6239a72e0db4b20e3
SimHash 5a7ed4404790

Groups

msnbot

Rule Path
Disallow /fresh/
Disallow /long/

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Disallow /fresh/
Disallow /long/

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Disallow /video/
Disallow /pics/

baiduspider

Rule Path
Disallow /*?
Disallow /fresh/
Disallow /long/
Disallow /tag/
Disallow /pornstars/
Disallow /video/
Disallow /best/*/1
Disallow /best/*/2
Disallow /best/*/3
Disallow /best/*/4
Disallow /best/*/5
Disallow /best/*/6
Disallow /best/*/7
Disallow /best/*/8
Disallow /best/*/9

yandex

Rule Path
Disallow /fresh/
Disallow /long/

ahrefsbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

cloudfind

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

webmeupbot

Rule Path
Disallow /

seokicks bot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

tinytestbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

*

Rule Path
Disallow /report/