sicip.net
robots.txt

Robots Exclusion Standard data for sicip.net

Resource Scan

Scan Details

Site Domain sicip.net
Base Domain sicip.net
Scan Status Ok
Last Scan2026-02-06T16:55:55+00:00
Next Scan 2026-02-13T16:55:55+00:00

Last Scan

Scanned2026-02-06T16:55:55+00:00
URL https://sicip.net/robots.txt
Domain IPs 138.128.190.75, 2607:fcd8:0:2::600
Response IP 138.128.190.75
Found Yes
Hash 7c18bc0db6667c50193191af6d06ef91607f798d1c0fb00fa1ca3285ac360d4b
SimHash 491acd767221

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /secure/

nutch

Rule Path
Disallow /

spiderbot/nutch-1.7

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

googlebot

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /