Log In Sign Up

sica.int
robots.txt

Robots Exclusion Standard data for sica.int

Resource Scan

Scan Details

Site Domain sica.int
Base Domain sica.int
Scan Status Ok
Last Scan5/7/2025, 2:19:40 AM
Next Scan 6/6/2025, 2:19:40 AM

Last Scan

Scanned5/7/2025, 2:19:40 AM
URL https://www.sica.int/robots.txt
Domain IPs 104.26.12.17, 104.26.13.17, 172.67.74.41, 2606:4700:20::681a:c11, 2606:4700:20::681a:d11, 2606:4700:20::ac43:4a29
Response IP 104.26.13.17
Found Yes
Hash d3c47b3102c31c831a8dc5ada9411dbeaa350b213e54ee0c9af2270f4077060b
SimHash f00ac11226b3

Groups

ahrefsbot

Rule Path
Disallow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /content/images/*/main/*.jpg

googlebot-image

Rule Path
Allow /content/images/*/main/*.png

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

openindexspider

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

yeti

Rule Path
Disallow /

buck

Rule Path
Disallow /

phantomjs

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

seznambot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.sica.int/sitemap.xml
Back to top