delcolibraries.org
robots.txt

Robots Exclusion Standard data for delcolibraries.org

Resource Scan

Scan Details

Site Domain delcolibraries.org
Base Domain delcolibraries.org
Scan Status Ok
Last Scan2025-09-09T00:05:00+00:00
Next Scan 2025-10-09T00:05:00+00:00

Last Scan

Scanned2025-09-09T00:05:00+00:00
URL https://delcolibraries.org/robots.txt
Redirect https://www.delcolibraries.org/robots.txt
Redirect Domain www.delcolibraries.org
Redirect Base delcolibraries.org
Domain IPs 44.225.190.210
Redirect IPs 18.161.111.34, 18.161.111.38, 18.161.111.57, 18.161.111.67, 2600:9000:21f8:2200:16:3ea7:b780:93a1, 2600:9000:21f8:2600:16:3ea7:b780:93a1, 2600:9000:21f8:4800:16:3ea7:b780:93a1, 2600:9000:21f8:4e00:16:3ea7:b780:93a1, 2600:9000:21f8:5200:16:3ea7:b780:93a1, 2600:9000:21f8:7400:16:3ea7:b780:93a1, 2600:9000:21f8:ac00:16:3ea7:b780:93a1, 2600:9000:21f8:c200:16:3ea7:b780:93a1
Response IP 18.165.72.61
Found Yes
Hash 9c621fd38cba498cd913ac1fa4d39d65627e13a170ec5ba09222371387104ba8
SimHash e2984b4da036

Groups

*

Rule Path
Disallow /wp-login.php
Disallow /xmlrpc.php
Disallow /*%28offset%29
Disallow /*SearchText%3D
Disallow /*SearchSort%3D
Disallow /*type%5B*%5D%3D
Disallow /*filter%5B*%5D%3D

anthropic-ai

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

Comments

  • crawler related