nbc25news.com
robots.txt

Robots Exclusion Standard data for nbc25news.com

Resource Scan

Scan Details

Site Domain nbc25news.com
Base Domain nbc25news.com
Scan Status Ok
Last Scan2026-02-13T08:23:46+00:00
Next Scan 2026-02-20T08:23:46+00:00

Last Scan

Scanned2026-02-13T08:23:46+00:00
URL https://nbc25news.com/robots.txt
Domain IPs 3.164.182.22, 3.164.182.35, 3.164.182.69, 3.164.182.85
Response IP 13.226.2.28
Found Yes
Hash 48836ba75318ca8123f939350dcd058c9dc25bc8d3db287f1d45773e9aa6cb8b
SimHash 5114c951e453

Groups

*

Rule Path
Allow /
Disallow /resources/ftptransfer/

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /news/
Disallow /sports/
Disallow /features/

ia_archiver-web.archive.org

Rule Path
Disallow /news/
Disallow /sports/
Disallow /features/