Log In Sign Up

topics.silive.com
robots.txt

Robots Exclusion Standard data for topics.silive.com

Resource Scan

Scan Details

Site Domain topics.silive.com
Base Domain silive.com
Scan Status Ok
Last Scan2024-11-09T03:56:21+00:00
Next Scan 2024-11-16T03:56:21+00:00

Last Scan

Scanned2024-11-09T03:56:21+00:00
URL https://topics.silive.com/robots.txt
Redirect https://www.silive.com/robots.txt
Redirect Domain www.silive.com
Redirect Base silive.com
Domain IPs 23.209.46.70, 23.209.46.75, 2600:1413:b000:13::b857:c192, 2600:1413:b000:13::b857:c194
Redirect IPs 23.52.171.136, 23.52.171.137, 2600:1413:b000:13::b857:c192, 2600:1413:b000:13::b857:c194
Response IP 23.45.207.168
Found Yes
Hash f974ab30e83fe3e8e4f7ccaca4080aae227d8df69136910858e16ff25dee73d3
SimHash 3115694186e2

Groups

meta-externalagent

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

*

Rule Path
Disallow /workspace-betting/*

bingbot

Rule Path
Disallow /home-beta/
Disallow /test/
Disallow /preview/
Disallow /staging/
Disallow /clients/
Disallow /auctions/
Disallow /cgi-bin/
Disallow /printer/
Disallow /*/print.html*
Disallow /*/print.ssf*
Disallow /landingpages/
Disallow /oil-spill-reports/
Disallow /premium/
Disallow /contests/scrape/
Disallow /news/advance/index.ssf?%2Fbase%2Fnews%2F1176538514249150.xml&amp%3Bcoll=1
Disallow /news/advance/index.ssf?%2Fbase%2Fnews%2F117681121166360.xml&amp%3Bcoll=1
Disallow /westshore/index.ssf/2009/02/staten_island_psychic_reaches.html
Disallow /northshore/index.ssf/2010/09/jury_still_out_on_new_forest_a.html
Disallow /opinion/2019/02/want-to-stop-fake-news-pay-for-the-real-thing-commentary.html
Disallow /puzzles-palace/
Disallow /silive_sandbox_blog_-_azywusko
Disallow /toprailapp/
Disallow /weddings/index.ssf/2017/07/silive_wedding_july17.html
Disallow /news/2017/12/your_mental_health_struggle_do.html
Disallow /*mt-preview-*
Disallow /sponsor-content-index-test
Disallow /sponsor-content-test
Disallow /sponsor-content-test
Disallow /sponsor-content-p-test
Disallow /puzzles-games/
Disallow /comics/
Disallow /puzzle-society/
Disallow /go-comics/
Disallow /comics-kingdom/
Disallow /entertainment/2018/03/christina_aguilera_owns_her_si.html

googlebot

Rule Path
Disallow /home-beta/
Disallow /test/
Disallow /preview/
Disallow /staging/
Disallow /clients/
Disallow /auctions/
Disallow /cgi-bin/
Disallow /printer/
Disallow /*/print.html*
Disallow /*/print.ssf*
Disallow /landingpages/
Disallow /oil-spill-reports/
Disallow /premium/
Disallow /contests/scrape/
Disallow /news/advance/index.ssf?%2Fbase%2Fnews%2F1176538514249150.xml&amp%3Bcoll=1
Disallow /news/advance/index.ssf?%2Fbase%2Fnews%2F117681121166360.xml&amp%3Bcoll=1
Disallow /westshore/index.ssf/2009/02/staten_island_psychic_reaches.html
Disallow /northshore/index.ssf/2010/09/jury_still_out_on_new_forest_a.html
Disallow /opinion/2019/02/want-to-stop-fake-news-pay-for-the-real-thing-commentary.html
Disallow /puzzles-palace/
Disallow /silive_sandbox_blog_-_azywusko
Disallow /toprailapp/
Disallow /weddings/index.ssf/2017/07/silive_wedding_july17.html
Disallow /news/2017/12/your_mental_health_struggle_do.html
Disallow /*mt-preview-*
Disallow /sponsor-content-index-test
Disallow /sponsor-content-test
Disallow /sponsor-content-test
Disallow /sponsor-content-p-test
Disallow /puzzles-games/
Disallow /comics/
Disallow /puzzle-society/
Disallow /go-comics/
Disallow /comics-kingdom/
Disallow /entertainment/2018/03/christina_aguilera_owns_her_si.html

proximic

Rule Path
Disallow /home-beta/
Disallow /test/

slurp

Rule Path
Disallow /home-beta/
Disallow /test/
Disallow /preview/
Disallow /staging/
Disallow /clients/
Disallow /auctions/
Disallow /cgi-bin/
Disallow /printer/
Disallow /*/print.html*
Disallow /*/print.ssf*
Disallow /landingpages/
Disallow /premium/
Disallow /contests/scrape/
Disallow /news/advance/index.ssf?%2Fbase%2Fnews%2F1176538514249150.xml&amp%3Bcoll=1
Disallow /news/advance/index.ssf?%2Fbase%2Fnews%2F117681121166360.xml&amp%3Bcoll=1
Disallow /westshore/index.ssf/2009/02/staten_island_psychic_reaches.html
Disallow /northshore/index.ssf/2010/09/jury_still_out_on_new_forest_a.html
Disallow /opinion/2019/02/want-to-stop-fake-news-pay-for-the-real-thing-commentary.html
Disallow /puzzles-palace/
Disallow /silive_sandbox_blog_-_azywusko
Disallow /toprailapp/
Disallow /weddings/index.ssf/2017/07/silive_wedding_july17.html
Disallow /news/2017/12/your_mental_health_struggle_do.html
Disallow /*mt-preview-*
Disallow /sponsor-content-index-test
Disallow /sponsor-content-test
Disallow /sponsor-content-test
Disallow /sponsor-content-p-test
Disallow /puzzles-games/
Disallow /comics/
Disallow /puzzle-society/
Disallow /go-comics/
Disallow /comics-kingdom/
Disallow /entertainment/2018/03/christina_aguilera_owns_her_si.html

twitterbot

Rule Path
Disallow /home-beta/
Disallow /test/
Disallow /preview/
Disallow /staging/
Disallow /clients/
Disallow /auctions/
Disallow /cgi-bin/
Disallow /printer/
Disallow /*/print.html*
Disallow /*/print.ssf*
Disallow /landingpages/
Disallow /oil-spill-reports/
Disallow /premium/
Disallow /contests/scrape/
Disallow /news/advance/index.ssf?%2Fbase%2Fnews%2F1176538514249150.xml&amp%3Bcoll=1
Disallow /news/advance/index.ssf?%2Fbase%2Fnews%2F117681121166360.xml&amp%3Bcoll=1
Disallow /westshore/index.ssf/2009/02/staten_island_psychic_reaches.html
Disallow /northshore/index.ssf/2010/09/jury_still_out_on_new_forest_a.html
Disallow /opinion/2019/02/want-to-stop-fake-news-pay-for-the-real-thing-commentary.html
Disallow /puzzles-palace/
Disallow /silive_sandbox_blog_-_azywusko
Disallow /toprailapp/
Disallow /weddings/index.ssf/2017/07/silive_wedding_july17.html
Disallow /news/2017/12/your_mental_health_struggle_do.html
Disallow /*mt-preview-*
Disallow /sponsor-content-index-test
Disallow /sponsor-content-test
Disallow /sponsor-content-test
Disallow /sponsor-content-p-test
Disallow /puzzles-games/
Disallow /comics/
Disallow /puzzle-society/
Disallow /go-comics/
Disallow /comics-kingdom/
Disallow /entertainment/2018/03/christina_aguilera_owns_her_si.html

*

Rule Path
Disallow /digitalsubscription/imagedownload/
Back to top

Other Records

Field Value
sitemap https://www.silive.com/arc/outboundfeeds/rss-latest/?outputType=xml
sitemap https://www.silive.com/arc/outboundfeeds/sitemap-index/?outputType=xml
sitemap https://www.silive.com/arc/outboundfeeds/news-sitemap-index/?outputType=xml
Back to top