tvklan.al
robots.txt

Robots Exclusion Standard data for tvklan.al

Resource Scan

Scan Details

Site Domain tvklan.al
Base Domain tvklan.al
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-01-30T07:47:11+00:00
Next Scan 2026-03-31T07:47:11+00:00

Last Successful Scan

Scanned2025-10-30T09:49:43+00:00
URL https://tvklan.al/robots.txt
Domain IPs 104.26.14.124, 104.26.15.124, 172.67.72.82, 2606:4700:20::681a:e7c, 2606:4700:20::681a:f7c, 2606:4700:20::ac43:4852
Response IP 104.26.14.124
Found Yes
Hash 529b6fd1823e7bda05027b4d4d3c42dea43f2a43cd0cdf08faef2ef2697386db
SimHash d4136950a0a2

Groups

*

Rule Path
Disallow /wp-json/
Disallow /search/
Disallow /search$
Disallow /search?

magpie-crawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

perplexity-user

Rule Path
Disallow /

google-cloudvertexbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

yandexadditional

Rule Path
Disallow /

yandexadditionalbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://tvklan.al/sitemaps/index.xml