Log In Sign Up

xiptv.cat
robots.txt

Robots Exclusion Standard data for xiptv.cat

Resource Scan

Scan Details

Site Domain xiptv.cat
Base Domain xiptv.cat
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-28T02:53:46+00:00
Next Scan 2024-12-27T02:53:46+00:00

Last Successful Scan

Scanned2023-02-13T15:57:16+00:00
URL http://xiptv.cat/robots.txt
Redirect https://www.alacarta.cat/robots.txt
Redirect Domain www.alacarta.cat
Redirect Base alacarta.cat
Domain IPs 13.81.66.35
Redirect IPs 13.81.66.35
Response IP 13.81.66.35
Found Yes
Hash e8e8f0d2725e90b620cc6feccebf02604fd5a420e72511bff325003384b4dcba
SimHash 4286dd62445b

Groups

panscient.com

Rule Path
Disallow /

vscooter

Rule Path
Disallow /

psbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

yandex

Rule Path
Disallow /

taptubot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

facebot

Rule Path
Allow *

twitterbot

Rule Path
Allow *

twengabot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

infopath

Rule Path
Disallow /

infopath.2

Rule Path
Disallow /

swebot

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

searchmetericsbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

ip-web-crawler.com

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

aboundex

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

spbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

a6-indexer

Rule Path
Disallow /

riddler

Rule Path
Disallow /

loadtimebot

Rule Path
Disallow /

obot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

advbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

lssrocketcrawler

Rule Path
Disallow /

gsa-crawler

Rule Path
Disallow /

nutch

Rule Path
Disallow /

tbot-nutch

Rule Path
Disallow /

thunderstone

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

ranksonicbot

Rule Path
Disallow /

betabot

Rule Path
Disallow /

parsijoo-bot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

gocrawl

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

applebot

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

rankactivelinkbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

seeker

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

yoozbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /*.jpg
Disallow /*.JPG
Disallow /*.png
Disallow /*.PDF
Disallow /*.pdf
Disallow /*.mp3
Disallow /*.MOV
Disallow /*.mov
Disallow /*.AVI
Disallow /*.avi
Disallow /disclaimer.html
Disallow /security.html
Disallow /poweredby.html
Disallow /about_smythies.html
Disallow /unused_link.html
Disallow /old_pages.html
Disallow /index_0*
Disallow /*index_0*$
Disallow /digital_camera/
Disallow /lab/
Disallow /xmas_*
Disallow /~doug/archives/
Disallow /~doug/linux/ubuntu-docs/help.ubuntu.com/
Disallow /bot_trap.html

googlebot

Rule Path
Disallow /*.jpg$
Disallow /*.JPG$
Disallow /*.png$
Disallow /*.PDF$
Disallow /*.pdf$
Disallow /*.mp3$
Disallow /*.MOV$
Disallow /*.mov$
Disallow /*.AVI$
Disallow /*.avi$
Disallow /index_0*$
Disallow /*index_0*$
Disallow /xmas_*
Disallow /~doug/archives/
Disallow /~doug/linux/ubuntu-docs/help.ubuntu.com/
Disallow /~doug/2010.01.23/
Disallow /~doug/2007.11.20/
Disallow /~doug/2004.06.26/
Disallow /digital_camera/
Disallow /old_pages.html
Disallow /unused_link.html
Disallow /disclaimer.html
Disallow /security.html
Disallow /about_smythies.html
Disallow /poweredby.html
Disallow /DSCN*.htm
Disallow /bot_trap.html

msnbot

Rule Path
Disallow /*.jpg$
Disallow /*.JPG$
Disallow /*.png$
Disallow /*.PDF$
Disallow /*.pdf$
Disallow /*.mp3$
Disallow /*.MOV$
Disallow /*.mov$
Disallow /*.AVI$
Disallow /*.avi$
Disallow /disclaimer.html
Disallow /security.html
Disallow /poweredby.html
Disallow /about_smythies.html
Disallow /unused_link.html
Disallow /old_pages.html
Disallow /index_0*
Disallow /*index_0*$
Disallow /digital_camera/
Disallow /lab/
Disallow /xmas_*
Disallow /~doug/archives/
Disallow /~doug/linux/ubuntu-docs/help.ubuntu.com/
Disallow /bot_trap.html

bingbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 1800

qwantify

Rule Path
Disallow /*.jpg
Disallow /*.JPG
Disallow /*.png
Disallow /*.PDF
Disallow /*.pdf
Disallow /*.mp3
Disallow /*.MOV
Disallow /*.mov
Disallow /*.AVI
Disallow /*.avi
Disallow /disclaimer.html
Disallow /security.html
Disallow /poweredby.html
Disallow /about_smythies.html
Disallow /unused_link.html
Disallow /old_pages.html
Disallow /index_0*
Disallow /*index_0*$
Disallow /digital_camera/
Disallow /lab/
Disallow /xmas_*
Disallow /~doug/archives/
Disallow /~doug/linux/ubuntu-docs/help.ubuntu.com/
Disallow /bot_trap.html

Other Records

Field Value
crawl-delay 3600

*

Rule Path
Disallow /*.jpg
Disallow /*.JPG
Disallow /*.png
Disallow /*.PDF
Disallow /*.pdf
Disallow /*.mp3
Disallow /*.MOV
Disallow /*.mov
Disallow /*.AVI
Disallow /*.avi
Disallow /disclaimer.html
Disallow /security.html
Disallow /poweredby.html
Disallow /about_smythies.html
Disallow /unused_link.html
Disallow /old_pages.html
Disallow /index_0*
Disallow /*index_0*$
Disallow /digital_camera/
Disallow /lab/
Disallow /xmas_*
Disallow /~doug/archives/
Disallow /~doug/linux/ubuntu-docs/help.ubuntu.com/
Disallow /bot_trap.html
Disallow /

Other Records

Field Value
crawl-delay 1800
Back to top

Comments

  • Disallow: /*.jpg$
  • Disallow: /*.JPG$
  • Disallow: /*.png$
  • Disallow: /*.PDF$
  • Disallow: /*.pdf$
  • Disallow: /*.mp3$
  • Disallow: /*.MOV$
  • Disallow: /*.mov$
  • Disallow: /*.AVI$
  • Disallow: /*.avi$
  • Disallow: /disclaimer.html
  • Disallow: /security.html
  • Disallow: /poweredby.html
  • Disallow: /about_smythies.html
  • Disallow: /unused_link.html
  • Disallow: /old_pages.html
  • Disallow: /index_0*
  • Disallow: /*index_0*$
  • Disallow: /digital_camera/
  • Disallow: /lab/
  • Disallow: /xmas_*
  • Disallow: /~doug/archives/
  • Disallow: /~doug/linux/ubuntu-docs/help.ubuntu.com/
  • Disallow: /bot_trap.html
Back to top