catchit.com
robots.txt

Robots Exclusion Standard data for catchit.com

Resource Scan

Scan Details

Site Domain catchit.com
Base Domain catchit.com
Scan Status Ok
Last Scan2025-12-29T01:52:54+00:00
Next Scan 2026-01-12T01:52:54+00:00

Last Scan

Scanned2025-12-29T01:52:54+00:00
URL https://catchit.com/robots.txt
Redirect https://www.catchit.com/robots.txt
Redirect Domain www.catchit.com
Redirect Base catchit.com
Domain IPs 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133
Redirect IPs 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133
Response IP 146.75.46.133
Found Yes
Hash 054ca5d41afc25d126ec9650c2dcce1e53fa56b98efc2cbaba864dafb0dd6834
SimHash 4352c9e0eec7

Groups

*

Rule Path
Disallow /es/
Disallow /us/
Disallow /tracking/
Disallow /tracker/
Disallow /miatrapalo/
Disallow /hoteles/pics/
Disallow /hotels/feed-rss/
Disallow /hotels/hide/
Disallow /hotels/profile/lite/*
Disallow /hoteles/profile/reviews/*
Disallow /de/hotels/feed-rss/
Disallow /de/hotels/hide/
Disallow /de/hotels/profile/lite/*
Disallow /de/hotels/profile/reviews/*
Disallow /fr/hotels/feed-rss/
Disallow /fr/hotels/hide/
Disallow /fr/hotels/profile/lite/*
Disallow /fr/hotels/profile/reviews/*
Disallow /pt/hoteis/feed-rss/
Disallow /pt/hoteis/hide/
Disallow /pt/hoteis/profile/lite/*
Disallow /pt/hoteis/profile/reviews/*
Disallow /it/hotel/feed-rss/
Disallow /it/hotel/hide/
Disallow /it/hotel/profile/lite/*
Disallow /it/hotel/profile/reviews/*
Disallow */tracking/
Disallow */miatrapalo/
Disallow *_search/
Disallow */results/
Disallow */resultados/
Disallow */common/photo/map/hoteles*
Disallow */common/photo/hotelmap*

megaindex.ru

Rule Path
Disallow /

npbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

grub

Rule Path
Disallow /

larbin

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webtrafficexpress

Rule Path
Disallow /

vayalacreep

Rule Path
Disallow /

mercator

Rule Path
Disallow /

httrack

Rule Path
Disallow /

pabloelrobot

Rule Path
Disallow /

wget

Rule Path
Disallow /

teleport

Rule Path
Disallow /

superbot

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

nocilla

Rule Path
Disallow /

henrythemiragorobot

Rule Path
Disallow /

harvest-ng/1.0.2

Rule Path
Disallow /

abachobot

Rule Path
Disallow /

bumblebee@relevare.com

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

true_robot/1.0

Rule Path
Disallow /

true_robot

Rule Path
Disallow /

szukacz/1.4

Rule Path
Disallow /

zeus

Rule Path
Disallow /

zeus 32297 webster pro v2.9 win32

Rule Path
Disallow /

openfind

Rule Path
Disallow /

Comments

  • CATCHIT
  • Blocked bots

Warnings

  • 2 invalid lines.