travel-images.com
robots.txt

Robots Exclusion Standard data for travel-images.com

Resource Scan

Scan Details

Site Domain travel-images.com
Base Domain travel-images.com
Scan Status Ok
Last Scan2025-06-02T13:08:50+00:00
Next Scan 2025-06-09T13:08:50+00:00

Last Scan

Scanned2025-06-02T13:08:50+00:00
URL https://travel-images.com/robots.txt
Domain IPs 66.96.149.18
Response IP 66.96.149.18
Found Yes
Hash e63b65029eae6842989b09e776fe6b365279777b93a1e1b5c1e28dd4d30546da
SimHash aa21dee58423

Groups

*

Rule Path
Disallow /naveg/
Disallow *.doc
Disallow *.txt
Disallow *.zip
Disallow *-map.jpg
Disallow maps*.*
Disallow ba-ti.html
Disallow ba-azerb.html

the python robot

Rule Path
Disallow /

calif

Rule Path
Disallow /

crawl

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow

mediapartners-google*

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

googlebot-mediapartners

Rule Path
Disallow

google*

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot

Rule Path
Disallow

msnbot

Rule Path
Disallow

msn*

Rule Path
Disallow

bing

Rule Path
Disallow

doubleclick

Rule Path
Disallow

msnbot

Rule Path
Disallow

yahoo*

Rule Path
Disallow

yahoo

Rule Path
Disallow

aol*

Rule Path
Disallow

aol

Rule Path
Disallow

a9

Rule Path
Disallow

jeeves

Rule Path
Disallow

voila

Rule Path
Disallow

ibm_planetwide

Rule Path
Disallow

voila

Rule Path
Disallow

lycos

Rule Path
Disallow

nomad

Rule Path
Disallow

exabot

Rule Path
Disallow

atomz

Rule Path
Disallow

gulliver

Rule Path
Disallow

slurp

Rule Path
Disallow

ingrid

Rule Path
Disallow

ilsebot/1.0

Rule Path
Disallow

infoseek

Rule Path
Disallow

scooter

Rule Path
Disallow

accoona

Rule Path
Disallow

abachobot

Rule Path
Disallow

aport

Rule Path
Disallow

arachnoidea

Rule Path
Disallow

baiduspider

Rule Path
Disallow

docomo

Rule Path
Disallow

exactseek

Rule Path
Disallow

iltrovatore

Rule Path
Disallow

ilsebot

Rule Path
Disallow

mozilla

Rule Path
Disallow

opera

Rule Path
Disallow

teoma

Rule Path
Disallow

verzeichnis_1

Rule Path
Disallow

verzeichnis_2

Rule Path
Disallow

exalead

Rule Path
Disallow

exalead.com

Rule Path
Disallow

exaleadng

Rule Path
Disallow

Other Records

Field Value
sitemap http://travel-images.com/urllist.txt