Log In Sign Up

pussycamworld.com
robots.txt

Robots Exclusion Standard data for pussycamworld.com

Resource Scan

Scan Details

Site Domain pussycamworld.com
Base Domain pussycamworld.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-09-10T12:06:37+00:00
Next Scan 2024-12-09T12:06:37+00:00

Last Successful Scan

Scanned2024-01-22T09:37:19+00:00
URL https://pussycamworld.com/robots.txt
Domain IPs 66.85.156.2
Response IP 66.85.156.2
Found Yes
Hash 039d35e65d64582e3398a78604b6f69a95082ef7d566326b61aad729c0df3535
SimHash 64ca1062c3a2

Groups

*

Rule Path
Disallow /online.js

adbeat_bot

Rule Path
Disallow /

agentlinkspammer

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ahrefsbot/4.0

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

aihitbot/1.0

Rule Path
Disallow /

aihitbot/1.1

Rule Path
Disallow /

acoon

Rule Path
Disallow /

arachmo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baiduspider+(+http://www.baidu.com/search/spider.htm)

Rule Path
Disallow /

baiduspider/2.0;+http://www.baidu.com/search/spider.html

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

mozilla/5.0(compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

comodospider/nutch-1.2

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

exabot/3.0

Rule Path
Disallow /

exalead

Rule Path
Disallow /

exalead crawler

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

ezooms/1.0

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mozilla/5.0 (compatible; ezooms/1.0; ezooms.bot[at]gmail[dot]com)

Rule Path
Disallow /

findlinks/2.6

Rule Path
Disallow /

java/1.6.0_04

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

kaloogabot

Rule Path
Disallow /

mail.ru_bot/2.0

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

mail.ru_bot/2.0; +http://go.mail.ru/help/robots

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mj12bot/v1.4.3

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

ichiro 3.0

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

netcraftsurveyagent/1.0

Rule Path
Disallow /

openwebindex/nutch-1.6

Rule Path
Disallow /

openwebindex

Rule Path
Disallow /

panoptastudybot

Rule Path
Disallow /

checks.panopta.com

Rule Path
Disallow /

psbot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

seznambot/1.0

Rule Path
Disallow /

seznambot/1.1

Rule Path
Disallow /

seznambot/3.0

Rule Path
Disallow /

sistrixcrawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sosospider+

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sosospider/2.0

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

wada.vn

Rule Path
Disallow /

wada.vn vietnamese search

Rule Path
Disallow /

wada.vn vietnamese search/2.1

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandex/1.01.001

Rule Path
Disallow /

yandexbot/3.0-mirrordetector

Rule Path
Disallow /

yandeximages/3.0

Rule Path
Disallow /

yandexsomething/1.

Rule Path
Disallow /

yandex.com

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

youdaobot/1.0

Rule Path
Disallow /

youdaobot/1.0

Rule Path
Disallow /

zao

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /cgi-bin/
Disallow /cgi-sys/

*

No rules defined. All paths allowed.

Back to top

Other Records

Field Value
sitemap /sitemap.xml
Back to top

Comments

  • Adbeat ads
  • AgentLinkSpammer
  • AhrefsBot ads
  • aiHitBot Ukraine or Russia
  • Acoon Germany
  • Arachmo Japan
  • Baiduspider China and Japan
  • careerbot Germany
  • COMODOSpider/Nutch-1.2 United Kingdom
  • EasouSpider - China
  • Exabot/3.0 - France proxy scraper
  • Exalead proxy scraper France
  • Ezooms and dotbot
  • findlinks/2.6 Germany http://wortschatz.uni-leipzig.de/findlinks
  • Java/1.6.0_04
  • JikeSpider China
  • KaloogaBot Netherlands contextual advertising
  • Mail.RU_Bot/2.0 Russia
  • Mail.RU Russia
  • Mail.Ru Russia
  • MJ12bot United Kingdom
  • MJ12bot/v1.4.3 United Kingdon
  • Ichiro Japan
  • Ichiro 3.0 Japan
  • NetcraftSurveyAgent/1.0
  • OpenWebIndex/Nutch-1.6 Germany
  • panoptaStudyBot checks.panopta.com monitor
  • panoptaStudyBot checks.panopta.com monitor
  • picsearch Sweden searches for pictures
  • plukkie Dutch (botje.nl)/Belgium (botje.be)/France (botje.fr)/United Kingdom (botje.co.uk) search engine
  • SeznamBot Czech Republic
  • SeznamBot/3.0
  • SistrixCrawler Germany DE
  • Sogou
  • Sosospider - China http://help.soso.com/webspider.htm
  • Sosospider - China
  • Sosospider/2.0 - China may not obey robots.txt
  • 360Spider China
  • SurveyBot
  • Wada.vn Vietnamese Search/2.1
  • Yandex
  • YisouSpider China
  • YoudaoBot/1.0 China
  • YoudaoBot China
  • Zao - Japan
Back to top

Warnings

  • 8 invalid lines.
Back to top