pcgames.de
robots.txt
Robots Exclusion Standard data for pcgames.de
Resource Scan
Scan Details
| Site Domain | pcgames.de |
| Base Domain | pcgames.de |
| Scan Status | Ok |
| Last Scan | 2024-11-09T18:42:40+00:00 |
| Next Scan | 2024-11-16T18:42:40+00:00 |
Last Scan
| Scanned | 2024-11-09T18:42:40+00:00 |
| URL | https://pcgames.de/robots.txt |
| Redirect | https://www.pcgames.de/robots.txt |
| Redirect Domain | www.pcgames.de |
| Redirect Base | pcgames.de |
| Domain IPs | 104.22.34.145, 104.22.35.145, 172.67.39.218, 2606:4700:10::6816:2291, 2606:4700:10::6816:2391, 2606:4700:10::ac43:27da |
| Redirect IPs | 104.22.34.145, 104.22.35.145, 172.67.39.218, 2606:4700:10::6816:2291, 2606:4700:10::6816:2391, 2606:4700:10::ac43:27da |
| Response IP | 104.22.35.145 |
| Found | Yes |
| Hash | 8169cebbf867888647cce809df1d9b902782b398a819f4189ea5fd562bbfca3f |
| SimHash | 779f50bac883 |
Groups
ccbot
acontbot
amznkassocbot
aboutusbot
acoon-robot
advista
aqua_products
backdoorbot/1.0
baiduspider
blogpulselive
blowfish/1.0
bookmark search tool
botalot
builtbottough
bullseye/1.0
bunnyslippers
cfnetwork
cabot
cazoodlebot
charlotte
cheesebot
cherrypicker
cherrypickerelite/1.0
cherrypickerse/1.0
cizilla
copyrightcheck
crescent
crescent internet toolpak http ole control v.1.0
crystalsemanticsbot
custo
dittospyder
djangotraineebot
download ninja
emailcollector
emailsiphon
emailwolf
erocrawler
euripbot
eurobot
exdomain
exabot
extractorpro
fairad client
fairshare
fetch
flaming attackbot
foobot
gaisbot
galbot
getright/4.2
gigabot
httrack
harvest/1.5
heinrichdermiragorobot
ia_archiver
ia_archiver/1.6
irlbot
infonavirobot
iron33/1.0.2
jennybot
jobroboter
kenjin spider
keyword density/0.9
lnspiderguy
lexibot
linguee
linkscan/8.1a unix
linkwalker
linkextractorpro
miixpc
miixpc/4.2
msiecrawler
magpierss
mail.ru
mata hari
meltwater
microsoft url control
microsoft url control - 5.01.4511
microsoft url control - 6.00.8169
microsoft.url.control
mister pix
mozilla/4.0 (compatible; bullseye; windows 95)
nicerspro
nebullabot
nerdbynature.bot
netants
netmechanic
netluchs
nutch
ocelli
offline explorer
openbot
openfind
openfind data gathere
oracle ultra search
pagesinventory
perman
peterbot
plukkie
propowerbot/2.14
prowebwalker
python-urllib
queryn metasearch
rma
radiation retriever 1.1
repomonkey
repomonkey bait & tackle/v1.01
ruky-bot
seokicks-robot
scoutjet
sensis web crawler
sitesnagger
snapbot
snoopy
sosospider
spankbot
speedy
surveybot
surveybot_ignoreip
szukacz/1.4
tasapspider
teleport
teleportpro
telesoft
tencenttraveler
the intraformant
thenomad
touche
true_robot
true_robot/1.0
url control
url_spider_pro
urly warning
vci
vci webviewer vci webviewer win32
www-collector-e
web image collector
webauto
webbandit
webbandit/3.50
webcopier
webenhancer
webreaper
websauger
webstripper
webzip
webzip
webzip/4.0
webmastercoffee
webmasterworldforumbot
website quester
webster pro
wget
wget/1.5.3
wget/1.6
xenu
xenu's
xenu's link sleuth 1.1c
yandex bot
youdaobot
zealbot
zeus
zeus 32297 webster pro v2.9 win32
zeus link scout
zyborg
adnbot
asterias
b2w/0.1
cityreview
cosmos
crawly
discobot
dotbot
dotbot
echobot
envolk
eventax
findlinks
flatlandbot
fyberspider
gonzo*
grub
grub-client
heise-it-markt-crawler
hloader
httplib
humanlinks
iccrawler
ichiro
iearthworm
infometrics-bot
jobs.de-robot
kalooga
larbin
laycat
libweb/clshttp
libwww
linko
looksmart
lwp-request
lwp-trivial
lwp-trivial/1.34
moget
moget/2.1
naughtyrobot
psbot
search17
searchlink
searchpreview
semager
sitecheck.internetseer.com
spanner
stalker
suggybot
suzuran
thesubot
tocrawl/urldispatcher
trendiction
turingos
uipbot
uipbot/1.0 (uipbot@semasio.net)
wget
woriobot
heritrix
trendictionbot
ahrefsbot
sentibot
landau-media-spider
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /preisvergleich/ |
| Disallow | /Fun-Stuff-Thema-216569/Bilder/ |
| Disallow | /external/cose/ |
| Disallow | /Computec25/ |
| Disallow | /*?menu= |
| Disallow | /Suche/ |
| Disallow | /commoncfm/flowneu/ |
| Disallow | /commoncfm/sharer.php |
| Disallow | /datenschutz/ |
| Disallow | /Alle-Artikel/ |
| Disallow | /Artikel-Archiv/ |
| Disallow | /Tags/ |
| Disallow | /tags/ |
| Disallow | /external/newsletter_v3/ |
| Disallow | /common/pageflip/ |
| Disallow | /webservices/aref/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.pcgames.de/sitemapIndex.xml |
| sitemap | https://www.pcgames.de/sitemapNews.xml |
Warnings
- 2 invalid lines.
Comments