ausbildungsanzeigen.de
robots.txt

Robots Exclusion Standard data for ausbildungsanzeigen.de

Resource Scan

Scan Details

Site Domain ausbildungsanzeigen.de
Base Domain ausbildungsanzeigen.de
Scan Status Ok
Last Scan2024-09-26T01:52:10+00:00
Next Scan 2024-10-03T01:52:10+00:00

Last Scan

Scanned2024-09-26T01:52:10+00:00
URL https://ausbildungsanzeigen.de/robots.txt
Redirect https://www.ausbildungsanzeigen.de/robots.txt
Redirect Domain www.ausbildungsanzeigen.de
Redirect Base ausbildungsanzeigen.de
Domain IPs 136.243.84.232
Redirect IPs 136.243.84.232
Response IP 136.243.84.232
Found Yes
Hash 430baa24b89d693f53a39dba3516783c313fc703614097a46673a8ac2f6e1543
SimHash 6344f4a3d235

Groups

*

Rule Path
Disallow /go/*
Disallow /impressum.html
Disallow /datenschutz.html
Disallow /ads.html
Disallow /agb_ideenkraftwerk_gmbh.pdf
Disallow /agb.html

bingbot

Rule Path
Disallow /go/*
Disallow /impressum.html
Disallow /datenschutz.html
Disallow /ads.html
Disallow /agb_ideenkraftwerk_gmbh.pdf
Disallow /agb.html

Other Records

Field Value
crawl-delay 3

iccrawler - icjobs

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow

jobboersebot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /stellenanzeigen-inserieren/firmenlogos/
Disallow /stellenanzeigen-inserieren/firmenlogos/*
Disallow /images/
Disallow /images/*
Disallow /img/
Disallow /img/*

msnbot-media

Rule Path
Disallow /stellenanzeigen-inserieren/firmenlogos/
Disallow /stellenanzeigen-inserieren/firmenlogos/*
Disallow /images/
Disallow /images/*
Disallow /img/
Disallow /img/*

psbot

Rule Path
Disallow /stellenanzeigen-inserieren/firmenlogos/
Disallow /stellenanzeigen-inserieren/firmenlogos/*
Disallow /images/
Disallow /images/*
Disallow /img/
Disallow /img/*

magpie-crawler

Rule Path
Disallow /stellenanzeigen-inserieren/firmenlogos/
Disallow /stellenanzeigen-inserieren/firmenlogos/*
Disallow /images/
Disallow /images/*
Disallow /img/
Disallow /img/*

yandeximages

Rule Path
Disallow /stellenanzeigen-inserieren/firmenlogos/
Disallow /stellenanzeigen-inserieren/firmenlogos/*
Disallow /images/
Disallow /images/*
Disallow /img/
Disallow /img/*

yandexmedia

Rule Path
Disallow /stellenanzeigen-inserieren/firmenlogos/
Disallow /stellenanzeigen-inserieren/firmenlogos/*
Disallow /images/
Disallow /images/*
Disallow /img/
Disallow /img/*

yahoo-mmcrawler

Rule Path
Disallow /stellenanzeigen-inserieren/firmenlogos/
Disallow /stellenanzeigen-inserieren/firmenlogos/*
Disallow /images/
Disallow /images/*
Disallow /img/
Disallow /img/*

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

cliqzbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

femtosearchbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mtrobot

Rule Path
Disallow /