ausbildungsstellen.de
robots.txt

Robots Exclusion Standard data for ausbildungsstellen.de

Resource Scan

Scan Details

Site Domain ausbildungsstellen.de
Base Domain ausbildungsstellen.de
Scan Status Ok
Last Scan2024-10-03T01:54:00+00:00
Next Scan 2024-10-10T01:54:00+00:00

Last Scan

Scanned2024-10-03T01:54:00+00:00
URL https://ausbildungsstellen.de/robots.txt
Redirect https://www.ausbildungsstellen.de/robots.txt
Redirect Domain www.ausbildungsstellen.de
Redirect Base ausbildungsstellen.de
Domain IPs 178.63.39.225, 2a01:4f8:222:198d::4
Redirect IPs 178.63.39.225, 2a01:4f8:222:198d::4
Response IP 178.63.39.225
Found Yes
Hash a4bc9f03ac011900f01089dd9b96b8efb3b56778126ae4a88d42c1440dd6965e
SimHash 477c50c08119

Groups

*

Rule Path
Disallow /suche.html
Disallow /suche.html*
Disallow /homepage.php
Disallow /homepage.php*
Disallow /ads.html
Disallow /agb.html
Disallow /job.php
Disallow /job.php*
Disallow /ausbildungsbewertungen/schreiben/
Disallow /ausbildungsbewertungen/schreiben/*
Disallow /ausbildungsbewertungen/schreiben.html
Disallow /ausbildungsbewertungen/schreiben.html*
Disallow /videos/
Disallow /videos/*
Disallow /agb_ideenkraftwerk_gmbh.pdf

bingbot

Rule Path
Disallow /suche.html
Disallow /suche.html*
Disallow /homepage.php
Disallow /homepage.php*
Disallow /ads.html
Disallow /agb.html
Disallow /job.php
Disallow /job.php*
Disallow /ausbildungsbewertungen/schreiben/
Disallow /ausbildungsbewertungen/schreiben/*
Disallow /ausbildungsbewertungen/schreiben.html
Disallow /ausbildungsbewertungen/schreiben.html*
Disallow /videos/
Disallow /videos/*
Disallow /agb_ideenkraftwerk_gmbh.pdf

Other Records

Field Value
crawl-delay 2

mediapartners-google

Rule Path
Disallow

sistrix

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

ahrefsbot

Rule Path
Disallow /

femtosearchbot

Rule Path
Disallow /

mtrobot

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

gigabot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

teoma

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

bleriot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60