heute-wohnen.de
robots.txt

Robots Exclusion Standard data for heute-wohnen.de

Resource Scan

Scan Details

Site Domain heute-wohnen.de
Base Domain heute-wohnen.de
Scan Status Ok
Last Scan2024-06-22T18:08:05+00:00
Next Scan 2024-07-22T18:08:05+00:00

Last Scan

Scanned2024-06-22T18:08:05+00:00
URL https://heute-wohnen.de/robots.txt
Redirect https://www.heute-wohnen.de/robots.txt
Redirect Domain www.heute-wohnen.de
Redirect Base heute-wohnen.de
Domain IPs 77.75.251.50
Redirect IPs 77.75.251.50
Response IP 77.75.251.50
Found Yes
Hash d3c3944b12858bed5365ac48dd4869ef845221eea98e7aa580b62f2c0491a367
SimHash 4a02d417fe15

Groups

*

Rule Path
Disallow /pdf/
Disallow /address_book_process.php
Disallow /account.php
Disallow /account_edit.php
Disallow /account_edit_process.php
Disallow /account_history.php
Disallow /account_history_info.php
Disallow /address_book.php
Disallow /checkout_process.php
Disallow /checkout_shipping.php
Disallow /checkout_address.php
Disallow /checkout_confirmation.php
Disallow /checkout_payment.php
Disallow /checkout_success.php
Disallow /checkout_amazon_callback.php
Disallow /cookie_usage.php
Disallow /create_account.php
Disallow /create_account_process.php
Disallow /create_account_success.php
Disallow /download.php
Disallow /info_shopping_cart.php
Disallow /login.php
Disallow /logoff.php
Disallow /password_forgotten.php
Disallow /popup_image.php
Disallow /popup_search_help.php
Disallow /product_notifications.php
Disallow /product_reviews_write.php
Disallow /redirect.php
Disallow /shopping_cart.php
Disallow /tell_a_friend.php

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 12

idealo-bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 12

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

aranhabot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 6

turnitinbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

yandex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

wozukaufenbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.heute-wohnen.de/google_sitemap.php
sitemap http://www.heute-wohnen.de/blog/category-sitemap.xml
sitemap http://www.heute-wohnen.de/blog/post-sitemap.xml
sitemap http://www.heute-wohnen.de/blog/page-sitemap.xml