Log In Sign Up

boyens-medien.de
robots.txt

Robots Exclusion Standard data for boyens-medien.de

Resource Scan

Scan Details

Site Domain boyens-medien.de
Base Domain boyens-medien.de
Scan Status Ok
Last Scan2024-06-02T10:06:45+00:00
Next Scan 2024-06-09T10:06:45+00:00

Last Scan

Scanned2024-06-02T10:06:45+00:00
URL https://boyens-medien.de/robots.txt
Redirect https://www.boyens-medien.de/robots.txt
Redirect Domain www.boyens-medien.de
Redirect Base boyens-medien.de
Domain IPs 94.127.225.120
Redirect IPs 94.127.225.120
Response IP 94.127.225.120
Found Yes
Hash 6c22eb0495fb88bfd8ae057b81bc31a52525893ab383fc55bffefa2532e1dd3b
SimHash b03075584997

Groups

*

Rule Path
Allow /

*

Rule Path
Disallow /fileadmin/_processed_/csm_anke_hansen_18651582c6.jpg

*

Rule Path
Disallow /fileadmin/_processed_/csm_julia_lansink_371dfb6974.jpg

*

Rule Path
Disallow /fileadmin/vertrieb/leserreisen/veranstalter/Julia_Lansink_Leserreisen_01.jpg

*

Rule Path
Disallow /fileadmin/_processed_/9/7/csm_Frau_Lansink_neu_236d26438f.jpg

*

Rule Path
Disallow /uploads/pics/Julia_Lansink_Leserreisen.jpg

*

Rule Path
Disallow /uploads/pics/julia_lansink_01.jpg

*

Rule Path
Disallow /uploads/tx_botrauer/img/262075-A_300.jpg

*

Rule Path
Disallow /uploads/tx_botrauer/img/263300-A_300.jpg

*

Rule Path
Disallow /typo3temp/_processed_/csm_IMG_6665_c57fd_c_27d36c6df5.jpg

*

Rule Path
Disallow /typo3temp/_processed_/csm_Oranje_20boven_ff3ae_46f9769524.jpg

blexbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

*

Rule Path
Disallow /*?tx_kesearch_pi1*

*

Rule Path
Disallow /suche*

*

Rule Path
Disallow /trauerportal/anzeige/*

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Comments

  • Legal notice: boyens-medien.de expressly reserves the right to use its content for commercial text and data mining (ยง 44b Urheberrechtsgesetz).
  • The use of robots or other automated means to access boyens-medien.de or collect or mine data without the express permission of boyens-medien.de is strictly prohibited.
  • boyens-medien.de may, in its discretion, permit certain automated access to certain boyens-medien.de pages,
  • If you would like to apply for permission to crawl boyens-medien.de, collect or use data, please email info@boyens-medien.de
Back to top