daftartelefon.com
robots.txt

Robots Exclusion Standard data for daftartelefon.com

Resource Scan

Scan Details

Site Domain daftartelefon.com
Base Domain daftartelefon.com
Scan Status Ok
Last Scan2024-09-19T16:05:48+00:00
Next Scan 2024-10-19T16:05:48+00:00

Last Scan

Scanned2024-09-19T16:05:48+00:00
URL https://daftartelefon.com/robots.txt
Domain IPs 194.5.188.240
Response IP 194.5.188.240
Found Yes
Hash 48ba34a3e2fa58bc2b7363df2c6344b166aa00832c566325f207de98fcdd5bc7
SimHash a80e6563c1b2

Groups

*

Rule Path
Allow /*.js***************
Allow /*.css**************
Allow /*.webp**************
Allow /*.png**************
Allow /*.jpg**************
Allow /*.jpeg**************
Allow /*.gif**************
Allow /*.eot**************
Allow /*.woff**************
Allow /*.woff2**************
Allow /*.ttf**************
Allow /*.svg**************
Allow /*.otf**************
Allow /*.pdf**************
Allow /*.PNG**************
Allow /*.JPG**************
Allow /*.JPEG**************
Allow /*.mp3**************
Allow /*.mp4**************
Allow /*.webm**************
Allow /*.pdf**************
Disallow /administrator/
Disallow /cache/
Disallow /cgi-bin/
Disallow /cli/
Disallow /includes/
Disallow /language/
Disallow /libraries/
Disallow /logs/
Disallow /tmp/
Disallow /test/
Disallow /test/*
Disallow /*/edit

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

w3c_validator

Rule Path
Disallow /

w3c_css_validator

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

acoon

Rule Path
Disallow /

pingdom

Rule Path
Disallow /

proximic

Rule Path
Disallow /

phantom.js bot

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

faviconizer

Rule Path
Disallow /

yie8

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

a6-indexer/1.0

Rule Path
Disallow /

a6-indexer

Rule Path
Disallow /

butterfly

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

voltron

Rule Path
Disallow /

obot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

Other Records

Field Value
sitemap https://daftartelefon.com/index.php?option=com_jmap&view=sitemap&format=xml&dataset=1
sitemap https://daftartelefon.com/index.php?option=com_jmap&view=sitemap&format=xml&dataset=2
sitemap https://daftartelefon.com/index.php?option=com_jmap&view=sitemap&format=xml&dataset=6
sitemap https://daftartelefon.com/index.php?option=com_jmap&view=sitemap&format=xml&dataset=8

Comments

  • Please don't remove folders from disallow.
  • The allows at the top allow any of the mimetypes listed to be crawled within any folder
  • using long-tail wildcards, these ignore the disallows for the folders below.
  • This gives full render for the search engines whilst preventing full crawls of system
  • folders
  • THIS ALLOWS FULL RENDER AT ENGINES
  • THESE FOLDERS SHOULD NEVER BE CRAWLED