baby-und-familie.de
robots.txt

Robots Exclusion Standard data for baby-und-familie.de

Resource Scan

Scan Details

Site Domain baby-und-familie.de
Base Domain baby-und-familie.de
Scan Status Ok
Last Scan2024-11-12T04:41:49+00:00
Next Scan 2024-11-19T04:41:49+00:00

Last Scan

Scanned2024-11-12T04:41:49+00:00
URL https://baby-und-familie.de/robots.txt
Redirect https://www.apotheken-umschau.de/robots.txt
Redirect Domain www.apotheken-umschau.de
Redirect Base apotheken-umschau.de
Domain IPs 83.135.48.65
Redirect IPs 83.135.48.65
Response IP 83.135.48.65
Found Yes
Hash e3abd985a8ce15f7c699bb473a310cdce5b35a3412e7af85b1359a64d3903fa8
SimHash 70d299c1e017

Groups

linkchecker

Rule Path
Disallow /uploads/
Disallow /ajax/
Disallow /services/
Disallow /external/polls/
Disallow /vendor/

sitebot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

slurp

Rule Path
Disallow /uploads/
Disallow /ajax/
Disallow /services/
Disallow /external/polls/
Disallow /vendor/

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

applebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

youbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

*

Rule Path
Disallow /uploads/
Disallow /ajax/
Disallow /services/
Disallow /external/polls/
Disallow /vendor/

Other Records

Field Value
sitemap https://www.apotheken-umschau.de/sitemap_news/
sitemap https://www.apotheken-umschau.de/sitemap/sitemap_articles/
sitemap https://www.apotheken-umschau.de/sitemap_video/
sitemap https://www.apotheken-umschau.de/sitemap_index

Comments

  • robots.txt for portal 'Apotheken Umschau'
  • Yahoo Slurp crawler
  • AI Bots
  • Screaming Frog
  • All User Agents
  • Sitemaps