de.linuxcapable.com
robots.txt

Robots Exclusion Standard data for de.linuxcapable.com

Resource Scan

Scan Details

Site Domain de.linuxcapable.com
Base Domain linuxcapable.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-16T03:56:03+00:00
Next Scan 2024-09-23T03:56:03+00:00

Last Successful Scan

Scanned2024-08-25T03:54:46+00:00
URL https://de.linuxcapable.com/robots.txt
Redirect https://linuxcapable.com/de/robots.txt
Redirect Domain linuxcapable.com
Redirect Base linuxcapable.com
Domain IPs 104.26.4.191, 104.26.5.191, 172.67.69.138, 2606:4700:20::681a:4bf, 2606:4700:20::681a:5bf, 2606:4700:20::ac43:458a
Redirect IPs 104.26.4.191, 104.26.5.191, 172.67.69.138, 2606:4700:20::681a:4bf, 2606:4700:20::681a:5bf, 2606:4700:20::ac43:458a
Response IP 172.67.69.138
Found Yes
Hash 916b4d366c82d12c600441d670ab1b80460946671d47a899f017397e6dd226a2
SimHash 691e1951a496

Groups

*

Rule Path
Allow /wp-admin/admin-ajax.php
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /xmlrpc.php

alphaai

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

archivebot

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot
cohere-ai

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

httrack 3.0

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

omgili

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /

rssingbot

Rule Path
Disallow /

voltron

Rule Path
Disallow /

Other Records

Field Value
sitemap https://linuxcapable.com/sitemap_index.xml
sitemap https://linuxcapable.com/post-sitemap1.xml
sitemap https://linuxcapable.com/post-sitemap2.xml
sitemap https://linuxcapable.com/post-sitemap3.xml
sitemap https://linuxcapable.com/page-sitemap.xml
sitemap https://linuxcapable.com/category-sitemap.xml

Warnings

  • 3 invalid lines.