Log In Sign Up

tldr.sus.fr
robots.txt

Robots Exclusion Standard data for tldr.sus.fr

Resource Scan

Scan Details

Site Domain tldr.sus.fr
Base Domain sus.fr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-09-07T16:03:24+00:00
Next Scan 2025-10-07T16:03:24+00:00

Last Successful Scan

Scanned2025-07-17T08:01:22+00:00
URL https://tldr.sus.fr/robots.txt
Domain IPs 185.151.30.169, 2a07:7800::169
Response IP 185.151.30.169
Found Yes
Hash 84150390ccd3cf6ab391a2c33bc9eaebd9b49f4c6fc7e6d9baed3c42aeb20c74
SimHash 38d4691145e0

Groups

ia_archiver

Rule Path
Allow /

archive.org_bot

Rule Path
Allow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

xenu&

Product Comment
xenu& 039;s
Rule Path
Disallow /

xenu&

Product Comment
xenu& 039;s Link Sleuth 1.1c
Rule Path
Disallow /

httrack

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

um-ln

Rule Path
Disallow /

yak

Rule Path
Disallow /

sogou

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mrgbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

orbbot

Rule Path
Disallow /

repolookoutbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

r6_commentscrawler

Rule Path
Disallow /

r6_feedcrawler

Rule Path
Disallow /

twingly recon

Rule Path
Disallow /

netpeakcheckerbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Back to top

Other Records

Field Value
sitemap https://www.oh.mg/sitemap.xml
sitemap https://www.oh.mg/news-sitemap.xml
Back to top

Comments

  • Allow the internet archiver to access my whole site
  • Fuck all the SEO Bros
  • Fuck the AI Bros
Back to top