Rendered at 16:06:05 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
jayknight 22 hours ago [-]
I've recently switched to SilverBullet[1] as my day-to-day note taking and task tracking app, because of it's very powerful ability to embed little queries and scripts directly in a page to build lists/tables/links that based on data from other pages. This looks like an interesting way to leverage markdown libraries in different ways. Nice!
Cool Hadn't seen this before and will check it out - and add to the Way of Markdown
digdugdirk 18 hours ago [-]
Same! Still getting used to it, but the benefits of being able to host my notes on my Tailscale network are huge. Having a built-in Lua engine to customize the interface and content is a really interesting concept, though I'll admit I haven't explored it beyond adding an AI assisted voice to text button. Still, it's been fun using it!
cxr 21 hours ago [-]
Since this presupposes Markdown of the flavor that comes with YAML frontmatter, and that frontmatter is doing all of the heavy lifting, this is more like a YAML database than a "Markdown database".
In any case, nowhere is it indicated why this approach (no matter what you call it) should be preferred over e.g. recfiles.
rufuspollock 7 hours ago [-]
Yes it is markdown plus frontmatter that makes this work.
Point on comparison with other formats is really well taken. I think the main argument i would have is ubiquity of markdown and markdown tooling.
xamde 20 hours ago [-]
Or just embed structured data in any text file document, not limited to Markdown.
Logseq does this too - or used to. They found that for the rich operations they want to support (live collaboration, richer data), persisting the data in a graph structure makes more sense to them. It probably adds some conveniences, but the file model is just unbeatable for simplicity.
rufuspollock 7 hours ago [-]
I really liked Logseq though i've ended up an Obsidian (and now mostly AI + markdown) user.
Specifically, my observation was lots of people use a collection of markdown files (or other data formats) in a git repository to crowdsource data. This ties in really nicely with a lot of static site generators.
I was interested in seeing if a tool could help data reuse and help people who aren't comfortable with git contribute.
lsb 18 hours ago [-]
As we have more powerful AI plus agents that can run on your phone, keeping everything as a pile of markdown plus some compute on top makes a lot of sense
I told AI to improve on that format and told me having to read all files just to get the frontmatter data is optimizable, sooo... to the trenches B)
AI > The problem with Frontmatter+YAML is O(n) read complexity. To know if a file is a "Person" or a "Meeting Note," the parser must read the entire file into memory, parse the YAML block, and then discard the content. For a vault of 10,000+ files, this becomes a massive I/O bottleneck
skybrian 17 hours ago [-]
Maybe store it in a Sqlite database and present records to the user as frontmatter + Markdown for import/export/editing?
adsharma 16 hours ago [-]
Many competent embedded graph DBs out there. More suitable than graphs on SQLite projects.
adsharma 16 hours ago [-]
For what benefit though?
Much better off storing this in an embedded graph database and use cypher.
MVCC, Indexes, Strong typing, WAL, changefeeds.
The simplicity of markdown and JSON are deceptive. SQLite is a solid choice, but then you deal with graph-relational impedance mismatch.
solarkraft 8 hours ago [-]
Simplicity. It’s a bunch of files, trivial to edit, version and develop against. This is important for things sometimes intended to stay relevant for a long time.
I’d flip the question towards the benefits more complicated solutions give you. One conclusion might be “not enough”.
Your filesystem is very much not a database, and you risk data corruption and loss by using markdown files as a database, especially if you have multiple programs reading and writing to and from the same files. Things will start getting weird and you will start rebuilding an ACID compliant database engine in your application layer.
That said, I love markdown. It's great. Simple.
torisima 4 hours ago [-]
That is absolutely correct.
For applications requiring simultaneous writes from multiple programs (concurrency) or ACID transaction properties, using Markdown files as a substitute for a database is undoubtedly an anti-pattern and entails a risk of data corruption.
rufuspollock 2 hours ago [-]
You're absolutely right ... and this is more for a personal database (the way most people use Notion).
Strictly we could have titled this: the markdown database pattern for small personal databases.
The point was that for personal use e.g. keeping a list of projects, or books etc this is perfect. Or even for maintaining this list on a website.
[1] https://silverbullet.md/
In any case, nowhere is it indicated why this approach (no matter what you call it) should be preferred over e.g. recfiles.
Point on comparison with other formats is really well taken. I think the main argument i would have is ubiquity of markdown and markdown tooling.
https://ddot.it
Disclaimer: My project
A “summary” field that can act as a preview when listing your files so your agent’s context doesn’t get jammed.
Specifically, my observation was lots of people use a collection of markdown files (or other data formats) in a git repository to crowdsource data. This ties in really nicely with a lot of static site generators.
I was interested in seeing if a tool could help data reuse and help people who aren't comfortable with git contribute.
I told AI to improve on that format and told me having to read all files just to get the frontmatter data is optimizable, sooo... to the trenches B)
AI > The problem with Frontmatter+YAML is O(n) read complexity. To know if a file is a "Person" or a "Meeting Note," the parser must read the entire file into memory, parse the YAML block, and then discard the content. For a vault of 10,000+ files, this becomes a massive I/O bottleneck
Much better off storing this in an embedded graph database and use cypher.
MVCC, Indexes, Strong typing, WAL, changefeeds.
The simplicity of markdown and JSON are deceptive. SQLite is a solid choice, but then you deal with graph-relational impedance mismatch.
I’d flip the question towards the benefits more complicated solutions give you. One conclusion might be “not enough”.
Your filesystem is very much not a database, and you risk data corruption and loss by using markdown files as a database, especially if you have multiple programs reading and writing to and from the same files. Things will start getting weird and you will start rebuilding an ACID compliant database engine in your application layer.
That said, I love markdown. It's great. Simple.
For applications requiring simultaneous writes from multiple programs (concurrency) or ACID transaction properties, using Markdown files as a substitute for a database is undoubtedly an anti-pattern and entails a risk of data corruption.
Strictly we could have titled this: the markdown database pattern for small personal databases.
The point was that for personal use e.g. keeping a list of projects, or books etc this is perfect. Or even for maintaining this list on a website.
It isn't for multiple simultaneous writes