The organizers presented this at COSCUP 25 [1]. It's both extremely ambitious and impressive. They said there are basically no restrictions on who can join -- high schoolers to postdocs, and it's completely based on personal aptitude. The fastest students can finish in a few months, and some go on for years. Students who graduate finish an actual chip that (probably among other things) runs a video game.
I can't find the video of the talk where either Eelco Dolstra (nix) or Todd Gamblin (spack) talks about this, but IIRC it's a design decision; you generally don't go spelunking in the nix store, but if you do, and you ls /nix/store, you'll see a huge list of packages, and due to the hash being a constant length, you can visually separate the tails of the package names like
Spack, another deterministic builder / package manager, IIRC uses the reversed order so the hash is at the tail. Pros/cons under different search / inspection conditions.
Comparing nix style (hash head, package tail), and spack style (package head hash tail), and speaking from my own limited experience, the need arises in different cases, which also depends on the ease of tooling,
sometimes I'm grepping in /nix/store and you have (as shown earlier) a list of derivation paths like this
$ ls /nix/store | grep nodejs-2 | head | sed 's/^/ /'
you get the package name immediately from the left, which is nice, and you can pipe that straight to `sort`, but where the hash starts is more jagged on the right so there's a bit more noise when you're looking at the numbers. In the end the information is identical and it's a UX difference.
Tradeoffs wise I think they both made the right choice. Because for nix, the packages are almost always in /nix/store, so the path length including the hash is almost always the same.
For spack you can place your packages anywhere so the base directories can be highly variable, and so it's sensible to have the package names immediately after the package directory.
Or, I'm just trying to rationalize the choices each designer made post-hoc. But after using both I appreciate the design considerations that went in. In the end, humans are inefficient. When I make name / version / hash identifiers in my own applications I end up using one or the either design.
Super interested in this as well (and thank you for Caddy)
How does this handle data updating / fixing? My use case is importing data that's semi structured. Say you get data from a 3rd party provider from one dump, and it's for an event called "jog". Then they update their data dump format so "jog" becomes subdivided into "light run" vs "intense walk", and they also applied it retroactively. In this case you'd have to reimport a load of overlapping data.
I saw the FAQ and it only talks about imports not strictly additive.
I am dealing with similar use cases of evolving data and don't want to deal with SQL updating, and end up working entirely in plain text. One advantage is that you can use git to enable time traveling (for a single user it still works reasonably).
In the advanced import settings, you can customize what makes an item unique or a duplicate. You can also configure how to handle duplicates. By default, duplicates are skipped. But they can also be updated, and you can customize what gets updated and which of the two values to keep.
But yes, updates do run an UPDATE query, so they're irreversible. I explored schemas that were purely additive, so that you could traverse through mutations of the timeline, but this got messy real fast, and made exploring (reading) the timeline more complex/slow/error-prone. I do think it would be cool though, and I may still revisit that, because I think it could be quite beneficial.
Thanks for the reply! I'll have to try this out... it almost looks like what perkeep was meant to become.
One interesting scenario re time traveling is if we use an LLM somewhere in data derivation. Say there's a secondary processor of e.g. journal notes that yield one kind of feature extraction, but the model gets updated at some point, then the output possibilities expand very quickly. We might also allow human intervention/correction, which should take priority and resist overwrites. Assuming we're caching these data then they'll also land somewhere in the database and unless provenance is first class, they'll appear just as ground truth as any other.
Bitemporal databases look interesting but the amount of scaffolding above sqlite makes the data harder to manage.
So if I keep ground truth data as text, looks like I'm going to have an import pipeline into timelinize, and basically ensure that there's a stable pkey (almost certainly timestamp + qualifier), and always overwrite. Seems feasible, pretty exciting!
Have you heard of XTDB / Bitemporality? The basic idea is to make time 2-dimensional, where each record has both a System Time range and a Valid Time range. Designed as a write-only db with full auditability for compliance purposes.
With 2D time you can ask complex questions about what you knew when, with simpler questions automatically extended into a question about the current time. Like:
"What is the price?" -> "What is the price today, as of today?"
"What was the price in 2022" -> "What was the price in 2022, as of today?"
"What was the price in 2022, as of 2023?"
You probably don't want to just switch to XTDB, but if you pursue this idea I think you should look into 2D time as I think it is schematically the correct conceptualization for this problem.
Yeah, I did actually pursue this for a time (heh), but I might revisit it later. It was too much complexity for debateable value-add, though the value is growing on me.
I was a long time org mode user and used babel extensively [1]. It was a marvelous experience, but the market chose markdown. I still drop into org-mode some times for specific features, and the babel integration is still in its own league, but technically there's nothing preventing something similar for markdown.
I pulled some org-babel code to make a code fence evaluator for markdown a while back [2] but haven't found myself needing it that much. So without a need to wrangle reports or run exports, I think it's a 1% feature for text file connoisseurs.
[1] https://ess.r-project.org/ with babel is an experience I find superior to jupyter tools even after all these years
Conforming to “the market” is just a pandoc call away. I export to markdown, pdf, html and even docx all the time when i need to share with colleagues. I prefer writing in org-mode because of the complete ecosystem it offers inside emacs. But also just as plaintext I really miss src blocks in markdown.
Yup, never had no problem with that. My notes are in Org - if I ever need to grab something to share with a colleague, my yanking command is advised - with a prefix it calls Pandoc and converts to markdown and vice-versa - to get something of markdown into my notes - it all takes just a keypress.
Following "the market" often feels like reinventing yourself over and over only for the sake of throwing that knowledge into the garbage pile of deprecation - do you guys remember the days when market dictated the use of Dreamweaver, Silverlight, SOAP and SVN?
If market tomorrow barks that everyone should be using "Blockchain-ML" or "UltraScript Deluxe" formats, I'm not gonna convert million lines of my notes to whatever crap gets trendy. When I say "Org-mode", there's no confusion - it will remain the same even a decade later, but Markdown? Which one is it - CommonMark, GFM, MultiMarkdown, Kramdown...? It looks like we're not done "perfecting" it, maybe markdown doomed to remain in 'beta'... forever?
Sure, the market chose Markdown, but this simply led me to the conclusion that the market isn’t worth following. Of course the mismatch creates some friction, but the benefits of org-mode, for me personally, easily outweigh that.
I advocate (and use) nix + home manager too but when you need a one-off change or test and realize you need to do the whole commit and switch thing, or when you are debugging and spelunk through the read only nix store, or when you set up a new (non-nix) computer and do the nix install, home-manager install, run the switch and get a deluge of errors...
it's simultaneously awesome but "can I really recommend this to <colleague>?"
With nix + home manager, you can use `mkOutOfStoreSymlink` to make symlinks between the dotfile repo and the target destination in `.config`. I've found this to be the most ergonomic way to have nix-managed dotfiles. Because the out-of-store dotfile repo is symlinked, you can make little changes to your system without doing the whole commit and switch dance.
For example, here's a snippet pulled from my dotfiles that does this for multiple dotfiles at once:
At the same time it often feels like a veneer of control, like you can control exactly where to place the door, but what's in the messy room (like emacs profiles if you do that) might be hidden behind the very nice and solid door.
It's like in python projects I lock python3 and uv, and beyond that it's wild west. Still beats everything else, still feels a bit incomplete, and still feels somewhat unresolvable.
The configs I tweak are git, bash and emacs, and each has their own way to load extra config from a local file. You can use this for stateful config local to a machine and out of the nix store.
It depends on buy-in from the tool so it’s not a panacea but it works for my use cases at least. I also don’t like to switch config for every change and it turns out I rarely have to.
I wanted to track my progress to the highest level of detail possible. The motor motions are somewhat similar, but slightly more complex, than typing on the keyboard. But since I already type fluently I wouldn't be able to get as fresh data, so the soroban was a nice other task.
The main goals were to track learning progress and truly experience what it meant to have a virtual abacus in my mind's eye.
Author here. Very surprised to see this on HN years after I stopped!
Also, thanks for the UI critique, points taken! The design decision was to hide nitty gritty detail about the background unless the reader was extra interested, in hopes of making an article with gradual / adjustable depth so readers who aren't as interested can get a tl;dr for each section (I still don't know how a nice UI to do this, although LLMs make the idea more in reach now).
I love this idea for how we could have dynamic LoD for articles. I've thought about this a lot over the years and also never landed on a great ux. Maybe llms will help us; it would be great to see these tools shape novel, powerful ux (in the hands of the people).
Thanks for your articles. About your "The surprising benefits of disciplined, deliberate self-logging", I couldn't understand and it seemed cryptic how you're logging? The biologger.md is throwing 404. Could you explain?
Wow, thanks for digging! Looks like I haven't committed that file! It's an incomplete description of the android app that I wrote to output the jsonl in the self-logging piece going through some comparisons of existing options in 2019 and design decisions for the app. In short, it's like myfitnesspal tailored to my use case. 3 years later, the data is still a sprawl, but the logging strategy is stable.
Choose core.async.flow when you want a declarative, monitorable process graph over channels (pause/resume, error channel, flow monitor).
Choose Manifold when you need deferreds/streams as general primitives, precise per-op backpressure, error-aware chaining, or you’re in the Aleph/Netty stack.
very abstract but based on this, the answer is no.
