This post introduces ButlerOS, a virtual operating system that runs multi-players applications on the web. In two parts, I present:
- The project’s overview, and,
- A motivating use-case; a remote desktop that looks like this:
Hobby Project
Before I begin, let me set the expectations. Butler is a hobby project inspired by the SerenityOS project. I am working on it on my free time as a challenge and learning experience. Moreover I am considering alternative runtimes and protocols. Therefore the implementation is subject to change.
That being said, the project reached a point where it is now enjoyable, and I’m happy to share its current state with you.
Modular Operating System
Butler’s goal is to implement a virtual operating system with a strong focus on concurrency. Butler is not a computer hardware operating system. Instead, Butler is an application environment that runs on top of an existing system.
Thus I created the following modules:
Starting with the Core
layer, I designed low level interfaces for the host system:
-
Process
for thread management, powered by the ki library. -
Pipe
for inter-process communication using stm. -
File
for directory and file objects. -
Network
to serve Web Application Interface wai.
Then in the Main
layer, I implemented higher level interfaces on top of the Core
primitive:
-
Scheduler
for the process hierarchy and supervision. -
Display
for graphical user interfaces. -
Session
for users. -
Frame
for the a data exchange format.
Finally, the App
layer provides the application environment:
-
Window
for app instance. -
Cursor
for user pointer. -
Sound
to produce and receive audio. -
Agent
to forward local agent.
I find it valuable to model these APIs after regular operating system constructs. They are available through a single top level Butler module. The next section introduces the application environment.
Application Actor
Butler applications are implemented as standalone processes that operate a AppEvent pipe. Checkout the App definition. Applications start with a AppContext provided by the display.
In order to run multiple application concurrently, Butler features a data exchange format to share a single socket per client. Applications output:
-
HTMX payload, by suffixing the application id. For example the UI are mounted on:
<div id="w-$pid">
. -
Binary payload, by encoding the application id at the beginning of the buffer:
| $pid | server-data |
Applications receive:
-
HTMX trigger, by suffixing the trigger name with the application id:
{"HEADERS": {"HX-Trigger": "$trigger-$pid"}}
-
Binary payload, using the same application id encoding:
| $pid | client-data |
The key design is that clients are dumb terminals and the state is controlled server side. A good example is the text editor app named Noter. This app leverages the whole application environment to implement an etherpad like text editor.
Haskell RunTime System
In this section I explain the choice of using Haskell. Beyond the language’s common features, here are two reasons for using Haskell:
- The language lets you break down any problem into smaller chunks, using let or where bindings. I find this syntax and way of thinking very helpful for simplifying complex software.
- The REPL evaluates modification almost instantaneously. I run development code through ghcid, and thanks to a simple client’s reconnect handler, the services are hot reloaded in milliseconds when the source changes.
However, even though the Haskell ecosystem is quickly improving, it is not a mainstream language, and it can be rough around the edges. Here are a couple of alternative languages that I am also considering:
- Rust, because it is very similar to Haskell. Though it can be more complicated: for example, some expressions don’t seem to be decomposable into sub expressions without triggering the borrow checker. And the lack of REPL to evaluate the code interactively is frustrating.
- Gleam, because it leverages the BEAM virtual machine, which is a great fit for application actors.
That being said, Haskell works great for Butler. So far I have only used its most basic features, and the project source is only 5k lines of code according to polyglot.
HTML5 Graphical User Interface
In this section I explain the choice of using HTML5 for the GUI. I choose HTML5 because it is an ubiquitous standard and the Web API offers a fully featured application environment. Thanks to the hypermedia paradigm enabled by htmx, it is a great fit for Butler.
Though it would be interesting to investigate morphdom and rich client alternatives, such as Web GPU or Flutter.
Application Demos
Here is a list of demos I am using to validate the core implementation:
- Butler.App.Noter: a text editor.
- Butler.App.FileManager: a file manager.
- Butler.App.Mumbler: a voice chat.
- Butler.App.Tabletop: a free form tabletop simulator.
- Butler.App.Terminal: a legacy terminal.
- Butler.App.NoVnc: a VNC client.
- And a few other system applications you can find in the Butler.App directory.
I am mostly focusing on the desktop use-case, but I also helped my colleague fboucher use Butler to create a game named HazardHunter.
Road-map
A terminal and VNC client are already provided to access non Butler applications, and perhaps it is possible to re-implemented these legacy system as Butler native applications. For example, I am looking forward adding:
- Programming REPL.
- Media player named “Ampler”.
- IRC and Matrix client.
- Agenda and calendar.
- More video games.
Moreover, I would like to develop the Butler APIs to include:
- Application testing framework, to simulate actor’s inputs and validate the outputs.
- User’s mailbox and notification system.
- Application debugging strace.
- External identity providers.
- Authorization system.
Finally, it would be interesting to investigate if and how an app registry could be implemented. For example as a list of packages that could be dynamically loaded/unloaded into a running instance.
Conclusion
The actor framework looks like a great solution for web applications, and I am quite happy with the Butler’s implementation. Moreover, I am excited about the community discussion for moving Towards an actor framework for Haskell. My implementation is rather limited, and I would be happy to use a common library similar to the erlang/otp.
Butler may sound like a crazy project, but I believe it is important to pursue fun goals that appear impossible. I would like to quote the author of SerenityOS Andreas Kling: “Never underestimate the power of a bunch of nerds building something just for fun.”
Finally I would like to thank elmiko for his early feedback and encouragement.
Cheers!