I think OCaml is a great language. While it’s always too simplistic to put languages on a straight line, I feel it’s alright to say that it’s somewhere in between Java/C#/Go and Rust/Haskell for a couple of reasons:

• Immutability is the default, but you can use types that are mutable if you need/want to
• You can throw exceptions if you want, or play it “safe” with a Result type
• Concurrency can be handled through monadic concurrency libraries (async/lwt) or the new effect handlers system/library (eio), which is very different to the lower-level “spawn a thread/goroutine” way of doing things

I could go on about differences and nuances (in particular the module system, which I love), but today I wanted to talk about my experience of actually trying to do something with OCaml.

The CRUD Web Service

As with most other things done in this day and age, I’m going to make a web service that is mostly CRUD. The architecture will be pretty plain, without any bells or whistles:

The intent is to just throw a binary (and possibly some resource files) up on a machine behind nginx, with a backup script of the SQLite3 database every night. Sounds straightforward enough, and I think it’s a great entry to new languages.

I’d like to skip ReasonML for now. HTMX and/or pure HTML is just fine to begin with in my experience. I can look at how to introduce that later on.

Opam/Dune/Merlin is Great

Installing Opam, Dune and Merlin and getting the “Hello World”-example set up, even with dependencies, is pretty straightforward. It’s been a while since I’ve done it, and sure, there is a tiny bit of configuration of your shell and your editor, but it feels like the setup of any other language out there these days.

The next step is to figure out what kind of web server I should use. I’m not really a fan of web frameworks, but I think they’re alright when you’re new to a language and just want to get something up and running – especially if it’s for a hobby project.

Dream is Well Documented and Looks Good

After roughly 5 seconds of Googling, I found Dream, which looks very well polished. It has a lot more features than I initially need and don’t really care that much about, but two things I see on the example page make me happy:

1. An example with SQLite3
2. An example with TyXML (producing type-safe HTML with OCaml)

That’s the barebones stuff I need to get started, and if I need more, I have the option to look at the other examples – there are loads of them!

Taking It Up For a Spin

Starting up a new Dune project is fairly straightforward:

$ dune init proj myapp
$ cd myapp
$ dune exec -w myapp

Adding a dependency on Dream is also easy: change the line in dune-project like so:

@@ 18,24 18,24 @@

(package
 (name myapp)
 (synopsis "A short synopsis")
 (description "A longer description")
- (depends ocaml dune)
+ (depends ocaml dune dream)
 (tags
  (topics "to describe" your project)))

Then issue

$ opam install . --deps-only

to get the dependencies installed.

Thorny Dependencies

The SQL library that seems to be the de-facto standard, and is what’s used in the Dream examples, is called Caqti. This is where my bigger issues started to appear: When I began following caqti-study, I was thrown off by a library not being found.

File "lib/dune", line 3, characters 39-53:
3 |  (libraries caqti caqti-driver-sqlite3 caqti-lwt.unix))
                                           ^^^^^^^^^^^^^^
Error: Library "caqti-lwt.unix" not found.
-> required by ...

Time for bug hunting I guess.

Investigating a bit, I found out that caqti-study was (naturally) updated to the last version of Caqti, v2.1. When Caqti moved from 1.x → 2.x, they moved the connection logic away from caqti-lwt to the sublibrary caqti-lwt.unix. However, Dream depends on Caqti 1.x and wouldn’t work with 2.x.

A little infuriating, but not the end of the world. Let’s follow the Dream example instead then, and stay with Caqti 1.x for now…

Or maybe not? The same issue appears if copy the Dream example:

Entering directory '/home/jeannikl/projects/dream'
File "src/sql/dune", line 7, characters 2-16:
7 |   caqti-lwt.unix
      ^^^^^^^^^^^^^^
Error: Library "caqti-lwt.unix" not found.
-> required by ...

Seems like version 2.x of Caqti is supported by Dream in the main branch, as done in PR #302, but not yet deployed to Opam.

Hmm… Since the latest released Dream version is 1.0.0~alpha5, 1.0.0 (or a new alpha) should be around the corner. I’d like to avoid having to upgrade the breaking changes from Caqti v1 to v2 when I bump Dream as well, especially when all recent documentation is about version 2. So now I need to find a way to depend on Dream from the last commit pushed to GitHub.

It’s Not Esy

While investigating how to depend on packages that are not in the opam repository, I stumbled upon Esy. Esy appeals to me very much, as it manages packages for your projects in an isolated environment, instead of having all your dependencies smeared together in a global one. I didn’t consider it to begin with because I just wanted to get something up and going, but since I had to pin stuff I may as well inspect it.

I quickly ended up backtracking from that decision: Following the instructions to install Esy itself was straightforward, but the “Hello World”-example gave me this:

$ git clone https://github.com/esy-ocaml/hello-reason.git
$ cd hello-reason
$ esy
... lots of output ...
    signals_nat.c:184:13: error: variably modified 'sig_alt_stack' at file scope
      184 | static char sig_alt_stack[SIGSTKSZ];
          |             ^~~~~~~~~~~~~
    make[3]: *** [Makefile:367: signals_nat.n.o] Error 1
    make[3]: *** Waiting for unfinished jobs....
    make[3]: Leaving directory '/home/jeannikl/.esy/3/b/ocaml-4.12.0-bfdfbfff/runtime'
    make[2]: *** [Makefile:765: makeruntimeopt] Error 2
    make[2]: Leaving directory '/home/jeannikl/.esy/3/b/ocaml-4.12.0-bfdfbfff'
    make[1]: *** [Makefile:215: opt.opt] Error 2
    make[1]: Leaving directory '/home/jeannikl/.esy/3/b/ocaml-4.12.0-bfdfbfff'
    make: *** [Makefile:275: world.opt] Error 2
    error: command failed: './esy-build' (exited with 2)
    esy-build-package: exiting with errors above...

  building ocaml@4.12.0
esy: exiting due to errors above

Yikes, building OCaml failed with the Hello World example? We’ll come back to the cause for this failure, but at the moment I wasn’t too interested in investigating this. Not only because it’s further down the rabbit hole, but also because I’d likely have to tune my emacs/tuareg/Merlin setup, and at this point I’d rather get something up and running.

Back to Opam

Another reason was that, once I knew how to do it, pinning Opam towards a Git repository was very straightforward. The short answer is

$ opam pin dream --dev-repo

if you want to pin the latest commit, but you can replace --dev-repo with a URL to the commit you’d like to pin to.

Great, we finally got Dream up and running with SQLite3! By testing on the side, I know I can get either JSON or HTML out the door without as much fiddling around as I’ve used on the database side.

ppx_rapper

However, before I start writing code with this setup, I wanted to see if there’s anything else I should use alongside Caqti. Many people seem to recommend ppx_rapper so I looked into that for a bit. ppx_rapper allows me to write things like this

let users =
  [%rapper
    get_opt
      {sql|
SELECT @int{id}, @string{username}, @bool{following}, @string?{bio}
FROM users
WHERE following = %bool{following}
  AND username IN (%list{ %int{ids}})
      |sql}]

which seems pretty handy. So let’s try to get that one up and working as well.

The installation tells me to install ppx_rapper and ppx_rapper_lwt, so I’ll do just that:

$ opam install ppx_rapper ppx_rapper_lwt
The following actions will be performed:
  ∗ install   pg_query             0.9.7          [required by ppx_rapper]
  ⊘ remove    dream                1.0.0~alpha5*  [conflicts with caqti]
  ↘ downgrade caqti                2.1.1 to 1.9.0 [required by ppx_rapper]
  ∗ install   ppx_rapper           3.1.0
  ↘ downgrade caqti-lwt            2.1.1 to 1.9.0 [required by ppx_rapper_lwt]
  ↘ downgrade caqti-driver-sqlite3 2.1.1 to 1.9.0 [uses caqti]
  ↘ downgrade caqti-async          2.1.1 to 1.9.0 [uses caqti]
  ∗ install   ppx_rapper_lwt       3.1.0
===== ∗ 3   ↘ 4   ⊘ 1 =====
Do you want to continue? [Y/n]

oh no. Turns out ppx_rapper isn’t updated to work with Caqti 2.x either. But even worse, you can’t just opam pin ppx_rapper --dev-repo yourself out of the situation.

So I dug in and investigated, as one does. There’s fortunately not too much to look at, I just needed to look a the pull requests and find this one. That one adds support for Caqti 2.x, and there are installation steps in there as well for the ones patiently waiting. So let’s go ahead and follow those:

$ opam pin add 'git+https://github.com/roddyyaga/ppx_rapper#2222edbbe68db7ba1ab0c7a2688c227ea5c0f230'
  [...]

# compiling src/postgres/src_port_snprintf.c
# src/postgres/src_port_snprintf.c:374:1: error: conflicting types for ‘strchrnul’; have ‘const char *(const char *, int)’
#   374 | strchrnul(const char *s, int c)
#       | ^~~~~~~~~
# In file included from ./src/postgres/include/c.h:61,
#                  from src/postgres/src_port_snprintf.c:62:
# /usr/include/string.h:286:14: note: previous declaration of ‘strchrnul’ with type ‘char *(const char *, int)’
#   286 | extern char *strchrnul (const char *__s, int __c)
#       |              ^~~~~~~~~
# gmake: *** [Makefile:140: src/postgres/src_port_snprintf.o] Error 1

<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ λ build pg_query 0.9.7
└─

For wanting to try out OCaml, there’s been a terrifying amount of errors from C compilation so far. And, somewhat annoyingly, this one is related to a Postgres feature in ppx_rapper, though I’m going to use SQLite3.

After investigating ppx_rapper, the pattern to fix the issue seems to be fairly similar now:

1. Find the original repo for pg_query
2. Find the PR that fixes the build issue (#17)
3. Pin the PR’s commit by issuing

   $ opam pin add 'git+https://github.com/repo#shasum
   

In this case,

$ opam pin add 'git+https://github.com/roddyyaga/pg_query-ocaml#eff9c46bf58ba966b1cf3f23c77a6566492d37ab'

was all it took to fix the issue.

aaand we’re finally up and running!

So What Failed, Exactly?

That was the experience without going into the issues in detail, so let’s do that now. I think the summary is that all of the issues had effectively two out of these three properties:

1. Build issues because of breaking changes upstream
2. Stale/abandoned projects
3. Stuff slipping through the cracks

Both the build issues with Esy and pg_query were related to “breaking” changes in glibc. The “Hello World” Esy project depended on an OCaml version that in turn depended upon a glibc constant that turned dynamic, whereas pg_query has some C code that depends on glibc NOT defining the function strchrnul, which it does from 2.38 and onwards.

Work on the Esy package manager is still ongoing and pretty active, so I would assume that the hello-reason GitHub repo just slipped through the cracks. In fact, I’m pretty sure that’s indeed the case, as a pull request to fix the issue was merged just 4 days ago.

The same can’t be said for pg_query: It seems like the author has been silent for quite some time. This is also the author of ppx_rapper, which has the same fate: A caqti dependency on 1.x prevents you from using it without pinning it to a PR commit. However, both pg_query and ppx_rapper are more or less done from what I can tell, but unfortunately some of their upstream dependencies had breaking changes.

Dream is only indirectly in the “build issues because of breaking changes upstream”-category: The examples you follow for SQL integration fail because 1.0.0~alpha5 depends on Caqti 1.x. Is it stale? Well, it seems like work on the project is coming in bursts, so presumably not – though the “examples are further ahead than the release”-issue is a bit jarring.

Managing Stale Projects

As I said, some projects seem to be abandoned/stale, as the authors haven’t added any commits nor replied to issues or PRs. But let me be very clear here: The authors of these open-source projects have no obligation to maintain them, and there is a myriad of reasons why a developer won’t¹. Rather, there has to be some system in place so that someone can take over the baton.

I’d assume there is some way to transfer ownership, as the opam repository is a Git repository where new deploys are pull requests. However, I think it’s more reasonable to fork off the project rather than to transfer the ownership. How, or even if, you can denote that in the opam repository, seems to be unclear to me.

One thing I like with both Maven and Go’s way of fetching dependencies is that they are namespaced: This makes it easier to denote whether something is official, and also makes it easier to fork without polluting the repository with more names. For example, go get github.com/aws/aws-sdk-go tells me that this is an official AWS library for Go, whereas opam install aws doesn’t tell me anything about the authors. And because there’s no namespace, if I were to fork this library, I’d have to call it hypirion-aws or something. From what I can’t see, I can’t mark it as a fork of aws, and if aws got abandoned and my fork became the standard, people would still do opam install aws if they were new to the language. Not really a good solution in my opinion.

An Unlucky but Unfortunate Start

Regardless of abandonment and forking, I think I just got unlucky. I’d assume most other OCaml libraries are stable and relatively well-updated, with no glibc/C issues present.

However, that bad luck isn’t only present on my machine: Everyone wanting to use Dream will experience some of these issues, perhaps all of them. That makes the first impression of OCaml as a language with a lot of abandoned projects and broken builds – which will make a lot of developers backtrack and go back to their original language. Learning a new language is one thing, but having to fight the package manager for what seems to be a trivial setup is the straw that breaks the camel’s back for many.

… that being said, OCaml openly states that it’s not exactly there yet on their “Is OCaml Web yet?”-page:

I’m not 100% sure what the solution to this is. It’s easy to say “fork and fix the stale projects”, but it’s a chicken and egg problem: You need people to use the language to have contributors to open-source, and those usually come when you have something you can use.

Part of the solution is probably foundations that promote and support developers so that they can work on open-source projects a couple of days every month. It seems like the OCaml Software Foundation is doing that to some extent, but they’re clearly not looking for monetary support from individuals. Perhaps a foundation akin to Clojurists Together could focus more on open-source projects, rather than teaching, core development and community events, which seems to be the main focus of the OCaml Software Foundation.

Not Recommended… Yet

For all of these reasons, I’d not unequivocally recommend OCaml for web development right now. I’d only consider it if you either

• are very tenacious and want to learn OCaml,
• think OCaml is the holy grail or something, or
• enjoy tinkering with setups and investigating build issues

Most people aren’t there, so if you just want to make something, I’d say another language (maybe Rust or Haskell) would suit your needs better.

That being said, the immediate issues I faced shouldn’t be hard to solve. I’m trying to see if I can help out fixing them, or at least make the issue less iffy for people trying to do the same thing. If that’s the case and the rest of my experience is smooth, I would recommend OCaml for small web services – with the usual caveat related to API support.