In the world of human-readable data formats (ie not programming languages), the best one I ever used was Jane Street's sexplib[1] s-expression format.
It was concise and expressive. There was a direct way to describe variants (types with multiple constructors), which is always awkward in JSON, but the format was still surprisingly low-noise for reading and editing by hand. I remember you could even use it as a lightweight markup format:
(here is some text with (em formatting) information)
And, unlike certain other formats I won't name, it has comments!
It also helps that Emacs with Paredit makes editing s-expressions flow. The tool doesn't need to know anything about the sexplib format specifically; just relying on basic s-expression structure gives us fluid but simple structural editing.
I am constantly sad that nobody else uses this sort of format, and I have to deal with a mixture of JSON, YAML, TOML and other ad-hoc formats instead.
Agree, sexp is quite nice. That was my favorite before json came around. Not that I like json particularly, it just ate the world so it's easier to go along with it.
The author seems to be using "markup language" as a concept basically synonymous with a configuration language, or something that is not a programming language. A markup language is a language used to "mark up" text with formatting and structure. This may sound like a terminology nitpick, but I would argue the reason why for example XML is not a great configuration language is that it was designed to be something else – a markup language.
I think the author is probably aligning with "is" rather than "ought". So a language made this list if it IS being used as a configuration format, regardless of whether it OUGHT to be used as one.
Which is a way of deciding that makes sense given that I think the purpose of this article is "use my language instead". Getting lost in the weeds about each language's original design intent would bloat the article without meaningfully contributing to their thesis.
Given that it's an essential/definitional issue, I would have preferred that the author at least showed awareness of the differences between languages intended for marking up text and languages intended to represent data in a structured way.
Back when XML was first being developed, I was really anticipating having a standardized, easier-to-implement successor to SGML (which was hampered by its complexity and by the cost of the ISO standard) in the text markup space. It was that disappointing it ended up filling the vacuum in the space of serialized representations for for structured data, then getting rejected when it wasn't quite as suitable for that as alternatives such as JSON.
I don't like this article all that much but I do like the MAML format the author wrote this to promote. It seems like it’s everything that’s good about JSON, plus everything fixed for human writing, and nothing more.
Hats off, wish my package.json and build.yml were package.maml and build.maml.
MAML is an improvement from JSON, but does not improve the data types much (although it does make one improvement, which is that it distinguishes integers from floating point), although the syntax is improved.
Depending on the kind of configuration, different formats might be helpful.
I made up TER, which supports all ASN.1 types (although in some cases there is not a built-in syntax for them and you must write them in terms of other types; however, extensions can be made for your specific application) and is not limited to Unicode, nor does it limit integers to 64-bits (and you can write integers in any base up to thirty-six). You can convert TER to DER (I wrote a program to do this; it is not really intended for an application to use the full specification of TER itself except for using an external program to convert to DER and then use that instead), and you might also make a subset of TER for specific applications.
But, like any other one, it also has advantages and disadvantages (including some that are mentioned in that article; if they look at TER they would probably have criticisms about that too).
There are two kind of criticisms with JSON, having to do with the data types, and having to do with the syntax, and I had mentioned some of these in the past (some of them are commonly mentioned by others too, but some are less commonly mentioned by others).
Another configuration format is the X resource manager format.
No mention of The Configuration Complexity Clock [1] which I always think deserves a link, but credit to the author for actually keeping it slim and readable over (IMHO) most of the newer additions to the landscape which either declare themselves as 'obvious' and aren't or just add features such as Pkl's extends keyword, moving us further round the complexity clock.
Of course just adding multiline strings is the start of the rabbit hole, now you need to think about leading line breaks, trailing line breaks, intermediate line breaks, whitespace chomping, and- oh heavens I've reinvented YAML, I think I need to lie down.
[1] https://mikehadlow.blogspot.com/2012/05/configuration-comple...
One kinda-exception I'd like to raise: Cases where you'd like to use the regular proper programming language from the very beginning, but there are trust issues, and there's no good/reliable sandboxing option.
For example, B2B stuff where every customer has their own idiosyncratic sets of rules for if-this-then-that, which change at a different cadence than your releases.
In those cases, it's less that configuration slowly becomes too complex and evolves into code, and more that code is wanted from the get-go but configuration is the compromise.
> every customer has their own idiosyncratic sets of rules for if-this-then-that
So Varnish Configuration Language? It's definitely an awkward case that doesn't seem to fit neatly into the landscape.
Thanks for this - it expresses something I've felt for a while. Solve too hard and you end up back at midnight.
I'm sure there's no relation to The Doomsday Clock.
I wonder if "the right solution" is a programming language that is fast, concise, trivially easy to run, and outputs some efficient binary format like protobuf.
Programming languages have comments and control flow, multiple popular implementations, and can have nice literals. Lack of Turing completeness is actually not a terribly useful feature if you trust the input (and you should probably just use protobufs or similar for untrusted inputs in that case.)
> Programming languages have . . . control flow
I do not want control flow of any kind in my configuration file. Nor do I want expressions or any kind of evaluation.
Greppability is a must-have feature for me. As is simplicity - I don’t want to have to deal with internalizing interpreter mechanics, rules for precedence, variable scope, etc just to figure out what config values my program is going to be provided with.
Any time I’ve been forced to work with a system which used a general or restricted programming language to express configuration, it’s been a nightmare.
There is also .ini, it's simpler than toml, though has more inconsistent implementations and formats. But the '[[table]]' is just '[table]' and I like the '[table.subtable]' syntax better.
But since we're criticizing formats ;-) maml https://maml.dev is fairly good good.
* I like multiline blocks and comment style
* I do not like optional commas, either require them or not. I'd lean toward requiring them.
* Not sure about ordered object keys. How should languages represent that, a map/dict kind of a data structure won't work it would have to be an ordered dict/list of kvs etc. Two objects {"a":"x", "b":"y"} and {"b":"y", "a":"x"} will be different then. I get the idea, but I am leaning toward not liking it by the time I finished typing all that.
* I like booleans and null. Good for not having ons and offs those are just annoying.
I've actually been getting a lot of mileage out of textproto: https://protobuf.dev/reference/protobuf/textformat-spec/
Well typed, simple syntax. Maps are annoying though.
I agree on this in general. The awkward things here are that:
a) textprotos aren't really touted by the protobuf folks as a thing to use outside Google. I'm not 100% sure why this is.
b) inside Google, there's a perception that you shouldn't use textprotos for much other than hardcoding proto values inside the monorepo (where there aren't really schema-versioning concerns). I think this perception is misplaced, you just have to be aware that a given schema is used in textprotos. Which is usually an easy thing to be aware of. This is just because the schema-versioning concerns are different than with binary photos (e.g. field renames are now breaking).
c) IIRC most parsers unconditionally reject unknown fields. I think the reason for this is highlighted in the docs: you can't safely go from a textproto with an unknown field to a binary serialisation of that proto. IIRC there are some parsers that let you parse unknown fields anyway but then I think you're a bit more tied to a specific implementation than you'd like...
One strange but quite handy alternative is actually the JSON representation. You can use .proto files as a schema but then serialise the values to JSON as there's a canonical mapping. Then you get something that's human readable but with the type safety of protos. Although of course it's not really writable since... It's JSON.
Jsonnet solves a different problem though.
Need to configure 5 services with hundreds of replicas in 7 data centers? Some values depend on the service, some on the data center and some on the combination thereof? Maybe also overrides for a bunch of problemstic machines?
And you also want a manageable config language which doesn't turn into a full blown Turing tar pit?
Then jsonnet is for you.
So it's not entirely fair to compare it in the "pleasant syntax" contest. It's like putting a Unimog into a ranking of city cars.
What about bash? https://wiki.archlinux.org/title/PKGBUILD
What is the name of the style found in (e.g.) ISC BIND9:
* https://www.zytrax.com/books/dns/ch6/#master
* https://bind9.readthedocs.io/en/latest/manpages.html#named-c...
ISC-style? (E/A)BNF-ish?
Personally I like (a) the open/close braces for better stanza navigation, (b) all statements/lines ending with semicolon.
(Something similar was used in (now-EOL) ISC DCHPd, and they've moved to "extended JSON" for their new DHCP server, KEA.)
I like json
I kind of wish it had two and only two features: defined ordering, and comments
As to languages for configuration (specifically), I wish there was a good specification for a program config, where you would have one well-defined configuration file for defaults, and a user-specified configuration file for the values you have set/overriden.
say default.config:
{"player_name":"player", "mouse_speed":5}
{"player_name":"me"}
That's a pretty common config pattern. SublimeText does that, and when you edit the settings it opens a 2 paned editor with the default config on the left and user config on the right.
> Well, this is some sort of programming language with dynamic types. But there are so many good programming languages, so I don't know why this one needs to be used.
RE: Jsonnet and others: because it has nice guarantees, like lack of arbitrary I/O and pure execution.
See: https://sre.google/workbook/configuration-specifics/#pitfall...
Personally I have embraced using sqlite as my configuration infrastructure, as the more I've used configuration formats, the more I've felt they are glorified databases.
Glorified? They're just text files.
The reason they're called configuration files is because they're often meant to be edited directly by the user. Which is why there are so many configuration formats, since everyone has a different opinion on the ideal UI/UX.
You can store configuration in an actual database, but asking your users to use SQL to change it is sadistic. In that case you really need a UI to allow users to change configuration, which defeats the purpose of the simplicity of configuration files.
I agree that YAML has its issues. But saying it's bad because it has too many features seems misguided. Do those advanced features get in the way of using a simple YAML file? At least not in my experience.
That being said TOML would be my choice.
Yes, the features in YAML absolutely get in the way. The "Norway problem" for example ("no" translates to "False" in versions prior to YAML 1.2), but that is just the one more people ran into. Here is a nice overview of some of the problems:
https://ruudvanasseldonk.com/2023/01/11/the-yaml-document-fr...
I would argue that most, if not all, of those problems stem from too many features.
Very interesting (and opinionated) overview of the existing configuration languages.
At the very bottom, author is presenting its own language (maml.dev). It is yet another JSON with comments, multiline strings, and optional commas. It's the most boring part of the page.
A good configuration syntax isn’t supposed to be exciting.
agreed, but then even some third-party project is too exciting.
My choice is "whatever is available in the standard library" - which used to be INI, JSON or YAML. But with python 3.11 there is also TOML, which is nice...
There's a direct correlation between how exciting and fun a configuration syntax looks and how much you'll despise it after a year of having to deal with the fallout of those 'exciting features'.
Speaking of which, Pkl looks absolutely fascinating, I might have to check that out...
No but you'd have to do a lot of work to convince me it's really any better then basic YAML (yaml without going nuts with anchors and indirection).
The Kubernetes ecosystem has made me hate YAML so much. Maybe I am just not using the tooling correctly but anchors all over the place, templating, etc.
I have yet to see a solution to that type of thing though as every config language exists on a spectrum of syntax to full blown programing language.
YAML isn't really the problem with Kubernetes though. The Kubernetes problem is that the config is complex and heavily typed, but also essentially ad hoc - outside some basic constructs there's conventions but they're all different or evolved and not uniform.
It does get a lot easier to deal with when you have VS Code tied into CRD definitions for example, butcimmnot convinced config is the problem so much as just Kubernetes is all things to all people and as a result what any given configuration actually does is almost completely random and likely to hit a bunch of edge cases.
Personally, I like configuration languages that don't let you "go nuts" in the first place.
My problem with YAML is that even when you try to keep it simple it tries its best to trip you up with some obscure feature you never wanted to deal with
If you want to use JSON as a configuration language, why keep the top level braces and indentation?
I didn't see any indication in the article, or a glance at the spec, that indentation is significant.
Middle Aged Man Language - too many braces.
A configuration language for what? What kind of configurations?
So, yaml with squiggles. Roger roger.
JSON sans commas, but with extra strings.
The author dismisses many entries, saying "it's a programming language, I want a config language, or I'd use Javascript". I completely disagree with the author, given my experience.
- You want bits of programmability in your config language. Else you end up with copy-pasting pieces and adding comments like "must be the same value as foo.bar below!". This sucks enough that you may end up with a config generator for some custom meta-config language, which you need to produce and maintain. You very certainly need symbolic constants, it's great to have simple arithmetics and string concatenation, and ideally you need a way to map a config fragment over a list.
- You don't want a Turing-complete language as your config language. You also don't want to allow any I/O or other effects in that language. You don't want to allow your config parser to do anything but to form a tree of keys and values in memory after reading the config. Javascript is right out. XML is also right out, because entities allow for external resource links, that is, uncontrolled I/O; this was a source of many exploits.
- Nevertheless, you want your configs to be modular when needed, allowing to graft into your main config imported values from some common templates, specific overrides, etc. Direct inclusion is a poor but serviceable approach.
- You want the values in your config be typed. They can be strings by default to save some typing. But you want your config parser to be able to complain when you write a string instead of a number, of a float instead of an integer, or an integer out of representable range (the bane of large numbers in JSON), or a date that cannot be parsed, or a null for a value that cannot be null. This saves you from a ton of nasty surprises when configs are updated.
- You want your config language to have explicit delimiters, so that incompletely copy-pasted constructs would be detected. Hence YAML is a bad choice, JSON5 is a much better choice.
- Of course, you absolutely want comments. JSON is a serialization format, not a human-operated config format.
This is why, I think, CUE, Dhall, and Jsonnet are doing more or less the right thing, but they are a tad large. Maybe something simpler and more compact exists, with some limited programmability and easy syntax.
For the reasons you mention (mainly the first one), it can sometimes be useful to use a separate programming language with functions that are used to create the required format, without needing the configuration format itself to do these things.
Just some opinions with no serious arguments. Okay XML is in the past but so what? There are still use cases for it. And Pkl/CUE/Dhall/Starlark looking like full-blown programming languages is a feature, but being deliberately weaker than say TypeScript is also a feature (no arbitrary I/O, in some cases no Turing completeness). I don’t think the author understands or appreciates why.
I've often wondered if PostScript or PDF contained the roots of a very good config language. Perhaps it simply is (PDF docs at least) but nobody regards it as such.
My guess is the RPN nature would be a no-go for many people. Nevertheless: comments, dicts, arrays, good string syntax, numerics, binary data, etc. Maybe that makes it too complicated.
PostScript is not so bad as a programming language, and it has a binary format as well as a text format, although the limits of numbers and some other things can be too small for many applications. The syntax of TER is based on PostScript (although it is not actually compatible, for many reasons; for example, the syntax "1.0" does not denote a number in TER, but "+1.0" does).
Shallow. Especially “ The era of XML is in the past… now it's dead.”
That’s not an argument against the merits of XML, it’s just a fashion declaration.
It’s also wrong. Podcasting boils down to XML files. It even heavily uses XML’s extension mechanism. XML is the basis for RSS and Atom. XML is the basis for LibreOffice and modern Microsoft Office files. XML is at the core of the epub digital book standard. And on and on.
XML may not be ideal for config (reasonable people can disagree on the topic) but it’s not dead.
It’s also interesting that he declares JSON “won” then adds a bunch of XML features to it like ordered entries and not having to quote element/attribute names.
JSON “won” for web apis - e.g. browser-server data interchange. It is not, and was never claimed to be, good for serial documents like config, or for when you need an extensible format (the “X” in XML). It’s fine for what it is, and so is XML, which, when it “won”, was similarly overused, like for web apis.
Are those examples using XML because it was the style at the time though?
The perfect configuration system would be able to support any number of parameters, enforce arbitrary validation rules, be searchable, provide metadata, examples, descriptions for each parameter, provide an audit log of who/why something was changed, support rollbacks, and so on.
The problem is that folks want a flat human readable file format to solve all these problems. That's a pipe dream.
JSON and CSV are pretty close to "syntactically optimal" in terms of losslessly storing nested and tabular data, respectively, in a human readable format. We ought to just stick with those and think about how to design effective configuration systems on top of them.
Yes this ^
It totally is a pipe dream to find some magic flat human readable file that does all this, you need to think of it as a config system not a config file.
So then why aren't we questioning our core assumption here that the solution has to be the shape of a human readable file format at all and accepting "syntactically optimal" as the best we can come up with?
Other than that text files are a compatible format to work with for the tools that coders use to interact with everything (text editor/git) why does it need to be a text file at all? or even human readable? Should we even call csv human readable... ?
The perfect config system you describe could equally be implemented on top of any kind of storage (file, db, whatever) with a configuration editor ui to drive it all. And thanks to llm's even for totally one-off custom configuration files making a small web editor for them is basically free if you can give it the schema and some examples.
And ui doesn't mean it can't be keyboard driven (just to preempt people griping about using a mouse). But it does mean that you can present your config in the way that makes the most sense to actually do the configuration instead of having to work around any limitations of a syntax. An example is the configs out there with the trailing comment "foo bar #foo bar sets the the foo bars the bar in the blaze." Maybe you've configured foo 100 times, you don't need the comment to know what foo sets but the syntax demands it so you have to see it every time. In a dedicated visual config editor the comment can be pulled up only when needed.
Will your config "file" end up with some extra cruft to store things about how to show it properly in the ui? totally. but we don't care because we don't actually edit that "file" directly anymore remember?
To be honest, all I see is https://xkcd.com/927/
All config formats are bad. You either don't need all the features at all, except key:value. Or you quickly run into weird limitations and quirks.
What I rather like instead, are custom build english-like DSL's, like:
- https://man.openbsd.org/pf.conf - https://man.openbsd.org/smtpd.conf - https://man.openbsd.org/httpd.conf - … and many more
OpenBSD's daemons' configs are pleasant. But then you have to maintain a DSL for even small projects. Most people aren't going to want to implement and maintain a whole one-off config language for their small project. Which is why having a common, minimal option is nice.
Here will be a list of ... things I don't like about them.
I think this proposed format is terrible. Almost like JSON, but not quite, so both humans and LLMs will be confused by it, and forget the rules.
name-of-config-language
meh.