Introducing IOD file format
If you're looking for a configuration file format, you might want to try IOD (short for "INI on drugs"). IOD is basically INI, with more precise specification and some extra features. If you are not using any of the extra features, you're practically using the good ol' INI format.
IOD tries to capitalize on the INI's popularity and simple syntax, and on top of that tries to define more precisely the format and to add some features that makes it more convenient and powerful to use.
Format is more precise. Because there is no single, official specification for INI, variants emerge and they support different set of features with different behaviors and incompatible syntax. IOD tries to define things more precisely, for example:
- Whitespace is explicitly allowed in various places:
; unindented comment ; indented comment is allowed [indented section] indented key name=value key name = whitespace around equal sign or end of line
; blank line is allowed
[ whitespace inside section name ]
key=val - Comment marker # (pound sign) is allowed in addition to ; (semicolon)
- Comments are allowed after a key value, or at the end of a section line
[section] ; a comment name=value ; another comment
- Double-quoted key values are allowed, and interpreted as JSON string.
- Other things that some INI parses support/allow are explicitly disallowed. These include: heredoc (supported by Config::IniFiles), line continuation/multiline value (use double quote and "\n" instead).
Encoding. Encoding allows specifying complex values or nonprintable characters. There are several encodings defined: json, base64, and hex. With json encoding, you can set a key value to undef (null), string with escape characters, and arrays/hashes. With base64 and hex, you can encode binary data.
Encoding is written using the !NAME prefix before the key value. For example:
name1 = !json "two\nline" name2 = !hex 0aff0d name3 = !base64 c29tZSB2YWx1ZQ==
A shortcut syntax for JSON string is allowed using the abovementioned double quotes:
name1 = "two\nline"
Encoding is planned for section name too, if there is such a demand.
Section merging. This feature is useful to avoid repetition. Suppose you have a set of common or default values that you want to apply to multiple sections:
[common] foo=1 bar=2;!merge common
[section1]
foo=11[section2]
baz=23; stop merging
;!merge[section3]
baz=33
Then section1 will have these values {foo=>11, bar=>2}, section2 will have {foo=>1, bar=>2, baz=>23}, while section3 will have {baz=>33}.
Includes. This is also another common feature of configuration file, to avoid repetition:
;!include another-file.conf
Expression. Expression is useful to add some dynamicity to the config file. The most common case is to refer to some other key. For example:
[plan1] disk_quota=1000 price=20[plan2]
disk_quota=!e val("plan1.disk_quota") * 1.5
price=25
You can parse IOD with Config::IOD::Reader which weighs in at just about 370 lines and requiring nothing but JSON and MIME::Base64 (and those modules are only loaded when necessary).
Another module is planned to be released: Config::IOD which, in addition to reading IOD files, will have the ability to write/modify IOD files (add/remove sections and keys, setting key values, and perhaps also format and add/remove comments). Config::IOD will be a round-trip parser, meaning you can modify an IOD file without modifying key/section ordering, formatting, or comments.
As always, comments and inputs welcome.
UPDATE 2014-08-30: Mention expression.
#perl #indonesia
Is there a writer too or is it just another configuration file Format library that only supports reading and leaves the user alone who might want to store configuration changes in files?
As mentioned in the second-to-last paragraph, a writer is coming :)
The only thing I really don't like (and why I often rather use YAML) is that the actual data determines your data structure. Same problem with Config::General, for example.
The decision if you'll get a single value or an array is determined by the data. One value => string, more values => list.
If the parser gives you the result as a data structure, you will get a string or an array reference.
So when using the config data structure, you have to check whether you got a reference or a string before using the value. This is tedious.
What would be really helpful is to have a possibility to declare (inside the config file) a key as always having a list of values.
That's incidentally one of the advantages of Config::MVP, which has an INI parser with similar rules to IOD.
Usually, Each section in a Config::MVP based INI file has a corresponding Perl Package, and each Perl Package can define an 'mvp_multivalue_args' method, which returns a list of parameters that are expected to be arrays.
Thus, regardless if a user specifies one or more, if its an array, the consuming code gets an array.
And if a user specifies a property that is not supported in array form, the user gets told they've done something wrong.
Yup, I personally don't really like INI's multivalue syntax (using multiple lines of assignment) for the exact same reason you put forth. But most INI parsers support it, so ...
In IOD you can also use this syntax:
[section]
fruits = ["apple","orange","avocado"]
As for YAML, I actually like it too. But I've had sysadmins and users often fumbling on the syntax. The two most common are: indentation, and forgetting to add whitespace after colon in a mapping or after a comma in a list.
Have you taken a look at [TOML](https://github.com/toml-lang/toml)? IMO it provides a much more sensible paradigm than INI, while still being intuitive enough for most people. There are already two parser for it in Perl.
I have. I guess TOML is okay for configuration format too, although it diverges quite a bit from INI. My goal with IOD is to be compatible enough with INI, to be interchangeble enough. Also, I'm not seeing a TOML round-trip parser yet, something which I intend to create for IOD.
> In IOD you can also use this syntax:
>
> [section]
> fruits = ["apple","orange","avocado"]
But does this not simply re-expose the problem tinita was complaining about?
It forces the consuming code to determine on a case-by-case basis whether it supports an array, and subsequently requires a lot of boiler plate code to automatically switch between single, and multiple values.
It doesn't seem quite the same. In a traditional INI file:
"bar" is an array, but is "foo" a simple string, or is it an array with just one element? The INI parser will probably treat it as a simple string. If the application wants "foo" to be an array, it needs to include code to "promote" a string into a one-element array.
With IOD, there is at least a way when writing the config file to unambiguously say "this is an array, even if it's only got one element at the moment".
Also, note that in the case of "actual data determining your data structure" IOD is not different from YAML which trinita uses: whether a parameter is scalar or array is still determined by the config. Config::MVP is different in that it uses some sort of "outside schema" to specify that a parameter needs to have a certain form/value.