Schleswig-Holstein, Germany’s most northern state, is starting its switch from Microsoft Office to LibreOffice, and is planning to move from Windows to Linux on the 30,000 PCs it uses for local government functions.
Concerns over data security are also front and center in the Minister-President’s statement, especially data that may make its way to other countries. Back in 2021, when the transition plans were first being drawn up, the hardware requirements for Windows 11 were also mentioned as a reason to move away from Microsoft.
Saunders noted that “the reasons for switching to Linux and LibreOffice are different today. Back when LiMux started, it was mostly seen as a way to save money. Now the focus is far more on data protection, privacy and security. Consider that the European Data Protection Supervisor (EDPS) recently found that the European Commission’s use of Microsoft 365 breaches data protection law for EU institutions and bodies.”
Maybe soon a unified CSV handling might be possible.
I can confidently say that CSV support is one of those problems that even the brightest computer scientists will be pondering for the decades to come.
Supporting CSVs sounds like an easy problem, but it’s not. It’s like a whole different complexity type. Time complexity, space complexity, and now, the dreaded subclass between spec complexity and organisational complexity.
You can’t just make the users agree which delimiter to use and how quotes are supposed to work. That’s nearly impossible. No no no.
Commas are too common, we should go with semicolons. And
\nand UTF-8 by default. And a header that defines changes from defaults, plus metadata such as data logger model and settings. These are some significant quality-of-life improvements but I’d guess it will take another file extension before that happens.I just don’t like that CSV exists as a format and has no standards currently. If you remove commas from CSV then you’re taking the C out of CSV.
SCSV (semicolon separated values) at least sounds like an upgrade to CSV. Or maybe just use something that is flexible but is standard like JSON?
Yeah, SCSV would work, with a .ssv file extension for FAT compatibility.
JSON is overkill, tabular data is often recorded by 8-bit devices. Yes, you can use a dishwasher to cook salmon, but building a dishwasher is difficult and it can break in many more places. Each piece of salmon also needs to be carefully wrapped.
Yeah, I get what you mean. I’m so overprotective of my dishwasher I actually pre-scrub plates very quickly so not to clog the dishwasher (which is pretty similar to sanitizing inputs for putting them in a database I guess). 😊 It’s still much faster than doing the dishes by hands.
But the point is something simple can run on a simple device with minimal supervision.
At that point why not use TSV?
ASCII 0x1f, unit separator and 0x1e, record separator. There’s also 0x1d group separator and 0x1c file separator.
Both CSV and TSV have been a mistake from the start it’s not like they’d be suitable for binary data anyway and not using ASCII control codes specifically made for in-band messaging of record fields means they ate into the printable characters (and yes \n and \t are printable, they move the print head that’s a printing action).
If you want binary compatibility either use bencode or throw ASN.1 at it. The important thing is to have a simple enough data model, don’t try to save code in the base compatibility version, evaluate the whole sheet before export if you have to. Using sqlite as interchange format is a bit hacky, but honestly defensible especially with the code (which kinda is the spec) being public domain.