the “linux problems page”

Isak already commented on this, but I’ll do so as well as it was me who wrote that page and so I should take the blame for it. Reading Thomas’ blog entry hurt, because I don’t want people to think the project is ‘arrogant’, and it especially wasn’t intended as a slam on Python. I’ve been using Python myself a fair bit lately for various university assignments, and it’s been great.

Firstly a bit of context – as it says at the top, that page was written for a very specific meeting and should probably have been deleted or archived after that. It is insulting, partly because at the time I was feeling pretty bitter and cynical about open source and Linux especially, but mostly because there had already been (polite) conversations with the relevant projects and in each case they had been entirely unwilling to even consider change. This wasn’t due to lack of technical arguments on our part, but was usually a result of either not seeing the point (who needs binary compatibility when we have distros?) or in a few cases because it was felt that better API/ABI compatiblity would only help “evil” proprietary software.

The effect the UNIX wars had are well documented – the fragmentation and pointless incompatibilities UNIX vendors used to try and lock people into their particular flavor allowed Microsoft to step in and take over with Windows NT, a comparably well integrated and consistent solution. United we stand, divided we fall, and yet here I was writing a page about all the myriad little differences that divide the Linux software ecosystem. Seemed like history was repeating itself. I’m a bit more positive these days (a bit ) and as time went by I just forgot about the page.

More context. Virtually every single one of the points on that page has a back story which hasn’t been fully documented anywhere. Ironically I think the only one there which hasn’t been raised with the relevant upstream projects is libPython, partly because it was only encountered very shortly before the page had to be written, and partly because the primary incompatibilities are caused by distributions modifying the default configure parameters. A library which changes its exported ABI depending on how it’s compiled is never a good thing IMHO, but nobody forced distributions to configure it differently. I did file a bug with Red Hat about them using this configure switch, and IIRC it was quickly closed as not being a bug because it was felt that the (presumed?) performance improvement was worth the loss of compatibility. Not an answer I’m unfamiliar with.

The back stories behind these points are twisted and have taken place over a period of years in many different forums. In virtually every case, especially with respect to things like ELF, GCC, binutils and glibc, the issues were originally raised politely, bugs were filed, and patches were written. Sometimes discovering these issues took a great deal of effort – for instance, the GCC bugzilla bug linked to from the page took literally weeks of investigation to track down (as if compiler bugs weren’t hard enough to find, compiler bugs that cause heap/stack corruption are even worse). So imagine how I felt when it turned out that a well documented and publicised feature of GCC that we had been relying upon on in autopackage had never worked reliably, and apparently been implemented and documented without ever once testing it on a real program? Weeks of my time wasted, my end users experiencing crashes (which were blamed on us of course), no clear solution in sight and an upstream who basically didn’t care

That’s the sort of story which has been repeated over and over in different situations and with different projects. I don’t care to name them all, I did too much of that lately. In each case the story was the same – either nobody cared, or they did care but not really enough to make the necessary changes, or in the worst case they felt it was a good thing because it hurt evil proprietary software (apparently if you develop open source software you’re quite happy with requiring your users to compile things!).

4 years of this sort of conversation does make one bitter, cynical, frustrated and yeah … maybe arrogant too. I can see why it would look that way, though IMHO what people are actually seeing is frustration. Why bother holding back when nothing else that has been said ever had an impact? Might as well let off some steam.

Concern about this kind of thinking is one reason I floated the idea of somebody else becoming the autopackage maintainer and me taking a back seat. As of yet nobody has, but obviously once somebody with the necessary drive and enthusiasm comes along I’d be happy to ease them in.

other stuff

I can’t seem to post to LJ today, it just says “invalid form submission”. So here’s a quick reply to some of the comments there. Daniel Stone pointed out that Python programs which don’t use the C API aren’t affected by that. That’s true, but even in its previous burned-out-and-rude form the wiki page made it clear that those programs weren’t a problem. In practice most of the times I’ve looked at packaging a desktop program that uses Python it’s used C extensions in some way, often for performance or integration (eg using objects from libegg). So in practice this issue makes it hard to distribute Python programs in binary form, in much the same way that the shifting C++ ABI makes it hard to distribute C++ apps. It can be done, in certain situations and with some apps, but there are gotchas.

Somebody else suggested that the comparison between autopackage users and Debian packagers was invalid, because one group is users and one group is packagers, and Debian is bigger anyway. But I think it’s a valid comparison, because the conversation was something like “Me: Here’s a patch that’ll make things easier for people using your autopackage ….. DPs: Don’t do that, it’ll make things less convenient for us”. So in this case comparing the relative convenience of the two groups seems legitimate (unless you believe that a packagers time is more valuable than users time, which I think would be a tough argument to make).