When I asked for ideas on what I should write about more, all the comments were from autopackage people asking me to write more about that. OK then. So be it.
## where are we now?
|
|
The good. We have a quality product. It usually works as it was intended to work, we have a well defined target audience and these people are always happy with our work. We have a huge well of evangelical support from some of these users, and it’s good to see. Autopackage always gets great reviews from the media. When it came to controversial issues we were bold, and we justified our decisions with rational debate and argument. We mastered then documented binary compatibility. High profile projects like AbiWord, Gaim and Inkscape use what we made already, others like Mozilla want to. These things are a personal achievement for every one of us, we should rightly be proud of them. |
| The bad. Autopackage has not gained widespread acceptance as we had hoped it would. Few packages exist, fewer still are of high quality and frequently updated. Whilst a few big, well known projects do use autopackage, in some cases this is not what it seems – Gaim developers except marv seem to dislike the project, and have been entirely obstructive. The Gimp package is not maintained by Gimp developers, who seem to believe that binary distribution is not their problem at all. Meanwhile, distribution acceptance sits at zero. Any mention of autopackage at all on mainstream distribution developer lists triggers instant flamewar, which further alienates developers who may not have had a strong opinion on the topic. CodeWeavers, my own employer, do not use it. | |
| The ugly. Some distribution people, notably Debian developers, outright hate us or insult us. This is part of a wider theme of upstream/downstream tension. Debian is by far the worst here and deserves to be singled out – their packagers routinely anger high profile developers, and the practice of subtly breaking software continues unabated. When autopackage 1.0 was released and we got a lot of media attention, one Debian developer implied that they would deliberately add autopackage runtimes modified against our wishes to their repositories. This behaviour is unacceptable, but is the status quo and as such is not seriously challenged. |
## how did we get here?
The good. A strong vision of easy to use software installation, commitment to that vision, and a team with diverse skills.
The bad. Inability to change the prevailing Linux community culture in which binary packages are Not Our Problem. Refusal to make it easier for non-programmers to produce autopackages (my refusal, basically).
The ugly. Various things have been suggested, everything from the /usr vs /usr/local issue being called wrong to skipping native package manager integration, but my personal belief is that this problem was inevitable and nothing we could have done to the product itself would have changed it. At a fundamental level, distribution packagers today have huge power and nobody likes to lose power. Period.
## where do we go from here?
At heart, this is a personal question. Its answer will be different for each one of us. Faced with the following facts, what can we do?
- Autopackage works well when used correctly
- Yet it has low acceptance
- Ideas for tackling this problem are thin on the ground
For now, I think the most important thing the project needs to do is just get 1.2 out of the door. It’s dragged on for a long time …. within a month or so, it’ll have been nearly a year (!!). This is mostly my fault, I’ve been absent and uninvolved for a long time now, mostly due to coursework loads which have taken away my hacking time, and partly due to waning interest. And of course Hongli and Curtis have been busy with university and preparing for a child (congratulations Curtis!).
After 1.2 is out there are various issues to consider.
- What should developer focus be on? Adding features like before, or on trying to tackle the causes of low acceptance?
- Is hostility towards non-upstream packagers hurting more than it helps? I still believe that the highest quality packages will come from the people making the software themselves, but a few times now (like trying to install mplayer a week ago) I found myself wishing for an autopackage, any autopackage would have done as long as it basically worked. But our literature and workflow is entirely oriented around the idea that upstream makes the packages. Concrete idea: redesign binary relocability to be independent of code changes by using a low level trick.
- Should I still be the maintainer? As you can see, I don’t really have the time or energy to push the project forward in the way it needs anymore. The obvious conclusion is that somebody else should be the leader. One thing I am certain of though is that at all times there must be one leader who has the final say. Letting things drift with no clear successor is not an option. …. and perhaps the most important one …
- Does it make sense to make a new distro?
## a new distro?
Yes. Perhaps even a new OS entirely.
At some point, you have to take a step back and look at the bigger picture. You must ask yourself “why am I doing this?” and “is this is the best way to achieve my goals?”. When I started autopackage, I had many goals but the primary one was simply to make Linux easier to use by making software installation easier to use. That was the goal. It was a bit narrow, because it accepted that “making Linux easier to use” was the be-all and end-all, but at the time I was enamoured of the free software movement and this goal seemed to make total sense.
There were actually many ways to achieve this ideal, for instance I could have become a Debian packager and joined the never-ending push for a complete repository. Or I could have developed some mad hackz to try and make Linux binaries run whatever the source, a kind of super alien. But I decided that the best way forward was to produce a new kind of package, one that would be like a binary version of the source tarball, and which had as its primary focus usability (then a new word in my vocabulary).
Looking back, it is now clear that as of Feb 2006 it has failed to achieve the original goal. As a toolkit it doesn’t make anything easier by itself, it only provides the tools for others to make things easier. And generally others aren’t interested in that.
But times have changed and making Linux easier to use is no longer the be all and end all that it once was. What does making Linux easier achieve? What is the purpose? Beating Windows was once enough, but Apple managed to do this pretty conclusively and the bar since moved – beating Windows is no longer enough, now you must beat Apple too.
Stopping spyware was once enough, and this is why my blog has often focussed on that. But developing an entirely new OS and entirely new set of apps to go along side them seems like an inefficient solution. Linux and MacOS are currently totally undefended against spyware, and platform designs that seem to hold up like J2ME are so radically different to existing desktop models that it’s hard to imagine retro-fitting them onto Linux or Windows.
And of course, there is the original motivation behind GNU, that of existing in a world of only free software. A laudable aim, but one not in practice very compatible with reality. Whilst I could use only free software, that would be terribly inconvenient. I use the proprietary nVidia driver, the proprietary Flash player, I play proprietary games, my computer is powered by a proprietary BIOS and the circuitry inside my CPU is likewise proprietary. Putting up with inconvenience to use more free software (given that being 100% free is impossible) doesn’t always seem like a good trade.
There are clearly some goals that would be best served by creating a new distro. For instance, it would solve the problem of not having any distros support autopackage. It could solve other problems too, like no distro offering an easy (Windows-based) installer/uninstaller for the OS itself. But it’s tough to see how it could solve the cultural issues with the Linux community or the problem of binary kernel drivers.
Other ideas – shaking up the way software is managed on Linux is one possibility. For instance, union mounts change fundamental assumptions about namespace design, which haven’t really been explored yet. You could have software installed to individual prefixes in /software yet still appear to be mounted in /usr, partly solving the binary relocatability issue. You could go for the J2ME approach of totally sandboxed applications.
There are lots of potential routes for the project to explore. But I would shy away from simply adding new features as the roadmap originally implied.
Most importantly the goals and solutions must be carefully considered. So what is the goal? the solution? Good question. One I’ll leave for another time, I think.
March 3, 2006 at 5:51 pm |
Mike,
First of all, thank you for your work and passion towards Autopackage. Although it is not a popular idea for the ‘purists’ Linux needs something like Autopackage. There was a time when these ‘purists’ were the rebels and now…. Autopackage….. 0install….. Klik…. Gobolinux…. Conary…..
Second, I just wanted to let you know that I also am considering using Autopackage in a distro. Of course I have little idea what I am doing. I am just a wanabe developer. But my idea is to build a distro using one of a number of more easily customized distros that is scaled back so that only core software is included and managed by the exsisting pm and all ‘add-on’ software is managed by Autopackage or 0install, or Klik. Part of the reason is because I would like Christian developers to have a place to showcase their work, but also an easy place for other distros to pick up quality programs without having to repackage them.
Third….. well… just keep up the good work! Thanks!
March 4, 2006 at 2:17 am |
great reading ~ keep going mike!
March 4, 2006 at 6:39 am |
If you want people to adopt something, it has to solve a problem they have. Autopackages of the Gimp don’t solve any problem for the Gimp developers (distributions already package their stuff for them) or for Debian (they have plenty of volunteers happy to package Gimp).
Smaller programs and smaller distributions might be better early adopters. Debian will adopt it when users keep complaining “Why can I do this in and not in Debian?”. After all, they do support user installation of third-party firefox extensions, so there is a precedent.
MPlayer seems an obvious application to try, as it’s both high profile and difficult for normal distributions to include. Didn’t there used to be an autopackage of it? What happened to that?
Finally, while autopackage, zero-install and similar systems aren’t widely used at the moment, they’re still having a big impact. Now every discussion of installation on Linux mentions them and people are thinking seriously about the issues (in much the same way that arch made people think about distributed version control systems, even if most people didn’t end up using arch).
March 4, 2006 at 7:46 am |
I really think you’ve made too many points to be addressed by a mere blog comment, but here goes:
1. Making packaging simple is absolutely essential. Trying to enforce high-quality packages by making the process difficult is the wrong approach.
2. The key to evangelism is concentrating on the mac angle – making software as easy to install as it is on OS X. This is one of the few angles that works consistently with non-idiots in the Free Software world.
3. Debian’s egalitarian nature is the reason you perceive the project as a whole to be hostile. Debian is a huge project, with a great deal of developers, and the majority of them are completely unqualified to be making commentary on software they don’t maintain. As a result people often see the vocal idiots (pick basically any non-GNOME dev in Debian making comments on usability or desktop software, for instance) before the rest. Autopackage has stronger support amongst those users and developers closely aligned to Ubuntu, for instance.
4. Making a new distro is insane. In the modern world the only distros that actually matter are Red Hat and Debian and their biggest derivatives (SUSE and Ubuntu mostly). The time for creating new distros to test packaging systems was the twentieth century.
I’d personally recommend:
1. pushing Autopackage harder than crack to people thinking about software installation on Ubuntu. With the Debian XULRunner problems, and Firefox 3 using Autopackage by default, you’re going to have a good opportunity to use Mozilla as a lever to get into Ubuntu.
2. A less steep learning curve to actually packaging software. Building and packaging are totally different operations, performed by different people and requiring different skills; you’ve picked the right level to target builders, but not packagers.
3. Avoiding protracted flame wars at all costs. Stick with comparisons to MacOS, appeal on technical grounds, don’t spar with idiots.
– Chris
March 4, 2006 at 10:29 am |
Wow, you’ve finally gotten your example of a thing that was broken by changes made in Debian: the user-agent in xulrunner.
A thing that was broken in the *first release* of the package, uploaded to the *unstable* distribution, and which will surely be fixed before long.
How can you hope to be taken seriously by distributions when making such ridiculous claims?
Nobody dared to say it here, so you should be aware of it: if nobody uses autopackage, it’s because it’s not useful. And when it overwrites files in /usr, ignoring the packaging system, it becomes not only unuseful but broken.
March 4, 2006 at 1:07 pm |
Thomas Leonard:
“Didn’t there used to be an autopackage of it? What happened to that?”
I made the mplayer autopackage. It has been abandoned since I lost interest. Anyone’s welcome to take over though.
March 4, 2006 at 1:22 pm |
I think this from Chris makes a great point:
If you/we can get Autopackage in a currently “big name” distribution then more people will begin to see it. I used Autopackage to install a game (Globulation I think) and it was the EASIEST install I have ever done.
I am mainly a Windows user who toys with Linux so I do not really know the ins and outs of managing packages. I want to install software in Linux like I do in Windows…. Double click… Next … Next … Finish. Double click to run the program.
For Linux to be adopted in the mainstream as a reliable and realistic Desktop Operating System then package management needs to be this simple.
Great Article!!!
-Will
March 4, 2006 at 1:50 pm |
I once thought, and still think, that what’s needed is a GUI package manager, that supports autopackage, and is abstracted to support whatever distro’s packages it is shipped with. Make it good enough and all the distros without GUI package managers (everyone but the commercial three), will bundle it. And then you have autopackage support in the GUI of many many distros. And crucially the user would probably be much happier to install autopackages, and might demand developers to provide them more often.
I actually started writing such a thing last year, but like most of my projects it got abandoned.
I agree it’s important for people-who-can to make the packages, otherwise you’d end up with 500 autopackages for every piece of software, and by that point even if the developers of the app make one, it might be crowded out by all the others.
March 4, 2006 at 1:51 pm |
BTW great use of images in this entry
March 4, 2006 at 4:59 pm |
Sorry dudes, i think you’re definitely losing your time. I spent several years with my Linux-from-scratch, and I’ve had enough experiences when keeping my system updated: install new version of library X, an suddently program Y stops working. Recompile Y, but it was a C++ program and now it shows a C++ ABI incompatibility with library Z. Ouch!
Now I’m running a “real” distribution, and I’m glad that I can simply install an RPM/DEB/whatever it uses. I know that upgrades won’t break my system, because installing a new, ABI-incompatible version of a library library will cause clear dependency conflicts with all the programs that depend on it. Security updates Just Work(TM) without hassle and without leaving unpatched programs or libraries around.
*This* is what I call ease of use. When I see an interesting program on the web, I just fire up the package manager, click on the package and start using it. If the program is not there yet, 99.9% of the times it’s so young and buggy that I can simply ignore it. When I want to remove something, my package manager will help me. Could it be simpler?
I can’t understand how autopackage could make my life easier.
(as a side note: I’ve played the role of the user here, but I’m also a developer, and the task of maintaing an “universal” binary package of an application just scares me even more. I can also estimate the huge amount of work gone into autopackage, and that’s why I say: dudes, it’s broken. It doesn’t solve any problem, and even brings its own ones. Use your skills to do something else. Don’t waste your time.)
March 4, 2006 at 4:59 pm |
Quote from point 2 of issues to consider once autopackage 1.2 is out the door:
“But our literature and workflow is entirely oriented around the idea that upstream makes the packages. Concrete idea: redesign binary relocability to be independent of code changes by using a low level trick.”
I am not a developer, packager or programmer.
But..It appears to me that this assumption is and remains contrary to the social structure of linux distributions, starting with binary compatability and who actually compiles the binaries the people use. Usually not the developer/programmer, if it’s a non-commercial product.
There really are different aspects of the the history and past decisions about each distribution that require that someone knowlegeable about the distribution package the software, and that distribution just is not going to care about binary compatibility with other distributions. They are committed to building their own binaries, for their own distribution…and they’ve got enough trouble doing that to spend any energy worrying about other distributions, let alone binary compatiility. I would call that a mere historical fact.
Lets note that the distributions tend to build/package/compile on their own, using their own methods fpr software packaging, and that a new system such as autobuilder will tend to override particular decisions that have been made and are now policy for a particular distribution..and erhaps the policy is merely a work around for some particular, current difficulty unique to that distribution. This seems to be some part of the friction encoountered with packagers.
It seems to me that the success of autopackage is predicated on creating a new social construct entitled “new linux distribution with binary compatibilities to others.” That requires people interested in packaging that way.
Are you ready to create a new distribution, and locate the people to make it work, to make autobuilder a successful project?
March 4, 2006 at 5:48 pm |
Joe, there are all sorts of reasons why autopackage was built and why nearly 400 people a day install their first package using it (http://autopackage.org/stats.php).
If the utopia of the perfectly complete and up to date distribution existed then I’d have never found myself frustrated by attempting to install things, and never sat down to build autopackage in the first place. Unfortunately, it doesn’t exist. It didn’t then, it still doesn’t today.
For instance, try installing the Banshee audio player on Fedora Core 4. In trying to do that, I managed to trash my distribution entirely, because the packaging of Mono for Fedora is weird and tried to upgrade me to GNOME 2.12 (which failed). Mono was excluded from Fedora for “business and strategic reasons”, apparently, and in fact distros exclude inconvenient software all the time. Whether it’s because of license, strategy or just personal beliefs of the developers, every distro has software that cannot be obtained via the repositories.
Actually, I wanted a well put together Mono setup so I switched to OpenSUSE. Now I have a modern Mono, and Banshee is installed out of the box. Hurray! Except … wait …. now I can’t install mplayer, because there are no RPMs for 10.1 betas, and because the GCC in OpenSUSE 10.1 refuses to compile mplayer. Suck.
I’ve spent the last three years pointing out examples of where the system falls flat on its face, time and time again. I can’t be bothered doing this anymore. If you don’t want to read the FAQ on the website then fine, but please don’t comment on autopackage matters until you have done so.
March 4, 2006 at 5:52 pm |
Josselin, lots of people “use” autopackage, as I said we get about 400 new users per day. Not bad considering there are only a handful of packages out there! The word “use” in this context is talking about developers, who often have your unfortunate mindset.
FWIW I have many examples of Debian breaking software subtly, this one was chosen because it’s the most recent one I’m aware of. And “it was fixed” is a stupid defence, if “gladium” hadn’t blogged about his change then it would have gone unnoticed for months until somebody tracked down why some web apps were rejecting Debian users and nobody else. The problem is one inherant in the packaging system – unqualified people are modifying software for no good reason (aesthetic in this case).
March 4, 2006 at 6:41 pm |
Mike, your comment shows that you’re a geek. I’m a geek too. I want to emphasize that you (like me) have an approach to computers that is very different from the one of 99% of the people. John A.V.G. User doesn’t spend his time installing applications, he just wants his music to play, his DVDs to show, his documents to document. What’s “Banshee Audio Player”? John A.V.G. User just clicks on the first icon in the “Multimedia” menu, because he knows it will play his MP3 collection.
If you ask Joe A.V.G. User “would you like to try another application?”, he will answer: “why?”. Even if he’s convinced that it’s nicer than the one he’s currently using, he will *absolutely* refuse to install it if there’s the slightest possibility to have something broken as a result.
By its very nature, autopackage will always have a chance to screw something up in the distribution it’s being used on. Or it may not work. Your very FAQ confirms this, and the situation is not going to change, simply because autopackage is based on unrealistic assumptions. Take the sentence “Integration with other software on the system is a matter for upstream”: this is what happened ten years ago, when developers packaged their software in ways that created problems on one distribution or another. Time has lead to spacialization (distribution-agnostic developer vs. distribution-specific packager) and the result is an enormous increase in quality, stability and integration of modern distributions. Most of the ones saying “distribution-specific packaging sucks” maybe don’t realize how bad things were before, and why.
Maybe 400 people are using autopackage every day, but it’s not so much. They are surely not relatives of John A.V.G. User, and if/when they will realize the limitations of autopackage they will probably switch to something else (like GNU Stow, maybe).
At the same time, I don’t see developers rushing to take the packaging task out of the hands of the distribution maintainers… Why spend time and energy to create half-baked packages, when someone else could do a better job (better for the developer and for users, I mean…). And if I’m really forced to create packages… Well, I can always create a static RPM and use alien on it. 1000 different package types at the price of one!
March 4, 2006 at 7:11 pm |
Hmm, I think you over-estimate the difficulty of making a package that works everywhere and underestimate what a mess distributions regularly make of packaging software. I’ve wasted hours helping users figure out “bugs” in Wine that turned out to be bad packaging. There’s no reason for this, “bad packaging” simply does not exist on the Mac for instance. You drag it to /Applicatons, double click it, and it works. Simple!
March 4, 2006 at 7:38 pm |
Mike,
I’ve said this before on IRC, but i’ll say it again: i believe you need to get the word out more. That means going to FOSS conferences, gatherings; in your uni or somewhere else, these things are important.
Of course anyone could talk about AP in an event (i’ve done it myself), but fact is you’re one of the guys behind it that knows it ALL; you will be able to talk about it a zillion times better than someone from outside, and thus will reach more minds.
Autopackage is great, and i too was one of the guys who thought when the project begun: “in a year everybody will be using this; this is just great”, and now i’ve just come to the obvious, old, conclusion: changes take time. We should’ve known this. So, i still think it’s a matter of time, and the project actually is still young.
Perhaps AP could grow more on commercial distributors. I feel lots of companies have a real hard time trying to figure out how to distribute its apps on Linux, and i say this from experience because i work for a company that makes software for both Windows and Linux.
So, AP could be a relief for those vendors – a package that is simple and works on all distros, just great. However, if a company is thinking about AP, the question that comes to their minds is: how mature is AP? Is it risky? We don’t know? Part of the answear will come with time (more people creating packages), and part of it comes with spreading the word in events.
As for developing a new distro… it’s too soon, and too risky for the project. AP is meant to work on all distros, not only 1, and that move could be misinterpreted by some. So let’s not lose faith.
March 4, 2006 at 7:45 pm |
I package stuff that “works everywhere” almost weekly at work. I share a precompiled .tar.bz2 among my collegues, to be unpacked under /usr/stow/, followed by a `stow program-1.2.3`. Stow won’t overwrite existing files, and I avoid messing the target system. Maybe I could make an autoexecutable script of it…
But the point is that the program is not integrated in the target distribution. It is a *huge* difference, expecially for desktop applications. Autopackage may be marginally better, but its basic approach is: just find a common denominator among distributions and stick with it. Even if it works in the medium/long term (don’t count on it…), users are not going to be happy. Have you seen the difference, for example, between the “standard” GNOME desktop and the customized versions provided by 90% of the distributions? How is the autopackage approach going to guarantee this added value?
I think that you’re also over-estimating bad packaging issues just because sometimes happened to you directly. It is quite a huge step from this to asking *all* developers to repudiate their packagers and do this work themselves… Expecially because 99% of them is just fine with the current packaging practices. Maybe more collaboration between upstream and downstream could have solved your problems…
Regarding the Mac (and Windows, for that matter): doing this kind of “installation” is easy when programs have little or no dependencies among them or with the rest of the system. This is not the case of modern Linux ditributions, though, nor for modern desktop environments (see Epiphany that depends from Firefox libraries, for example).
Making hundreds of applications interact correctly without quirks for the end user is a *huge* task, performed daily by distribution developers. A few self-contained applications may be installed without being checked by them (via autopackage, ./configure;make;make install, whatever), but every little bit of “linux from scratch” introduced into the system will increase its entropy until a complete reinstallation is required. Microsoft Windows anyone?…
March 4, 2006 at 8:40 pm |
If you think that current Linux distros are any different from Windows on the “I messed something up and nothing short of a reinstall will fix it” scale, you’ve obviously never had a piece of hardware that wasn’t automagically configured on boot or by the installer.
And for god’s sake leave a link or an email address (or at least your full name). There is literally nothing more irritating on the Internet than arguing with the anonymous.
– Chris
March 5, 2006 at 4:04 am |
Chris, I’m managing several pieces of hardware without out-of-the-shelf support on Linux. I try hard to keep the required changes as small as possible, because I know that sooner or later they will conflict with my distribution (e.g. when doing a system update). It’s the only way to avoid the the-system-is-b0rked-lets-reinstall-everything-from-scratch syndrome.
This kind of customization is always a “last resort” approach. Making it the default (via autopackage, stow, whatever) is going to create *far* more problems than the ones being solved, at least for the 99% of the users. If you don’t believe this, then you’ve never seen a person that just wants to *use* his/her computer without hassle, like he/she does with the cellphone, the electric oven, the dishwasher, the hi-fi…
March 5, 2006 at 4:35 am |
Joe Said:
Now I’m running a ‘real’ distribution, and I’m glad that I can simply install an RPM/DEB/whatever it uses. I know that upgrades won’t break my system, because installing a new, ABI-incompatible version of a library library will cause clear dependency conflicts with all the programs that depend on it.
Having your system refuse to install a program because of conflicts is indeed better than messing up your system. But it’s still pretty poor.
I’m a Debian user, and I often find that I can’t install one program because APT wants to remove a dozen others first (Qt3 vs Qt4 is a good example).
I can’t speak for autopackage, but in Zero Install all libraries are parallel installable. As with stow, every version goes in its own directory. Unlike stow, no global symlinks are needed, so installing one package will never affect anything else (running it might, but that’s another issue).
So if you want to install a Qt4 application and you already have a load of Qt3 apps… no problem. You’ll have Qt3 and Qt4 on your system. And if you have Qt4 but you want to install a Qt2 app, that will work too, because our package metadata gives download locations for all versions, not just the latest, so we can automatically find and download a version that works. (this is just an example; we don’t have any Qt programs available through Zero Install yet)
I guess that as more autopackages become available, autopackage will have to grow some similar system, because trying to maintain thousands of packages that all work with the same versions of libraries isn’t going to work. It isn’t working very well today even in Debian’s tightly controlled system, and it certainly won’t work in the world at large. I’m sure the autopackage guys can comment on how they handle this better than I can, though.
March 5, 2006 at 8:36 am |
What autopackage does in the situation of Qt 3 vs Qt 4 is something we haven’t really explored yet, as using the systems Qt (which is what you want) requires the C++ support that’s in 1.2
In the case of library instability there are several approaches a packager can take.
Firstly, they can just depend on the new library and trust that it’s parallel installable. If the system is working as designed then Qt 4 should have a different soname to Qt 3 and so be able to co-exist on the filing system properly. I do not know why Debian thinks Qt 4 and Qt 3 conflict yet (presumably) GTK 2 and GTK 1 do not. I have both Qt3 and Qt4 installed at once on my SUSE system.
Alternatively, they could bundle the new version of the library with the package itself. That’s appropriate if it’s not widespread yet.
Alternatively, if the new library exports mostly the same API/ABI as the old library then relaytool can be used (theoretically) to link against both versions of the library at once, and whichever is present will be used. Actually I never got around to adding that feature, but it’s easy to do (like a 10 minute patch). This works when the part of the library that changed incompatibly is different to the part you use.
Long term the whole plan was to have a desktop Linux platform so instability would become less of an issue. But that never happened.
March 5, 2006 at 11:28 am |
Google says that the libqt3 and libqt4 conflict in Debian was solved 6 months ago:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=306879
It was not due to the libraries themselves, but to the programs being distributed together with the libraries (moc, qtconfig, etc): they weren’t expected to coexist by the upstream authors. The Debian packagers used a Debian-specific way to solve the issue (the alternatives system). Maybe SUSE folks simply renamed the executables.
I don’t want to upset you, but I must notice that the solutions you proposed simply addressed a nonexistent issue (library clashes between different QT versions, ABI conflicts, etc). Maybe a deeper knowledge of the problems could bring simpler solutions… But this in turn requires people to invest some time in packaging software for specific environments/distributions…
March 5, 2006 at 2:21 pm |
Joe, I’m not sure who you are, but I’d be willing to bet a fair bit that we have a lot more collective experience of library conflicts and ABI instability than you do. To say these problems “don’t exist” is to deny the facts.
I’m not sure I can be bothered arguing this. The evidence and rationales for the project are well documented on the website and wiki, which you apparently haven’t read at all.
March 7, 2006 at 4:32 pm |
[...] Apparently I have been “entirely obstructive” with regards to gaim and autopackage.[1] I consider this a complement, as I tend to agree more with (Mr.?) Josselin Mouette’s post[2] than not. Mr. Hearn is essentially just a troll, sometimes amusing, sometimes annoying, but always just a troll. [...]
March 8, 2006 at 8:32 pm |
I think that most of what’s holding autopackage back is political.
Meme’s to Destroy:
1. Autopackage is written in bash so it must be a hack. There’s a good reason for writing it in bash — this lets it run on every desktop distro out there.
2. Autopackage means losing all of the convenience (automattic security updates, etc.) of repositories. Autopackage can live a peaceful coexistence with repositories. They are not mutually exclusive.
3. All linux distributions should be considered separate operating systems. Yes, there are cases where this is a good and valid distinction to make, for example, the linux running on a cell phone and the linux running on my desktop can probably safely be called different OSes. But the differences between SuSE/Fedora/Mandrake/Ubuntu are not significant enough to be different operating systems.
I’ve been wanting to contribute to autopackage in my spare time for a long time because I strongly disagree with the #3 meme, but college keeps me busy. That said I’ve still thought about the linux packaging problem a lot, and below are some ideas that I would think about, both for improving general autopackage quality and for adoption:
1. Think more about developers. You’ve stated in the past that developers need to gain an appreciation for API stability. How can autopackage encourage that with actual code? Suggestion: An autopackage tool that lets you mark APIs as “stable” and notifies you when a build breaks the API, e.g. if a function is changed from being inline. When the developer chose to break the API, it would handle the version numbering change automatically. This would be extremely useful even to non-autopackagers and could help spur support.
2. Think more about developers. There needs to be a GUI autopackage creation tool. I know there’s a rough one in the blogs — but this should be a center of development. It should be full featured and make it easier for the developer to do all of the steps to his code necessary to make an autopackage, walk him/her through BinReloc, etc.
3. Think more about developers. Compromise with them. Don’t force them to use the autopackage installer. Develop an API, libautopackage, against which they can build their own installers that will properly register that their software is installed. But provide the current autopackage installer as a skeleton example. All of the file copying and other sensitive ops should be done in libautopackage, but they could have their own GUI around it. They could use this to display EULAs, but they could also use it to show screenshots of a game as it installs or ask users to register, etc. I know the autopackage team has a UI vision — but I doubt you have UI foresight — i.e., implement every feasible installer feature 3rd parties will want.
4. Think more about developers. It has to be acknowledged that the world of open source software development is drastically different than closed source. Autopackage has to face that developers are less coordinated, libraries change more often, and there will be less guaruntees. Autopackage should be trying to attack on as many levels of the toolchain as possible to work around this. Suggestions:
-There’s no reason apps shouldn’t be able to be compiled with a static version of a library but use the dynamic version if it is available and API compatible.
-Arbitrary .so’s should be subsitutable at runtime as long as they have the same API as the original .so. You obviously can’t protect against API breakage from behavior changes, but at least you can make sure the function signatures match up.
5. Think more about developers. They’re sensitive to feeling pushed around. The biggest attractor should be how sexy autopackage is, not trying to spread the word. Most distributions have already heard of autopackage and made their judgement on it, premature or not.
Short version: Keep developing to lower the barriers to adoption. Right now it takes A) some work to learn how to use autopackage and B) for developers to accept more than they might be willing (can’t use their own installer).
March 9, 2006 at 9:35 am |
Joe, wow, lots of good ideas there. A few brief comments:
* BinReloc might die at some point. I am looking at the possibility of a kernel patch to add /proc/self/exedir … if the kernel guys find that acceptable then apbuild could link in emulation code for it transparently so making a relocatable build becomes “./configure –prefix=/proc/self/exedir/../ && make && make install DESTDIR=/whereever”. Binreloc causes problems currently, it’d be nice to make this transparent.
* Focus on GUI creator tool – yes, there’s been a lot of discussion about this on the lists lately. Making it easier (especially for non upstream and non developers) to build packages is likely to become a focus in future.
* Tools and docs to help devs keep a stable library interface: YES! We should so do this. I actually already wrote a document on shared library versioning, but it needs to be merged into the autopackage site proper, and automated tools would be useful too:
http://plan99.net/~mike/writing-shared-libraries.html
* You can already bundle a library statically and have the systems be used if it’s missing, in a variety of ways. It needs to be better documented though and examples need to be provided.
* Making it easier for people to write their own installers on top ….. interesting idea. Right now you can write frontends independent of the backend so this allows for quite some experimentation with UI ideas (as well as having qt + gtk + console frontends). I’m not sure a libautopackage makes sense as the database may disappear or change form at some point in the future.
March 10, 2006 at 4:49 pm |
Have you guys thought about trying to be a google SoC program? It definitely sounds like that way you could pull in more outside help that way.
I’d like to help with this stuff but during the school year I’m usually pretty much sapped. I had a bit of discussion with firefly last night in #autopackage (under the name dataangel) about the static-when-unavailable, dynamic-when-available library approach. I wasn’t aware that there were already ways to do this.
Please keep hacking on autopackage. Not being able to install arbitrary software when I want is the #1 cause of me swapping around distros and I’m sick of it.
March 18, 2006 at 6:04 pm |
It was refreshing to read Mike’s words about proprietary software. It is unrealistic to think that all software can be free. My dayjob is making proprietary software on Linux and most of the time it is like pulling teeth. I have developed for windows and solaris and it is much easier.
Also thanks to Joseph for stating that the various Linux distros are really separate operating systems. I know they are. Actually I would go as far as saying that different versions of the same distro are also separate operating systems. There might well be enough changes in a new version of a distro that the software has to be separately built, installers modified and tested for it.
Most ISVs cannot afford to support more than a handful of operating systems. We try to be better. We test our software on about a dozen Linux distributions. (And now I’m counting e.g. SUSE 9.3 and SUSE 10.0 as separate distributions.) We are constantly running against there not being enough hours in a day to test a nightly build on all distros, even when we are throwing more hardware to the problem.
Currently we distribute our software as RPMs but we do our best to strip all dependency information from them because dependencies calculated on our build machine are not correct if the RPM is installed on a distro or version which is different from the build machine. Basically we are just using RPM as a glorified version of tar or zip archive (glorified because it can remove the files it added to the system.)
We cannot rely on a lot of other software on the system because it might not be installed or it might not be compatible. We carry with us a version of perl and java jre because parts of our software is written in perl and java. The only things we dare rely on are kernel, /bin/sh and libc (and that’s stretching it a bit.) We tried to use C++ but soon discovered that we have to ship our own libstdc++.so or drop C++.
I would be interested in distributing our software as an autopackage. I’m not sure what it offers to us but it can’t be much worse than RPM and sounds like it could be better. Can I make the autopackage install under /opt by default? Can I make an autopackage which is not relocatable? Making everything fully relocatable for a large software package with lots of modules is a huge task. E.g. how could you make perl relocatable? It needs to find its files and the search path is fixed during compile time.
March 19, 2006 at 5:49 pm |
Sami,
Yes it can install under /opt by default if you like.
Yes, you can make autopackages that aren’t relocatable, but I’d ask you to look at alternatives first. If it really is too hard then it can be done, but users quite like the ability of autopackages to install to any directory (ie their $HOME) so I don’t want to see lots of unrelocatable packages popping up. I’m hoping to work on a better (easier) approach to relocatability this easter.
September 20, 2006 at 5:17 am |
I’m a relatively new Linux user. I downloaded a copy of Fedora Core 5 and installed it on my system. I’ve broken this distro a few times with software right off the repositories for Fedora Core 5. Being a formor MacOS and Windows XP user, I knew nothing else to do but to completely reinstall Linux. Not sure if this is what Mike is talking about (I’m a user, not so much a develper), but I really like Linux, and having an installer to handle software for me instead of me spending hours upon the internet trying to figure it out would make Linux more appealing to people who are sick of Windows.
I know my view is completely end user, but that’s what I am. I want to put in software that works, I know that works, and without much hassle, such as trying to determine if there’s a binary for my distro, and if not, how to compile and build from the source. Isn’t this the reason GUI installers (like Installshield for Windows) were created on the other platforms?
Furthermore, I am disatisfied with the complete lack of real game support for Linux. There are a handful of native ports, and the WINE groups who can’t make every Windows game run? You really expect a proprietary software companies like game developers to make games for Linux, if they aren’t sure it will run on every major distrobution? You really expect them if they were to attempt to, to build binaries for each distro. Because I’m sure as hell they won’t release their source for you to compile and install yourself.
I don’t know your view, but if Linux is truly “an operating system for everyone” then it should be able to do everything its competitors can do, on equal grounds, if not better.
October 20, 2006 at 3:03 pm |
I think I fit the “Gaim developers except marv” qualification, but I don’t think I’ve been “obstructive”. I tried to help at least with the GtkSpell stuff, but the other developers have a point: Why should we, as an upstream project, have to add junk to our source files to support autopackage?
There are two changes necessary to Autopackage (as of when I last looked at it, and my limited understanding) that would make me happy:
1. Autopackage needs to have the ability to add patches to upstream source before building. If it already does this, then we should look at reworking the way autopackage is used in Gaim so that we can address, for example, your GtkSpell concerns. That way, people who don’t use Autopackage don’t have to deal with its changes.
2. Autopackage should integrate with the system package manager for packages installed system-wide. This is really tricky to get right, but I think it’d be really beneficial. As a user, I’d like to be able to install a newer version of Banshee via an autopackage, but then when my distro caught up, it’d be replaced with a distro-provided package. I suppose the big problem is that the autopackage might enable some features that the distro package doesn’t, so there’d need to be a way to have it take precedence over the packages from the distro itself.
Also, of course, if Autopackage is trashing existing files, that’s bad. But it sounds like you have this addressed already.
July 31, 2007 at 4:45 am |
Anger Management
Anger Management