I glanced at Slashdot yesterday while bored and found an interesting article on the CDE packaging tool. An impressive piece of work, CDE automatically packages up all of your dependencies into a self-contained directory structure. If I wanted to package up Dungeons of Dredmor and to be able to guarantee that it would work on your computer, all I would have to do is to run CDE on my computer, where I know Dredmor works, and sha-zam! Instant package.
This is something I have been fretting about: how do we distribute Dredmor for Linux?
Never mind figuring out where to sell it, how do we even package up a binary that will work on everybody’s computers? CDE gives a good estimate for just how much stuff you need to pack with your binary in order to guarantee that it will work. Look at the “hello world” example in the video cast. That’s a *lot* of touched files – not atypical, but certainly enough that I wouldn’t want to have to go through the trial-and-error process needed to figure out what I needed to pack into my MojoSetup redistributable. I can’t trust CDE to do the work, either – CDE isn’t very careful about making sure that it doesn’t include anything in its package which would violate the GPL, so there’s a chance that a GPL’ed library could sneak its way into a redistributable package. For games using OpenGL, I would be very worried about CDE getting the bright idea to start including chunks of DRI, or proprietary GL drivers, in its redistributable as well. Ugh.
Still, you know what? It’s a nice idea, and it gives me some indication of what I will have to do in order to make sure that a Linux Dredmor binary runs everywhere.
The comments on Slashdot, however, are pretty sad and predictable. The Linux community, in their infinite wisdom, proceeds to flame the hell out of CDE. Most of their objections are tangentially related to the objections that have already been aired for, say, the FatELF project which aimed to add support to Linux for Universal Binaries. As with FatELF, everybody’s solution is the same: “We should all just be using package management.”
Here is what I want to say, and let my words be carried down from the mountaintops, written on tiny stone tablets:
Package management is not a universal panacea.
Package management does not guarantee that I can take my game, package it, sell it, and guarantee that the resulting binary will execute on your computer, my computer, and Derek’s computer, running Fedora, Slackware, and Ubuntu respectively. Package management isn’t standard across distributions, and package management is not an acceptable installation tool for commercial, closed-source software produced by independent software vendors. Package management doesn’t work on distributions that don’t exist yet – back when I started in the Linux biz, Ubuntu didn’t exist yet.
Closed source software exists. ISVs exist. Games are going to continue to be closed-source for all of their lifespan, or at least until their creators feel that they’ve shipped enough units to justify open sourcing it. Get used to it.
What is worse is that there is no *standard* way that I can build a game, as a binary, and guarantee that the resulting binary will work across all distributions. The best case scenario is that I include shared libraries for everything, target a “close enough” version of libc, write a shell-script that munges the library loading path, and then hope and pray. As a consequence, most Loki games no longer run on modern Linux distributions. (Conversely, I can build Dredmor on my Windows 7 machine and the resulting binary runs on Windows 95, 98, 2000, XP, Vista, and 7.)
I am not saying CDE is a solution. Of course it’s not. Arguably, FatELF may not have been a solution either; I’m sure some of the objections raised on the LKML and other places were perfectly reasonable, and I’m also sure that most of them could have been worked around through reasoned discussion and technical debate. Both of these projects are starting points for coming up with a solution, and every time anybody – usually somebody *outside* of the mainstream community of users, like Ryan or Philip – tries to make my life easier, I get to watch as the Linux community demonstrates that it is totally uninterested in providing ISVs with a mechanism that will let them install software, across distributions and across systems, and which will guarantee that the resulting binary will work. At the same time, everyone likes to complain about how no commercial game developers want to support Linux… and then wonder why Windows continues to dominate on the desktop.
I have supported Linux game development for ten years, and I will continue to support Linux game development at Gaslamp. That said, some days it’s sure not easy.
I am an avid linux gamer too, and I’ve never had any problems with (free or proprietary) games, that are packed in a way you described above:
1) 32-bit ELF binary
2) some crucial libraries in $prefix/libs (openal, SDL)
3) linked against system’s libc, libx11, libGL, etc.
4) script to setup LD_LIBRARY_PATH and run your game
That is pretty standard way, whatever you say. Loki games do run on today’s systems as many old binaries do (not all, but many).
Also, you could release patches, if something changes in the world of linux. And after some 10 years you could release the source, and the community will make the patches itself! 😉
There are probably good reasons to do a 64-bit ELF as well, actually, which means two builds. This isn’t such a big deal for Dredmor, as it’s pretty low-tech, but I can imagine our next project being much happier without having to worry about the 32-bit memory address limit.
That said, I disagree with you about patches. What if, for whatever reason, Gaslamp Games goes under after releasing Dredmor? What if I am hit by a bus after Dredmor ships, and I’m carrying the only copy of the Dredmor source tree with me on a laptop? (What if David just can’t be bothered to compile the Dredmor source code after I’m dead?) What if Gaslamp gets bought out by a Linux-unfriendly company, and I decide to part ways with them soon after? There are any number of reasons why relying on IHVs to patch legacy software, over time, in order to ensure that their software works with new versions of the kernel and glibc is a ridiculous proposition. As the fall of Loki indicates: things happen.
And if Loki Software is still running on today’s systems, why are there pages such as this one dedicated to binary patching? http://members.shaw.ca/dan.mckay/LokiInst.html
Joel Spolsky puts it well: http://www.joelonsoftware.com/articles/APIWar.html – Microsoft lost the API war as soon as it stopped guaranteeing forwards compatibility. Linux never had forwards compatibility to begin with.
Well, most of the problems with Loki games you’ve linked to, relate to installation and updating (no gtk1 in host system, installer bugs, etc.).
64-bit ELF? I’ve run 32-bit (linux native) Neverwinter Nights on a 64-bit system without a hassle, so that’s not a requirement to me (but good to have, okay).
As for the patches, hehe… Just keep your sources “on the cloud”: a corporate dropbox account, a private SVN server, etc… 🙂 This really helps, if you’re dead! 😉
I’m not an expert in distributing closed code, but could a little bit of open source be a good thing here? I’m not saying that you should open source any of your game engine, but perhaps you could create an LGPL shim library that encapsulates all of the functionality you require out of the host system. That way you could create binaries that link against this library only and then the distros could custom patch that shim layer into perpetuity to support newer APIs.
So, for example, instead of linking to openal’s makesound() (yes, I’m sure that doesn’t exist) you would link to the open libdredmore’s (or whatever) makesound() implementation, which would take the same arguments and pass it to openal. As you release the shim would probably be 1:1 with whatever library you’re using, but as time goes on and libraries go obsolete, change their APIs etc. it’s up to the distros to translate your libdredmore makesound() call into something that works on modern systems.
Again, this is just an off-the-cuff idea, and I’m sure that you’re using a lot of library functionality so libdredmore might be quite a large shim but you would be completely invulnerable to libraries breaking by allowing the community to do the work for you.
I agree that a standard way to package and distribute closed-source software across linux distributions would be great. I’m also pessimistic about the odds of it happening, because any solution would have to work through the package manager to be acceptable to the distros. This is because, to the distros, all software should be managed by the package manager.
Because linux is itself, if a few people disagree with an aspect of an adopted solution, they can just write their own and use it instead. Thus, a universal solution is unlikely. But it sure would be nice…
That said, I very much appreciate the work gaslamp games is putting into supporting linux, and can’t wait for it to be finished!
I would get in conctact with Ubuntu and get it put into the Ubuntu Software Centre, would be an easy way to reach a lot of people!
As a 64-bit Slackware user, I would like to see a 64-bit binary distributed with a standard Slackbuild (www.slackbuilds.org). The nice thing about slackbuilds is its just a script that uses the standard make process to build a distribution package.
As far as a universal distribution format is concerned, there is none, save the standard make scripts. If you want to please most people in the Linux community, give the people who maintain the distributions what they need to construct your program in their preferred package format, and they will take care of you.
I would advise to simply use wine.
– It provides an uniform platform on top of the various linuxes. It is the wine package that has to be distrib-native, and bother with the various versions of linux libraries.
– It works! Picasa & Google Earth and many others work ell this way already.
– It already works for Dungeon of Dredmor from a quick test. You will only have to work of small bugfixes for making it work perfectly for wine, not a full port. For what I saw, only the fullscren does not work (checking the box has no effect)
You could go the same way as propritary drivers go; you could release a source package bundled with binary-only parts containing your precious code?
The source part provides an interface between your code and the platform supposed to run it. Leave it to the distribution vendors to make what ever patches is needed for their strange setups. This way your code will work on setups you havn’t even heard about and you can release a single binary package and say that if someones distribution is too strange to handle it, worst case they can always just compile on their own..
Again, this assumes:
a) the distributions care (hint: they don’t. Their money comes from server market and support for major companies, not desktops. They didn’t care back when Desktop Linux was supposed to be a thing, either. The exception is Debian, and we don’t pass their Purity Test.)
b) making a hybrid binary/source package would make life easier. (In fact, it would make matters worse. See the brouhaha that exists with binary kernel drivers.)
I’ve always been curious about why games companies don’t simply open source the game engine, and sell the content? The content (textures, models, etc.) is what is really needed to play the game, so open sourcing the engine wouldn’t allow people to play without paying, they’d still have to buy the content. Like how id does it with DooM. You have the open source engine, but you still have to purchase the .wad files to play the game.
So, how come so few game companies do it that way?
Because the company is providing you with two things, not just one. They’re providing you with the content, sure, but they’re *also* providing you with the framework to make content. If they open source either one of these things, they’re making a conscious decision not to try to earn money from the work that their asset took to create.
Since developing the engine and the framework took us basically a year and a half of solid work, if we don’t actually sell it, we’re deciding to basically donate all of that work, and all of the money it cost us to sustain ourselves and our pets while we did it.
If we can’t make money on it, we can’t (and won’t) do it. It’s too expensive.
Doom wasn’t actually free when it was first released, as far as I can recall. Not even the engine was. It had an inherent value, so id sold it. Their content also had value, so they sold that, too.
After enough time, (4 years later) the engine wasn’t particularly valuable, as the techniques became easier to reproduce, so at a certain point it was economically valuable to them to release the engine for free, as they were going to gain new fans (who might then buy their content) as well as positive PR to propel their next wholly commercial products. Plus open source was (and is) cool, so there’s that.
Now, there’s one other option that id didn’t do. You can sell the platform and make the content free. Almost no one does this, because the content of games doesn’t depreciate in value. Good writing (so to speak) is always good writing. Therefore, it makes even less sense than the former case, and you have more to lose because of that lack of depreciation.
TL;DR: One day when we can all afford to have health insurance and our own homes, I’ll consider it.