Quote:
|
There've been many discussions of late about the challenges of scaling app installation to a very large number of apps, including making app packages much easier to automatically audit and sandbox, and making them safer and quicker to install without the possibility of system-level breakage. This is needed to make it quicker and easier for app developers to get their software into Ubuntu. There are several pieces to this (including a good deal of prior discussion in https://wiki.ubuntu.com/AppDevUploadProcess), but it seems clear that one of those is some kind of simplified packaging format which we can offer to third-party application developers. This is not aimed at changing packages that are already part of the Ubuntu archive; for the most part our existing system works well for those, and they tend to have non-trivial dependency structures. We'll continue to use dpkg and apt for building the Ubuntu operating system, syncing with Debian, and so on. There's no point developing a packaging system for apps and making it have the full panoply of features needed for the Ubuntu archive: it'd just be second-system-effect on top of our current packaging system. So the scope of what I've been considering is purely leaf apps built on a fixed "base system", which in the case of the initial target of the Ubuntu phone/tablet work would be the run-time part of the Ubuntu SDK. The radically-reduced dependency structure means that most apps will be Ubuntu-SDK-specific to start with, although I'd like to make sure that the package format design includes enough support up-front to allow this to be useful for other platforms that define suitable base system profiles in future. So, at Steve Langasek's request, I've been putting together a proof of concept of a low-level app package installer and packaging format. Highlights of what it can do so far are: * no dependencies between apps; single implicit dependency on the base system by way of a Click-Base-System field * installs each app to an entirely separate directory * entirely declarative: maintainer scripts are forbidden * base package manager overhead, i.e. the time required to install a trivial package containing a single small file, is about 0.15 seconds on a newish x86 laptop and about 0.6 seconds on a Nexus 7 (and that's with the current prototype implementation in Python; a later implementation could be in C and would then be faster still) * not limited to installing as root, although there may be similar constraints elsewhere to ensure that apps can't edit their own code at run-time * packages built by feeding the intended output directory tree to a simple Python tool, plus a manifest.json file * building packages requires only the Python standard library, with the intent that it should be possible to build these packages quite easily on non-Ubuntu or even non-Linux systems * binary packaging format sufficiently similar to existing one that we could add support to higher-level tools with minimal effort * strawman design for hooks into system packages, which will be entirely declarative from the app's point of view * unit-tested from the start Obvious items I still need to work on: * produce a strawman hooks implementation with some real worked examples * integrate (or demonstrate how to integrate) the container isolation properties worked on elsewhere * Click-Base-System field is skeletally simple right now, and may need to be expanded to at least leave open the possibility of multiple flavours of base system (see also GNOME's profiles idea) * adjust unpack handling to avoid problems with project renames and name clashes, and to unpack each version into its own directory and flip symlinks to allow for multi-user independence * integrate into the Ubuntu SDK, as well as providing examples of how it can be integrated into other build systems too To do, but not in this project: * package acquisition * indexing support and other frontend things (i.e. the app store) * upload handling So far I've demonstrated to my own satisfaction that we can get acceptable performance and isolation characteristics with very little work; the installer part on its own is under 300 lines of Python right now, which I consider useful since bugginess is often proportional to code size. I should have the prototype ready for people to look at in time for UDS next week, and I'll ensure that there's a session scheduled for this. Is there anything else people can think of that a system like this needs to consider? Footnote: Why not use one of the existing systems out there? We still might, but at present it's not clear that it would make a whole lot of difference. Many of the existing app packaging systems are more of a reflection of the system they were built for than anything else. If you look at, say, Android's APK format, it's essentially a zip file with a manifest and some conventions about Dalvik class installation and the like. Most of the other mobile app formats are similar. Things like Listaller might be a reasonable fit, but I'm worried about importing things like a full dependency solver into this: much though I love dependency-based packaging systems, they necessarily involve scanning the system packaging database at some point and I would much prefer app packages to be as independent of that as possible, mainly for performance but also to be as bulletproof as possible. GNOME app bundles appear to have a rather different sandboxing model than we need (e.g. enforced restrictions on dependencies), we probably want better system integration, and perhaps other differences; but that said I like the coarse-grained profile model of dependencies (and unknowingly imitated it), and it might make sense to share some profile names. Barry spent some time looking at 0install, and it wasn't too bad a fit but we would still need to solve many of the same system integration problems. The proof of concept I wrote also isn't entirely new code. It's tiny due to using .deb as a container format (minus maintainer scripts, full dependencies, etc.), so I get to save effort by using dpkg to unpack things, which leaves us room to selectively use more of its features in future if we want to. This is quite a compelling approach to me for various reasons. I could be persuaded to look at something else, but I don't think there's actually much code to share at the packaging format level anyway, as it's so simple; quite possibly more at the sandboxing level, but that's really a separate layer. Furthermore, the priority for this system at present is for Ubuntu phone/tablet app packages, which haven't actually been built yet. As a result, I'm less worried about fragmentation than I might otherwise be. I would like to make sure that it's usable elsewhere too, although somebody would need to work out what that means in terms of base system profiles. Although I'm very familiar with dpkg and traditional package management, I only started looking at this side of things a couple of weeks ago, so I'm sure I'm behind on research. Pointers welcome. |