Given the mass epidemics of spyware, viruses and bots we’ve seen over the past few years it’s not surprising people are looking at ways to deal with rogue software. The old world assumption that programs are a tool and the user an experienced craftsman no longer holds – arguably it hasn’t for decades – and that changes everything.
How can we stop rogue programs? One way is to lock them down so rather than having full access to the entire system (as programs running on XP Home have today), they only have the permissions they need to do their job. This is hard. It mixes up different concepts that use the same implementation – least privilege is used to stop hackers gaining extra influence by hacking a program, yet people also want to use it to stop evil programmers doing things they shouldn’t.
The hard part isn’t really the technology: SELinux lets us specify to a fine level what a program can and cannot do. The hard part is deciding what constitutes “evil”. Right now, applications have full access to the entire system, at all times. There is no defence against evil software, not on Windows, OS X or Linux. This applies even if the programs don’t run as root, as typically they are installed as root and that means they can mark themselves suid-root if they so wish. For autopackages you can install to $HOME to slightly mitigate this risk, but the root vs user distinction isn’t enough to stop the majority of “evil” programs out there.
What is Evil?
We can start with a very basic idea of what evilness is, and work upwards from there. To start with, let’s say that evil software is software that tries to tamper with the core operating system internals. An obvious example is adware like Aurora. This program employs several tricks like hiding a respawner inside svchost.exe and modifying winlogon.exe internal registry entries, forcing itself to load at Windows startup in such a way that makes it near-impossible to delete (even in safe-mode!)
This sort of random hooking of OS entrypoints is pretty clearly Evil. It has no legitimate uses for non operating system developers, as there are documented ways to load your program at startup.
Embedding yourself into the system so tightly you cannot be removed is one way to tamper with OS internals. There are others, like the browser helper objects (BHOs) and winsock LSP chains which can be used to filter or rewrite web pages arbitrarily.
Examples on Linux would include patching the kernel system call table, overwriting or modifying “init”, and other techniques favoured by crackers.
Stopping it
We can draw up a simple set of rules to protect the operating system core that are likely to completely eliminate undesirable application behaviour with minimal risk of breaking backwards compatibility. Obviously, doing this requires some distinction between an application and an operating system component but autopackage vs rpm/deb already provides that.
Here’s a quick stab at such rules. Programs (including their install scripts) should not be able to:
- Load kernel modules
- Modify or overwrite existing files
- ptrace an existing process running at a different privilege level
- Add files to peoples home directories EXCEPT for menu items, desktop icons, and at few other well defined integration points.
Of course, there are several ways to implement these rules. One is SELinux policy, which will become an option once Fedora Core 5 ships. Another way is to use the NeXT/Apple approach of “interpreted integration”, in which a package is an inert thing containing files that are loaded and interpreted by the system shell. Installers can’t modify anything the OS doesn’t let them modify. For autopackage, SELinux is probably the only feasable method, though I slightly prefer the NeXT approach.
