In a previous post, I made a pretty pathetic attempt to define Evil. I suck at it, which is weird because it’s something I do every day – we all do. Every time we interact with somebody we have to make a decision on whether to trust them or not. Buying some food from a stall: are they likely to sell me an under-cooked burger? Lending some money to a friend: are you likely to get it back? Telling somebody a secret: will it stay a secret for long? Listening to somebody tell theirs: are they lying? We make hundreds of trust decisions every day and never even think about it because it feels so natural.
So you could be forgiven for asking why desktop computing is nearing collapse under the weight of untrustworthy software. Of course in our lives sometimes we’ll be conned or cheated or lied to. Hopefully, not very often. But for many people, if their computer was a person it’d be going straight to jail for serious fraud: their computer deceives them, cheats them, annoys them and sometimes downright steals from them every single day. Computers aren’t built that way, they get that way because – ignoring the influence of instant code execution (ICE) exploits – people make bad trust decisions.
WHY do people trust Kazaa, a program designed specifically to let you break the law? WHY do they act surprised when shortly after installing it pop up adverts start appearing? I think we now have to delve into the realm of psychology.
When people make a trust decision with a real world person or entity (such as a company), we base that decision on many factors:
- Past experience (how well we know them) – of course this assumes we can link consequences with their actions. IE if we don’t know we’re being cheated this one can lead us astray
- Advice from people we already trust
- Strength of motivation (do they have something I want?)
- Attractiveness. In a person this can be how smartly they are dressed, for a company or website how professional it looks
- Synthesis of various visual cues. For instance, does a website have a privacy policy, does it have contact details ….. for people, how are they acting, do they seem nervous, how easy is it to contact them, how did you meet etc.
- Understanding of what they might do to you – it’s pretty obvious that if you lend somebody your phone in the middle of the night on a dark street with nobody around, they may do a runner. That’s common sense.
Problem! Most of these factors are less effective with software. For past experience, you have to go on your experiences of the company or group who made it. Have you used other programs by these people? Were they good? Most critically of all, if something bad happened shortly after you used their program, did you figure out where it came from?
Advice from people we trust is very valid – why do you think Firefox got so popular so quick – but this only works for commodity software like web browsers or chat programs. There is so much software in the world that the chances of personally knowing somebody who can vouch for a program is slim. Large auction sites like eBay have the same problem, a reputation system is used there to make up for lack of personal trust with sheer weight of numbers.
Strength of motivation ….. aaah this is so important. People install filesharing apps sometimes despite knowing the risks because they want the program so bad! Not much you can do there, except to reduce the risk.
Attractiveness is not a useful cue with software, because GUI software tends to look the same due to the widget toolkit imposing a certain visual style.
Perhaps the biggest problem is users not understanding what rogue software can do. People who are attacked by dialers are often shocked that a computer program was able to run up a huge phone bill by dialling numbers in Russia or Cuba behind their backs. Yet to any computer engineer this fact is obvious: the modem is controlled by software, and any software can control the modem.
The following quote from Security and Usability sums up the challenge nicely:
Users often want to make use of things they don’t completely trust. For example, it’s reasonable for people to want to run programs or visit web pages without having to understand and audit their source code. Instead of trusting the unknown entity, users trust an agent (such as a secure operating system or web browser) to protect their interests by placing constraints on the unknown entity. The agents challenge is to determine the correct constraints.
Think about that.
December 5, 2005 at 12:42 pm |
Nobody in the Free Software Movement asks users to “understand and audit their source code”. The Free Software Movement’s message says that people should have the freedom to do so. When users run proprietary software they are denied the freedom to do this for themselves or to get someone else to do it for them. The only people who can do this are those motivated to take advantage of the users.
Users deserve software freedom (the freedom to inspect, run, share, and modify computer software) and proprietary software, regardless of ostensible purpose, is untrustworthy.
–J.B. Nicholson-Owens (jbn@forestfield.org)
December 6, 2005 at 7:38 am |
It depends how you define “untrustworthy”. I can’t read minds, but I still trust my friends. This seems no different to software – I can’t always see what makes it tick but I can still trust it.
If you want to get theoretical, then even if I can see /some/ source code that doesn’t mean it actually equals the binaries I’m running – even if I compile it myself the well known story of the cracked compiler shows that at some point you just have to have blind faith that your setup is trustworthy.
Also, being open source doesn’t necessarily make software an example of ethics – this blog runs WordPress:
http://www.waxy.org/archive/2005/03/30/wordpres.shtml
December 9, 2005 at 1:39 am |
The Free Software Movement focuses on the social effect on people by asking the most important question one can ask–how should we treat other people? That movement’s answer is to create and sustain a community where we can become interdependent by granting one another the freedom to run, inspect, share, and modify computer software. This community is not possible with proprietary software because you don’t have the rights you need (nor the source code) to grant those rights to others or engage in those acts yourself.
The Open Source Movement doesn’t talk about ethics. That’s one of the issues that the founders of that movement decided not to raise. That movement speaks to businesses, chiefly, about how to better leverage a pool of talented free labor. That movement’s founders decided early on that software freedom talk would get in the way of communicating their philosophy to their intended audience. That’s why that movement talks about the speed, cost, and lack of bugs developed in the Open Source methodology.
December 9, 2005 at 4:45 am |
You’re nitpicking at terminology. Yes, I’m aware that some people use Stallmans distinction between the free software movement and the open source movement: I don’t, for anybody who isn’t deeply involved with the community it’s a confusing abuse of equivalent (to them) terms.