Quite a lot of people, including frequent contributor Anton Philidor in last week's discussion of Microsoft's $3 response to Gelernter, claim that the availability of source code for open source products makes it easier for bad guys to attack them.
Consider, on this, what Bruce Schneier says in the introduction to the second edition of his Applied Cryptography about the difference between security and obscurity:
If I take a letter, lock it in a safe, hide the safe somewhere in New York, then tell you to read the letter, that's not security. That's obscurity. On the other hand, if I take a letter and lock it in a safe, and then give you the safe along with the design specifications of the safe and a hundred identical safes with their combinations so that you and the world's best safecrackers can study the locking mechanism -and you still can't open the safe and read the letter - that's security.
That's the simple answer: open source lets developers see their methods and their code as others see them - enabling them to understand and correct vulnerabilities that might well be invisible to people working in more proprietary environments.
There's also a more complex answer - one which helps explain why bad guys working with Microsoft binaries can find exploit opportunities that aren't visible to Microsoft's own code review teams.
in the tradition of complex answers it is, of course, completely counter-intuitive: it is actually easier to build an expert system capable of finding exploit opportunities in binaries than in high level code.
Makes no sense, right? but here's the thing: compilation reduces variability - and variability is the biggest factor in finding a reliable exploit opportunity.
In other words, the less variability there is in actual process behaviour, the easier it is to find any theoretical vulnerabilities in that code; and therefore, particularly in x86 environments where exploit opportunities in code are usually trivially convertible to actual exploits, the easier it is to craft and execute an attack - and notice, please, that Zovi's $10,000 Mac attack from last week worked on Windows Vista too, but would not have worked against that same MacOS X utility running on PowerPC.
The basis for an expert systems engine scanning binaries for vulnerabilities is a variation on decompilation. Most compilers go through two stages - compile to an assembler, and then translate that to object code for the intended CPU. Optimisations and code cleanup aside, both processes work by inserting programmer defined and computed values into standard blocks of machine code and then assembling these into the right order. Thus the rules for the expert system looking for a specific kind of exploit opportunity like a buffer overflow start with a long block that says:
if(encounter_some_sequence) {assembler code block could include: some long list of possible constructions }
Here the goal of the thing is first to bring that number of possibilities down to something small, and then to evaluate the plugin values with each possibility in the context of the assembler structure called to see what might be unintentionally executable and where.
The critical thing here isn't the new vulnerabilities introduced into otherwise good code by the assembler, but the number of possibilities. Notice too, that this has nothing to do with the human readability or otherwise of the code - to a program, hex isn't any harder to work with than plain text. From the attacker's perspective, the enemy is variability not visibility: how does some sequence actually play out in terms of stack or memory operations when the program runs? - and that's why widespread use of Microsoft's standardised IDEs and compilers has contributed almost as much as poor coding practices to the ease of exploit discovery in Wintel environments.
And it's also, of course, the ultimate reason obscurity is a very poor form of defence.