← journal

Basics of the Unix Philosophy

Highlights

The Unix philosophy (like successful folk traditions in other engineering disciplines) is bottom-up, not top-down.

Doug McIlroy, the inventor of Unix pipes and one of the founders of the Unix tradition, had this to say at the time [McIlroy78]:

(i) Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new features.

(ii) Expect the output of every program to become the input to another, as yet unknown, program. Don’t clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don’t insist on interactive input.

(iii) Design and build software, even operating systems, to be tried early, ideally within weeks. Don’t hesitate to throw away the clumsy parts and rebuild them.

(iv) Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you’ve finished using them.

He later summarized it this way (quoted in A Quarter Century of Unix [Salus]):

This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.

  1. Rule of Modularity: Write simple parts connected by clean interfaces.
  2. Rule of Clarity: Clarity is better than cleverness.
  3. Rule of Composition: Design programs to be connected to other programs.
  4. Rule of Separation: Separate policy from mechanism; separate interfaces from engines.
  5. Rule of Simplicity: Design for simplicity; add complexity only where you must.
  6. Rule of Parsimony: Write a big program only when it is clear by demonstration that nothing else will do.
  7. Rule of Transparency: Design for visibility to make inspection and debugging easier.
  8. Rule of Robustness: Robustness is the child of transparency and simplicity.
  9. Rule of Representation: Fold knowledge into data so program logic can be stupid and robust.
  10. Rule of Least Surprise: In interface design, always do the least surprising thing.
  11. Rule of Silence: When a program has nothing surprising to say, it should say nothing.
  12. Rule of Repair: When you must fail, fail noisily and as soon as possible.
  13. Rule of Economy: Programmer time is expensive; conserve it in preference to machine time.
  14. Rule of Generation: Avoid hand-hacking; write programs to write programs when you can.
  15. Rule of Optimization: Prototype before polishing. Get it working before you optimize it.
  16. Rule of Diversity: Distrust all claims for “one true way”.
  17. Rule of Extensibility: Design for the future, because it will be here sooner than you think.

As Brian Kernighan once observed, “Controlling complexity is the essence of computer programming” [Kernighan-Plauger].

The only way to write complex software that won’t fall on its face is to hold its global complexity down — to build it out of simple parts connected by well-defined interfaces, so that most problems are local and you can have some hope of upgrading a part without breaking the whole.

Unix tradition strongly encourages writing programs that read and write simple, textual, stream-oriented, device-independent formats.

don’t write programs that accept and emit simple text streams, it’s much more difficult to hook the programs together.

To make programs composable, make them independent.

separate your application into cooperating front-end and back-end processes communicating through a specialized application protocol over sockets

Many pressures tend to make programs more complicated (and therefore more expensive and buggy). One such pressure is technical machismo. Programmers are bright people who are (often justly) proud of their ability to handle complexity and juggle abstractions. Often they compete with their peers to see who can build the most intricate and beautiful complexities. Just as often, their ability to design outstrips their ability to implement and debug, and the result is expensive failure.

Many a good design has been smothered under marketing’s pile of “checklist features” — features that, often, no customer will ever use.

encourage a software culture that knows that small is beautiful, that actively resists bloat and complexity:

Software is said to be robust when it performs well under unexpected conditions which stress the designer’s assumptions, as well as under normal conditions.

Most software is fragile and buggy because most programs are too complicated for a human brain to understand all at once.

the way to make robust programs is to make their internals easy for human beings to reason about. There are two main ways to do that: transparency and simplicity.

software is_transparent_ when you can look at it and immediately see what is going on. It is _simple_when what is going on is uncomplicated enough for a human brain to reason about all the potential cases without strain.

Modularity (simple parts, clean interfaces) is a way to organize programs to make them simpler.

Software should be transparent in the way that it fails, as well as in normal operation.

Therefore, write your software to cope with incorrect inputs and its own execution errors as gracefully as possible. But when it cannot, make it fail in a way that makes diagnosis of the problem as easy as possible.

Rushing to optimize before the bottlenecks are known may be the only error to have ruined more designs than feature creep.

get your design right with an un-optimized, slow, memory-intensive implementation before you try to tune. Then, tune systematically, looking for the places where you can buy big performance wins with the smallest possible increases in local complexity.

If it is unwise to trust other people’s claims for “one true way”, it’s even more foolish to believe them about your own designs.

Never assume you have the final answer.

When you design code, organize it so future developers will be able to plug new functions into the architecture without having to scrap and rebuild the architecture

You owe this grace to people who will use and maintain your code after you.

When you design for the future, the sanity you save may be your own.