Excerpts from "UNIX: A History and a Memoir"
Author: Brian Kernighan
Why did Unix and everything associated with it meet with such success? How did the experiment of a few people grow into a project that literally changed the world?
The first thought that crossed my mind when I picked up the book was: "345 pages?! Seriously?" I expected to see a work comparable to something monumental, like Daniel Yergin's "The Prize" at three thousand pages — this is UNIX and Bell Labs after all. Apparently, the author expected such a reaction and warns from the first pages that a more detailed history should be sought in the official Bell Labs publication (specifically, seven volumes and nearly five thousand pages) "A History of Science and Engineering in the Bell System," as well as Jon Gertner's "The Idea Factory" and James Gleick's "The Information," which excellently describe what was happening in the field of computer science.
Bell Labs
In 1966, Bell Labs employed 3,000 people, of which at least a thousand held doctoral degrees in physics, chemistry, mathematics, and various engineering disciplines. And between 15,000 and 25,000 belonged to various hardware and systems development groups for the Bell System.
Initially, most research at Bell Labs was conducted in the fields of physics, chemistry, materials science, and communication systems. Employees could choose their own research topics, as there were so many pressing problems that it was easy to find something interesting and simultaneously useful for both the Bell System and the world at large.
The scientific and technical achievements of Bell Labs changed the world. The first of these was the transistor, invented in 1947 by John Bardeen, Walter Brattain, and William Shockley, who were trying to improve amplifiers for long-distance telephone lines. The transistor resulted from fundamental research into the properties of semiconductor materials while searching for less energy-intensive and more reliable physical devices than the vacuum tubes that were ubiquitously used in 1940s communications equipment and, incidentally, formed the basis of first-generation computers.
In 1956, the invention of the transistor was awarded the Nobel Prize in Physics. This is one of nine Nobel Prizes awarded for work at least partially carried out at Bell Labs.
Other important inventions include negative feedback amplifiers, solar cells, lasers, mobile phones, communication satellites, and charge-coupled devices (which power the cameras built into phones).
AT&T held a monopoly on telephone service in most of the United States. However, its rates were regulated by federal and state authorities, and the company could not engage in business unrelated to providing telephone service. AT&T's services had to be provided to anyone, regardless of the difficulty of access or profitability. As compensation, the company had a stable and predictable overall rate of return. As part of this arrangement, AT&T directed some of its revenue to Bell Labs for the further improvement of communication services. Essentially, the Labs were funded by a modest tax on all phone calls in the country. According to a document by A. Michael Noll, AT&T spent about 2.8% of its revenue on research and development, and about 0.3% on fundamental research. I don't know if such a scheme would work today, but for decades this arrangement generated both continuous improvements in the telephone system and a significant number of fundamental scientific discoveries. The defining factor was stable funding. For AT&T, it ensured a long-term perspective, and for Bell Labs employees, it provided the opportunity to conduct research without the need for immediate returns. Nothing like this exists in the modern world — planning often doesn't look beyond a few months, and a great deal of effort is spent on thinking about next quarter's financial results.
Bell Labs became a pioneer in the design, creation, and improvement of communication systems. This general term covers everything from the design of household devices like telephones to the infrastructure of switching systems, radio relay towers, and fiber optic cables.
Sometimes solving such a range of practical problems drives progress in fundamental sciences.
Bell Labs began working with computers in the 1930s, and by the late 1950s, large computer centers had appeared there.
UNIX
In 1964, the CTSS (Compatible Time-Sharing System) developed at MIT was widely used. The CTSS operating system proved to be such a productive programming environment that MIT decided to create an improved version of it and brought in Bell Labs, which had extensive experience building its own systems since the early 1950s, to design the Multics system (Multiplexed Information and Computing Service).
Multics was maintained and used until 2000, although it did not gain widespread adoption. It spawned many good ideas, but its most significant contribution turned out to be unexpected: how it influenced the modest Unix operating system, whose development was partly a reaction to the complexity of Multics. After exiting the project, Bell Labs employees who had worked on this operating system had to find themselves a new occupation. Many wanted to continue developing operating systems, but Bell Labs management, disappointed with the Multics outcome, was not interested in purchasing equipment for a similar project.
According to the author, such caution on the part of management had a positive influence on UNIX development, because in this case, to obtain resources, people had to think through their proposals and concentrate on the idea they were promoting. Limited resources often force ideal planning of the work process.
In attempts to persuade management to fund the purchase of equipment, the UNIX team resorted to a trick. Bell Labs, as a large and productive research company, generated many patent applications — roughly one patent per day. Such an application was a text document with strict formatting requirements, such as line numbering per page. No existing system could format documents this way, so the patent department planned to purchase equipment from a company promising software that would eventually solve such problems. At that time, line numbering had not yet been implemented. Joe Ossanna proposed a different plan. The patent department could use the PDP-11 computer for preparing patent applications, because the Unix group would write the software to output applications in the required format. There was no mention of working on operating systems. This combined approach allowed overcoming the remaining objections from management. The money for the PDP-11 came from Max Mathews, director of the Speech and Acoustics Research Center. It turned out that one of his superiors, Lee McMahon, was very interested in text processing and helped advance Ossanna's plan.
1969 marks the beginning of the UNIX system development.
Developers: Ken Thompson, Dennis Ritchie, and Doug McIlroy.
The author claims that one of the drivers of UNIX's success was clear documentation.
Dennis was excellent at writing technical documentation. He had an elegant style of his own, could express thoughts wonderfully, and possessed inimitable wit. Without doubt, the success of the C language and the Unix operating system can be partially explained by Dennis's writing skill.
The first edition of Unix was up and running by the end of 1971. Over the next few years, roughly every six months, a new edition of the manual came out, each time describing new functionality, new tools, and new languages. The Sixth Edition of the operating system, whose manual is dated May 1975, was the first to leave Bell Labs and had a profound impact on the world.
The first edition of Unix had just over 30 system calls, half of which were related to the file system.
All UNIX tools were, of course, written in C, as it was an expressive and efficient language. Today, few people know and remember that in those days, when computer storage capacity was measured in kilobytes rather than gigabytes, efficiency in both time and space was crucial. Every byte was counted, and it mattered at what level each instruction was executed, so a language that could account for both aspects was a practical necessity.
After several years of use within division 1127, the Unix operating system began to spread throughout Bell Labs and beyond, mostly through universities that could obtain the source code by signing a non-disclosure agreement. There was no talk of open source yet. The system could only be used for educational purposes, and license holders could discuss their experience with Unix only with other licensed users. However, the community grew quickly, user groups appeared around the world, and major technical innovations were added, such as ports to various hardware and new internet access mechanisms.
The workstation market emerged thanks to technological improvements that allowed packing serious computing power into a small unit and selling it at a modest price. The full price of a system became acceptable partly due to the availability of software, including the operating system. New manufacturers no longer needed to create an operating system from scratch; it was enough to port Unix and related programs to the processor being used. Thus, the development of the workstation market was greatly facilitated by the availability of Unix.
The prototype of the tool that made people see "tools" as something more than "programs" was the grep utility, designed for pattern search. The author of the first version was Ken Thompson.
The name of the grep utility even made it into the Oxford English Dictionary.
Unix is characterized by the concept of programs as tools. Writing small programs that do one thing well, instead of large programs that try to do many things, has many advantages. Of course, there are cases where large programs make sense, but it's much more convenient to have a set of small tools that can be combined in new ways, and this operation is accessible even to ordinary users.
Essentially, this is about modularity at the program level, similar to modularity of functions within a program. This approach can be formulated as "divide and conquer," since individual components don't interact with each other. It gives the user the ability to combine tools to their liking, which is difficult to achieve in a large program trying to solve too many problems within a single package.
A funny story related to the grep utility happened in 1972. One Bell Labs employee noticed that if you turn a pocket calculator upside down, some of the digits displayed look like letters. For example, 3 becomes E, 7 becomes L. Since we had a dictionary on our computer, we were asked to compile a list of words that can be typed on an upside-down calculator. As a member of the research division, I was happy to help solve a practical problem. I asked which letters could be obtained from upside-down digits. It turned out to be the letters BEHILOS. Then I typed the command grep 'ˆ[behilos]*$' /usr/dict/web2. The file /usr/dict/web2 contains the second edition of Webster's International Dictionary, that is, 234,936 words, one per line. The character string in quotes is a regular expression or pattern that specifies strings consisting of any combination of the seven specified letters. I got a list of 263 words. English is my native language, but there were many words in that list that I had never seen before. I printed the list and gave it to my colleague. I think the result satisfied him, because there were no further questions or requests. But I was left with this story as a wonderful demonstration of the value of tools like grep and regular expressions.
Undoubtedly, a market for selling Unix existed. One could even say that AT&T unintentionally created it by giving licenses to universities. After all, students who became accustomed to working in this OS wanted to continue using it after graduation, once they found jobs in commercial companies that could afford to buy paid licenses. From 1984, USL made great efforts to turn Unix into a professional commercial product. The culmination was System V Release 4, or SVR4. AT&T invested significant resources in making this version a standard implementation and carefully defining compatibility for both code and object modules. I think it was the attention to standards and compatibility that was important. Not all of AT&T's attempts to profit from Unix were successful, but the standardization of Unix was a great gift to the entire community.
Looking back, all these legal disputes were a side effect of AT&T's hasty and almost accidental decision to make Unix available to universities. Having spread from universities, where it was used for free, to companies willing to pay for it, Unix acquired commercial potential. However, it was already too late for effective copyright protection. Even after AT&T imposed restrictions on the source code, the system call interface remained in the public domain, and community members had enough experience that creating new versions unencumbered by AT&T licenses became commonplace. A similar situation arose with application software such as compilers, editors, and all other tools. Metaphorically speaking, AT&T was trying to lock the safe after all the money had been taken out.
Unix also became the main driving force behind the development of the internet. The University of California, Berkeley developed the Berkeley Software Distribution (BSD) — an extended version of Unix implemented using the TCP/IP internet protocol suite. The development was based on the Sixth Edition of Unix, which Bell Labs distributed with source code to universities and research institutes in 1975, giving rise to the "open source" culture. The BSD Unix operating system helped implement the internet.
Due to AT&T's attempts to profit from software, the Unix license included more and more restrictions. Among other things, it limited the use of Unix in universities, which opened the way for BSD. At the same time, the ongoing wars between AT&T and UC Berkeley prompted others to try their own Unix-like systems. Independently created versions were free from commercial restrictions because they only used the system call interface, not someone else's code.
As I said in the preface, the birth of Unix, which changed the world of computers, was probably a singularity resulting from a unique combination of circumstances. I doubt we'll see anything like it again in the OS space, but there will certainly be other cases where a handful of talented people with good ideas and a favorable environment will change the world with their inventions.
Linux
Linus did not expect a great future for his hobby system, just as Ken and Dennis did not foresee the success of Unix. What started with a few thousand lines of code now numbers over 20 million lines.
Currently, Linux is the standard OS capable of running on any computer. It is installed on billions of devices (for example, on all Android phones). It runs a significant portion of internet infrastructure, including servers of giants like Google, Facebook, Amazon, etc. It is found in many devices implementing the Internet of Things (IoT): my car, my TV, your Alexa and Kindle, and Nest thermostats all run Linux. Looking at the other end of the computing power spectrum, you'll find this OS on the top 500 supercomputers in the world. However, it is poorly represented on the laptop and PC market. They mostly run Windows and MacOS.
Interesting Facts
- Everything typed on the teletype was sent to the computer, and the response information was printed (in uppercase) on a long roll of paper.
In the 1980s, Microsoft distributed a version of Unix called Xenix.
It's interesting to think what world we would be living in now if, instead of its own MS-DOS operating system, Microsoft had promoted Xenix and if dealing with AT&T had been easier. According to The Unix Heritage Society community, Xenix was later bought by Santa Cruz Operation (SCO), and from the mid to late 1980s, it was the most common variant of Unix in terms of the number of machines it was installed on.
The essence of shared computing resources is not about entering programs by pressing keys instead of using punch cards, but about stimulating close communication.