Computers From Hardware to Software

Hello, my name is Brian. I'm a systems engineer at Cloudflare and I work on the content delivery team. Hi, I'm Zaidoon and I also work on the content delivery team here at Cloudflare as a systems engineer. In this talk, Zaidoon and I will be walking through how computers are built and what software is. We'll start with the physics of transistors and end up with the motivation for operating systems. Zaidoon will be asking the questions and I'll be answering them. Okay, let's start, shall we? How about we start at the basics and talk about what a transistor is? Right, okay. Transistors are little blocks of material that have a physical property that applying a voltage to it can alter its electrical properties. Unlike a resistor which unconditionally resists current or a conductor which unconditionally allows current to pass through it, a transistor can be transitioned into a state of high resistance or low resistance by applying electricity to it at a special location. I see, but what is a special location? Geometrically, the source and destination terminals are usually on opposite ends and they're separated from each other by just the body of the transistor. The switching signal is applied laterally directly into the body of the transistor and that's what alters the electrical properties of the body of the transistor. A signal has to be constantly applied, usually, and then that either increases the resistance of the circuit across the input and output terminals or it decreases it. Okay, so a transistor is basically just an electric switch? Yeah, pretty much, and depending on the construction and the materials, geometry, and electricity applied, it can just act as an amplifier too. Got it, but I read that CPUs have billions of transistors, so what are they all doing? See, transistors are arranged into little groups where the inputs of some transistors are connected to the outputs of some of the others, and these little groups simulate a table lookup. So, the tables are pretty simple. For instance, we could say that two high signals should produce a high signal and any other two signals should just produce a low signal, and then we have like a table with four rows. So, four rows in this logical table, one for each simulated input and output combination, right? Yeah, that's right, and that particular table would be the table for AND, right? The signal is on if both first input signal and second input signal are on. You could simulate other tables, like OR, or even negation, which would just be a two -row table where only one output and one input, so that they're opposites of each other. You could even simulate things like exclusive OR, which is only on if either one or the other input is on, but not both and not neither. I see. So, processors achieve everything they do by implementing in logic, but how does the addition work, for example? Well, addition and multiplication and division, they're all implemented in logic, and processors operate, the process to operate on binary numbers. So, those are just numbers that have digits that are either 0 or 1, which you could encode that as a low or high signal. So, you can think of it like on and off, and you can feed those inputs into a few tables to achieve addition. For instance, the addition table for binary numbers is just the tables for exclusive OR and AND. Okay, but how does that actually work? Okay. So, of the two output columns in the addition table, the low bit of the result of the addition is 1 if we added a 1 to a 0 or a 0 to a 1 in that column, and it's a 0 otherwise, right? 0 plus 0 is 0, 0 plus 1 is 1, 1 plus 0 is 1, and 1 plus 1 would have 0 in that column with a carry. The high bit is only ever going to be a 1 if we had to carry from the previous column. So, it's a 1 if both of the inputs are 1. Otherwise, it's a 0. So, the low column is exclusive OR, and the high column is just AND. And so, with basic logic, we can add two 1-bit binary numbers to produce a 2 -bit result. I see. So, and I imagine that adding more bits just means changing the logic somehow, correct? Yeah, you got it. A circuit that does just what I described is called a half adder. Because it doesn't allow us to actually carry a bit into it externally. So, it can only really be used for, like, the head of that chain. The middle links of the chain need to be able to feed the carry bit from the previous link into its result, and those are called pull adders. And with that, we can add numbers of arbitrary numbers of bits, although computers typically only operate on multiples of 8-bits called bytes. I see. But what's so special about a byte? The adding circuit has to be connected to the rest of the facilities of the processor in order to be useful. So, how do you input values into the circuit? How do you extract values from the circuit, you know? If you have many different functions, different kinds of arithmetic and logic functions, for instance, you would need to send a signal to choose between them when you apply the signal to communicate the inputs. And you need all that to be, it can't be dynamic because it's baked into the chip. So, it needs to be sort of static. So, you can't have variable length bits, you know, for every different kind of length that you might require, like 5-bits for here, 9 -bits for there. It's a waste of space. So, you need to pick something, or at least you need to pick to have a few multiples of something that reduces the amount of circuitry because it already takes so many transistors to even build simple logic, let alone something like division. So, that's pretty much why byte is significant. It's a good convention for reducing the amount of circuitry that chips have to have. Got it. And then, how do we end up getting the actual results? So, after that, you'd expect to receive the output of the operation. But because it's likely that you'll feed the output of one operation into the inputs of another, it makes sense to store it in the processor, in like a named location, so you can refer to that rather than having to receive the full value from the processor, just so you can send it right back. Those named places are called registers. And to simplify the circuitry, they all operate on the same number of bits. Different processors could handle different numbers of bits, but most processors nowadays are operating on 64 -bits. Oh, so you can actually build memory out of transistors? Yeah, you can. If you feed the output of some of the transistors into the control signal of others, you can create like this, basically what's like an electric latch. You kind of loop them up together, and then you can make memory circuits that way. So, the output will always return the last signal received on the input. I see. So, processors only know how to do arithmetic on 64-bit binary numbers then? Well, it's possible to do arithmetic on fewer bits by ignoring part of the result, which can be achieved by zeroing it out. We could zero out the top half of the result, except possibly the last bit of the top half, which is a carry bit for the bottom half, just by applying other operators like AND. The trick would be to do a bitwise AND on all the digits with a 64-bit binary value that has the first 31 bits set to zero and the rest to one. So, doing that sort of forces everything else that we don't care about to be zeroed out. Although, because operations like those are common, they're baked into their own instruction. There's also a wide variety of different processor architectures. Some support 32 -bit, some support 16-bit, or even 8-bit as a standard register size. And, you know, depending on how powerful the processor that you're trying to program is, you might run into that. Modern desktops usually support 64-bits and have really very intricate, involved, comprehensive instruction sets that do lots of stuff. But ARM is based on a reduced instruction set. It's far smaller and simpler, but I'd say that the capabilities of each instruction are kind of, they compose better with each other. So, you can get a lot out of the little bit that they provide. So, in some sense, it's more efficient use of circuitry on the processor, which means less power. There's like a lot of different kind of development. The number, 64-bit, 32-bit, it's, it was a historical choice. There's nothing super secret or magical about it. I see. So, we basically have circuitry to do the addition, but how would we go about to do a subtraction? Subtraction is actually just implemented by overloading addition. So, it turns out you don't need dedicated circuitry for subtraction. If you just agree to represent negative numbers as one plus the negation of the corresponding positive number, so that representation is called two's complement. For instance, 0, 1, 0 in binary is two. Its negation is 1, 0, 1. And add one is 1, 1, 0. All right. We can arbitrarily choose to treat that as the number negative two. So, 0, 1, 0 is two, and 1, 1, 0 is negative two. We would want to do that because it lets us reuse the addition circuitry. We don't have to actually do anything. Just choosing that representation means that when we add two negative numbers or add a negative number to a positive number without any change to the result, and if we apply the same interpretation, it's meaningful. It makes sense. So, for instance, negative two plus one is negative one. And 1, 1, 0, which is negative two, plus 0, 0, 1, which is one, is 1, 1, 1. And that should represent negative one. I see. But I thought 1, 1, 1, 1 would represent eight in binary. So, eight would be the largest possible positive number representable in three bits. If we didn't use half of the available representations to represent negative values, we are doing that trick of adding one to the negation. Half of our representable numbers are negative. So, one of our representations is zero, and that means just the remaining are positive. Right? So, what makes them negative and positive is just that the output after using the standard addition circuitry. For instance, negative one plus one is zero, and if we add 1, 1, 1, which I said from before is negative one, to 0, 0, 1, then we get 0, 0, 0, plus a carry bit, but we can ignore that. So, the real magic in it is that the encoding is consistent with arithmetic. It's consistent with arithmetic on negative and positive numbers without actually changing any circuitry. I see. So, this is just a hack that lets us reuse the addition circuitry to add values it wasn't really designed for. Yeah, that's pretty much it. Okay, what if I wanted to add numbers that require maybe more than 64 bits? Because earlier we talked about 32-bit addition and 64-bit addition. You could add them in pieces by setting up the right sequence of operations on the registers and making sure to inspect the carry bits and carry it along yourself. What if the numbers has more bits than all the bits that the registers can have? Um, well, we need some way to load the numbers into the processor in pieces. We need to store the first 64 bits of each number in a couple registers, perform an operation, like addition in this case, and send its result to a third register, and then load the next 64 bits of each number into the same couple registers. Do the operation again, but this time respect the possibility of carry as an input from the third register. Okay, that sounds familiar to the half adder and the full adder concept, right? Yeah, it's exactly the same. We're just implementing the logic ourselves by sequencing the operations the right way. In general, actually, that's going to be how anything that runs on the processor runs. The processor is very dumb, but very, very fast. So, anything complex has to be broken up into little pieces, little simple pieces that the processor can understand, and the only intelligence it has is that it's able to run billions of them every second. Got it, but how do you get values into the processor? You said we load them. Yeah, that's right. Remember, we drive addition by sending a signal to control which input and output registers we want it to operate on, and a signal for selecting the actual operation of addition. Load would just be a signal to represent load, and then there'd be a signal that goes along with that that represents the output register, and there is a signal to represent the actual bits of the input value that we want to load into it. But how do we know what signals the processor understands if there's so many signals in different processors? Yeah, it's real hard. The convention that the processor understands for the signals will be baked into its design and published in a very large book or multiple books. I think the IA32 architecture and the books that AMD publishes, they are huge tomes with thousands of pages and just reams and reams of stuff. I mean, they nowadays have thousands of different operations. You could imagine if you had very few operations and registers, you might be able to express all possible arithmetic operations in the bits of a single byte. But generally, processors nowadays, especially for things like desktop and laptop and like your iPhone and stuff, they have many, many registers, many, many operations, and so the opcodes are a lot longer than one byte. Just for example, the load operation I described needs to be long enough to encapsulate the value that it's loading. I see. So, we have load, but what about store? And how do we get those values out of the processor exactly? So, what does it mean for the processor to store something? So far as the processor is concerned, it only has static dedicated wires leading out. Its whole world is static. There's no, can't add more wires and take away more wires while it's executing or anything. So, it can only send a value along some of the wires and possibly something like an address along some of the other wires, right? Think about it. There's got to be a device somewhere that understands what these addresses mean, because the intent here is to only store the values and not operate on them. That device only needs the registry circuitry, right? It doesn't need to have any kind of sophisticated operations. It just needs to have the same latching circuitry we have for registers. And yeah. So, that's important because it means there can be a very intense reduction in the amount of circuitry. I see. But why is this reduction in circuitry important? What do we gain out of it? The reduction in circuitry means we can have more registers and a smaller physical package. So, this memory device can have many, many of them. Although generally we call them memory locations rather than registers, they're the same concept. It uses transistors as a latch to keep this state of some number of bits in the same state that they were when we last applied the current or signal to them. If you turn off the power, they'll go away. Just like how we use a few bits in the opcode to tell the processor which register to serve as the output, the processor uses some number of bits to tell the memory device which memory location should serve as the output. Although usually the number of bits used is quite a bit higher because there's a far greater number of memory locations than there are registers. And generally the memory locations have room for a single byte. So, the smallest addressable unit of storage is one byte. I see. So, we have to store the bytes of the result in our big addition one byte at a time then? Yeah. Yeah, pretty much. Conceptually at least. There's usually a pretty wide memory bus between the processor and the memory. Because it would be really inefficient to have to store, to like send off a bunch of different bits that represent the address and then only like eight bits to represent the value, then most of your communication is just going to be to communicate address bits and that's not a great or terrific thing to have. It's too much overhead. But generally speaking, you can think of it like that. You can store as little as one byte at a time if you'd like. For most processes, generally though, that sort of illusion is, it's just that, it's just an illusion. There's this thing called the word size. So, I said before 64-bit. Most registers are 64-bit. For a processor that has a standard 64-bit addition and multiplication and such, you could say that it's a word size of 64-bit. So, that's the word that it speaks. It knows, it knows, it's basically, its vocabulary is that wide. So, when it talks to the memory device, it's going to be talking 64 bits at a time in terms of the values that it stores. So, there's going to be some number of bits. Modern computers use 48 for the addressing and then 64 for the value. And so, although theoretically, you can address something as small as eight bits, which would be one byte. Realistically, what actually happens is you're accessing something that's 64 bits wide, storing that. I see. Wait, but there are so many opcodes involved in loading and storing these bytes. Do the signals we send to the processor come from memory as well? Yeah, there's a special register called the instruction counter in the CPU that records the address of the next byte to be loaded into the CPU and interpreted as an opcode or a piece of an opcode. After doing the load, the counter is incremented. And so, the processor constantly loads a stream of bytes from memory and interprets them as instructions. The counter itself is not usually implemented in logic. It's typically just driven from something called a clock, which is an oscillating signal for each bit. Maybe your clock has 64 bits. You have 64 different oscillating signals. And the frequency of each one of those lines is configured in such a way that everything just lines up. And it makes it look like it's a counter that keeps incrementing. The reason to do it that way is because fundamentally, you only need something that keeps incrementing. You need this clock that keeps counting up and then rolls over. But you don't need all the circuitry for doing an incrementation or an addition to do that. You just need some oscillating signals to build something like that. Got it. So then, could we use a load opcode to load a value into the instruction counter register, for example? Yeah. You could load a value into the instruction counter register. You could actually load one that's the result of a series of logic operations and gives you the memory location of some other sequence of opcodes so that you can order in order to execute those opcodes based on the result of some sort of other operation. Right? And based on the specific logic operations that you were using, you could yield different locations of different sequences of opcodes based on some inputs. And that's actually a really, really common pattern and a really common thing to do. So there's dedicated opcodes for that. And they're called conditional jumps. And that allows you to execute the opcodes stored in one memory location if the result of executing some other opcodes indicated that the number loaded into one register was bigger than the number loaded into another. And then you could execute the opcodes stored in a different memory location if that isn't the case. So basically, that gives you this sort of conditional jumping is what gives you the ability to tell a computer to do some stuff based on a result. Look at the value. If the value is greater than this, do path A. Otherwise, do path B. Got it. So we have something called conditional jumps, which, again, given a condition, does something. But what about when we wanted to jump anyways? Do we have something like an unconditional jump? Yeah. You can use that to build an infinite loop. You'd obviously want to have some kind of conditional jump within the loop so that you can exit the loop when a certain condition is met. Otherwise, you just loop forever. Got it. So then what happens when a processor loops forever like that? Is there like an instruction counter that gets reset to zero? Or do you reset the computer when you reset the computer? Or what exactly happens? Generally, yes. So it gets reset to a predefined value and executes the opcode stored in that memory location when the computer is rebooted. There are other ways of grabbing the processor's attention, though. For instance, a separate dedicated wire can be used to signal to the processor that some specific peripheral needs its attention. Peripheral could be like mice, keyboard, network, card. Like a disk, something like that. Monitor. So main memory is usually optimized to be fairly quick and dense, but it's not persistent. As I said before, registers can't store values unless power is applied. So when the power is turned off, the data is lost. So different kind of memory is required for persistence. And that is generally called storage. And disk drives and flash drives are storage, for instance. I see. So does the processor have to be directly involved with the store and read operations of each of those storage devices? It would be a waste of time to involve the processor with the operation of loading and storing data from those storage devices into memory. So they're generally allowed to do that on their own when given the instruction to do so. But they need a way to notify the processor when they're finished. And that's achieved using a special dedicated signal. It's like literally a dedicated pin on the processor. If you were to turn the processor over and look at the pins on the back, there's going to be a pin dedicated for interrupts from the mouse or an interrupt from the keyboard or an interrupt from disk. Those are called hardware interrupts. You can share an interrupt line, but things get a little bit tricky if you do. So usually they're separate. So the processor is able to do things even when the peripherals do work, too. What if they both try to write to the same location at the same time, for example? Okay. So like if the disk is trying to load something into one memory location that a network card is trying to load memory into at the same time, right? Like you're trying to write to the same effective location. What would happen would be up to the memory device. But generally, there's no guarantee engineered into the memory device that ensures something specific happens. The situation needs to actually just be avoided. So typically we only allow peripherals to have direct memory access and only a specific range of addresses that have been assigned to it. And it constantly notifies the processor when it has no more room to write. So it fills up its buffer, the amount of memory it has available to write, and its range of addresses. And then it uses the dedicated wire that it has directly to the processor to tap it on the shoulder and say, I need your help because I have no more room. The processor receives the interrupt signal, then it decides what to do based on the exact interrupt. Remember, it has one interrupt line per thing, generally, per thing that it's interested in allowing to interrupt it, right? In the case mentioned earlier, it would be the dedicated interrupt for disk writer. By design, the processor will execute some code at a specific memory location that corresponds to the specific interrupt. And that would be called a hardware interrupt handler. But how are these interrupts configured exactly? In order to keep the circuitry and the processor simple, usually the memory locations for all these interrupts are all very, very close to each other and arranged as a table with a constant row size. And that's called interrupt vector table. The values that can be stored in these rows are usually quite small, but they're big enough to store the opcode for a jump. And that usually what you do is you have that jump jump into a sequence of opcodes towards somewhere else. And that sequence effectively handles the interrupt. And it's called the interrupt service routine. So, for instance, if disk is saying I need help because I've got no more room to write into my buffer, please help me. It interrupts the processor, regardless of what it's doing. It jumps into a hardware interrupt vector table, finds another jump, executes that instruction, goes into an interrupt handler that way. And then typically the bytes are just going to be copied out of the range of bytes that are allocated specifically to that storage device. That way it can write over the bytes that it was using in the buffer and you don't lose any data. Whatever you need to do with the bytes you copied out, you can do whatever you need with. I see. And does the storage device also write one byte at a time to memory? It depends on the peripheral and the type of storage and computer architecture. But most modern storage devices write chunks of bytes called pages. Yeah. A page could be like 4,096 bytes, for instance. It's a bit tricky because what they actually might be doing underneath is probably it's definitely going to be writing things at whatever width the memory device supports, right? So, if the memory device supports things that are if it supports like a 48-bit address and 64-bit data width and expects you to write things by driving the signals on those wires, that's what's actually going to be writing at a time. But in terms of what let's say is logically addressable, storage devices don't address one byte. They address something much larger. And usually that's called a page. They address pages. Got it. So, let's say the processor is doing something and then it gets interrupted by, you know, an event. If it handles that event, then what happens afterwards? Yeah, it goes back to what it was doing after being interrupted. So, obviously, there's like limits to what you can do in the interrupt handler, right? So, if basically the code that called you, I'm sorry, maybe more clear, the code that was running at the time when the interrupt was fired, what happens to the registers that it was using, right? Because it could have values stored in different registers and it might have been using intermediate results of those. So, the interrupt handler has to make sure not to mess up the computer for all those other things that might be running. For all the other opcode that's working. So, there's limits to what can be done inside of an interrupt handler. And hardware isn't the only thing that can interrupt a processor. Software can interrupt it, too. And when that occurs, the processor is engineered to temporarily save the instruction counter to a known place and jump to a predefined memory location to handle that exact software interrupt. And that can be done intentionally using a special instruction, which is designed to execute software interrupt handlers. Or it can be done unintentionally. For instance, most modern processors support arithmetic like division. And division by zero isn't defined. You can't go forward with that, right? Because you have to store the result of that somewhere. And if you're storing the result, it has to be encodable, right? And we don't have a way to encode the result of dividing by zero. So, when it occurs, the processor executes a specific interrupt handler. It knows that that's happened because it can see that it's had to divide by zero. So, it stops what it's doing and it goes to the interrupt handler that's been preconfigured and the code has been prewritten and loaded into memory. It's already there. And it starts executing the interrupt handler. And that tells the processor what it needs to do next to sort of rescue itself from that situation. Software interrupts caused by errors like that are called faults. So, if let's say we're running a software interrupt right now, can that software interrupt have another software interrupt within it? The goal of the software interrupt handler should be to directly handle the interrupt immediately without deferring to other routines. So, if the situation you just described does happen, that's called a double fault. It could just be that there's a fault or software interrupt that occurs while handling another fault or software interrupt. The double fault interrupt handler is sort of like a dedicated handler for that situation. And its only goal is to rescue the system from that situation. So, if a fault or interrupt occurs while doing that, then you have a triple fault. And generally what most processors do is simply reset when that occurs. Yeah. So, if we can't invoke software interrupts from within software interrupts, then how do we break up code? It could be broken down into sequences of opcodes and managed by using jobs. But then we won't have to make sure that the registers are written to. And being used by one chunk of opcode isn't being written by another. And how do we manage that complexity exactly? Right. So, one way is to store the content of the registers in a known place before modifying them. And to restore them after you're finished. And although there are different ways to achieve that, a specific pattern called the procedure pattern is pretty common. So, a procedure is a series of opcodes that expects a specific register to contain the address that it should jump back to when it's finished. To perform a jump to a procedure is called calling procedure. So, before calling procedure, we set an address in a specific register by convention. And it's going to be used by the series of opcodes that we're calling, the series of opcodes we're calling a procedure. It's going to use that to jump back after finishing its operations. But where do we jump back to exactly? So, the jump is almost always back to the code that called the procedure. To make sure that the to make sure that the procedure can actually use all the registers without overwriting registers that were in use by the calling code, it stores the register values. And in a known place before it starts executing its own operations. Then it loads the register values from that place back into the registers before it jumps back. So, it tries to do this like magic trick where it makes the caller think that it didn't even use the registers. Although it's often useful for a procedure to be able to communicate a value to calling code. So, maybe it doesn't restore all the registers. Maybe one of them is dedicated for the procedure to talk to send a message back to the thing that called it. And that's usually just also there's a convention for which register that is. Right. So, we keep talking about this known place, but where is this known place exactly? So, strictly speaking, there's no reason that these known places have to actually be next to each other. But because this pattern is so common and such a useful method for breaking up complexity, there are dedicated opcodes to simultaneously write register values and perform the jump or read register values and perform the jump back. Because a procedure can't be interrupted by the code that actually called it. And because the register stores always happen before the register loads, we're just conceptually pushing register values and popping register values from like a logical stack. And so, the location that we are using to store these register values is called the stack. And it grows in memory like a stack. Depending on how you want to orient it, it can grow up or down. And so, there's like this stack counter that I'm sorry, like a stack pointer that points at the current head of the stack. And there's some record there for how wide the stack is. And like the actual register values that get pushed on, along with any other values that might get pushed on, those are called stack frames. So, frames are pushed on the stack one frame at a time and popped off the stack one frame at a time. It's also sometimes useful for the procedure to take some values as input from the calling code. And those are often passed in registers that aren't saved into the stack frame. So, they're just available. They'll disappear once the register values are restored. So, only the procedure can actually see them. And those inputs are called parameters. I see. So, but what if we need to pass data from a caller to a procedure? And let's say this data can't fit in those registers. How do we handle that? Well, we can pass the address of the data as a parameter. So, most processes support loading data from a memory address. Rather than having to encapsulate the data to be loaded within an opcode. Okay. And then how do we store that data in the first place, though? So, let's say if each procedure does what it wants without knowing what the other procedure is doing, isn't there a possibility they overwrite each other's data? Um, so, coordination can be organized beforehand, like statically, by baking it into the procedures themselves. That would be great. If not, then they have to defer to some kind of an authority that manages memory. And those are called memory allocators. The reason why they're called that is because the region of memory that memory allocators can reserve is called the heap. When a procedure needs some additional memory, it just it asks the memory allocator for the memory. And the memory allocator ensures that we don't have those situations that you've described. It asks, it basically provides, it's just a procedure. It provided the amount of memory as a parameter, and it gives you back the address of the return. It gives you back in the return value the address of the memory that you asked for. But most memory allocators also provide a deallocation procedure. And it's the engineer's responsibility, the person who's writing the code, must remember to write the code to eventually call the deallocation procedure to deallocate any memory that was allocated after it's no longer needed. And it can be challenging to know when something is no longer needed based on the code. And a lot of bugs happen just because of these weird memory issues. And it can be very hard to track down. Right. So what if two procedures need to use the same peripheral, like writing to a disk, for example? Another authority must be responsible for managing access to disks, and in general, any shared resource. One way to reduce contention is to allow readers of a page of storage to not block each other. To do that, the readers need to express their intent to access a page of data in a kind of like read-only mode. And then the authority which manages access to disks can show whether the page has already been loaded into memory and loaded if necessary. Okay. And how does this ensure readers don't interfere with each other at all? The readers can all be given the address of the page without danger because they've announced their intent to only read the data. So there's like some amount of trust there, just so far based on what we've been talking about. You would probably want some circuitry there in the CPU to ensure that a store operation or modification to the memory isn't actually tried. Nobody tries to perform that. And if they do, then you probably want to call some sort of interrupt handler, right? You would interrupt the CPU with a software interrupt. But it's a pretty common pattern to do that. So many computer architectures support the concept of virtual memory, which is a mapping that makes it possible to treat main memory as being like quite a bit larger than it actually is. That actually sounds like an amazing concept, but how does virtual memory actually work? So there are virtual addresses that correspond to addresses used in memory like RAM. And then there are virtual addresses that correspond to addresses used in storage like disk. And these various address types are called physical addresses. And there must be a mapping from the virtual addresses to the physical addresses. And when an access is made to a virtual address that corresponds to a page of storage that has not been loaded, then a page fault occurs. I see. So there's a fault. That means we have a page fault handler. Yeah, that's right. The interrupt handler for a page fault loads the page into main memory and records the mapping between the virtual address and the physical address in a data structure called a page table. And to keep things really fast, most processes are engineered to contain an associative cache called the translation lookaside buffer that stores the most recently used mappings from the page table. If a memory access occurs and could not be found in the translation lookaside buffer, then the page table is referenced to see whether a mapping exists. So if no mapping exists and the page loaded into memory? If no mapping exists because the page is not in memory, it is loaded into memory, even if that means making room by evicting another page in my memory. So the eviction necessitates writing the evicted page back to storage. And if no mapping exists because the address is invalid, a segmentation fault will occur and the fault will be handled by the corresponding interrupt handler. Got it. Is there any way to reduce contention besides this? Apart from this, there are ways to diminish contention by sharding the logic access of a disk into separate units, which are just called files. So that the procedures, they can just contend over files rather than the entire disk. Different kinds of storage have different access performance characteristics. For instance, platter -based disk drives, although they're becoming much, much less common, they have a seek time, which effectively means that contention may still occur under a heavy load unless the disk management authority organizes accesses in such a way that optimizes for those particular characteristics. Even SSDs have their own quirks and characteristics that need to be optimized for. I see. So what if I want to execute two procedures at the same time? Can I give one more time than the other, for example? Unless there are multiple processor cores that can execute procedures simultaneously, only an illusion of two procedures executing at the same time can be created. So how do you create an illusion of that exactly? To create that illusion, a little process of time needs to be given to each procedure, and you just keep rapidly doing that back and forth. And a special software interrupt is configured to regularly interrupt the processor so that an interrupt handler can select which procedure gets time. And when switching context from one procedure to another, it's necessary to store the contacts so they can be resumed when necessary. And what goes into that context? Actually, the context generally contains not only the working state of the procedure, but a reference to the collection of procedures, and we just call it a program. So one of the procedures serves as the entry point, which is the first thing that is called when the program is executed. The procedures within a program can all call each other, and to call procedures in other programs the addresses of the procedures must be included in the program at the time the program is compiled, and that's called static linking. Or at the time that the program is executed, that's called dynamic linking. Got it. And how does priority work with the procedures in the program? So when a program is being executed, it's called a process, and the process is associated with a priority, not individual procedures. So by storing a record of the priority for each process, the interrupt handler is responsible for giving processes little slices of time that it can make sure to respect the desired priority. If a process becomes blocked on I-O and cannot make progress, it is put into a state that causes the time slice giving interrupt handler to not give it any time slices until it is in a non-block state. So the interrupt handler for I-O, for input output, may mark the process as non-blocked once the data it requires is available to be read. Got it. But can programs do more than one thing at a time on a multi-core processor, for example? In this case, each process owns additional state for objects called threads, and two procedures executing under the context of two different threads may execute in parallel. And each thread has its own stack. So the threads of each process are given slices of time of the physical core of the processor while respecting the configured priority of the process. If support exists in the interrupt handlers, then an individual thread may enter into a blocked state without blocking the entire process. Got it. Okay, that makes sense. So then our process is just used to manage how much time is given to each executing program? Processes are also used to separate each application so that they don't interfere with each other. The process tries to read a virtual memory address that does not exist, then the process can be aborted without affecting the execution of other processes, for instance. If we know that a process enters an infinite loop and there's no escape from that, then the execution of the process can just be stopped. And you can do that without stopping or affecting any other process. The programs that each process are executing might not be completely trusted either. So some state can be tracked and find what resources a process is capable of accessing or utilizing. Okay, doesn't that mean we have to trust some kind of code to ensure that the processes aren't accessing resources they shouldn't be accessing and to make sure that they are responsive and operate as intended? Yeah, and that's the goal of an operating system. The operating system schedules and manages processes, balancing their needs and ensuring that their collected dependencies are satisfied. If you do any kind of dynamic loading, making sure everything is that needs to be in memory for all the different things that might be requiring it are in memory. Even while it's swapping pages in and out of main memory, it provides a consistent interface to various devices and peripherals so that programs don't have to have like super specialized code for specific storage devices or mice or keyboards. Operating system does all the initial configuration that the processor expects, like setting up page tables, contains interrupt handlers like handling for handling IO and key presses and stuff, and it serves as a set of trusted code that every process might find useful and can rely on. So having an operating system that makes it so that we can run multiple programs reliably. Having an operating system also effectively lowers the difficulty of programming the computer because of the uniform set of utilities it provides to engineers and makes it easier for programs to be shared because the uniform environment it provides to programs. A lot of the most difficult to program utilities have already been written and exist within the operating system. So it means that there's more code reuse and smaller less difficult to write programs overall. If there's a vulnerability in the operating system, it can just be patched once rather than having to do a security audit on many different programs. So then the value of an operating system is that it reduces the cost of developing and operating a program, especially when there are more programs than computers. Yeah, effectively it makes the computer hardware fungible. So we can replace the hardware at will without modifying the programs that we've written. So that increases the longevity and reach of all the programs that target that operating system. It liberates the developers of software from the hardware manufacturers are liberated from the specific requirements and quirks of individual programs or software packages and can focus on working closely with operating systems developers to make improvements or changes to new generations of hardware. So it's a win-win situation. I see. So then would there always be utility in making programs and applications easier and safer to write and use, regardless of how you make that happen? Right. So yeah, that's the key bit. That's like the crux of it. And that's exactly what Cloudflare is set up to do, to make the Internet and applications that people have written that use it safer, faster, more reliable. In some sense, Cloudflare is becoming like an operating system for the Internet. Delivering content is fundamental, like serving up pages from disk. But it's only the tip of the iceberg. It's just the beginning of what's possible. I mean, a lot of what remains to be done to bring it to parity with what a real operating system would do. For instance, to make Internet applications much safer, balance their consumption of resources, like network resources, based on a priority. I mean, that's not been done. Provide visibility and instrumentation to make operating them easier. We're in the process of doing that. And to ensure computers and engineering effort are used efficiently. I mean, over time, the win-win scenario created by Cloudflare will make the Internet more valuable to everyone. Well, this has been great. Thank you, Brian, for this. Thank you, everyone, for watching.