When One Computer Is Not Enough The Fascinating World of Distributed Systems

1. What happens when the problem you want to solve becomes too big for any one computer?

We can divide the tasks for solving our problem into multiple computers. A problem may be “too big” for several reasons: the storage of a single computer may not be sufficient, the calculation may take too long for a single computer, or there may be too many incoming requests that should be dealt with simultaneously.

MongoDB, Inc. (n.d.). Sharding. https://www.mongodb.com/docs/manual/sharding/

2. Suppose 1,000 computers work together. Do you now have one computer that is 1,000 times more powerful? Why or why not?

Not exactly. For example, a task usually will not be done 1000 times faster. Because some steps may only be done after some previous steps have been completed, and these may not be done concurrently. According to Amdahl’s Law, there is a portion of the execution that is serial (i.e. cannot be parallelized); therefore, no matter how many computers are used, the execution will take at least that much time. Also, transferring results across computers takes additional time, which also limits the speeding factor.

Cornell University Center for Advanced Computing. (n.d.). Amdahl’s law. Cornell Virtual Workshop. https://cvw.cac.cornell.edu/parallel/efficiency/amdahls-law

3. Can 1,000 computers agree on something if some of them fail or even lie?

Yes, they can. Distributed systems use a consensus protocol that may allow it to operate if the majority of the computers continue to work, as in the case of Raft. Also, systems can still reach a conclusion amidst mis/disinformation, and such systems are called Byzantine Fault Tolerant. As I understood, to overcome the Byzantine problem, honest computers agree on what the majority suggests, and exclude the proposal of the minority. This can only be achieved if fewer than one thirds of the computers are providing mis/disinformation, however.

Kleppmann, M. (2021). Distributed systems [Lecture notes]. University of Cambridge. https://www.cl.cam.ac.uk/teaching/2122/ConcDisSys/dist-sys-notes.pdf

HashiCorp. (n.d.). Consensus protocol. https://developer.hashicorp.com/nomad/docs/architecture/cluster/consensus

4. When you use ChatGPT, Google, Instagram, or an online game, where is the computation actually happening?

Some of the computation happens in my own device, and some of it happens in remote computers, in platforms’ servers. If we share a content, also the devices of receivers do computing as well.

5. If you could make millions of computers behave like one dependable machine, what could humanity build that we cannot build today?

We could store and evaluate much more information, and this would allow us to model Earth in much more detail and preciseness. This greater understanding of reality may help us better utilize physical phenomena, for example, in designing transportation vehicles. It would also allow us to conduct a greater amount of tests, which may include testing computational replications of reality, and this would foster scientific discovery: we could better model biological systems and test them.Additionally, we could make more accurate predictions (about natural disasters, for example) as we have more data.