What happens when the problem you want to solve becomes too big for any one computer? you hit the physical limit, whether it is RAM, CPU, or disk. you can upgrade to a bigger machine, but you will eventually run into the same problem. the only way forward is to split the work across more machines. this is the primiary idea of distributed systems, which you can learn more about thru MIT's course here

Suppose you have 1k computers working together, do you now have one computer that is 1000x more powerful? No, network communication between the computers would hinder performance, and syncing between the machines would create a slowing overhead. so you do get a performance boost, but not a linear one. this is discussed more deeply in this short book

can 1k computers agree on smth if some of them fail or even lie yes, some algorthims allow that, and they are called consensus algorithms. working nodes agree on a value even if some crash or lie. check this interactive visualization for more info

when you use chatgpt, where is the computation actually happening? in data centers, not your phone or device

if 1k computers mean 1000x more power, what could be donething like real-time global scale simulations. or training ai models far larger than today's. the dream is infinite computing power with zero single point of failure.