At some point, a single machine runs out of memory, storage or processing power, no matter how good its hardware is. When that happens, the only way forward is to split the problem across multiple machines and have them work on pieces of it at the same time, coordinating with each other over a network. This is the basic idea behind distributed computing, and it's how modern services handle data and workloads that are far too large for any single computer to process alone.
No. Some parts of almost every task have to happen in a fixed order, like combining results at the end or waiting for one step to finish before the next can start, and that portion can't be split across machines. This limitation is described by Amdahl's Law, which shows that the speedup from adding more processors is capped by whatever fraction of the work must stay sequential. Coordinating 1,000 machines also adds its own overhead since they need to communicate, wait on each other and handle network delays.
Yes, but it requires special algorithms designed for exactly that problem. This is known as the Byzantine Generals Problem: if some machines in a network fail, send conflicting information or act maliciously, how can the honest majority still reach one shared, correct decision? Byzantine fault-tolerant algorithms solve this by having nodes cross-check messages and vote, typically able to reach correct agreement as long as fewer than a third of the machines are faulty or dishonest. This is the same principle blockchains rely on to keep a consistent ledger across thousands of untrusted nodes.
Almost none of the heavy computation happens on my own phone or laptop. My device mostly just displays the interface and sends requests over the internet to massive data centers, sometimes called "the cloud", where thousands of servers actually run the search, generate the response, store the photos or process the game logic. My device is essentially a window into a distributed system that spans machines all over the world.
We could build systems that model and predict extremely complex phenomena in close to real time, like things that currently take supercomputers weeks and still only approximate. It could also mean truly reliable global infrastructure, such as financial or medical record systems that never go down and never disagree about the true state of the data, even if individual machines anywhere in the network fail or get compromised.