Weekly Research

Updated September 27 by Akhyar Kamili

Computer Networks

Computer Networks (or just “Networks”) were, like most great technological advances, initially developed for military purposes. Back in late 1950s, the first usage of networks came from the necessity of radar computers to communicate with each other. In 1959, the Soviet Union’s Anatolii Ivanovich Kitov proposed a plan of organizing the country’s military and economy on the basis of connected computers.

Computer Networks was discovered by the public in the 1960s. At the time, it was cutting edge technology for computers to share bits of data over a cable – well, this was the time when computers take space of whole rooms to operate.

In Networks, a device that can terminate (receive), route, and originate data are referred to as a “node”. Therefore, learning Networks is all about learning how to connect these nodes. In the transmission medium level, we now use physical cables and electromagnetic waves to connect nodes. For the sake of reliability and speed, countries connect to each other’s networks by creating physical network cables, laying them on the ground, underground or in the seabed. Wireless connections, on the other hand, are used only for local area networks (LANs) or Wide Area Network (WANs) -- we’ll revisit these terms later – or sometimes satellite communication in places where cable networks are not available.

On top of these physical infrastructures, we use layers of networking software to communicate with each other. Generally, data is sent over the network in packets, which means they are broken down to small pieces, sent one by one through the network, and then routed or reconstructed in the receiving node. The most popular networks around, the Internet, runs on top of these layers. On the very low-level, there is the Network Interface Layer, which manages connection of nodes that are physically connected. On top of that, runs the Internet Layer. This layer agrees on the IP protocol to have data packets and send them accordingly. Then, on top of this layer, comes the transport layer. For the internet, this is served with the TCP protocol. It is at this level that most user applications are programmed in. The transport layer regulates the flow of information, agrees on handling missing packets, connects with addresses, and breaks down data to data packets. Finally, comes the application layer – the one we use for our web consumption. On the final layer, the World Wide Web uses HTTP protocol to agree on how web applications communicate with the clients and servers. However, it is a mistake to confuse the WWW to the internet. The World Wide Web is not the only application in the internet. For example, WhatsApp have their own protocol – they do not use HTTP. Understanding how this works will help us in designing our applications.

Links: https://en.wikipedia.org/wiki/Communications_protocol, https://en.wikipedia.org/wiki/Computer_network, https://en.wikipedia.org/wiki/Link_layer

Questions:
1. How do we decide, in non-strict communications, how much data we are able to afford to lose?
2. Why don't we make the IPv6 as our main protocol yet?
3. Is there any new developments that might make the internet faster?


Embedded Systems

Not all computers are made equal, are they? Different technological needs can be meet by different kind of computers, like today. For example, printers only need to translate documents into printing commands. If we accomplish this using a personal computer with 8GB of RAM and an i7 processor, that would certainly be overkill. Thus, to do simple, less-resource-taking tasks, we don’t have to use big and bulky computers like the one we use for gaming or making presentations. And a lot of times in our daily lives, we found ourselves dealing with Embedded Systems.

Embedded Systems are computer systems which operate within another system, typically mechanical or electrical ones. Their main function is not to operate as a standalone computer, hence the name. As such, this term describes a lot of devices. Since the recognition of it in the Apollo guidance computer in 1966, the use of these systems have increased exponentially. According to Barr (2009), up to 98% of microprocessors are manufactured for embedded systems. To simplify, they are the heart of most of automated services we take for granted today. The whole manufacturing industry, whether in food, textile, cars, and anything that uses large-scale automated production must use embedded systems for their manufacturing machines. These small computers would be receiving commands, moving mechanical or electrical parts, and communicate with one another. MRIs, EDCs and ATMs are also some type of embedded systems. With the recent popularity of Internet of Things, embedded systems might soon be in every single tool we use.

Most of Embedded Systems today are based on microcontrollers, a small computer on a single integrated circuit complete with a processor, memory, and programmable I/O devices. This compactness means they can have a smaller size compared to our general computers. Since embedded systems are expected to do specific tasks, they can be a lot more optimal than using general devices, for example, by adjusting the processing power available to the bare minimum. To make up for the cost in customizing, embedded systems are often produced in a large scale, pushing the price-per-unit down.

For the sake of these optimizations, embedded machines must be programmed in low-level languages, such as C, and the software must be optimized to match the specific hardware. At an even lower level, embedded systems have limited processing power, memory, and storage. They are also often expected to run for years without malfunctioning. As such, creating a proper software with these low-level constraints is very challenging for many programmers.

Benefits do come with a cost, don’t they?

Sources: Barr, M. (2009). "Real men program in C". Embedded Systems Design. TechInsights (United Business Media). p. 2. Retrieved 2016-09-07

Links
1. http://leeseshia.org/
2. a href="https://en.wikipedia.org/wiki/Microcontroller
3. https://en.wikipedia.org/wiki/Embedded_system

Questions:
1. What are some recent improvements on this particular field?
2. Where are Embedded Systems going?
3. What are kernels, exokernels and monolithic kernels?


Cloud Computing

One of the ways we often use to reduce expenses are resource sharing. It’s the reason why we have public transportations, shared rooms, and computer clusters here at Carnegie Mellon. It’s similar to the idea of computer clusters: Instead of having to hand one set of computer (including the software licenses!) to each person, we can instead have a set of public computers to use, and share resources there. And one of the modern application of this principle is Cloud Computing.

Cloud Computing takes resource sharing to another level. Using the “cloud’ and some internet connection, we can remotely share data storage and processing resources, such as applications or even OSes. This also eases collaboration. With the cloud, one can use a computer with weak resources to remotely work on heavy tasks.

But what is the cloud?

Mell and Grance (2011) in The NIST Definition of Cloud Computing defined it as follows:
Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. Cloud Computing basically is a term coined to refer the remote, on-demand use of computing resources. Take the service Canva, for example. Canva provides a light, web-based graphic editing software. When you, as a user, want to draw or edit an image, Canva would send the user the necessary tools, and then renders and saves the image within their computers. Instead of using your computer to do the heavy, mechanical process of image rendering, Canva does it for you.

How does it work?

The general idea is simple: After a connection is established, remote users can send instructions – with a certain protocol – to the servers so that it can use the computing resources available at that server. Now, to approach this, there are many different models of cloud computing.

To scratch the surface, we can mention IaaS (Infrastructure as a Service). IaaS is the most basic application of Cloud Computing. It provides nearly every kind of resource a computer can give to a user, including processing, storage, and networks. This may include a whole operating system, complete with a specified storage and customized applications. Users are not able to customize the underlying cloud infrastructure.

However, aside from that, users can act as if they are in a computer of their own. How? This could be achieved by the help Virtual Machines. Generally, a VM client is installed in the computer(s) of the user(s), and a VM server is installed in the serving computers. To start, the VM client would make a request to the VM server. After authentication, the VM server would grant access to the user(s).

A different kind of model is SaaS (Software as a Service). This is the model that is being used by Canva, on which the cloud provider gives access to applications through a thin (lightweight) client, or another program interface. Emails are also an example of this.

Cloud computing is nowhere near new, the execution was already started almost 50 years ago. It would be slightly incorrect to say that cloud computing is the future, since it is already happening now, and keeps improving better and better.

Questions
1. How does an IaaS cloud server restrict its user?
2. What protocols are used for VM? At which layer is it working on?
3. Can an IaaS cloud server manage multiple users at once? If so, how?

Links
Canva
Cloud Computing (Wikipedia)
SaaS
IaaS


Machine Learning

Machine Learning started as the study of pattern recognition and learning theory. As the name suggests, machine learning is concerned with making machines – especially computers – to learn and improve itself as it gain more and more “experience” from tasks it executed and inputs it processed. This allows computers to learn by themselves instead of creating specific mathematical models and explicitly programming them to do abstract or complicated problems. Machine learning is not new, but it’s currently receiving a lot of public and business attention due to recent breakthroughs in the field. Some of the current successful examples of machine learning are search engines, self-driving cars, as well as speech recognition used by personal assistants such as Siri or Cortana. E-commerce companies such as Amazon has used machine learning to enhance the efficiency of advertisements to users.

Machine learning’s recent breakthrough can be attributed to a few things. Today, more and more data is available for computers to process. Computational processing capabilities has also “reasonably” increased in the last 20 years. Data storage has reached a level where it’s so affordable and powerful that companies can have virtually endless storage. Google’s data center, for example, is estimated to hold over 10 exabytes, or around 10*2^30 Gigabytes. The more data is available, the easier it is for machines to improve themselves.

Just as machine learning is just one way to approach artificial intelligence, there are more than approaches to machine learning. The popular ones include Artificial Neural Networks, Deep Learning, and Feature Learning. Artificial Neural Networks (ANN) imitates animal neurons to learn. It can be characterized by:
1. Architecture, which describes the structure of the network
2. Activity Rule, which describes how neurons should interact with each other in response of an input.
3. Learning Rule

Deep Learning, which is based upon ANN, tries to model how our brain processes signals from senses with multi-layer representations of data. By knowing inputs and expected outputs, it’s able to find a relationship. Deep Learning is greatly used in creating computer vision (including image processing) as well as speech recognition which was mentioned earlier.

Feature Learning, on the other hand, is more than finding relationships between data. It’s a set of techniques to learn in a higher level of complexity: learning how to learn. It helps computers to create low-level features out of high-level data they analyzed.

However, as good as it might sound, Machine Learning does leave some philosophical questions for us to answer. Ethics, for one, is an important issue. A machine that learns from a biased set of data might produce biased decisions or analyses. How should we keep it neutral? Or should we? And interestingly, as we look for the answer, machines are, and will always be, getting “smarter”. Perhaps there will be a day when they will make those ethical judgements by themselves.

Links: https://en.wikipedia.org/wiki/Machine_learning, https://en.wikipedia.org/wiki/Artificial_neural_network, https://en.wikipedia.org/wiki/Deep_learning, https://en.wikipedia.org/wiki/Feature_learning

Questions:
1. Which approach is currently viewed as the best approach to machine learning?
2. What cutting-edge researches are going on in this field, right now?
3. Philosophical question: Is there a limit of how “smart” a machine can be?


Theory of Computation

Theory of Computation is a branch of Computer Science and Mathematics which deals with the modelling of problems into a model of computation using algorithms (step-by-step instructions carried by a computer) – including real-world limitations. It translates problems into mathematical models which simulates real-world computers at a reasonably understandable level. Theory of Computation is composed of crucial math properties of computer software and even hardware, and possess clear connection to engineering. It also helps us abstract out problems and understand computers, as well as stressing out the importance of efficient algorithms. Theory of Computation also provides us with understanding of the capabilities and limitations of a computer, and how to use it to our advantage. Theory of Computation has three main branch: Automata and Formal Language Theory, Computability Theory, and Computational Complexity Theory.

Automata theory is the study of abstract machines and problems that can be solved using it. Using concepts such as automatons - finite-state machines - helps us to solve problems by defining a solution as a mathematical model that is easy to implement to computers.  Formal Language Theory, on the other hand, is the basis of programming languages and syntaxes which allows us to give instructions to computers.

The second one, computability theory, also known as recursion theory, is a very strong mathematical field which studies the limitations of computers. For instance, Computability Theory identifies what kinds of problem are theoretically unsolvable by computers. It also deals with a more rigorous interpretation of algorithms.

Computational Complexity Theory lastly classifies problems according to their difficulty of being solved with computers – also called complexity. It measures complexity of problems based on the amount of resources necessary to solve it – such as storage and time It also measures the practical limits of what computers can do with a problem.

History

Originally, TOC was a part of Mathematics. However, as the complexity and depth of Computer Science increases, it was apparent that TOC deserves to have the attention as a more focused subject. During the last century, its popularity was imminent, and people like John Von Neumann and Alan Turing made significant contributions. The famous Turing Machine was invented by the latter. It is the basic mathematical model which allows today’s programming languages to exist.

Applications

Theory of Computation also provides us with understanding of the capabilities and limitations of a computer, and how to use it to our advantage. It’s important to understand the underlying principles of a computer program, because only then one can create a program easily – the way it’s supposed to be. It’ll also help to identify which problems cannot be solved by computers.

Questions!

Why is Computability Theory important?
Tehnically, how do we identify that a problem is unsolvable by computers?
What exactly is the Turing Machine?

Further Reading

Introduction to The Theory of Computation by Michael Sipser
Automata and Formal Language Theory
Computational Complexity Theory


Randy Pausch Summary

Randy Pausch was a Computer Science professor in Carnegie Mellon University. He was especially known for his lecture that was called "The Last Lecture", after he was diagnosed with terminal diagnosis because of pancreatic cancer.

5 things that I liked in his lecture:
- His funny intro, his attitude.
- Time is more important that money - I grabbed this on the early part of the lecture: you can get money with time, but not otherwise.
- His position in this lecture. He showed himself as a person who is at the end of his life, which makes his talk about time rally powerful.
- When he explains how much time we waste everyday. It struck me as a realization.
- When he emphasizes on reasoning "Why" we do things, and that we should do the right things.

You can read his full story here. He also has a website here.

“Time is all you have and you may find one day that you have less than you think.” - Randy Pausch.