16-711 Kinematics, Dynamics and Controls
Assignment #1
Tom Goff
Overview:
The goal of this assignment was to simulate two different types of robots, a one-legged hopper and a two-legged walker, and to develop control policies that would optimize either the efficiency or the robustness of the robots as they traveled some distance horizontally.
The efficiency of the robots was determined by summing the square of the forces applied at each time-step of the simulation multiplied by the length of each time-step and then dividing the sum by the distance traveled by the robot. The goal distance to travel, D, was 10 times the average length of the robot’s leg(s). There were constraints on the initial conditions that could be set and on modifications that could be made to the robots. This part of the assignment was a search to minimize the formula E = (F^2) / D, or to maximize E’= D / (F^2), for a simulation that successfully brought the robot to the goal distance without falling over. As such, this portion of the assignment was well suited for genetic algorithms and other search techniques.
The portion of the assignment related to robustness was focused on developing control policies that would allow the robots to successfully travel the target distance while dealing with challenges such as noise on force commands, correlated errors and sloping terrain. Rather than try to minimize the amount of force exerted by the robot, the goal was to maximize the ability of the robots to deal with errors without falling over. Unlike with the efficiency portion of the assignment, robustness of the robots is not easily quantified because being better at dealing with one type of obstacle may reduce the robot’s performance against other challenges. Additionally, the control policies that maximized robustness were more complex than those governing efficiency and would be much harder to derive through search methods. To derive policies to maximize robustness, I performed a sort of simulated annealing on the parameters and control conditions myself. Since I could apply an intuitive understanding of what was causing the robots to fail under certain conditions, I was able to quickly make significant changes to correct the failures. I also played with all of the parameter values, eventually converging on combinations that supported robust performance.
Hopper 
Part 1: Calculate efficiency for the example controllers provided for hopping and walking.
Part 2: Optimize efficiency by minimizing E = (F^2) / D.
Part 3: Maximize Robustness (introduce noise, correlated errors and slope)
Conclusions:
The way in which “efficiency” or “robustness” is defined has a tremendous impact on what sorts of solutions will work for a given system. If efficiency oriented part of this assignment used E = F/D to measure efficiency instead of relating to the square of forces, then solutions that involve large jumps and uneven exertion of leg forces over time become reasonable. Also, some of the goals of maximizing efficiency conflict with those of maximizing robustness, and so there is a tradeoff between the two qualities. Thus, when designing control systems, it is very important to keep in mind exactly what sort of behavior is desired from the system and to be careful about how metric for this behavior are defined. Trying to increase efficiency may reduce robustness, which could make the system susceptible to failure. When I tried using a genetic algorithm to optimize the Hopper’s efficiency, I realized the importance of being careful to specify what sorts of behaviors would not be acceptable. An added benefit of carefully setting such constraints is that it significantly reduces the state space that needs to be searched for a solution.
Given the highly intuitive nature of these problems, I was able to achieve good performance without using an automated search algorithm. However, for systems of greater complexity, it will become impossible to achieve good results simply by manually adjusting parameter values. In retrospect, I can appreciate the value in developing a sense of what sorts of features in a control policy would be well suited for a problem. Once a general layout of the control policy is made, with weighted functions relating parameters and behaviors, automated optimization techniques could take over the daunting task of determining exactly what the weights should be used for the different parameter relations and their impact on system behaviors.
My genetic algorithm approach to optimizing the Hopper’s efficiency failed because I had tried to use it simply to manipulate combinations of parameter values. Using my intuitions about the system, I could have created the control conditions that I felt would work for the Hopper, and then let the genetic algorithm handle the task of determining how the relations of parameters to behaviors should be weighted. I learned that automated optimization isn’t a “fix-all” tool, but rather that it should be used in combination with the insights of the system’s engineer in order to determine which ideas work and to fine tune parameters and their impact on the system behavior.