Mini Grand 2010

'''This article was written on April 26th and may make references to articles that have changed or been deleted. Browse through articles' history and select the most appropriate dates to find what this article referes to.'''

= Introduction =

The following document is an overview and postmortem of the 2010 Mini Grand Challenge submission from the Pennsylvania State University, University Park, Robotics Club. This document is designed to help future groups understand the current software and hardware associated with the Mini Grand, as well as improve future versions we choose to compete with.

If you are unfamiliar with the competition, please review review the Mini Grand Challenge article.

= 2010 Solution Overview  =

The 2010 Mini Grand solution was based on known and proven algorithms, with the exception of a new laser-vision approach for path finding, and a new underlining component system for code modularization. The hardware was a revised 2009 Mini Grand vehicle without any heavy modifications. The resulting system that was competed with had all the planed components, though significantly refined, and did not use the core laser-vision fusion technology.

The 2010 solution was to take an image from a front-facing vehicle camera, sample the road color from a valid known road section (as detected by the laser range finder), and segment the road into a binary image. This image, in which white was considered a drivable surface and black was non-drivable, was given to a path planning algorithm. A path-tracing algorithm would find the center of the road and project a best-fit line, in which the degree from the line's slop was given to the vehicle controller. The system's inputs were a camera, a laser range finding device, a GPS, and an Arduino to retrieve an on-board compass' data. The system's outputs were vehicle turn and speed through the same Arduino. In previous projects, several "unique" features included voice scripts and music, not included in the 2010 solution.

The implementation of this system was done through modular components of code. A component was a single executable which implemented a specific part of the solution (visions, laser interface, vehicle controller, etc. When ran, each component would take as inputs data from other components and output any needed data. Some inputs were simple scalar values, such as vehicle velocity or laser resolution, while other data structures were more complex, such as the raw laser scan data or road color data. The system was to be written in C++ for a fast run-time, and would run on Linux, for a wide-range of device support.

Software
The software for the 2010 solution was designed for simplicity, ease of development, and speed. With previous years, a robotics library such as Player/Stage was used to help with hardware access as well as provide a way to modularize code. Doing so allowed for cleaner and smaller implementations that could be tested individually through testing modules. Each module could receive input and send output on it's own time. Since each component runs independently (as threads in Player's design), a component may choose to speed up or slow down computation as needed. An example could be that a component that is waiting for fresh laser-collision data does not need to execute anything, and can thus sleep until it is notified of new data.

Instead of using Player/Stage, we chose to use a smaller specialized library named CMU IPC. CMU IPC is similar to Player, but does not contain any robotics-specific code. This library allows for quick inter process communication, on any major operating system, either locally or through an internet connection. Though this library is not robotics specific, it handles the needed inter process communication functionality in a treadless way, allowing for independent-process implementations of components. This is key is in prevention of whole-system failures: if a single process (rather than thread) failed, the entire solution does not collapse, and the failed process can be restarted by a daemon (background service) observer. This concept has been proven to work, and is now implemented in all major operating systems (See Minix 3 Design). CMU IPC has also been proven to work in large-scale projects (Original Mars Rovers) and similar competitions (University of Delaware's 2009 AUVSI IGVC Submission). Player formally defines many data structures to pass between components, such as images and laser scans. Though conceptually this is a powerful tool, it falls short in being helpful because of lack of documentation and complexity of these data structures. CMU IPC allows for run-time registration of any data format of any size, and thus allows the 2010 solution to pass any form of data between components in a much more developer-friendly way.

Each component, for the 2010 software solution, was a single class that derived from a pre-programmed base component class, aptly named "BaseComponent". Developers then had to overload a generic update function named "int Update(float)". This function was the core component cycle, which would be called as-needed by internal resource management code. Each component's goal was to implement a feature or algorithm, and would thus mostly implement that code in the update cycle. Each component was initialized with a target update frequency, calling that update cycle at an appropriate frequency. By setting the update speeds explicitly, we are able to better control update times and operating system resources management. If a component is updating too slowly and takes more time then it was allocated, the internal resource manager makes sure to update more frequently. If a component is updating very quickly, then the process is put to sleep more often, releasing system resources for other components. Each component would, once programmed, be compiled into an executable, through a common makefile, and could be independently executed when needed.

The software solution comprised of several modules, each satisfying a single need for the system. Some components were simple, such as the laser interface that directly returned laser-scanning data, while others were more complex such as the vehicle controller. Some modules that were too big to implement as a single file, had supporting code in other classes (most notably the vehicle controller does this). Each component, during development, had defined inputs and outputs, as well as target frequencies of update.


 * Base Component - All software components must derive from this class
 * Utilities code - Several special functions are needed globally, such as image passing & receiving code
 * VehicleController - Vehicle interface that sets velocity, gets bearing, etc 10 Hz
 * Arduino Server - The vehicle-side hardware code
 * Arduino Client - The computer-side hardware code
 * Laser Interface - A generic laser-range finder interface 20Hz
 * Collision Detection - An interface dealing with collision detection and avoidance 20Hz
 * GPS and Waypoint Interface - An interface with a Garmin GPS device and waypoint manager 5Hz
 * Main Controller - Main controller logic 10 Hz
 * Observer Client - A GUI to observe vehicle properties and system performance 30Hz
 * Laser Vision Fusion - Detects segments of road based on the laser range finder 5Hz
 * Road Detection - A visions based road detection algorithm based on sampling road data 10Hz
 * Projection Interface - Applies a projection transform on an image 20Hz
 * Forward Vehicle Transformation - A vehicle path planner based on vehicle dynamics 20Hz

At the project's completion, many of the components were either merged or removed for simplification and run-time speed ups. The orignal goal of the 2010 solution was to attempt a real-world implementation of laser-vision fusion, which seemed promising. Due to the complexity of the needed software, it was ultimately dropped and replaced by a simpler (though effective) color-range selection algorithm.


 * Base Component - Same as above
 * Utilities code - Same as above
 * Arduino Interface - Same as above; renamed from "VehicleController" 10Hz
 * Arduino Client - Simplified communication protocol
 * Arduino Server - Removed unused code
 * Laser Interface - Same as above 20Hz
 * Collision Detection - Does not check for collision within a box, and instead thresholds collision distances 20Hz
 * GPS and Waypoint Interface - Same as above 5Hz
 * Main Controller - Same as above 10Hz
 * Observer Client - Heavily reduced; did not use OpenGL GUI library 30Hz
 * Path Interface - Derived from Forward Vehicle Transformation; simple ray-casting path planner 10Hz
 * Road Detection - Same as above 20Hz

Each software component is defined in much more detail in the Mini Grand Challenge Architecture article. An explanation of how the component-level solution works is as follows (for the final 2010 solution): First, the Road Detection component queries an image from the front-facing vehicle camera. Within this component, a pre-set area of the road is selected as the color range of the road. This was originally intended to be done by the laser-vision fusion component, but was later removed. Once the road color is selected, the component then sets all road-like areas to white on the image and all non road-like areas to black. This binary image is given to the Path Interface, which returns an angle for which the vehicle should travel towards. This is interpreted by the Main Controller which moves the vehicle towards that direction. As the Laser Interface queries new data, it gives it to the Collision Interface for collision detection. If there is collision detection, the Main Controller stops all movement. In the case there isn't any collisions, the GPS and Waypoint Manager will tell the Main Controller how close it is to the end of the course and which waypoint we are currently traveling towards. The Main Controller communicates with the hardware for movement by sending commands to the Arduino Interface, which in-turn uses the Arduino client and server communication code. The performance and memory usage of each component can be observed by the Observer Client.

Defense of System Design
The system's approach, of using laser-vision fusion for road selection was originally proven to work well for Stanford's 2005 DARPA Grand Challenge entry. It is unknown if this system would work on a much smaller scale for the Mini Grand vehicle, but was experimented with in the past within the Robotics Club. The current path selection code, based on OpenCV, works well but depends on the road always occupying a set lower rectangle of the image. If the laser-vision fusion component were to work, it could always pick out a starting region on the road to hand to the existing path selection code, and thus help the vehicle better choose its path. Another critical issue with path selection is the variance of colors based on shadows on the road. The laser scanner knows no differentiation between shadows and light, and would again help eliminate that variance. Due to the complexity of this component, it was dropped for time constraints. However, this is a true core component of the solution and should be pursued in future Mini Grand versions, but is not critical to a functioning robot.

Though the vehicle failed to run through the course correctly, the visions and path planning code worked well in unison at the competition. This was shown when the main controller component printed the intended vehicle speed and direction. It was observed that the vehicle was turning correctly in software at the appropriate times and positions for several given situation, but the front-vehicle wheels could not respond quickly enough. CMU IPC was tested and the issue was not in software, but in hardware. The front-vehicle wheels were responding too slowly (up to a three to five second delay) for turning commands. The debugging windows, shown by the road and path selection code revealed, at run-time, correct road selection and correct path selection, though it needs refining.

Defense of Player/Stage Removal
Player/Stage has several major faults, in both design and philosophy, that prevents it from being close to any ideal library for our system design. Player, a robotics interface library, is designed to modularize code and load them as libraries at run-time, running in independent threads. Stage, a robotics simulation platform, is designed to take these modules (with modification) and simulate their functionality, testing inputs and outputs. Player's point of failures are with its complexity of development and usage, platform restriction, unclean code, undocumented memory management usage, and poor error handling.

Player forces developers to use pre-defined message structures, disallowing the creation of new structures unless Player is recompiled. Even if a message format does exist that a developer would like to use, it is usually overly complex, poorly documented, and larger than needed. Sending and receiving messages can be a complex task, as each message may have internal structures that need to be individually manages. Player's memory model is rarely detailed in this case. Running player, with it's complex configuration system, does give a developer a large range of settings to play with, but is far too daunting and complex for new developers, which the Mini Grand is intended for. The Mini Grand is designed to be an introduction to high-level robotics, for those who only have experience with table-top robotics; Player is simply too complex for new developers to understand and use effectively.

Player, especially with the Penn State Robotics Club AUVSI IGVC 2009 vehicle, has proven to be slow with large data transmissions. During that year's solution, we attempted to localize the robot through a simple SLAM-based algorithm, which requires a map of all collision data. These maps were of high-resolution of large open areas, leading to high memory usage. Even though the code was designed with paging in mind, to optimize memory usage with the operating system, it was found that data transmission through Player was a huge bottleneck.

Error handling is critical to a large project of this size. If a single component fails in player, the entire system crashes. You can work around this by implementing each Player module as it's own process, but Player was never intended to do that kind of design.

Stage, the simulation environment for Player modules, is also overly complex and difficult to use. It is easier to test individual components than test an entire system in Stage, making it somewhat of an unneeded tool.

Defense of CMU IPC
CMU IPC is a library designed to send complex data structures between processes or computers. A process is an independent thread of execution, managed by the operating system. A thread is similar, but managed by a process. If a process dies, all threads die, but if any given process dies, no other processes are affected. Thus, we choose each component to run in its own process to prevent major run-time failures, and with such a design, a method of inter process communication was needed.

CMU IPC has been proven to work in major projects (See above references to the Mars Rovers and 2009 AUVSI IGVC winner). It is able to send complex messages and define new ones at run-time. It is also a very clean and small implementation, which would help new developers with the project understand how to write code with it. CMU IPC is also very fast, no matter what operating system it is run on. It takes advantage of each operating system's IPC implementation, and optimizes for that system.

The CMU IPC messaging model is also simple. A central server takes all sent messages and forwards them to the listening components. Any component many listen to any other component. Each unique message has it's own buffer, in which older messages may or may not be buffered or removed depending on their age. CMU IPC can also work on a decentralized system, in which a component knows directly where to send it's messages, instead of using a central server.

We chose to wrap CMU IPC communication into a base class that all components must derive from. This hides much of the IPC functions into a simple get/set function protocol to use. It can take any data type and structure, and send it to any listening component at run-time.

Hardware
No major changes to the hardware occurred. The hardware was fully documented, and now includes an initialization procedure list.

= Points of Failure =

The following is a list of point of failures for the 2010 Mini Grand, for both software and hardware.

Front Wheels
Front actuator was too slow for movement. Though it did move to the right angles, there was too much latency between setting an angle and rotating to face that angle. A large servo, used in the past, should be used again, but with software controllers to prevent the burnout issues we have faced in the past.

Laser Range Finder
The Hokuyo laser, though powerful, did not read any correct data simply because it is not able to work in our-doors light frequencies. A different laser range finder should be used. A cheap alternative could be a stereo-visions camera, which is now sold at a much more competitive price (and could help with laser-vision fusion).

The collision interface also had a too-simple approach when it came to collision detection. There should be a geometric shape infront of the vehicle that is considered a collision. If any collision point is within this geometric shape, we can then declare a collision.

GPS / Waypoint Manager
The GPS device has an awkward physical connection system. It currently requires a cigarette-lighter 12v connection system, a serial-to-USB adapter, and specialized code. We should find a more simple replacement for it.

The waypoint manager code failed and did not return correct distance to the next waypoint. We should be reviewing this code to understand why the correct number of meters was note returned.

BaseComponent/IPC Messaging System
Each message sent between components has the same name, even if the variable sent in the message is different. That way, when component A sets component B as a dependency, component A receives all messages from component B. There as several issues with this design: we may receive more data than we want, and the internal IPC buffer doesn't know how to differentiate unique messages, since they all have the same title. This is where the original latency in the initial version of BaseComponent existed, since the queue would overflow with messages, and older messages (with unique data) was removed. BaseComponent also has the ability to throttle itself as well as release system resources if a component is able to.

It was noted that BaseComponent's model of initialization and update functions was a good approach for real-time/robotics systems. It could implement any algorithm, state machine, has a good update structures, and overval was a good abstract model of it. This model was also similar enough to the Arduino's programming language structure.

A possible solution is to make each data type name unique for each component that sends it out. Doing this allows for CMU IPC to better manage our messaging system, as well as substantially improve the latency.

Developer Training
There was a major lack of developer training. Though there is heavy documentation, there were never any one-on-one training sessions. It was expected students would be able to read through the Doxygen documents and the RoboWiki tutorials, most students are not used to having to do this much research to start a project. In the future, the project leader should be doing a three-week tutorial session demonstrating to students how to use the component design introduced this year into the project.

Tutorial sessions should be as follows:
 * Teach how to use linux through VM
 * Create a component and explain component communication
 * Sample: One component takes an image, second parses second image
 * Show how to move vehicle

Reasons of Intimidation
We lost quite a few developers due to intimidation by the complexity of the project. Formal reasons:
 * Lack of Linux knowledge
 * Perception of difficulty
 * Lack of interest
 * Ownership of code (Scared of being responsible for code)
 * Have teams work on code, instead of individuals, to reduce fear and intimation by ownership
 * Have these teams to focus during meetings, not "goof around"

Documentation
Most of the Mini Grand documentation was on the Penn State Robotics Club's RoboWiki. There were by far too many documents scattered about the site, without any central location. Any future Mini Grand versions should have the documentation coalesced and centralized. Doxygen was also a powerfull tool, but was not followed enough. Source code formatting was not consistent, in which it should be.

To help with more clear documentation, and thus help get new students involved, there should only be two to three maximum pages relating to the Mini Grand project. One to describe the project, one for our solution and detailed component list, and a final with all the associated tools needed to develop code.

Be careful of forcing students into using (design, protocol, etc) specification.

SVN
SVN, or a type of source control, is a tool used for multiple developers to work on a central storage of code. For the 2010 Mini Grand project, members had to commit any changes to the SVN server. Though we did not implement a type of continuous integration with the SVN, it should become a tool used in the future for measuring correctness of code. There was a new tool introduced called SVN stat, which showed statistics for each developer and the project as a whole. This was good at measuring the amount of code written per developer, but was not efficient at enticing developers to write more code.

Code Review and Testing
There was no formal code review or testing sessions for this year's Mini Grand, but is a critical need for any future versions. We should never have a single person or group work on code review or testing the entire time; many developers find this as tedious work, no matter how important it is.

Developers should never change implementation detail (though style / formatting can change) without doing it at as a team / formal sit-down session.

Virtual Machine Environment
To aid students in development, but who don't have access to a Linux installation, they had the choice of using a pre-built Linux Virtual Machine. This was helpful for development, but worked against many developers when attempting to test hardware connections because of difficulty with hardware recognition on the host operating system.

= Developer Notes =

Concluding remarks from core developers.

Adam Brockett
This was the commit message I used for the final code submission to SVN.

Here are a couple notes about how the competition went. It was a sunny day, early in the morning, which meant that shadows were present on the path, which was an added challenge for teams that used vision base algorithms (as we did). Additionally, we found out that the fact that our laser was labeled "for indoor use" meant that it was "for indoor use" -- it operates on a wavelength that is directly in the interference zone for normal daylight. Oops. That is why you will find some HAX in the CollisionInterface that ignore conditions when over 70% of the points are below the threshold.

But as far as actual competition went, I feel that we should be proud of our accomplishment. The RoadSelect work impeccably, and the PathInterface was impressive to see. Like I said before, the CollisionInterface was cut out of the final running, but worked in the lab. There is a bug with the distance calculation in the WaypointInterface. However, the robot was built with a linear actuator for steering. This provides us with lots of torque, but it adjusted too slowly. This meant that by the time our decision to go left was translated to wheel movement, it was too late and we were already too far off the path to recover. But the software worked very well.

Jeremy Bridon
Adam sums it up perfectly. The developer team should be very proud of the final project. What matters is to continue the development of code and stay away from anything too new or radical. We should be focuses on fixing the current problems, getting more student involvement, and eventually winning! There should be a huge emphasis of getting testing done as early as possible, and to push students to get coding done on-time.

= Conclusion =

Though the 2010 Mini Grand solution failed to be win, it proved that the overall solution approach worked. When the vehicle was manually pushed forward at speeds slow enough for the front-wheel movement to turn appropriately, the vehicle aligned itself in the center of the road. This also proved that the new underlying component software design was powerful, fast, and removed the unneeded complexity Player had brought in past attempts.

Future attempts for the Mini Grand should absolutely be based on the current code and design, with improvement to the needed sections based on the Points of Failure subsection of this article. The laser-vision fusion should be completed, as this is a core component never implemented and should help road-selection greatly improve the selection quality. This project, through the current design, is a powerful stepping stone between table-top robotics and the international level of robotics competitions. Code has been made much less daunting than in the past, and documentation is a powerful way to help those interested understand the current solution without too much of a technical background.