Pick up Mechanism

The system to be used in picking the balls up off the court consists of a single rotating shaft mounted on the front of the robot with 4 or 5 rows of stiff, plastic bristles that shall push balls up a curved ramp and into an onboard storage bin area in the chassis. The storage area is designed such that the bottom plate of the bin can be raised by use of a motorized pulley system to unload balls over top of the bristle wheel. The bristle wheel is in constant rotation when the solution is in operation. The figure below describes various details of the system in collection mode.

The build consists of aluminum frame that shall be tack welded and riveted to the existing Kodiak chassis. The side panels consists of the support framework for the system components, and make use of the existing mounting holes in the Kodiak chassis body. The rotary wheel is to be chain driven from a spur gear that is pre-mounted on the Kodiak chassis. All measurements for the existing Kodiak structure are available in the Kodiak database provided by the UW Robotics Team. Further torque analysis shall be done to determine the minimum required torque to rotate the bristle wheel and lift the maximum amount possible load (geometrics allows for a maximum of 4 balls to be lifted in any given 90 degree rotation of the bristle wheel).

A force analysis will be required to determine the optimal height of the pulley rod, and the motor torque required to lift the bin platform from a horizontal position to nearly vertical position, while considering the weight of the platform and ball load. The motor for the pulley system is to be mounted underneath the storage bin platform in front of the drive wheel motors (not viewable in the diagrams). The pulley support links are to made adjustable to accommodate some adjustability. The system gives unloaded balls a potential energy of 0.109 N at a height of 18.5 cm from ground level. This should allow enough feasible potential energy to displace the balls to a desired orientation in the drop-off bay to reload into the ball machine.

Vision System

A major aspect of the control of the robot is the vision system. Major strategic decisions will be made based on the feedback from the vision system. The system will be composed of an off-the-shelf web-camera, in tandem with the image processing capabilities of the Microsoft Robotics Studio software. The system will be composed of three main parts, blob detection, color segmentation, and overall analysis.

The first job of the vision system is blob detection. Blob detection will be the main sensory input for the “Look for Balls” process. Blob detection inside of Microsoft robots studios is quite advanced, and does not require extensive image processing knowledge to be used. The first step in blob detection is calibration.

Calibration of the blob detection system is key to overall performance, and is done very well by Microsoft robotics studio. In Microsoft robotics studio, there is a built in tool named “Blob Tracker Calibration”, which allows the user to select the color of the blob from a live video feed. This effectively makes the color selection field programmable, and will allow for calibration on-site, to avoid issues with lighting and other chaotic environmental conditions. The goal is to have the user program the blob detection color when the robot is initialized, as to always ensure optimal tracking conditions. This, along with the fact that the testing environment has uniform lighting conditions indoors, ensures stability in blob detection. Testing is carried out on site using the built in blob detection and is proven to be very accurate.

The blob detection algorithm is based speed. The system uses a live feed from a webcam to perform image analysis. The speed of image analysis is based on the speed of the system running the software. Testing proves an approximate rate of 20fps using the hardware selected for this project. The algorithm focuses on analyzing the colors in the image, and creating a convex hull around the colors that are to be tracked. Below is a video of the blob detection in Action:

The final stage of the vision system is to decide on which blob or single ball to pick up. Color segmentation is the process of segmenting an image into different clusters of the same color. This is important to the vision system, as it allows the robot to get a general idea of where the balls are distributed, and decide how to proceed based on analysis of this information.
Color segmentation will be done through Microsoft Robotics Studio’s built in functions. Much like blob tracking, Microsoft Robotics Studio provides a calibration tool for color segmentation, that is also field programmable. The actual color segmentation tool in Microsoft Robotics studio then runs on the live webcam feed. It should be noted that color segmentation is far more computationally intensive that blob tracking, and therefore will only be run once a blob of balls is found, to prevent unnecessary stress on the system.

The color segmentation tool returns an array of segments in the screen. This array contains the x and y coordinates of the centroid of each segment found. Based on this array, analysis can be performed to determine the best plan of attack to pick up the largest amount of balls. The first process in this decision is distance.

Based on the two fixed variables y (distance from the ground to the camera) and Φ (angle of camera from ground), analysis can be done on the Y position on the image to determine the distance from the ball. Initial calibration will be needed to determine the exact distance of the ball based on the placement of the ball in the camera view. It should be noted that due to the nature of an angled camera, the higher the value of h1 (from diagram), the farther the ball from the robot. This increases on an exponential scale, as the camera can see far into the horizon.
Once the distance to each ball segment is determined, the algorithm will then analyze the optimal decision on where to proceed. While distance is important, the distance of one ball to the next ball is also important, and will be weighed into the algorithm. Testing will be carried out to determine which weighting of distance to ball versus distance to next ball is optimal.
Finally, the segmentation algorithm will also be used to determine the approximate number of balls stored on the robot. Based on the fact that the robot will be aware when it is approaching and picking up a ball, a count will be maintained to allow the robot do determine when the maximum 15 balls are reached and that it should proceed to ball drop off.

The Problem

Professional tennis matches can last upwards of 3 hours, often with little more than 10 minutes of rest in between sets. The need to train for such endurance matches becomes difficult without a partner of adequate skill. For decades, tennis players have been using automatic tennis ball launching machines to train without the need of a partner.
Testing has concluded that in general, an automatic ball machine that holds 100 tennis balls will launch at an average rate of 10 balls every 29 seconds. Therefore, the total amount of time an average tennis ball machine can last without needing to be refilled is around 290 seconds
Therefore, the maximum time that a tennis player can play with an automatic ball machine is in the area of 4 minutes and 50 seconds.
After which, the player must stop all play and collect all the balls around the tennis court using a bottom-loading basket, as is standard procedure in tennis. This is a physically straining procedure, which is generally despised by avid tennis players.
From additional experimentation it would take in the area of 2:15 to manually pick up 85 balls. It can be determined that it will take the player two minutes to pick up all the balls after using the capacity of the tennis ball machine.
Therefore, it is determined that using current technology, a player will spend~35% of the total time collecting tennis balls, and not training. This is a large proportion of the time, and needs to be reduced to truly emulate a tennis match. There are certain methods of automatically collecting tennis balls, but these are not portable, and require extensive modifications to the tennis court.


The following objectives are adapted from the project goal statement. These objectives represent the
goals that the final design will ideally achieve:
  • Decrease training downtime by 75%
  • The solution will be adequately portable
  • 1 hour without requiring external power
  • 80% ball retrieval rate
  • Durability – Can be struck by a tennis ball

Professional tennis matches can last upwards of 3 hours, often with little more than 10 minutes of rest in between sets. The need to train for such endurance matches becomes difficult without a partner of adequate skill. For decades, tennis players have been using automatic tennis ball launching machines to train without the need of a partner.Therefore, the maximum time that a tennis player can play with an automatic ball machine is in the area of 4 minutes and 50 seconds.~35%


The environment for this system is narrowed to ensure ease of preliminary design. Through a partnership with the Northfield tennis club, a fixed environment of operation is chosen to be an indoor tennis court with artificial lighting. The lighting in the court is uniform. The tennis court also includes an automatic tennis ball machine. The exact model is a “silent partner”.

Tests are carried out using the silent partner and an average skilled player. After a number of tests, a general distribution of the location of the balls is calculated. It is determined that roughly 85% of the tennis balls will be found behind the tennis court’s base line, with the remaining tennis balls in front of the tennis net. The distribution is generally proportional against the back wall.