Drive System Reliability

and Trouble shooting

April 29, 2006

Championship

Atlanta, GA

Jay TenBrink and Patrick Major

Goodrich H.S. Martians 494 and More Martians 70

DaimlerChrysler and General Motors

Agenda

1. Introduction (why we are here)

2. Drive system reliability

3. Trouble shooting

4. Corrective action implementation

5. Design for reliability

6. Questions and answers

IntroductionJay TenBrink

Manager – FWDPT Chassis Engineering

Team 494 Engineering Coach since 2001

Patrick MajorOwner – Major Distributing Co.Team 494 Head Coach since 2001

My Favorite RobotTeam 494 Head Robot since 2003

Why we are here In 2003 team 494 developed the Robot Dynamometer

as a device for testing and developing robot drive systems. This device applies a varying resistance load to the drive wheels and measures the robot's speed. Throughout 2003,4 and 2005 seasons, this dyno was available for use to all teams at 3 regional events and the Championship. Of the more than 100 robots that have been tested approximately 20% exhibited a drive system problem of some degree. In many cases, root cause analysis took place on the dyno.

The Robot Dynamometer allowed this work to take place on a static robot under any simulated speed or load condition in a safe and controlled environment. Since many drive system problems are not evident when there is no load put on the system, conventional root cause analysis can be difficult, imprecise, time consuming, and even unsafe.

What we will share with you today:

Drive system reliability Examples of actual electrical and mechanical problems When, where, and why they occurred (we won’t say who)

Trouble shooting techniques (root cause analysis) Safety precautions and root cause analysis do’s and don’ts Helpful analysis tools

Corrective action implementation Appropriate repairs and avoiding collateral damage Verifying the fix

Design and construction methods for reliability Basic DFMEA (Design Failure Mode Effects and Analysis)

What we won’t be covering here:

Internal battery problems and how to avoid and diagnose them

Internal motor problems and fundamental drive system design flaws

However…

Many of the drive system fundamental issues we address can be applied to other robot systems (arms, claws, etc.)

Quote for the day

“It is good to learn from experience. It is better to learn from someone else’s experience”

Author: unknown

Terms Root cause - The heart of the issue, what started the whole

problem. Example: Did not crimp the wire terminal correctly.

Failure mode – The primary effect of the problem. Example: Poorly crimped power supply wire increases the resistance of the circuit, this decreases the voltage available to the entire robot.

Secondary failure mode – Cascading failure brought on by the primary failure mode. Example: When voltage drops, motor power is decreased.

Symptom - A sign or indication of a problem that is neither a root cause or a failure mode. Examples: Wires or terminals that are hot to the touch, melted insulation, smoke, noises, odor, etc.

Corrective action – what you did to address the root cause. Example: Fixed poorly crimped terminals.

Reliability Problems

What follows is a somewhat comprehensive list of actual problems that we have experienced first hand or witnessed during our last four years with the FIRST program.

Electrical Problems

1. Poor crimps on terminals (the grand daddy of them all)2. Loose screws on terminals (battery, controller, etc.)3. Mechanically damaged or cut wires4. Shorting to the frame5. Incorrect wire gauge or terminal size (too small)6. Terminal size too large for the wire gauge7. Loss of battery or battery cable becomes unplugged8. Incorrect fuse size or non-functioning fuse9. Damaged or discharged battery10. Metal debris in electrical devices11. Cracked or broken control cable connectors or device12. Cold solder joint on electrical device terminal13. Broken terminal pins on control wire connections

Mechanical Problems1. Drive train out of alignment and binding2. Drive sprocket to wheel hub slipping under load3. Wheel tread to drive wheel slipping under load –

especially on tank tread robots4. Bevel gears out of place due to axial loads5. Drill motor gear box in between gears6. Drive chains coming off of sprockets during a match7. Drive sprockets breaking (plastic sprockets)8. Fractures due to removing too much material during

weight loss (AKA swiss cheesing)9. Frame deflection under load from drive system10. Set screws loosen and back out of sprockets

Common Drive System Problem Symptoms

1. Electrical connections or wires are hot to the touch or melt insulation on terminals or wires

2. Metal or plastic debris is present on the robot from grinding, scraping, slipping gears, etc.

3. Robot lacks the power or speed it used to have

4. Robot only turns in one direction or will not move at all

5. Motors overheat and circuit breakers / fuses open

6. Main power supply circuit breaker opens

7. Sparks, smoke or foul smell is present

Trouble Shooting Do’s and Don’ts(Root Cause Analysis)

1. Do work safely:• Always wear safety glasses in the pit area or around live

robots, even if there is no action on your robot• Be aware of what your neighbors are doing (exercising or

testing their robot, grinding, testing, etc.)• Be aware of any stored energy device on your robot

(springs, cylinders, tank, battery, kinetic or potential energy devices, etc.)

• Be extra careful when powering up your robot in the pit area. Never leave the controls of a live robot unattended.

• Never make electrical repairs on a live robot. Electrical shorts can cause severe burns and damage components

• Keep loose objects out of the robot (sleeves, hair, lanyards, body parts, etc.) Don’t lean over the machine, walk around.

Trouble Shooting Do’s and Don’tscontinued

2. Do select a leader for your root cause analysis activities3. Do have a systematic plan of attack4. Do discuss the plan with the students and mentors and get

feedback5. Don’t rush, this almost always leads to careless mistakes6. Don’t allow the evidence to be tampered with or destroyed before

the analysis has been completed. This is crucial!7. Don’t wait to trouble shoot.

• Some evidence has a short half life, so observe the symptoms quickly and carefully (hot wires, open circuit breakers, smells, etc.)

8. Do use all of your senses (except taste) to analyze the evidence• Visually inspect for anything out of the ordinary, foreign matter,

melted solder, broken plastic, etc.• Listen and feel for scraping, vibration, clunking, etc.• Notice odors from hot wires, motors, burning rubber, etc. Don’t

sniff around batteries or inhale smoke.• Feel for components that may have gotten warm. Watch for

burns.

Trouble Shooting Do’s and Don’tscontinued

9. Do know what conditions were present at the time of the failure• Does it happen at max power going head to head with other robots?• Was the robot in a collision with another robot or field piece?• Is the failure repeatable or seemingly random?• Is the severity or rate of occurrence increasing or decreasing?• What events were coincident with the failure (shipping damage, new

battery, spilled Pepsi, etc.

10. Don’t have more people’s hands in the robot than is necessary

11. Do try to duplicate the failure mode in a “controlled environment” to confirm that you have the true root cause.

12. Do take measurements (current draw under load, voltage drop, continuity) before disassembly, adjustments, or repairs are attempted.

Trouble Shooting Do’s and Don’tscontinued

13. Don’t begin to implement corrective actions before you have determined the root cause.

14. Do try to duplicate the failure mode after the corrective actions are in place to verify your fix.

Helpful Tools and Materials for Analysis and Corrective Actions

1. Multimeter capable of indirect current measurement. Very useful. Available for under $200 from Newark Electronics.

2. Electrical connector crimping tools, soldering gun, tape, and assorted wiring connectors and wire.

3. Spare kit parts: spikes, victors, control cables, motors, etc.4. Spare parts that were custom built and critical to the drive system.5. Tie wraps and velcro6. Thermocouples to measure motor temperatures under heavy use.7. Robot dynamometer. If you don’t have one, you can use ours.

Corrective Action Implementation

1. Emergency corrective actions (containment)• Can you “contain the issue” to get you through the next match

(tie wraps, braces, screws, jumper in a new component, etc.)?• Can you remove or deactivate the damaged component and play

the round without it until it is fixed?• Can you continue to run with the problem without causing more

serious or permanent damage?

2. Long term corrective actions• Accidental damage:

• Can a reoccurrence be prevented by adding protection (a guard, shield, or modification) ?

• Craftsmanship problem:• After correcting a craftsmanship problem, check to see that there are

not additional areas. Don’t just fix the failed crimp, check all crimps.

Corrective Action ImplementationContinued

Long term corrective actions cont’d. Design flaw:

• Is it possible to eliminate or reduce the severity of the problem with a change to design, software, or driving style?

3. Avoiding collateral damage during analysis / repair• Remove or protect sensitive components (victors, spikes,

controller, etc.) from debris or damage during repairs with a towel, etc.

• Don’t lean on or reach over the robot, walk around it.• Do not place tools, heavy components, or drinks on the

robot.

Corrective Action ImplementationContinued

4. Verifying the fix Can the problem be reproduced after the “fix”

has been incorporated? Are any of the previous symptoms still

present (noises, hot wires, low power, etc.)?

DFMEA (Design Failure Mode and Effects Analysis)

Note: This is an analysis tool that is used during the design stage.• List all known possible design failure modes

• Let’s use a real example: drive chain breaks• Estimate probability (what are the chances of it happening).

• It is likely to happen: high probability (it did)• Assess the level of severity if it happened (the affects)

• If it happens, we loose power to that wheel • What secondary failure modes might be caused by the primary

failure mode.• The broken chain might get tangled up in the

wheel and lock it up. It might also get tangled up in the other drive chain.

• What is that probability and severity?• A tangled chain could bind a wheel and could

break a hub. Address problems with the highest probability and severity first.

• Monitor and maintain the chain tension (didn’t do this at first) and remove any threat of entanglement at the wheel. (did this)

Design for Reliability and Durability

1. Apply the KISS principle (keep it simple, silly)2. Don’t design in cascading failures (like in our example)3. Layout your wiring neatly and in a naturally protected

area. If it must be routed in a severe area, protect it.4. Label both ends of each wire, both power circuits and

control cables. This will assist you in diagnosing issues and will save you time in the long run.

5. Protect exposed components from damage with metal guards or lexan.

6. Positively secure your battery in the robot and tie wrap the power cables together prior to each match. In every regional at least one robot has a battery fall out.

Design for Reliability and Durability

7. Use self aligning bearings and couplers that allow for misalignment in the build or frame deflection.

8. Anticipate sudden shock loads from accidental contact at any angle. Severe side impacts can cause a drive chain to jump off of the sprockets.

9. Loctite* all set screws in place10. Include a method for adjusting drive track tension to

account for track stretch or wear.11. Include a method for adjusting drive chain tension.12. Include guides for long drive chains to keep them

from falling off of sprockets13. #25 pitch chain is lighter, but much less robust than

#35 chain

Craftsmanship• Wiring

1. Terminals must be properly crimped to achieve a solid mechanical and electrical connection. Do a pull test on several of your samples. Crimping is the chief source of problems.

2. Terminals need to be the proper size and screwed down tight at the device.

3. Make sure you meet or exceed the wire gauge sizes specified. 4. Make sure wiring is not routed over sharp edges, in pinch

zones, or areas where it will be bruised, stretched, or cut.5. Soldering of terminals should not be necessary if they are

crimped properly. Soldering can temporarily mask a poorly crimped terminal.

6. When routing wiring through tubing, or in any enclosed area, be careful not to drill through it.

7. Check for continuity from the battery to the frame (grounding out). before powering up your robot for the first time.

Craftsmanship

Construction:

Break sharp edges that can damage wires, cut tires, cause mechanisms to bind, or cause personal injury.

Build things true and square that need to be true and square: axles, motors, etc. Not everything needs precision.

Be careful not to weaken high stress areas during the “weight loss” phase of the program.

In general, follow sound building practices

Inspection and Maintenance

1. Create an inspection check list to go over every time the robot returns from a match to find damage & wear.

2. Exercise all robot features for proper function after the robot has been: shipped, idle overnight, repaired, modified, etc.

3. Keep a log of all repairs and modifications to your robot: damage, wear, regular service, modifications, etc.

4. Establish a battery maintenance and charging procedure with one person assigned to be in charge.

Team 494 Robot Dynamometer

Reliability and Trouble Shooting Drive Systems

Questions?

This presentation will be posted on the Martian website:

www.494martians.com