telephone power dependency do your phones work when the lights go out?
TRANSCRIPT
Telephone Power Dependency
Do your phones work when the lights go out?
WCDM - July 10, 2007 Ray J. Vaughan, MS 2
In the old days…
Your dial tone came from the Telephone company Central Office (CO).
The old telephone ran on voltage that came from huge batteries in the CO.
The batteries were kept charged by either the power company or huge generators with weeks of fuel.
WCDM - July 10, 2007 Ray J. Vaughan, MS 3
Now
Today’s technology comes with a price… electricity.
Now we need to provide power to our phones, our phone systems
And to the equipment that makes fiber optic networks possible.
WCDM - July 10, 2007 Ray J. Vaughan, MS 4
Low Tech Solution
First the easy solution to the big problemEducate the Public to get a simple or old
phone in the house.Does your phone have a display?
Answering machine? Cordless? It has a second cord that goes to an electrical outlet.
When the lights go out, these phones die.
WCDM - July 10, 2007 Ray J. Vaughan, MS 5
Excuses
“I have a generator”Will you be able to call for help if it doesn’t
start?How will you call for the Fire Department if it
catches fire?How many days of fuel will you have
available?Will you run it 24/7?
WCDM - July 10, 2007 Ray J. Vaughan, MS 6
Excuses
“I have a cell phone”How will you be charging it?Will the towers survive?
WCDM - July 10, 2007 Ray J. Vaughan, MS 7
Message:
Have at least one simple phone in the house.
Add this to your agency’s home disaster inventory list.
( and take the analog TV OFF the list… HDTV and EM http://rayvaughan.com/HDTV&EM.htm )
WCDM - July 10, 2007 Ray J. Vaughan, MS 8
Harder Problems to fix
The next problem doesn’t have such an easy fix.
WCDM - July 10, 2007 Ray J. Vaughan, MS 9
See the Light
Fiber Optic technology allows for huge amounts of voice and data over many miles on a hair-thin piece of glass.
BUT, Glass conducts light, not power.Something has to convert the digital light
signals back into analog electrical signals.
This conversion requires sophisticated electronic equipment that needs power.
WCDM - July 10, 2007 Ray J. Vaughan, MS 10
Remote Terminals
Covert Fiber Optic signals to regular phone lines
Powered by the local power utility4 to 8 hours of battery powerUsually no backup power source on siteSometimes called SLC, Subscriber Loop
Carrier
WCDM - July 10, 2007 Ray J. Vaughan, MS 11
Is this really a problem?
If there is a long (>4 hours) power outage who will be effected?
Everyone. Calling 911 will be impossible.
You. Your Public Safety Facility may be out of service.
Cellular Sites are interconnected with circuits fed by the same RTs
WCDM - July 10, 2007 Ray J. Vaughan, MS 12
Wilma
WCDM - July 10, 2007 Ray J. Vaughan, MS 13
War Stories
Hurricane Wilma October 2005South FloridaWeak Category 2 hurricane
$20.6 Billion in damage36 deaths in Florida
Wide area but minimal floodingComplete power failure in South Florida
3,241,000 customers, 6,000,000 people without power
Weeks to restore power to many areas
WCDM - July 10, 2007 Ray J. Vaughan, MS 14
How Bad was it?
http://ngs.woc.noaa.gov/storms/wilma/30112791.jpg
Photos by NOAA
WCDM - July 10, 2007 Ray J. Vaughan, MS 15
Communication Failure
As I had predicted, we had major communication failures.
The area effected by Wilma was so wide, resources available in the area were overwhelmed.
WCDM - July 10, 2007 Ray J. Vaughan, MS 16
Fire Stations
Our generators came onVoIP phones, delivered by T1, ran until
the RT diedBackup Hotline/Fax Line also on RT
Last resort: UHF Public Safety Radio SystemConventional, Simplex availableNo common points of failure
WCDM - July 10, 2007 Ray J. Vaughan, MS 17
K&B Radio Site
Hub site for South end of the CountyT1s to dispatch, other sites failed
Site in In-Cabinet Mode, 1 RX, 1 TXLong RT failurePortable generator set up
Fuel shortageMisconnection
WCDM - July 10, 2007 Ray J. Vaughan, MS 18
WCDM - July 10, 2007 Ray J. Vaughan, MS 19
WCDM - July 10, 2007 Ray J. Vaughan, MS 20
Outage Time Line
EventPower outage4 hour countdown startsEnd of wind event, earthquakeBatteries start to dieNo communicationGenerators arriveFuel shortages
WCDM - July 10, 2007 Ray J. Vaughan, MS 21
Katrina Facts
Estimates of the damage: Initially over 1.75 million people without
phone service 131 Central office effected and 19 of them
either destroyed or heavily damaged, serving a total of 187,000 lines.
Cost to restore/replace: $400-500 million Data and Photo Source: The Central Office
WCDM - July 10, 2007 Ray J. Vaughan, MS 22
WCDM - July 10, 2007 Ray J. Vaughan, MS 23
WCDM - July 10, 2007 Ray J. Vaughan, MS 24
WCDM - July 10, 2007 Ray J. Vaughan, MS 25
WCDM - July 10, 2007 Ray J. Vaughan, MS 26
Verizon Central Office, WTC
WCDM - July 10, 2007 Ray J. Vaughan, MS 27
Verizon Central Office, WTC
WCDM - July 10, 2007 Ray J. Vaughan, MS 28
Verizon Central Office, WTC
WCDM - July 10, 2007 Ray J. Vaughan, MS 29
Verizon Central Office, WTC
WCDM - July 10, 2007 Ray J. Vaughan, MS 30
Verizon Central Office, WTC
WCDM - July 10, 2007 Ray J. Vaughan, MS 31
Action Items
Establish a working relationship with your telco technical expert
Identify your critical locations Sorry, they’re not ALL critical
How will a Telcom outage effect your process? What’s convenient and what’s mandatory?
Define backup systems Public Safety Radio
WCDM - July 10, 2007 Ray J. Vaughan, MS 32
Action Items 2
Start with your most critical. Ask:Where do these lines/circuits need power to
operate?Your end. You better trust your own powerThe CO. Batteries, large generators.
Your concern: the ‘everywhere else’ Is there an automatic generator at each?Home-Run to CO? Or in a loop?
WCDM - July 10, 2007 Ray J. Vaughan, MS 33
Take a look
Learn what to look forIdentify the RTs in your areaLook for generatorsAsk questions
Your regular Telco Tech is a great resource for how things really are
WCDM - July 10, 2007 Ray J. Vaughan, MS 34
Types of RTs
Above GroundUnderground
Flooding?In-BuildingIn someone else’s building
Worst case. You depend on their planning and survival.
WCDM - July 10, 2007 Ray J. Vaughan, MS 35
Clues that it’s a RT
Power MeterHum or fan noiseThick cables radiating outFrequent Telco Trunks parkingAddress or RT ID stickers
WCDM - July 10, 2007 Ray J. Vaughan, MS 36
WCDM - July 10, 2007 Ray J. Vaughan, MS 37
WCDM - July 10, 2007 Ray J. Vaughan, MS 38
WCDM - July 10, 2007 Ray J. Vaughan, MS 39
On-Site RT
The best solution is to have your RT in your facility
You provide the primary powerYou provide the securityVerify fiber route
Route redundancyMake sure you trust everyone in your loop
WCDM - July 10, 2007 Ray J. Vaughan, MS 40
Case Study
K&B Radio SiteNow has an on-site RTIndependent of area RTOn our generatorRedundant fiber routes to CO
WCDM - July 10, 2007 Ray J. Vaughan, MS 41
WCDM - July 10, 2007 Ray J. Vaughan, MS 42
Single Points of Failure
Better to have multiple:Remote TerminalsGeneratorsFibers and Fiber routesCopper circuits (if close enough to CO)Communication Companies
WCDM - July 10, 2007 Ray J. Vaughan, MS 43
Generators
Subject to barometric pressure failure during storms
Over/Underspeed due to windWater in air intakeProjectile damageCheck the oil change frequency
Oil & Filters on siteDowntime for changes
WCDM - July 10, 2007 Ray J. Vaughan, MS 44
WCDM - July 10, 2007 Ray J. Vaughan, MS 45
Flooding
Many RTs and other Telecom facilities in Katrina were flooded. Fiber can be underwaterEquipment can’t be.
Identify any critical systems in your system that need to be moved higher
Even a CO can be at risk of flooding
WCDM - July 10, 2007 Ray J. Vaughan, MS 46
Terrorist Targets?
RTs could be an easy target for anyone wanting to effect communication in an area.
Most are easy to spot when you know what you’re looking for.
Exposed fiber, cables, powerEven simple vandalism at an RT can put
you out of service.
WCDM - July 10, 2007 Ray J. Vaughan, MS 47
Monitoring
Your Telco should monitor every RTInvestigate every alarm, even open
doorsRoll generator when AC power fails
Not when batteries fail
WCDM - July 10, 2007 Ray J. Vaughan, MS 48
Regulators:
Require permanent, automatic generators for all new RTs
Encourage RT clusteringRequire response levels
Ratios of portable Generators to RTsPhase out RTs without automatic power
backupMandate in-building RTs for all Public
Safety buildings
WCDM - July 10, 2007 Ray J. Vaughan, MS 49
More Information
My information site about Powering and Remote Terminals:
TelephoneFailure.com