opc-redundancy.pdf

Cogent Real-Time Systems Inc

Copyright 2011, Cogent Real-Time Systems Inc. www.cogentdatahub.com

RedundancyforOPC

byRobertMcIlvrideandAndewThomasCogentRealTimeSystemsInc.

Earlyonemorning,MelFarnsworthwassittinginthecontrolboothattheHardyAutomotivePartsassemblyline,drinkinghisfinalcupofcoffeebeforetheendoftheshift.Watchingthelinemetergraph,henoticedthattheyieldandefficiencytrendsfortheLine3haddroppedtozero.Helookeddownthroughthecontrolroomwindow,butLine3seemedtoberollingrightalong.Whatwastheproblem?Thelinewasrunningsmoothly,butMelwasntgettingthedataheneeded.SomewherebetweenthePLCsandhisHMIdisplaytherewasadatadisconnect.Maybeitwasafieldbusproblem,orabadnetworkconnection.PerhapsitwascausedbyhisOPCserver,orpossiblyevenhisHMIsystem.Whateverthereason,sinceMelsdataconnectionwasasinglechain,onebreakinthechainmeansthathedidntgethisdata.Tominimizethiskindofriskandensurethehighestpossibleavailability,missioncriticalsystemsoftenuseredundancy.WhatisRedundancy?Redundancyinaprocesscontrolsystemmeansthatsomeorallofthesystemisduplicated,orredundant.Thegoalistoeliminate,asmuchaspossible,anysinglepointoffailure.Whenapieceofequipmentoracommunicationlinkgoesdown,asimilaroridenticalcomponentisreadytotakeover.Therearethreetypesofredundantsystems,categorizedbyhowquicklyareplacement(orstandby)canbebroughtonline.Thesearecoldstandby,warmstandby,andhotstandby.Coldstandbyimpliesthattherewillbeasignificanttimedelayingettingthereplacementsystemupandrunning.Thehardwareandsoftwareareavailable,butmayhavetobebootedupandloadedwiththeappropriatedata.Picturetheoldendaysofsteamlocomotives.Thecoldstandbywastheextraengineintheroundhousethathadtobefiredupandbroughtintoservice.Coldstandbyisnotusuallyusedforcontrolsystemsunlessthedatachangesveryinfrequently.Warmstandbyhasafasterresponsetime,becausethebackup(redundant)systemisalwaysrunning,andregularlyupdatedwitharecentcopyofthedataset.Whenafailureoccursontheprimarysystem,theredundantsystemcandisconnectfromthefailedsystemandconnectinsteadtothebackupsystem.Thisallowsthesystemtorecoverfairlyquickly(withinseconds,usually),andcontinuethework.Somedatawillbelostduringthisdisconnect/reconnectcycle,butwarmstandbycanbeanacceptablesolutionwheresomedatalosscanbetolerated.Hotstandbymeansthatboththeprimaryandsecondarydatasystemsrunsimultaneously,andbothareprovidingidenticaldatastreamstothedownstreamclient.Theunderlyingphysical



systemisthesame,butthetwodatasystemsuseseparatehardwaretoensurethatthereisnosinglepointoffailure.Whentheprimarysystemfails,theswitchovertothesecondarysystemisintendedtobecompletelyseamless,orbumpless,withnodataloss.Hotstandbyisthebestchoiceforsystemsthatcannottoleratethedatalossofacoldorwarmstandbysystem.ATypicalRedundantOPCSystemWhatdoesredundancylooklikeinanOPCbasedsystem?AtypicalscenariowouldhavetwoOPCserversconnectedeithertoasingledeviceorPLC,orpossiblyduplicatedevicesorPLCs.ThosetwoOPCserverswouldthenconnecttosomekindofOPCredundancymanagementsoftwarewhich,inturn,offersasingleconnectiontotheOPCclient,suchasanHMI.TheredundancymanagerisresponsibleforswitchingtothesecondaryOPCserverwhenanyproblemariseswiththedatacomingfromtheprimaryOPCserver.ThisscenariocreatesaredundantdatastreamfromthephysicalsystemallthewaytotheHMI.

ThemostcommonuseofredundancyinOPCisforOPCDA,butitispossibletoconfigureredundantOPCA&EorOPCUAsystems.Theprinciplesarethesame.Sometimes,onlargesystems,itisnecessarytoconfiguremultipleredundantpairs.Redundancycanalsobeconfiguredoveranetwork,usingDCOMorOPCtunneling.Foranetworkedconfiguration,theredundancymanagerwouldnormallyresideontheOPCclientmachine,tominimizethenumberofpotentialpointsoffailure.Althoughcoldorwarmstandbymaybeusefulundersomecircumstances,typicallyanengineerorsystemintegratorimplementingaredundantOPCsystemislookingforhotstandby.Thisisthemostusefulkindofredundancyinaprocesscontrolsystem,andatthesametimethemostdifficulttoachieve.LetslookalittlemorecloselyatthatallimportanttaskoftheOPCredundancymanagerinahotstandbysystemmakingtheswitch.

cpeme-cd00101Highlight

cpeme-cd00101Highlight

cpeme-cd00101CalloutOPC Data Access is a group of standards that provides specifications for communicating real-time data from data acquisition devices such as PLCs to display and interface devices like Human-Machine Interfaces (HMI).



MakingtheSwitchPutsimply,ahotstandbyredundancymanagerreceivesdatafromtwoidenticalinputs,andsendsasingleoutputtotheOPCclient.Itistheredundancymanagersjobtodetermineatalltimeswhichofthetwodatastreamsisthebest,andswitchfromonetotheotherassoonaspossiblewheneverthestatuschanges.Theswitchcanbetriggeredbyanumberofdifferentkindsofevents:

Singlepointvaluechangetoorfromacertainvalue,achievingathreshold,etc. Singlepointqualitychangeforexample,fromGoodtoanyotherOPCquality. Multipleitemmonitoringifthequalityorvalueofanypointinagroupgoesbad. Rateofchangemonitoringifpointschangevaluemoreslowlythanexpected. Networkbreaksandtimeoutscheckedwithsomekindofheartbeatmechanism.

Oncetheswitchhasoccurred,thesystemortheredundancymanageritselfmighthavetheabilitytosendanalarmoremailmessage,orevenlaunchsomekindofdiagnosticorinvestigativeprogram.ItmightalsobeabletologdiagnosticinformationaboutthestateoftheprimaryOPCserverornetworkconnection.Andinasystemthatdistinguishesbetweenprimaryandsecondaryinputs,therewilloftenbeameanstofavortheprimaryinput,andswitchbacktoitwhenpossible,sometimesreferredtoasafallback.

PracticalConsiderationsTheideaofredundancyisnotdifficulttograsp,butimplementingittakessomethought.Aninitialdecisiononcold,warmorhotstandbywillimpactallaspectsoftheimplementation.Thechoiceofproperhardwareandsoftwareiscriticalforawellfunctioningsystem.Robustsystemarchitectureisalsoimportant,especiallyiftheconnectionisacrossanetwork.InadditiontoselectingOPCserversandplanningthenetworkinfrastructure(ifnecessary),animportantdecisionwillbethesoftwareusedtomanagetheredundancy.Goodredundancymanagementsoftwareshouldbeeasytouse,withnoprogrammingnecessary.Thetechnologyshouldbeuptodate,capableofrunningonthelatestversionofWindows.Thereshouldbeanabsoluteminimumchanceofdatalossduringaswitchover,evenoveranetwork.TheTimerPitfallInpracticeitisnotpossibletoachieveacompletelyseamlessswitchoverinallcases,evenwithahotstandbysystem.Forexample,ifanetworkfailureoccursontheprimaryconnection,acertainamountoftimewillpassbeforearedundancymanagercandetectthatfailure.Datatransmittedduringthisperiodwillfailtoarrive,buttheredundancymanagerwillnotbeabletodistinguishbetweenafailureandanormalpauseindataflow.



Manyredundancymanagersimplementtimerstoperiodicallycheckthenetworkconnectionstatustotrytominimizethisdelay,butaswitchovermechanismbasedonperiodictimerswillalwayssufferfromdataloss.Systemswithmultipletimingparameterswilloftenresultinadditivedelays,wherethefastestpossibleswitchoverforthesystemisthesumofthesetimingdelays.Inaddition,theuseoftimerstodetectnetworkfailurecanresultinaconfigurationproblemwherethesystemintegratormusttradeoffswitchoverlatencyagainstfalsepositivenetworkfailuredetection.Thiseffectivelybecomesatradeoffbetweensystemstabilityandresponsiveness.Usingtimerstoperiodicallycheckdatavaluesorqualities,orpolltheOPCservers,isalsoproblematicbecausetimersintroduceunnecessarylatencyintothesystem.Whereasanetworkfailuremustbedetectedbasedontiming,adatavalueorqualitychangecanbedetectedimmediatelyastheeventoccurs.Itisusuallybesttoavoidsystemsbasedontimebasedvaluechangedetection,anduseeventbasedobjectmonitoringinstead.ObjectandLinkMonitoringAgoodredundancymanagershouldbeabletosupportbothobjectmonitoringandlinkmonitoring.Objectmonitoringmeanstheabilitytomonitorindividualpoints,andmakeaswitchoverbasedonanevent.Forexample,ifadesignatedwatchdogtagchangesinasignificantway,suchasturningnegativeorgoingoveraspecifiedthreshold,itcantriggeraswitchtothesecondaryOPCserver.Ormaybeyoudliketomonitoragroupofpoints,andifthequalityofanyofthemgoestoBadorUnconnected,youcanswitch.Linkmonitoringisespeciallyusefulfornetworkedconnections.Yoursystemwillneedawaytodetectanetworkbreakveryquickly,topreventdataloss.Forhotstandbyonhighspeedsystemswithfastdataupdaterates,timeoutdetectionwithasubsecondresponserateisessential.Inanyevent,thesystemshouldbeabletodetectatimeoutforafailednetworkconnection,aswellasafailuretoreceivedata.Thisdistinctionisimportant.Itmaytakesecondsorevenminutestodetectacommunicationfailure,butaredundancymanagershouldbeabletodetectastoppageofdataflowinanamountoftimeveryclosetothetruedataratefromthephysicalsystem.Theredundancymanagershouldbeabletoswitchfromonesourcetotheotherbasedsolelyonanobservationthatdatahasnotarrivedfromtheprimaryconnection,buthasarrivedfromthebackupsystem.SomesystemsuseCOMtimeoutsforlinkmonitoring.Thismaybeacceptableforcircumstanceswhererelativelylongdataoutagesaretolerable,butwedonotrecommendrelyingonCOMtimeoutsforhotorwarmstandby.SmartSwitchoverThebehavioroftheredundancysystemduringaswitchovercanbesignificant.Forexample,supposetheprimaryandsecondaryconnectionshavebothfailedforsomereason.AtypicalredundancymanagerwillbeginacycleofattemptingtoattachtooneandthentheotherOPC



serveruntiloneofthemresponds.Theredundancymanagerwillflipflopbetweenthetwoindefinitely,injectingsleepperiodsbetweeneachflipfloptoreducesystemresourceload.Thissleepperiodisitselfasourceoflatency.Asmarterswitchovermodelistomaintainasourcehealthstatusthatallowstheredundancymanagertoonlyswitchoverwhenasourcestatuschanges.Thisallowstheredundancymanagertoeffectivelyidle,orperformsimultaneousreconnectionattempts,untilasourcestatuschanges,thenimmediatelyrespondwithoutintroducingextralatency.Smarterswitchinglogiccanresultinsubstantiallyreducedsystemloadandswitchovertimes.ForcedSwitchingvsPreferredSourceItisusefultobeabletoselectonedatasourceoveranother,evenifthecurrentlyattachedsourceishealthy.Anaveredundancymanagerwillforcetheusertoswitch,evenifthebackupsystemisnotavailable.Thiswillagainresultinaflipflopbehaviorastheredundancymanagerattemptstoswitchtotheunavailablebackupsource.Amuchbetterapproachisfortheredundancymanagertounderstandtheconceptofapreferredsourcethatcanbechangedatruntime.Ifthepreferredsourceisavailable,theredundancymanagerwillswitchtoit.Iftheuserwantstoswitchfromonesourcetoanother,hesimplychangesthepreferredsource.Ifthatsourceisavailable,theswitchwillbemade.Ifitisnot,theredundancymanagerwillmaketheswitchonlywhenitbecomesavailable.Thiseliminatestheflipflopbehaviorwhileatthesametimeeliminatingthedatalossassociatedwiththeminimumoftwoswitchcyclesthatthenaveredundancymanagerwillimpose.AccessingRawDataAgoodhotredundancysystemwillgivetheclientapplicationaccessnotjusttotheredundantdata,butalsototherawdatafrombothsources.Thisgivestheclientapplicationtheoptionofpresentingdiagnosticinformationaboutthesystemonthefarsideoftheredundancymanager.Mostredundancymanagershidethisinformationsothataclientapplicationwouldhavetomakeandmanagemultipleconnectionstoaccesstherawdata,ifitispossibleatall.OtheroptionsandfeaturesInadditiontotheabovecapabilities,agoodredundancymanagermayofferadditionalfeaturesforyourconvenience.Itmightprovidetheoptiontorefreshtheentiredatasetatswitchover.Maybeitwillsendoutemailsorevenlaunchadditionalprogramsateachswitchover.Thiscanbeusefulfornotifyingkeypersonnelofthesystemstatus.Itmaylogdiagnosticstoprovidevaluableinformationaboutthereasonsformakingtheswitch.Someredundancymanagerscanconnecttomultipleservers,andcreatemultipleredundantconnections.Otherscanletyouworkwithsubsetsofthedata.Anotherdesirablefeatureistheabilitytoassigntheprimaryandsecondarydatasources,andtotriggerafallbackfromthesecondarytotheprimarydatasourceoncetheproblemthatcausedtheswitchoverhasbeenresolved.



Ascontrolsystemscontinuetogrowincomplexity,andaswerelymoreandmoreonthem,MelFarnsworthssituationwillbecomemorecommon,andmorecostly.Ifdataconnectivityiscrucialtothesuccessofthecompany,itwouldbewisetoconsiderthepossibilityofinstallingaredundantsystem,andtoweightheoptionscarefullywhenchoosingthekeycomponents.

Foundedin1995,CogentRealTimeSystemsprovidesversatileandreliablemiddlewareproductstoenablerealtimedataintegrationandaccessforindustrial,embedded,andfinancialsystems.CustomersincludeSiemens,ABB,Honeywell,IBM,GE,Statoil,Goodyear,BASF,CadburyChocolate,andtheBankofCanada.Formoreinformation,pleasecontactCogentatinfo@cogent.caorvisitourwebsiteatwww.cogentdatahub.com.Youcanalsocallusat+1(905)7027851.

opc-redundancy.pdf

Documents