1

EuropeanImmuno-OncologyClinicCompanionGrantID34704573EugenePozniak,ManagingDirector&CEOSiyemiLearningUniversityofManchesterInnovationCentreArch29NorthCampusIncubatorSackvilleStreet,ManchesterM601QDUnitedKingdomAbstractSiyemiLearningproposestheEuropeanImmuno-OncologyClinicCompanion(EIOCC),aneeds-based,multicomponentinitiativedesignedtosupportandfacilitateincreasedknowledge,competence,confidence,andperformanceamongthetargetpopulationofEuropeanmedicaloncologistsandothermembersofthecancercareteaminvolvedintheuseofimmuno-oncologytherapies.TheEuropeanIOCCinitiativealignswiththemissionofSiyemiLearningbyfocusingonthreekeycomponents:needs-baseddesign,collaboration,andinterprofessionallearning.First,theprojectbeginswithalearnerself-assessmentactivitytorefineandvalidatetheeducationalgoalsandintendedresultsoftheoverallinitiative.Inthedevelopmentofindividualactivities,theEIOCCinitiativeleveragesSiyemiLearning’srelationshipswithONCOassist,atechnologypartner,aswellastheEuropeanSocietyofMedicalOncology(ESMO)andEuropeanOncologyNursingSociety(EONS)forbroadlearnerrecruitment.Together,themixofactivitiesselectedfortheEIOCCinitiativewilladdresstheneedsoftheinterprofessionalcancerteam,includingmedicaloncologists,oncologynurses,andotherhealthcareprofessionalsinvolvedintheemergingfieldofimmuno-oncology.Lastly,acomprehensiveplanforoutcomesassessmentanddissemination,includingpublicationinthecontinuingmedicaleducation(CME)literature,ensuresthatthelessonslearnedfromthisinitiativecancontinuetopropelthefieldsofCMEandlearningscienceforward.Keywords:Immuno-oncology,clinicalpathways,case-basedlearning,immune-relatedadverseevents,auditandfeedback

2

TableofContentsSection Page

OverallGoal&Objectives 3

CurrentAssessmentofNeed 3

TargetAudience 5

ProjectDesignandMethods 7

EvaluationDesign 10

DetailedWorkplan 12

References 13

OrganizationalDetail 17

Appendix:DeliverablesSchedule 20

Appendix:BudgetNarrative 20

3

OverallGoal&ObjectivesSiyemiLearningproposestheEuropeanImmuno-OncologyClinicCompanion(EIOCC),aneeds-based,multicomponentinitiativedesignedtosupportandfacilitateincreasedknowledge,competence,confidence,andperformanceamongmedicaloncologistsandothermembersofthecancercareteaminvolvedintheuseofimmuno-oncologytherapies.TheEuropeanIOCCinitiativealignswiththemissionofSiyemiLearningbyfocusingon:

• Needs-baseddesign:theprojectbeginswithalearnerself-assessmentactivitytorefineandvalidatetheeducationalgoalsandintendedresultsoftheoverallinitiative

• Collaboration:theinitiativeleveragesSiyemiLearning’srelationshipswithONCOassist,atechnologypartner,aswellastheEuropeanSocietyofMedicalOncology(ESMO)andEuropeanOncologyNursingSociety(EONS)forbroadlearnerrecruitment

• Interprofessionallearning:activitieswilladdresstheneedsoftheinterprofessionalcancerteam,includingmedicaloncologists,oncologynurses,andotherhealthcareprofessionalsinvolvedintheemergingfieldofimmuno-oncology

Basedonourpreliminaryneedsassessment,wehaveidentifiedkeylearningobjectivesfortheoverallEIOCCinitiative:assesstheroleoftheimmunesysteminregulatingantitumoractivity;evaluatethemechanismsofaction(MOAs)ofimmuno-oncologyagentsandtheirroleincancertreatment;applythelatestclinicalresearchtotheselectionandsequencingofimmuno-oncologytherapiesinappropriatepatientsusingcase-basedexamples;andcollaboratewithteammemberstoidentifyopportunitiestomanageimmune-relatedadverseevents(irAEs)andreducesymptomburdenduringtreatmentwithimmuno-oncologyagents.CurrentAssessmentofNeedSiyemiLearningwillbeemployingatwo-foldstrategyforneedsassessmenttoguidetheEuropeanIOCC.First,wereviewedpublishedoncologistsurveydatatoidentifyneedsinthreeareas:foundationalknowledgeofimmuno-oncology,complexityofimmunotherapyselectionandsequencing,andmanagementofirAEs.ThisbroadanalysisincorporatesthefulltargetaudienceofEuropeanoncologiststoreflectallpotentialEIOCClearners.TheTargetAudiencesectionsummarizesadditionalneedsandopportunitiesintheUK,Spain,andItaly.Second,theEIOCCinitiativewillbeginwithalearnerself-assessmentactivitydesignedtovalidateknowledge,competence,andperformancegapsrelatedtoimmuno-oncology(seeProjectDesignandMethods,Phase0).Findingsfromthelearnerself-assessmentwilldefinetheeducationalgoalsofadditionalEIOCCactivities(Phases1aand1b).I.PreliminaryNeedsAssessmentFindingsGapArea#1:FoundationalKnowledgeofImmuno-OncologyWhatShouldBe:Oncologistsmaketreatmentdecisionsforpatientswithcancerbasedonasolidunderstanding,andconfidenceinknowledge,oftheimmunesystem,theMOAsofimmuno-oncologytherapies,andtheevidence-basedroleoftheseagentsincancercare.1,2WhatIs:Europeanoncologistsdemonstratesubstantialgapsinfoundationalknowledgenecessarytosupporttreatmentdecisionsaroundimmuno-oncology.3Inasurveyof169medicaloncologistsfromsixEuropeancountries,only35%describedthemselvesas“well-informed”onthetopicsofcancerimmunotherapyandimmuno-oncology.3Self-reported

4

knowledgegapswereconsistentacrossregionsandsubtopics:basicpathophysiologyoftheimmunesystem,useofbiomarkerstoguidetheselectionofimmunotherapy;MOAsofimmunecheckpointinhibitors;andtheuniqueresponsekineticsandtoxicitiesofimmuno-oncologyagents.Gapsinknowledgecanimpedetheselectionofevidence-basedtherapy.Inanothersurveyofmedicaloncologists,57%reportedthatitwasunlikelythattheywoulduseanewimmunotherapeuticagentwithoutunderstandingitsMOA.4Oncologists’knowledgegapsalsointerferewitheffectivepatientcommunication.Inaglobalsurveyof895oncologists,only23%feltthattheirpatientswerefullyinformedabouttheircanceranditstreatment.5Oncologistswholackfoundationalknowledgeinimmuno-oncologyarenotabletoprovidethenecessaryeducationtopatientsandtheirfamiliestosupportshareddecision-making(SDM).EducationalNeed:Todeliverevidence-basedcare,oncologistsrequireupdatesonnewinsightsonimmunesystempathophysiologyasitrelatestocancertreatment,relevanttumorpathophysiology,expectedantitumorresponseofnewimmuno-oncologyagents,andothertopicsnecessarytosupportclinicaldecision-makinginroutineoncologypractice.6-9GapArea#2:ComplexityofImmunotherapySelectionandSequencingWhatShouldBe:Asongoingclinicaltrialsmatureandknowledgeaboutimmuno-oncologyagentsadvances,oncologistsareabletoabsorb,process,andapplynewevidencetodecisionsaboutbiomarkertesting,first-linetherapy,andsubsequent-linetreatmentselection.WhatIs:Advancesinimmuno-oncologyaredevelopingatarapidpace,leavingoncologiststodecipherthebesttreatmentapproachforeachindividualpatient.Clinicaltrialsareincorporatingnewendpointstoreflecttheuniqueresponsekineticsofimmunotherapiesrelativetochemotherapeuticagents,makingthepracticalinterpretationofnewresearchresultsespeciallychallenging.10,11Further,asimmunotherapytreatmentdecisionsgrowincomplexity,SDMremainsacentralcomponentofqualitycancercare.12Inasurveyof5,315patientswithcolorectalorlungcancer,thosewhoperceivedtheironcologiststobeincontroloftreatmentdecisions(versusSDM)weresignificantlylesslikelytoreportexcellentqualityofcareorexcellentphysiciancommunication.12ModelstosupportSDMinoncologypracticeareemerging,butchallengesremain.13Communicationbetweenpatientsandoncologistscanbehamperedbypatients’misconceptionsregardingtreatmentexpectations,withmanypatientsbelievingimmunotherapyoffersa“cure”despitebeingtreatedinanon-curativesetting.14EducationalNeed:Oncologistsneedpracticalguidanceonhowtoapplythelatestclinicalevidenceregardingtheoptimaluseofimmuno-oncologyagentsacrossthespectrumofcancertherapy,giventhereal-worldcontextofmultidisciplinarycareandSDM.GapArea#3:ManagementofImmune-RelatedAdverseEventsWhatShouldBe:Oncologistsapplyaclearunderstandingoftheuniquetoxicityprofilesofimmuno-oncologyagentstothemanagementofpatientsreceivingthesetherapies.WhatIs:Inadequatemanagementofimmune-mediatedtoxicitiescaninterferewithoptimaldosing,adherence,andtreatmenteffectiveness.15,16Untilrecently,oncologistshavenothadclearguidanceonthemanagementofirAEs.Theavailabilityofnewguidelines,however,presentsanopportunitytoelevatethestandardsofcareforpatientsundergoingtreatmentwithimmuno-oncologyagents.TheEuropeanSocietyofMedicalOncology(ESMO)publishednewguidelinesonthemanagementofimmunotherapytoxicitiesinJuly2017,17andjoint

5

guidelinesfromtheAmericanSocietyofClinicalOncology(ASCO)andtheNationalComprehensiveCancerNetwork(NCCN)areexpectedlaterthisyear.18Althoughnewguidelinesarewelcometoolsforfacilitatingevidence-basedcare,oncologistswillbechallengedtoincorporatenewanddetailedalgorithmsonsideeffectmonitoringandmanagement.EducationalNeed:Oncologistswillneedpracticalcase-basedguidanceonimplementingnewrecommendationsinthecontextofotherclinicaldecision-makingaroundimmunotherapy.Managingpatientexpectationsaroundimmune-relatedeventsthrougheducationandcounselingisessentialforsuccessfultreatment.19II.RationaleforLearnerSelf-Assessment—Althoughpublishedsurveydataarecriticalforunderstandingclinicalgapsrelatedtoimmuno-oncology,surveysconductedeventwotothreeyearsagomaynolongerreflectcurrentlevelsofknowledge.Giventherapidpaceofadvancesinthisfield,aswellastheaccompanyingfloodofinformationaftereachmajoroncologycongress,itisreasonabletoexpectsomegainsinbasicimmuno-oncologyknowledge.Yetsomeoncologistswhoarechallengedtostayup-to-datewiththelatestadvancesmayshowpersistentgapsinknowledgeandcompetence.TheEIOCCLearnerSelf-Assessment(Phase0)willvalidateandquantifycurrentgaps.TargetAudienceFortheEIOCCLearnerSelf-Assessment(Phase0)andEIOCCImmuno-OncologyClinicinaBox(Phase1a),thetargetaudienceincludesEuropeanmedicaloncologistsandothermembersofthemultidisciplinaryandinterprofessionalcancercareteam.Learnerrecruitmentplansaredescribedattheendofthissection.OncologistsintheUK,Italy,andSpainwereselectedasthetargetaudiencefortheEIOCCClinicMentorPilotProgram(Phase1b)basedon:1)theavailabilityofrobustbaselinedatadocumentingcurrentneedsinoncologycare;and2)thepresenceofnationalacademiccentersand/ormedicalsocietiesfocusedonimmuno-oncology,demonstratingtheavailabilityofnationallyrecognizedfacultychampionsandanestablishedframeworkforcollaboration.Italy:CurrentChallenges—IntheEuropeansurveyofimmunotherapyknowledgeandpracticegaps,Italianoncologists(n=30)demonstratedaneedforeducationacrossarangeoftopics.3Inparticular,onlyaminorityofItalianoncologistsdescribedthemselvesaswell-informedontopicssuchastheimmunesystemandcarcinogenesis(40%),theroleofbiomarkersinpredictingresponsetoimmunotherapy(43%),andthemechanismsofactionofagentstargetingtheCTLA-4andPD-1signalingpathways(37%and34%,respectively).3BarrierstoeffectivecommunicationappearstobeachallengeintheItalianoncologysetting.Inasurveyof341Italianpatientswithbreast,lung,orcolorectalcancer,patientsratedtheirlevelsofsatisfactionon19domainsofcancercare.20Ofthese,“treatmentinformation”and“informationonsymptomsandlifestyle”receivedthelowestratings,suggestingpersistentcommunicationgapswithpatientsaroundtreatmentexpectations.20

6

Italy:EmergingOpportunities—AttheVenetoInstituteOncologicoVeneto(IOV)inPadova,Italy,researcherslaunchedapilotprogramofinnovativecost-containmentstrategies,includingcentralizeddrug-daycompounding,toaddressbarrierstoipilimumabtreatmentamongpatientswithmetastaticmelanoma.21FindingsfromtheIOVprogramhighlighttheimportanceofmultidisciplinarycollaborationamongoncologists,pharmacists,nurses,andtechnicians,inimprovingaccesstonewimmuno-oncologytherapies.21InanotherNorthernItalianpilotprogram,researchersfromcancercentersinTrento,Meldola,andBergamoareassessingthefeasibilityofmobileplatform(Onco-TreC)designedtosupporttreatmentadherence,symptomreporting,andtoxicitymanagementinpatientsreceivingoralanticancertherapiessuchassunitinib.22Asacomplementtotheseresearchprojects,severalItalianorganizationsarebecomingprominentvoicesinimmuno-oncology.In2016,theItalianAssociationofThoracicOncologypublishedgeneralguidanceontheuseofimmunotherapyinnon-smallcelllungcancer(NSCLC),aswellasaspecificupdateontheroleofnivolumabinsecond-lineNSCLCtreatment.23,24TheItalianNetworkforTumorBiotherapy(NetworkItalianoperlaBioterapiadeiTumori;NIBIT)isanonprofitconsortiumofmorethan40academic,regulatory,andindustrialgroupsfocusedonadvancesintumorimmunologyandimmuno-oncologytherapies.25,26Spain:CurrentChallenges—IntheEuropeansurveyofimmunotherapyknowledgeandpracticepatterns,Spanishoncologists(n=30)demonstratedespeciallylowratesofself-reportedknowledge.3Only20%feltwell-informedonthetopicoftheimmunesystemandcarcinogenesis;30%feltwell-informedaboutanti-CTLA-4therapies;and0%describedthemselvesaswell-informedaboutagentstargetingthePD-1pathway.3Thesefindingsdemonstrateapronounceddeficitinfoundationalknowledgenecessarytoincorporateimmuno-oncologyagentsintoroutineclinicalpractice.Spain:EmergingOpportunities—TheSpanishGroupforCancerImmuno-Biotherapy(GrupoEspañoldeTerapiasInmuno-BiologicasenCancer;GÉTICA)isanonprofitsocietyfocusedonimmunotherapydevelopmentandclinicianeducation.27GÉTICA’sgrowingcontributionstothefieldofimmuno-oncologyincludehostingannualscientificmeetingsanddevelopingguidelinesforendocrine-relatedAEmanagementinpatientsreceivingimmunecheckpointinhibitors.28In2017,theSpanishSocietyofMedicalOncology(SociedadEspañoladeOncologíaMédica;SEOM)outlinedthecurrentandfutureneedsofSpanishoncologists.29Basedonasurveyof176oncologistsandfindingsfromanexpertadvisoryboard,theSEOMidentified29keyprioritiesfortheSpanishoncologycommunitytomeetthegrowingdemandandcomplexityofoncologyservices.29SeveralSEOMrecommendationsalignwiththegoalsandmethodsoftheproposedEIOCCinitiative:advancethedefinitionofagreedprotocols/therapyguidelinesandpromotetheirimplementation;define,establishandmeasureusefulindicatorsfortheassessmentofcaredeliveryqualityandtheimpactofinnovationincorporation;increasetheuseofinformationandcommunicationtechnologiesforpromotinginter/multidisciplinarywork;andpromoteresearchinbiomarkers,immunotherapyandcombinedtherapies.UnitedKingdom:CurrentChallenges—SurvivaloutcomesamongpatientsdiagnosedwithcancerintheUKarefallingbehindthoseistherestofEurope.30Inarecentanalysisofcancermortalityin30countries(28memberstatesoftheEUplusNorwayandSwitzerland),theUK

7

hadtheworstsurvivalratesforovariancancerandthesecondlowestsurvivalratesforlungandpancreaticcancers.30Health-systembarrierssuchaspoormedicationaccesscontributetothesediscrepanciesintreatmentoutcomes.IntheUK,theincorporationofnewercancertherapiesintooncologypracticeisgenerallyslowerthaninotherG5membercountries(France,Germany,Italy,Spain).Thisisparticularlytrueofnewertargetedtherapiesandcheckpointinhibitors.Todate,theUKhashadthelowestusageratesofbevacizumab,imatinib,anderlotinibamongG5countries,andisinthemiddleofthegroupforipilimumabusage.30Poorknowledgeofimmunotherapiesmaycontributetosuboptimalcancertreatmentoutcomes.IntheEuropeanimmunotherapysurvey,thesubgroupofUKoncologists(n=30)wereunlikelytodescribethemselvesaswell-informedontheroleofbiomarkersaspredictorsofresponsetoimmunotherapy(33%),themechanismsofactionofagentstargetingCTLA-4(33%)orPD-1(20%),orthemanagementofimmunotherapy-relatedAEs(37%).3UnitedKingdom:EmergingOpportunities—TheCancerImmunologyandImmunotherapyCentre(CIIC)(www.qehb.org/ciic)isacollaborativeofapproximately30academicandclinicalresearchcentersbasedattheUniversityofBirminghamandtheQueenElizabethHospitalBirmingham.31TheCIICmemberorganizationsare“focusedondevelopinginternationallyoutstandingbasictumorimmunologyresearchandtranslatingkeydiscoveriesintoimprovementsincancertreatment.”AnewCentreforCancerImmunologyattheUniversityofSouthamptonwillexpandtheimmuno-oncologyinfrastructurewithintheUK.32LearnerRecruitment—ThroughourcollaborationwithONCOassist,SiyemiLearningwillengagewithotherONCOassistpartnersforrobustlearnerrecruitment.RecruitmentcampaignswilltargettheESMOmemberlistofmorethan13,000Europeanoncologyprofessionals,aswellastheEONSmemberlistof20,000oncologynursesworkinginEurope.ProjectDesignandMethodsTheoverallstrategyfortheEIOCCinitiativeincludes4phases:learnerself-assessment(Phase0),tworoundsofeducationalinterventions(PhaseIaandIb),andoutcomesevaluationandreporting(Phase2).Theeducationalactivitieswereselectedbasedontheircombinationoffeasibility,proveneffectiveness,andexpectedimpactonclinicalpractice.Therationaleforindividualcomponentsisexploredbelow.AllactivitiesandtoolsdevelopedthroughtheEIOCCwillbemadeavailablepublicallyatnocost.Activity DescriptionPhase0:EIOCCLearnerSelf-AssessmentOne(1)case-basedself-assessment1.0credit

Theinitiativewillbeginwithacase-basedself-assessmente-learningactivitydesignedto:1)assessalignmentofcurrentclinicalpracticewithguidelinesandevidence,2)identifyadditionalclinicalgapsandeducationalneeds,and3)validatetheeducationalgoals,learningobjectives,anddesiredresultsfortheoverallinitiative.LearnerswillberecruitedviaESMO/EONSmailinglists,TheChristie,andotherESMODesignatedCentresofIntegratedOncologyandPalliativeCareacrossEurope.Findingsfromaminimumof100learnerswillbeusedtoshapePhaseI/IIcontent.

8

PhaseIa:EIOCCImmuno-OncologyClinicinaBoxFour(4)knowledge-buildingmodules0.25creditseach

Learnerswillbeinvitedtoan“AssessYourKnowledge”Quiz,andfilteredtoparticipateinupto4knowledge-buildingmodulesfocusedon:1)areviewoftheimmunesystemincancertherapy;2)MOAsofimmuno-oncologyagents;3)immune-responsekinetics;and4)irAEs.Additionalresourcestoincludeaforumforcasediscussion,clinicalguidelines,sampleinstitutionalprotocols/ordersets,andpatienteducationmaterials

Three(3)livewebcasts1.0hourseach

Webcastswillprovidelearnerswithpracticalstrategiesforimplementingimmuno-oncologyclinicsintheircentres,withfacultyQ&A.

Web-basedclinicalpathway(non-certifiedcompaniontool)

Anewweb-basedtoolpoweredbyONCOassistanddesignedtoputthe2017ESMOguidelines(andpossiblytheexpectedASCO/NCCNjointguidelines)forirAEmanagementandotherrelevantteam-basedalgorithms.

Four(4)case-basedactivities0.5hourseach

Eachcasewilladdressaspecificclinicalchallengeidentifiedinthelearnerself-assessmentactivityandduringwebcastQ&As.Caseswilldemonstratethepoint-of-careuseoftheONCOassistapptosupportclinicaldecision-makingandreflectlatestclinicalevidence,includingupdatespresentedatthe2018ASCO,ESMO,andotherannualmeetings.

PhaseIb:EIOCCClinicMentorPilotProgramThree(3)A&FsessionsatpartnerclinicsintheUK,Spain,Italy2.0hourseach

TheEIOCCfacultymentorsfromtheUK,Spain,andItalywillidentifypartnerfacilitiestohoston-sitefacultyvisitsthatincorporateanelectronicA&Finterventioncustomizedtotheneedsofeachsetting.Electronicdatawillbecollectedandevaluatedpriortothevisit(‘audit’),allowingtimefor‘feedback’duringtheclinicvisititself.Additionalfollow-upfeedbackwillincludeemailsfromthefacultyreinforcingtheoriginalfeedbackmessages,andaddressingnewquestionsthatariseduringtheonsitevisit.

PhaseII:EIOCCImpactAssessmentandDisseminationMeetingabstractsandJECMEmanuscripts

Impactassessmentanddisseminationviameetingabstracts(e.g.,ASCO/ESMO2018and2019)andJECMEmanuscripts.AdditionaldetailsintheEvaluationDesignsection.

A&F,auditandfeedback;EIOCC,EuropeanImmuno-OncologyClinicCompanion;ESMO,EuropeanSocietyofMedicalOncology;irAE,immune-relatedadverseevents;JECME,JournalofEuropeanCME;NCCN,NationalComprehensiveCancerNetwork.Web-BasedToolkits:RationalefortheImmuno-OncologyClinicinaBoxCliniciansworldwideincreasinglyrelyonweb-basedlearningforcontinuingprofessionaldevelopment.33Accordingtothe2016ACCMEDataReport,onlineenduringactivitiesnowaccountformorethan50%ofalllearnerinteractionswithACCME-accreditedproviders.33Inastudyof383physiciansinScotland,94.3%reportedusinginternetresourcesonthreeormoreworkingdaysperweek,and87.4%rated‘completingonlinelearningmodules’amongthemostvaluedonlineactivities.34Ina2017,theEuropeanCancerPatientCoalitionendorsedthewider

9

useofinformationcommunicationtechnologies,includingmobilehealth(mHealth)tools,todelivermorepersonalizedcancercare.35SiyemiLearninghasappliedthelatestevidenceinadultlearningsciencetoshapetheweb-basedphaseofthismulti-modal,multi-exposureinitiative.36-38Comparedwithsingleinterventions,multifacetededucationalprogramsaremorelikelytoachievesignificantimprovementsinknowledgeandself-reportedintentiontochangepracticebehavior.39-41SiyemiLearningwilldevelopadedicatedEIOCCportaltohostallcomponentsoftheImmuno-OncologyClinicinBox,amulticomponentframeworkdesignedformedicaloncologistsacrossEuropetoadopt,customize,andimplementtoaddresstheiruniqueinstitutionalneeds.ClinicalPathways:RationalefortheEIOCCPathwayforirAEManagementClinicalpathwaysaredetailedprotocolsthattranslatecomplexguidelinesintoclearalgorithmsforpatientsundergoingspecifictypesoftreatment.42Whenusedatthepointofcare,clinicaloncologypathwaysareeffectivetoolsforimprovingpatient-providercommunicationaboutcomplextreatmentoptions.43Oncologypathwaysareincreasinglybeingincorporatedintoroutinepractice,withindividualcancercenters,hospitalnetworks,payors,andotherstakeholdersdevelopingpathways.43Manysuccessstoriesarealsoemerging.AttheDana-FarberCancerInstitute(DFCI)inBoston,MA,theimplementationoftheDFCIPathwayforstageIVNSCLCsignificantlyreducedthecostofNSCLCcarewhilepreservingclinicaloutcomes,includingoverallsurvival.44IntheUnitedStates,anestimated25%ofcancerpatientsarenowtreatedunderclinicaloncologypathways,upfrom15%in2010.43In2017,ASCOpublishedcriteriafordevelopinghigh-qualityoncologypathwayprograms,withanemphasisonprogramsthatareexpert-driven,transparent,patient-focused,up-to-date,andevidence-based.43ASCOalsorecommendscost-effectivetechnology,integrateddecision-supporttools(e.g.,linkstoordersets),andachievableoutcomes(e.g.,expectationsregardingpathwayadherence).43SiyemiLearningwillintegratethesecriteriaintothedevelopmentofaweb-basedEIOCCpathwayforirAEmanagementincollaborationwithONCOassist,apoint-of-caremobileappdesignedbyoncologyprofessionalstofacilitateevidence-basedclinicaldecision-making.45Unlikeotherpoint-of-careapps,ONCOassististheonlyappthatisclassifiedasamedicaldeviceandCE-approvedforuseinEuropeanEconomicAreacountriestoaidinclinicaldecisions.SiyemiLearninghasobtainedguidancefromtheACCMEtoensurethattheEIOCCweb-basedtoolwillbedevelopedincompliancewithallACCMEstandards.Case-BasedLearning:RationalefortheEIOCCCaseSeriesCase-basedactivitiesandsimulationsofclinicaldecisionmakingalsoenhancededucationaleffectiveness.46Point-of-carefeedback,whichinvolvespromptinglearnerswithrelevantguidelinerecommendationsatthetimeofclinicaldecision-making,improvesknowledgetransferandfacilitatestheadoptionofclinicalpracticeguidelines.47EachcaseintheEIOCCserieswillfeatureatleastoneclinicaldecisionpointdesignedtoinstructlearnersontheappropriateuseoftheweb-basedEIOCCirAEManagementPathway.ThecaseswillalsomodelSDMtechniquesthatsupportthediscussionofimmuno-oncologytreatmentdecisions.

10

Audit&Feedback:RationalefortheEIOCCClinicMentorPilotProgramHealthcareprofessionalscanoverestimatetheircompliancewithperformancestandardsby20%to30%.39Auditandfeedback(A&F)isthepracticeofprovidingobjectiveclinicalperformancesummariestohealthcareprofessionalswiththegoalofsupportinganevidence-basedchangeinpracticebehavior.48IntraditionalA&Fprograms,the‘audit’portioninvolvesthedirectobservationofaphysician’sclinicalperformance,followedbyface-to-face‘feedback’regardinganyaspectsofhisorherpracticethatisinconsistentwiththedesiredstandardsofcare.48Arecentmeta-analysisof140A&Fstudiesidentifiedseveralfeaturesthatincreasethelikelihoodofachievingasignificanteffectonprofessionalclinicalbehavior:thepersonconductingtheauditisasupervisororcolleague;feedbackisgivenbothverballyandinwriting,isprovidedmorethanonce,andincludescleartargetsandanactionplan;andtheA&Ffocusesonaspectsofclinicalbehaviorwithlowbaselineperformance48Withtheincreasingavailabilityofhealthdatainelectronicformat,electronicA&Fisemergingasanotheroptionforsupportingbehaviorchange.49Comparedwithstandardindividual-basedA&F,electronicA&Ffacilitatesfeedbacktoentirecareteams,departments,andfacilities.Thisapproachmayhavemorerelevanceforteam-basedcare,wheremultiplehealthcareprofessionals(oncologists,oncologynurses,technicians)areresponsibleforpatientoutcomes,andcareishighlycoordinated.49TworecentrandomizedtrialsofA&FinterventionsintheUKdemonstratedthefeasibilityofimprovingsafeprescribingbehaviorsamonghealthcareprofessionalsusingelectronicprescribingdatacomparedwithnationalsafetybenchmarksasthefocusoftheA&F.50,51AspartoftheEIOCCClinicMentorPilotProgram,SiyemiLearningandtheEIOCCfacultymentorswillcollaboratewitheachpartnerfacilityintheUK,Spain,andItalytodevelopanapproachtoA&Fthatismostsuitabletoeachsetting.TheA&Fwillincludeelementsthatincreasethelikelihoodofimprovingprofessionalpractice:afocusonpracticeswithlowbaselineperformancelevels;useofverbalandwrittenfeedback;multipleoccurrencesoffeedback;andfeedbackthatincludescleartargetsandanactionplan.48EvaluationDesignSiyemiLearninglookedbeyondtheCME/CPDliteraturetoexaminecurrentbestpracticesinevaluatingboththeimplementationandimpactofhealthprograms.53Educationalinterventionsthatarenotimplementedappropriatelywillnothavethedesirededucationalimpact.53Therefore,ateachinitiativephase,wewillcollectdatatoassessimplementation(processmeasures)andmeasuretheeducationalimpact(outcomesmeasures).Theoutcomesassessmentplanwill:1)determinewhetherlearninggoalshavebeenmet;and2)quantifythemagnitudeofeducationaleffect.EffectsizesforeachactivitywillbereportedwithinthecontextofMoore’sexpandedoutcomesframework,uptoknowledge,competence,andperformance,whereapplicable.54Further,theimpactanalysiswillconsiderthreatstointernalvalidityofthepretest/posttestdesign,includinghistoryandmaturation.53Finally,wewillmeasuretheclinicalimpactoftheseinitiativesbycalculatingthenumberofcancerlives

11

reached(i.e.,numberofpatientstreatedbyeachparticipant)relativetothemagnitudeofincreaseintreatmentdecisionsthatalignwiththestandardsofcare.

Intervention ProcessMeasures

AssessmentandReportingofEducationalImpact

OutcomesMeasures EffectSize Use&Dissemination

Phase0:EIOCCLearnerSelf-AssessmentLearnerself-assessmentactivity

Targetaudiencereach:learnerparticipationEvaluationfeedback:contentappropriatetopractice

Knowledge:Recognitionofkeysignalingpathways(e.g.,checkpointinhibition);MOAsofimmuno-oncologyagents;immunotherapyresponsekinetics;prevalenceandpathophysiologyofirAEs.

None;goalistovalidatepresumedgapsandidentifyadditionalgaps

OutcomestovalidategapsandguidethedevelopmentofEIOCCPhaseIa&Ibcontent

PhaseIa:EIOCCImmuno-OncologyClinicinaBoxKnowledge-buildingmodules

Targetaudiencereach:learnerparticipationEvaluationfeedback:contentappropriatetopractice

Knowledge:Recognitionofkeysignalingpathways(e.g.,checkpointinhibition);MOAsofimmuno-oncologyagents;immunotherapyresponsekinetics;prevalenceandpathophysiologyofirAEs.

40-60%increaseinknowledge

Individualactivityoutcomestobereportedasmeetingabstracts(e.g.,ESMO2018and2019);outcomesfromthefullImmuno-OncologyClinicinaBoxportionoftheEIOCCinitiativetobesubmittedtoJECMEAdditionalfindingswillinformfocusof2018EIOCCinterventions

Livewebcasts

Knowledge:Recognitionoftheplaceofimmuno-oncologyagentsincurrentclinicalpathways;recognitionofopportunitiesforteam-basedcare.Confidence:self-reportedconfidenceinimmuno-oncologytreatmentselectionandirAEmanagement

20-40%increaseinknowledge20-40%increaseinconfidence

Case-basedactivities

Competence:Selectionofevidence-basedtreatment

20-40%increaseincompetence

12

choicesincase-basedclinicalvignettes.Confidence:self-reportedconfidenceinimmuno-oncologytreatmentselectionandirAEmanagement

20-40%increaseinconfidence

PhaseIb:EIOCCClinicMentorPilotProgramFacultyexpertvisitwithelectronicA&F

Targetaudiencereach:facilityandindividuallearnerparticipationEvaluationfeedback:contentappropriatetopractice

Performance:compliancewithdefinedstandardsofcare(referralforPD-L1testing;treatmentselectionconsistentwithguidelines;documentationofirAEseverity;irAEmonitoringandmanagementconsistentwithESMOguidelines.

10-15%inratesofcompliancetooutcomesmeasures

OutcomesfromthefullEIOCCClinicMentorPilotProgramtobesubmittedtoJECME

PhaseIa:EIOCCImmuno-OncologyClinicinaBox—Estimatesofeffectsizesforinternet-basedCMEactivitiesvaryconsiderably,althougheffectsizesof0.2,0.5,and0.8arerecognizedthresholdsforsmall,medium,andlargeeffects,respectively.41,46Inonemeta-analysisof48internet-basedCMEactivities,themeaneffectsizewas0.75,withanincreaseof45%inevidence-baseddecision-makinginresponsetoclinicalcasevignettes.55PhaseIb:EIOCCClinicMentorPilotProgram—TraditionalandelectronicA&Fcanresultinmodestbutclinicallymeaningfulchangesinpracticebehavior.48,49Inthemeta-analysisoftraditionalA&Fstudies,themedianchangewasa4.3%absoluteincreaseincompliancewithdesiredpractice(interquartilerange,0.5%to16%).48Inanothermeta-analysisofelectronicA&F,theweightedoddsratioofcompliancewithdesiredpracticewas1.93(95%CI,1.36-2.73)whencomparingelectronicA&Fversusnointervention.49Absolutedifferencesintheratesofcompliancerangedfrom9.4%to14.9%betweentheelectronicA&Finterventionandcontrolgroups.49Basedonthesefindings,SiyemiLearningwillidentifyappropriatedichotomousprocessmeasurestoassesswhetherthecareprovidedisincompliancewithspecificstandards(outcomes:yes/no).Theexpectedeffectsizeisanincreaseof10%to15%intheobservedrateofcompliancewithspecificstandardsrelatedtoimmuno-oncologytreatmentfollowingtheA&Fintervention.DetailedWorkplanSiyemiLearningwillcommenceworkstartingOctober2017withcompletioninDecember2018.ThemilestonesforeachphasearedescribedintheDeliverablesschedule.Basedonthesuccessofthe2018activities,theEIOCCprojectmayexpandtoincludeagrowinglibraryofadditionalresources,andextendedintootherregionsin2019andbeyond.

13

References1. PandyaPH,MurrayME,PollokKE,RenbargerJL.Theimmunesystemincancer

pathogenesis:potentialtherapeuticapproaches.JImmunolRes.2016;2016:4273943.2. WhitesideTL,DemariaS,Rodriguez-RuizME,ZarourHM,MeleroI.Emergingopportunities

andchallengesincancerimmunotherapy.ClinCancerRes.2016;22:1845-1855.3. BorrelloIM,SchafferMM,RoehrlE,MarshallJF.Identificationofdifferencesin

immunotherapyknowledgeandpracticepatternsamongoncologistsfromsixEuropeancountries.MolClinOncol.2014;2:269-274.

4. MortimerJ,BowserA,ObholzK.Useofcross-platformoutcomestodocumentimprovementsandongoinggapsincancerimmunotherapyknowledge.AllianceforContinuingEducationintheHealthProfessions(ACEhp)2016AnnualMeeting.January13-16,2016;Washington,DC.

5. CiardielloF,AdamsR,TaberneroJ,etal.Awareness,understanding,andadoptionofprecisionmedicinetodeliverpersonalizedtreatmentforpatientswithcancer:amultinationalsurveycomparisonofphysiciansandpatients.Oncologist.2016;21:292-300.

6. BursteinHJ,KrilovL,Aragon-ChingJB,etal.ClinicalCancerAdvances2017:AnnualreportonprogressagainstcancerfromtheAmericanSocietyofClinicalOncology.JClinOncol.2017:Jco2016715292.

7. ChiouVL,BurottoM.Pseudoprogressionandimmune-relatedresponseinsolidtumors.JClinOncol.2015;33:3541-3543.

8. NishinoM.Immune-relatedresponseevaluationsduringimmune-checkpointinhibitortherapy:establishinga“commonlanguage”forthenewarenaofcancertreatment.JImmunotherCancer.2016;4:30.

9. SharmaP,Hu-LieskovanS,WargoJA,RibasA.Primary,adaptive,andacquiredresistancetocancerimmunotherapy.Cell.2017;168:707-723.

10.SeymourL,BogaertsJ,PerroneA,etal.iRECIST:guidelinesforresponsecriteriaforuseintrialstestingimmunotherapeutics.TheLancetOncology.18:e143-e152.

11.PavlovicM.ChallengesforrelativeeffectivenessassessmentandearlyaccessofcancerimmunotherapiesinEurope.FrontMed.2016;3:56.

12.KehlKL,LandrumMB,AroraNK,etal.Associationofactualandpreferreddecisionroleswithpatient-reportedqualityofcare:shareddecisionmakingincancercare.JAMAOncol.2015;1:50-58.

13.D'AmbruosoSF,CoscarelliA,HurvitzS,etal.Useofasharedmentalmodelbyateamcomposedofoncology,palliativecare,andsupportivecareclinicianstofacilitateshareddecisionmakinginapatientwithadvancedcancer.JOncolPract.2016;12:1039-1045.

14.GillespyK,DixonMD,PentzRD.Communicationaboutimmunotherapy:barriersandinformationtodiscuss.AmericanSocietyofClinicalOncology(ASCO)2017AnnualMeeting.June2-6,2017;Chicago,IL.Abstract6543.

15.KroschinskyF,StolzelF,vonBoninS,etal.Newdrugs,newtoxicities:severesideeffectsofmoderntargetedandimmunotherapyofcancerandtheirmanagement.CritCare.2017;21:89.

14

16. ByunDJ,WolchokJD,RosenbergLM,GirotraM.Cancerimmunotherapy-immunecheckpointblockadeandassociatedendocrinopathies.NatRevEndocrinol.2017;13:195-207.

17. HaanenJBAG,CarbonnelF,RobertC,etal.Managementoftoxicitiesfromimmunotherapy:ESMOClinicalPracticeGuidelinesfordiagnosis,treatmentandfollow-up.AnnOncol.2017;28:119-142.

18. AmericanSocietyofClinicalOncology(ASCO)PressRelease.Guidelinesplannedonmanagementofimmunotherapysideeffects:ASCOandNCCNtocollaborateondevelopment.February17,2017.

19. WeberJS,PostowM,LaoCD,SchadendorfD.Managementofadverseeventsfollowingtreatmentwithanti-programmeddeath-1agents.Oncologist.2016;21:1230-1240.

20. CarradoriT,BraviF,ButeraDS,etal.Continuityofcareinoncology.QuantitativeanalysisofdatafrompatientstreatedintwodifferentsettingsinEmilia--Romagna.RecentiProgMed2017;108:288-293.

21. RussiA,Chiarion-SileniV,DamuzzoV,etal.Casestudyonanipilimumabcost-containmentstrategyinanItalianhospital.IntJTechnolAssessHealthCare.2017;Epubaheadofprint.

22. PassardiA,RizzoM,MainesF,etal.Optimisationandvalidationofaremotemonitoringsystem(Onco-TreC)forhome-basedmanagementoforalanticancertherapies:anItalianmulticentrefeasibilitystudy.BMJOpen.2017;7:1-7.

23. GridelliC,AsciertoPA,BarberisMC,etal.Immunotherapyofnon-smallcelllungcancer:reportfromaninternationalexpertspanelmeetingoftheItalianAssociationofThoracicOncology.ExpertOpinBiolTher.2016;16:1479-1489.

24. GridelliC,BesseB,BrahmerJR,CrinoL,FelipE,deMarinisF.Theevolvingroleofnivolumabinnon-small-celllungcancerforsecond-linetreatment:anewcornerstoneforourtreatmentalgorithms.ResultsfromaninternationalexpertspanelmeetingoftheItalianAssociationofThoracicOncology.ClinLungCancer.2016;17:161-168.

25. RussoV,AmadoriA,BregniM,etal.GoalsandobjectivesoftheItalianNetworkforTumorBiotherapy(NIBIT).CytokineGrowthFactorRev.2017:Epubaheadofprint.

26. NetworkItalianoperlaBioterapiadeiTumori.AboutNIBIT.www.nibit.org.27. GrupoEspañoldeTerapiasInmuno-BiologicasenCancer.AboutGÉTICA.www.getica.org.28. Gonzalez-RodriguezE,Rodriguez-AbreuD.Immunecheckpointinhibitors:reviewand

managementofendocrineadverseevents.Oncologist.2016;21:804-816.29. RiveraF,AndresR,FelipE,etal.MedicaloncologyfutureplanoftheSpanishSocietyof

MedicalOncology:challengesandfutureneedsoftheSpanishoncologists.ClinTranslOncol.2017;19:508-518.

30. JönssonB,HofmarcherT,LindgrenP,MoenF&WilkingN.ComparatorreportonpatientaccesstocancermedicinesinEuroperevisited−AUKperspective.InstituteforHealthEconomics(IHE)Report2017:1.IHE:Lund,Sweden.

31. QueenElizabethHospitalBirmingham(QEHB)CancerImmunologyandImmunotherapyCentre.www.qehb.org/ciic.

32. UniversityofSouthamptonCentreforCancerImmunology.www.southampton.ac.uk/youreit.33. AccreditationCouncilforContinuingMedicalEducation(ACCME).ACCMEAnnualReport--

2016.http://www.accme.org/news-publications/publications/annual-report-data/accme-2016-data-report.UpdatedJuly12,2017.

15

34. MacWalterG,McKayJ,BowieP.Utilisationofinternetresourcesforcontinuingprofessionaldevelopment:across-sectionalsurveyofgeneralpractitionersinScotland.BMCMedEduc.2016;16:24-33.

35. EuropeanCancerPatientCoalition.ValueofInnovationinOncology.Brussels,Belgium:EuropeanCancerPatientCoalition,2017.www.ecpc.org/innovation.

36. DavisSW,Oakley-GirvanI.mHealthEducationApplicationsAlongtheCancerContinuum.JCancerEduc.2015;30:388-394.

37. SanaieyNZ.Thecomparativestudyoftheeffectivenessofusinge-learning,blendedlearningandpresencelearningincontinuousmedicaleducation.WorldJMedSci.2014;10:488-493.

38. CerveroRM,GainesJK.TheimpactofCMEonphysicianperformanceandpatienthealthoutcomes:anupdatedsynthesisofsystematicreviews.JContinEducHealProf.2015;35:131-138.

39. GrimshawJM,EcclesMP,LavisJN,HillSJ,SquiresJE.Knowledgetranslationofresearchfindings.ImplementationScience.2012;7:50.

40. YamadaJ,ShorkeyA,BarwickM,WidgerK,StevensBJ.Theeffectivenessoftoolkitsasknowledgetranslationstrategiesforintegratingevidenceintoclinicalcare:asystematicreview.BMJOpen.2015;5:1-10.

41. DeAngelisG,DaviesB,KingJ,etal.Informationandcommunicationtechnologiesforthedisseminationofclinicalpracticeguidelinestohealthprofessionals:asystematicreview.JMIRMedicalEducation.2016;2:e16.

42. ZonRT,FrameJN,NeussMN,etal.AmericanSocietyofClinicalOncologypolicystatementonclinicalpathwaysinoncology.JOncolPract.2016;12:261-266.

43. ZonRT,EdgeSB,PageRD,etal.AmericanSocietyofClinicalOncologycriteriaforhigh-qualityclinicalpathwaysinoncology.JOncolPract.2017;13:207-210.

44. JackmanDM,ZhangY,DalbyC,etal.CostandsurvivalanalysisbeforeandafterimplementationofDana-FarberClinicalPathwaysforpatientswithstageIVnon–small-celllungcancer.JOncolPract.2017;13:e346-e352.

45. ONCOassistOncologyDecisionSupportSystem.www.oncoassist.com.46. CerveroRM,GainesJK.TheimpactofCMEonphysicianperformanceandpatienthealth

outcomes:anupdatedsynthesisofsystematicreviews.JContinEducHealthProf.2015;35:131-138.

47. MenziesS,DuzJ,KinchR.Knowledgetransferatpointofcare:investigatingnewstrategiesforimplementingguidelinerecommendations.JContinEducHealProf.2015;35Suppl1:S22-S23.

48. IversN,JamtvedtG,FlottorpS,etal.Auditandfeedback:effectsonprofessionalpracticeandhealthcareoutcomes.CochraneDatabaseSystRev.2012:Cd000259.

49. TutiT,NzingaJ,NjorogeM,etal.Asystematicreviewofelectronicauditandfeedback:interventioneffectivenessanduseofbehaviourchangetheory.ImplementSci.2017;12:61.

50. GuthrieB,KavanaghK,RobertsonC,etal.Datafeedbackandbehaviouralchangeinterventiontoimproveprimarycareprescribingsafety(EFIPPS):multicentre,threearm,clusterrandomisedcontrolledtrial.BMJ.2016;354.

16

51. ElouafkaouiP,YoungL,NewlandsR,etal.Anauditandfeedbackinterventionforreducingantibioticprescribingingeneraldentalpractice:theRAPiDclusterrandomisedcontrolledtrial.PLOSMedicine.2016;13:e1002115.

52. TheAngiogenesisFoundation.Criticalpathwaysforwardinthetreatmentofrenalcellcarcinoma:reportfromanExpertSummitonrenalcellcarcinoma.July30-31,2012;Washington,DC.TheAngiogenesisFoundation;Cambridge,MA.

53. GrembowskiD.ThePracticeofHealthProgramEvaluation;2ndEdition.SagePublicationsInc.2016.

54. ConverseL,BarrettK,RichE,ReschovskyJ.Methodsofobservingvariationsinphysicians'decisions:theopportunitiesofclinicalvignettes.JGenInternMed.2015;30Suppl3:S586-594.

55. CasebeerL,EnglerS,BennettN,etal.Acontrolledtrialoftheeffectivenessofinternetcontinuingmedicaleducation.BMCMedicine.2008;6:37-37.

20

Appendix:DeliverablesScheduleDeliverable Planning Launch Completion

Phase0:EIOCCLearnerSelf-Assessment

Case-basedself-assessment October2017 December2017 February2018

PhaseIa:EIOCCImmuno-OncologyClinicinaBox

Knowledge-buildingmodules January2018 April2018 June2018

Livewebcasts February2018 May2018 July2018

Web-basedirAEtool November2018 March2018 NA

Case-basedactivities January2018 April2018 July2018

PhaseIb:EIOCCClinicMentorPilotProgramClinicvisitswithA&F May2018 June2018 August2018PhaseII:EIOCCImpactAssessmentandDisseminationMeetingabstractsandJECMEmanuscripts April2018 July2018 December2018