methods for efficient and adaptive scheduling of...

Fran

cisc

o Po

zo M

ETHO

DS FO

R EFFICIEN

T AN

D A

DA

PTIV

E SCH

EDU

LING

OF N

EXT-G

ENER

ATIO

N TIM

E-TRIG

GER

ED N

ETWO

RK

S 2019

ISBN 78-91-7485-436-7ISSN 1651-4238

Address: P.O. Box 883, SE-721 23 Västerås. SwedenAddress: P.O. Box 325, SE-631 05 Eskilstuna. SwedenE-mail: [email protected] Web: www.mdh.se

Mälardalen University Doctoral Dissertation 296

Methods for Efficient and Adaptive

Scheduling of Next-Generation

Time-Triggered Networks

Francisco Pozo

Mälardalen University Press DissertationsNo. 296

METHODS FOR EFFICIENT AND ADAPTIVE SCHEDULINGOF NEXT-GENERATION TIME-TRIGGERED NETWORKS

Francisco Manuel Pozo Pérez

2019

School of Innovation, Design and Engineering




2019

School of Innovation, Design and Engineering

1

Copyright © Francisco Manuel Pozo Pérez, 2019 ISBN 978-91-7485-436-7ISSN 1651-4238Printed by E-Print, Stockholm, Sweden

Copyright © Francisco Manuel Pozo Pérez, 2019 ISBN 978-91-7485-436-7ISSN 1651-4238Printed by E-Print, Stockholm, Sweden

2




Akademisk avhandling

som för avläggande av teknologie doktorsexamen i datavetenskap vidAkademin för innovation, design och teknik kommer att offentligen försvarastorsdagen den 24 oktober 2019, 13.30 i Milos, Mälardalens högskola, Västerås.

Fakultetsopponent: Professor Petru Eles, Linköping University

Akademin för innovation, design och teknik




Akademisk avhandling

som för avläggande av teknologie doktorsexamen i datavetenskap vidAkademin för innovation, design och teknik kommer att offentligen försvarastorsdagen den 24 oktober 2019, 13.30 i Milos, Mälardalens högskola, Västerås.

Fakultetsopponent: Professor Petru Eles, Linköping University

Akademin för innovation, design och teknik

3

AbstractReal-time networks play a fundamental role in embedded systems. To meet timing requirements, provide low jitter and bounded latency in such networks the time-triggered communication paradigm is frequently applied in such networks. In this paradigm, a schedule specifying the transmission times of all the traffic is synthesized a priori. Given the steady increase in size and complexity of embedded systems, coupled with the addition of wireless communication, a new time-triggered network model of larger and mixed wired-wireless network isdeveloping. Developing such next-generation networks entails significant research challenges, especially concerning scalability, i.e., allowing generation of schedules of the very large next-generation networks in a reasonable time. A second challenge concerns a well-known limitation of the time-triggered paradigm: its lack of flexibility. Large networks exacerbate this problem, as the number of changes during network operation increases with the number of components, which renders static scheduling approaches unsuitable.

In this thesis, we first propose a remedy to the scalability challenge that the synthesis of next-generation network schedules introduces. We propose a family of divide-and-conquer approaches that segment the entire scheduling problem into small enough subproblems that can be effectively and efficiently solved by state-of-the-art schedulers. Second, we investigate how adaptive behaviours can be introduced into the time-triggered paradigm with the implementation of a Self-Healing Protocol. This protocol addresses the flexibility challenge by only updating a small segment of the schedule in response to changes during runtime. This provides a significant advantage compared to current approaches that fully reschedule the network. In the course of our research, we found that our protocol become more effective when the slack in the original schedule is evenly distributed during the schedule synthesis. As a consequence, we also propose a new scheduling approach that maximizes the distances between frames, increasing the success rate of our protocol.

The divide-and-conquer approaches developed in this thesis were able to synthesize schedules of two orders of magnitude more traffic and one order of magnitude more nodes in less than four hours. Moreover, when applied to current industrial size networks, they reduced the synthesis time from half an hour to less than one minute compared with state-of-the-art schedulers. The Self-Healing Protocol opened a path towards adaptive time-triggered being able to heal schedules online after link and switch failures in less than ten milliseconds.

ISBN 978-91-7485-436-7ISSN 1651-4238

AbstractReal-time networks play a fundamental role in embedded systems. To meet timing requirements, provide low jitter and bounded latency in such networks the time-triggered communication paradigm is frequently applied in such networks. In this paradigm, a schedule specifying the transmission times of all the traffic is synthesized a priori. Given the steady increase in size and complexity of embedded systems, coupled with the addition of wireless communication, a new time-triggered network model of larger and mixed wired-wireless network isdeveloping. Developing such next-generation networks entails significant research challenges, especially concerning scalability, i.e., allowing generation of schedules of the very large next-generation networks in a reasonable time. A second challenge concerns a well-known limitation of the time-triggered paradigm: its lack of flexibility. Large networks exacerbate this problem, as the number of changes during network operation increases with the number of components, which renders static scheduling approaches unsuitable.

In this thesis, we first propose a remedy to the scalability challenge that the synthesis of next-generation network schedules introduces. We propose a family of divide-and-conquer approaches that segment the entire scheduling problem into small enough subproblems that can be effectively and efficiently solved by state-of-the-art schedulers. Second, we investigate how adaptive behaviours can be introduced into the time-triggered paradigm with the implementation of a Self-Healing Protocol. This protocol addresses the flexibility challenge by only updating a small segment of the schedule in response to changes during runtime. This provides a significant advantage compared to current approaches that fully reschedule the network. In the course of our research, we found that our protocol become more effective when the slack in the original schedule is evenly distributed during the schedule synthesis. As a consequence, we also propose a new scheduling approach that maximizes the distances between frames, increasing the success rate of our protocol.

The divide-and-conquer approaches developed in this thesis were able to synthesize schedules of two orders of magnitude more traffic and one order of magnitude more nodes in less than four hours. Moreover, when applied to current industrial size networks, they reduced the synthesis time from half an hour to less than one minute compared with state-of-the-art schedulers. The Self-Healing Protocol opened a path towards adaptive time-triggered being able to heal schedules online after link and switch failures in less than ten milliseconds.

ISBN 978-91-7485-436-7ISSN 1651-4238

4

PopularvetenskapligSammanfattning

Manga datorbaserade system bestar av noder som for att fullfolja sina uppgifterkommunicerar via meddelanden som skickas over natverk. I vissa systemracker det inte med att informationen ar korrekt, aven timingen for kommu-nikationen ar viktig. Dessa system kallas for realtidssystem. Ett exempel arsjalvkorande bilar. Om en autonom bil bromsar for tidigt eller forsent kan denorsaka en olycka medforande alvarliga skador pa saval manniskor som fordon.En etablerad metod for att uppna korrekt timing ar via tidsstyrd kommunikationdar det vid systemets design skapas ett ett schema som styr nar meddelandenskickas. Typisk skickas meddelandena periodiskt, t.ex. var tionde millisekund.Vid drift foljer systemnoderna schemat for att forsakra sig om att alla med-delanden skickas nar det ar tankt. Man kan jamfora detta med tagtidtabellersom ser till att tag kan ta sig fran punkt A till punkt B via ett antal mellanlig-gande stationer utan kollisioner med andra tag och sa att passagerarna (i vartfall informationen) kommer fram till ratt plats vid ratt tid.

Manga existerande och tidigare realtidssystem ar relativt sma, och hittast.ex. i bilar, plan, tag eller industriella maskiner. Komplexiteten for dessatillampningar har kontinuerligt vaxt, och framtidens realtidssystem, t.ex. i fab-riker med ett stort antal samarbetande produktionsceller eller for styrning ochkontroll av trafik och energianvandning i smarta stader, forvantas vara betydligtmer komplexa an dagens. Denna avhandling behandlar tva huvudproblemensom uppstar nar man applicerar tidsstyrd kommunikation i stora system: hogkomplexitet och lag flexibilitet.

Att ta fram det statiska schema som ar karnan i tidsstyrd kommu-nikation ar ett mycket komplext problem. En metod ar att oversattaschemalaggningsproblemet till ett motsvarande matematiskt problem, for att

i

PopularvetenskapligSammanfattning

Manga datorbaserade system bestar av noder som for att fullfolja sina uppgifterkommunicerar via meddelanden som skickas over natverk. I vissa systemracker det inte med att informationen ar korrekt, aven timingen for kommu-nikationen ar viktig. Dessa system kallas for realtidssystem. Ett exempel arsjalvkorande bilar. Om en autonom bil bromsar for tidigt eller forsent kan denorsaka en olycka medforande alvarliga skador pa saval manniskor som fordon.En etablerad metod for att uppna korrekt timing ar via tidsstyrd kommunikationdar det vid systemets design skapas ett ett schema som styr nar meddelandenskickas. Typisk skickas meddelandena periodiskt, t.ex. var tionde millisekund.Vid drift foljer systemnoderna schemat for att forsakra sig om att alla med-delanden skickas nar det ar tankt. Man kan jamfora detta med tagtidtabellersom ser till att tag kan ta sig fran punkt A till punkt B via ett antal mellanlig-gande stationer utan kollisioner med andra tag och sa att passagerarna (i vartfall informationen) kommer fram till ratt plats vid ratt tid.

Manga existerande och tidigare realtidssystem ar relativt sma, och hittast.ex. i bilar, plan, tag eller industriella maskiner. Komplexiteten for dessatillampningar har kontinuerligt vaxt, och framtidens realtidssystem, t.ex. i fab-riker med ett stort antal samarbetande produktionsceller eller for styrning ochkontroll av trafik och energianvandning i smarta stader, forvantas vara betydligtmer komplexa an dagens. Denna avhandling behandlar tva huvudproblemensom uppstar nar man applicerar tidsstyrd kommunikation i stora system: hogkomplexitet och lag flexibilitet.

Att ta fram det statiska schema som ar karnan i tidsstyrd kommu-nikation ar ett mycket komplext problem. En metod ar att oversattaschemalaggningsproblemet till ett motsvarande matematiskt problem, for att

i

5

ii

sedan mata in det i en sa-kallad SMT losare, som returnerar overforingstidernafor alla meddelanden. Att losa detta problem for system som ar tiofalt storreen dagens. I denna avhandling presenterar vi en losning dar problemet de-las upp i mindre, mer hanterbara problem. For att t.ex. skapa ett schema foren hel timme, kan vi dela upp problemet i 60 scheman som vardera omfat-tar en minut. Den bristande flexibiliteten har sin grund i schemalaggningensbristande formaga att stodja forandringar i systemet, sasom att lagga till nyakomponenter eller hantera fel som uppstar. En mojlig losning for felhantering,som t.ex. anvands i flygplan, ar kommunikation over tre olika natverk – omfel uppstar i ett av natverken sa kommer informationen fram i ett av de andranatverken, men for natverk som t.ex. spanner over en hel stad sa ar den typenav losning alldeles for kostsam. Ett alternativ ar att vid fel utarbeta ett nyttschema som kringgar problemet. Men att ta fram ett nytt schema for ett stortsystem tar flera timmar, vilket ar alltfor lang tid. Vi foreslar en metod som baraandrar den del av schemat som paverkas av felet, vilket kan goras betydligtsnabbare.

Vi har utvarderat vara metoder for schemalaggning och felhantering i ettstort antal scenarier. Resultaten ar mycket lovande och indikerar att vi kanhantera tio ganger sa stora system jamfort med vad som kan hanteras medexisterande metoder. Vi kan dessutom med mycket stor sannolikhet justeraschemat vid kommunikationsfel utan att stora driften.

ii

sedan mata in det i en sa-kallad SMT losare, som returnerar overforingstidernafor alla meddelanden. Att losa detta problem for system som ar tiofalt storreen dagens. I denna avhandling presenterar vi en losning dar problemet de-las upp i mindre, mer hanterbara problem. For att t.ex. skapa ett schema foren hel timme, kan vi dela upp problemet i 60 scheman som vardera omfat-tar en minut. Den bristande flexibiliteten har sin grund i schemalaggningensbristande formaga att stodja forandringar i systemet, sasom att lagga till nyakomponenter eller hantera fel som uppstar. En mojlig losning for felhantering,som t.ex. anvands i flygplan, ar kommunikation over tre olika natverk – omfel uppstar i ett av natverken sa kommer informationen fram i ett av de andranatverken, men for natverk som t.ex. spanner over en hel stad sa ar den typenav losning alldeles for kostsam. Ett alternativ ar att vid fel utarbeta ett nyttschema som kringgar problemet. Men att ta fram ett nytt schema for ett stortsystem tar flera timmar, vilket ar alltfor lang tid. Vi foreslar en metod som baraandrar den del av schemat som paverkas av felet, vilket kan goras betydligtsnabbare.

Vi har utvarderat vara metoder for schemalaggning och felhantering i ettstort antal scenarier. Resultaten ar mycket lovande och indikerar att vi kanhantera tio ganger sa stora system jamfort med vad som kan hanteras medexisterande metoder. Vi kan dessutom med mycket stor sannolikhet justeraschemat vid kommunikationsfel utan att stora driften.

6

Abstract

Real-time networks play a fundamental role in embedded systems. To meettiming requirements, provide low jitter and bounded latency in such networksthe time-triggered communication paradigm is frequently applied in such net-works. In this paradigm, a schedule specifying the transmission times of all thetraffic is synthesized a priori. Given the steady increase in size and complexityof embedded systems, coupled with the addition of wireless communication, anew time-triggered network model of larger and mixed wired-wireless networkis developing. Developing such next-generation networks entails significantresearch challenges, especially concerning scalability, i.e., allowing generationof schedules of the very large next-generation networks in a reasonable time.A second challenge concerns a well-known limitation of the time-triggeredparadigm: its lack of flexibility. Large networks exacerbate this problem, asthe number of changes during network operation increases with the number ofcomponents, which renders static scheduling approaches unsuitable.

In this thesis, we first propose a remedy to the scalability challenge thatthe synthesis of next-generation network schedules introduces. We propose afamily of divide-and-conquer approaches that segment the entire schedulingproblem into small enough subproblems that can be effectively and efficientlysolved by state-of-the-art schedulers. Second, we investigate how adaptive be-haviours can be introduced into the time-triggered paradigm with the imple-mentation of a Self-Healing Protocol. This protocol addresses the flexibilitychallenge by only updating a small segment of the schedule in response tochanges during runtime. This provides a significant advantage compared tocurrent approaches that fully reschedule the network. In the course of our re-search, we found that our protocol become more effective when the slack inthe original schedule is evenly distributed during the schedule synthesis. As aconsequence, we also propose a new scheduling approach that maximizes thedistances between frames, increasing the success rate of our protocol.

iii

Abstract

Real-time networks play a fundamental role in embedded systems. To meettiming requirements, provide low jitter and bounded latency in such networksthe time-triggered communication paradigm is frequently applied in such net-works. In this paradigm, a schedule specifying the transmission times of all thetraffic is synthesized a priori. Given the steady increase in size and complexityof embedded systems, coupled with the addition of wireless communication, anew time-triggered network model of larger and mixed wired-wireless networkis developing. Developing such next-generation networks entails significantresearch challenges, especially concerning scalability, i.e., allowing generationof schedules of the very large next-generation networks in a reasonable time.A second challenge concerns a well-known limitation of the time-triggeredparadigm: its lack of flexibility. Large networks exacerbate this problem, asthe number of changes during network operation increases with the number ofcomponents, which renders static scheduling approaches unsuitable.

In this thesis, we first propose a remedy to the scalability challenge thatthe synthesis of next-generation network schedules introduces. We propose afamily of divide-and-conquer approaches that segment the entire schedulingproblem into small enough subproblems that can be effectively and efficientlysolved by state-of-the-art schedulers. Second, we investigate how adaptive be-haviours can be introduced into the time-triggered paradigm with the imple-mentation of a Self-Healing Protocol. This protocol addresses the flexibilitychallenge by only updating a small segment of the schedule in response tochanges during runtime. This provides a significant advantage compared tocurrent approaches that fully reschedule the network. In the course of our re-search, we found that our protocol become more effective when the slack inthe original schedule is evenly distributed during the schedule synthesis. As aconsequence, we also propose a new scheduling approach that maximizes thedistances between frames, increasing the success rate of our protocol.

iii

7

iv

The divide-and-conquer approaches developed in this thesis were able tosynthesize schedules of two orders of magnitude more traffic and one order ofmagnitude more nodes in less than four hours. Moreover, when applied to cur-rent industrial size networks, they reduced the synthesis time from half an hourto less than one minute compared with state-of-the-art schedulers. The Self-Healing Protocol opened a path towards adaptive time-triggered being able toheal schedules online after link and switch failures in less than ten millisec-onds.

iv

The divide-and-conquer approaches developed in this thesis were able tosynthesize schedules of two orders of magnitude more traffic and one order ofmagnitude more nodes in less than four hours. Moreover, when applied to cur-rent industrial size networks, they reduced the synthesis time from half an hourto less than one minute compared with state-of-the-art schedulers. The Self-Healing Protocol opened a path towards adaptive time-triggered being able toheal schedules online after link and switch failures in less than ten millisec-onds.

8

To my Family To my Family

9

Acknowledgments

I had the pleasure to start my studies belonging to the RetNet project, formedby a group of remarkable people with whom I could share the same experi-ences. The students (Ayhan, Elena, Marina and Pablo) were from scary andinhospitable northern places such as Bulgaria, Russia or even from far San-tander, Spain. But they invited me to their homes and showed me that northernpeople are similar to us, they just own warmer clothes. I regret that I could notreciprocate their invitations yet, but I have to admit it is an excellent incentiveto meet again in Mallorca. My supervisors also belonged to this project family,Hans, Guillermo and Wilfried, thanks for teaching me a bit about scheduling,and a lot about much more important skills. I also want to express my gratitudefor the rest of the project members, Mats, Elisabeth, Radu, Sasikumar, Johan,Arjan, Caroline, Christian, Ann-Catrin and Carolina for their support.

I also have shared many good experiences with many people during thewriting of this thesis, people from MDH, TTTech, RISE and ABB. I know youwant to read your name here. As a way to have some fun and a break fromthesis writing, I obtained a ranking of people I received more emails from,so here we go. Thanks to Guillermo, Hans, Pablo, Damir, Wilfried, Elena,Cristina, Svetlana, Voica, Marina, Federico, Thomas, Radu, Leo, Gita, Mats,Moris, Elisabeth, Saad, Momo, Ayhan, Predrag, Mehrdad, Sasikumar, Alessio,Irfan and many others I received less than 30 emails from.

I know David you won in the marrying and buying an apartment race, andIsma, you surprised us and won in the parenting race, but at least this one ismine. They also belong to an old group of friends in my hometown, FD (whosemeaning I am too ashamed to share here), we had and continue to have a greattime together. I have to admit that I chicken out when they decided to tattoothe group name for eternity, but now they are on my thesis, so there are thatmamones. I have left the best for the last, my family, thanks for understandingme in every decision I made. I know it has been hard when I went abroad, but

vii

Acknowledgments

I had the pleasure to start my studies belonging to the RetNet project, formedby a group of remarkable people with whom I could share the same experi-ences. The students (Ayhan, Elena, Marina and Pablo) were from scary andinhospitable northern places such as Bulgaria, Russia or even from far San-tander, Spain. But they invited me to their homes and showed me that northernpeople are similar to us, they just own warmer clothes. I regret that I could notreciprocate their invitations yet, but I have to admit it is an excellent incentiveto meet again in Mallorca. My supervisors also belonged to this project family,Hans, Guillermo and Wilfried, thanks for teaching me a bit about scheduling,and a lot about much more important skills. I also want to express my gratitudefor the rest of the project members, Mats, Elisabeth, Radu, Sasikumar, Johan,Arjan, Caroline, Christian, Ann-Catrin and Carolina for their support.

I also have shared many good experiences with many people during thewriting of this thesis, people from MDH, TTTech, RISE and ABB. I know youwant to read your name here. As a way to have some fun and a break fromthesis writing, I obtained a ranking of people I received more emails from,so here we go. Thanks to Guillermo, Hans, Pablo, Damir, Wilfried, Elena,Cristina, Svetlana, Voica, Marina, Federico, Thomas, Radu, Leo, Gita, Mats,Moris, Elisabeth, Saad, Momo, Ayhan, Predrag, Mehrdad, Sasikumar, Alessio,Irfan and many others I received less than 30 emails from.

I know David you won in the marrying and buying an apartment race, andIsma, you surprised us and won in the parenting race, but at least this one ismine. They also belong to an old group of friends in my hometown, FD (whosemeaning I am too ashamed to share here), we had and continue to have a greattime together. I have to admit that I chicken out when they decided to tattoothe group name for eternity, but now they are on my thesis, so there are thatmamones. I have left the best for the last, my family, thanks for understandingme in every decision I made. I know it has been hard when I went abroad, but

vii

11

viii

even thousands of kilometres away, I have always felt you were next to me, andwithout your support, I could not have achieved this.

This work was supported by the People Programme (Marie Curie Ac-tions) of the European Union’s Seventh Framework Programme; project Ret-Net (Grant Agreement Number 607727).

Francisco PozoSeptember, 2019

Vasteras, Sweden

viii

even thousands of kilometres away, I have always felt you were next to me, andwithout your support, I could not have achieved this.

This work was supported by the People Programme (Marie Curie Ac-tions) of the European Union’s Seventh Framework Programme; project Ret-Net (Grant Agreement Number 607727).

Francisco PozoSeptember, 2019

Vasteras, Sweden

12

List of publications

Papers included in the doctoral thesis1

Paper A SMT-based Synthesis of TTEthernet Schedules: a PerformanceStudy, Francisco Pozo, Guillermo Rodriguez-Navas, Hans Hansson,Wilfried Steiner. In Proceedings of the 10th IEEE International Sympo-sium on Industrial Embedded Systems (SIES), Siegen, Germany, June2015.

Paper B Methods for Large-Scale Time-Triggered Network Scheduling, Fran-cisco Pozo, Guillermo Rodriguez-Navas, Hans Hansson. MDPI Elec-tronics, June 2019.

Paper C Schedule Reparability: Enhancing Time-Triggered Network Recov-ery upon Link Failures, Francisco Pozo, Guillermo Rodriguez-Navasand Hans Hansson. In Proceedings of the 24th IEEE International Con-ference on Embedded and Real-Time Computing Systems and Applica-tions (RTCSA), Hakodate, Japan, August 2018.

Paper D Self-Healing Protocol: Repairing Schedules Online after LinkFailures in Time-Triggered Networks, Francisco Pozo, GuillermoRodriguez-Navas and Hans Hansson. MRTC Report, MDH-MRTC-327/2019-1-SE, Malardalen Real-Time Research Centre, MalardalenUniversity, September, 2019. Submitted to IEEE Transactions in Indus-trial Informatics, September, 2019, third review round.

Paper E Semi-Distributed Self-Healing Protocol for Online Schedule Re-pair after Network Failures, Francisco Pozo, Guillermo Rodriguez-Navas and Hans Hansson. MRTC Report, MDH-MRTC-326/2019-1-SE,

1The included articles have been reformatted to comply with the doctoral thesis layout.

ix

List of publications

Papers included in the doctoral thesis1

Paper A SMT-based Synthesis of TTEthernet Schedules: a PerformanceStudy, Francisco Pozo, Guillermo Rodriguez-Navas, Hans Hansson,Wilfried Steiner. In Proceedings of the 10th IEEE International Sympo-sium on Industrial Embedded Systems (SIES), Siegen, Germany, June2015.

Paper B Methods for Large-Scale Time-Triggered Network Scheduling, Fran-cisco Pozo, Guillermo Rodriguez-Navas, Hans Hansson. MDPI Elec-tronics, June 2019.

Paper C Schedule Reparability: Enhancing Time-Triggered Network Recov-ery upon Link Failures, Francisco Pozo, Guillermo Rodriguez-Navasand Hans Hansson. In Proceedings of the 24th IEEE International Con-ference on Embedded and Real-Time Computing Systems and Applica-tions (RTCSA), Hakodate, Japan, August 2018.

Paper D Self-Healing Protocol: Repairing Schedules Online after LinkFailures in Time-Triggered Networks, Francisco Pozo, GuillermoRodriguez-Navas and Hans Hansson. MRTC Report, MDH-MRTC-327/2019-1-SE, Malardalen Real-Time Research Centre, MalardalenUniversity, September, 2019. Submitted to IEEE Transactions in Indus-trial Informatics, September, 2019, third review round.

Paper E Semi-Distributed Self-Healing Protocol for Online Schedule Re-pair after Network Failures, Francisco Pozo, Guillermo Rodriguez-Navas and Hans Hansson. MRTC Report, MDH-MRTC-326/2019-1-SE,

1The included articles have been reformatted to comply with the doctoral thesis layout.

ix

13

x

Malardalen Real-Time Research Centre, Malardalen University, Septem-ber, 2019. Submitted to IEEE Transactions in Industrial Informatics,September 2019.

x

Malardalen Real-Time Research Centre, Malardalen University, Septem-ber, 2019. Submitted to IEEE Transactions in Industrial Informatics,September 2019.

14

xi

Additional papers, not included in the doctoralthesis

1. A Decomposition Approach for SMT-based Schedule Synthesis forTime-Triggered Networks, Francisco Pozo, Wilfried Steiner, GuillermoRodriguez-Navas, and Hans Hansson. In Proceedings of the 20th IEEEConference on Emerging Technologies and Factory Automation (ETFA),Luxembourg, September, 2015.

2. Current Techniques, Trends, and New Horizons in Avionics NetworksConfiguration, Wilfried Steiner, Marina Gutierrez, Zoltan Matyas, Fran-cisco Pozo, Guillermo Rodriguez-Navas. In Proceedings of the 34th

IEEE/AIAA Digital Avionics Systems Conference (DASC), Prague,Czech Republic, September 2015.

3. Period-Aware Segmented Synthesis of Schedules for Multi-Hop Time-Triggered Networks, Francisco Pozo, Guillermo Rodriguez-Navas, Wil-fried Steiner, and Hans Hansson. In Proceedings of the 22nd IEEE Inter-national Conference on Embedded Real-Time Computing Systems andApplications (RTCSA), Daegu, South Korea, August, 2016.

4. Next Generation Real-Time Networks Based on IT Technologies, Wil-fried Steiner, Pablo Gutierrez Peon, Marina Gutierrez, Ayhan Mehmed,Guillermo Rodriguez-Navas, Elena Lisova, Francisco Pozo. In Pro-ceedings of the 21st IEEE International Conference on Emergin Tech-nologies and Factory (ETFA), Berlin, Germany, September 2016.

5. Cognitive Radio for Improved Reliability in a Real-Time WirelessMAC Protocol based on TDMA, Pablo Peon Gutierrez, Pedro ManuelRodrıguez, Zaloa Fernandez, Francisco Pozo, Elisabeth Uhlemann,Inaki Val, and Wilfried Steiner. In Proceedings of the 22th IEEE Confer-ence on Emerging Technologies and Factory Automation (ETFA), Ly-massol, Cyprus, September, 2017.

6. Work-in-Progress: A Hot-Patching Protocol for Repairing Time-Triggered Network Schedules, Francisco Pozo, Guillermo Rodriguez-Navas and Hans Hansson. In Proceedings of the 24th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS),Porto, Portugal, April, 2018.

xi

Additional papers, not included in the doctoralthesis

1. A Decomposition Approach for SMT-based Schedule Synthesis forTime-Triggered Networks, Francisco Pozo, Wilfried Steiner, GuillermoRodriguez-Navas, and Hans Hansson. In Proceedings of the 20th IEEEConference on Emerging Technologies and Factory Automation (ETFA),Luxembourg, September, 2015.

2. Current Techniques, Trends, and New Horizons in Avionics NetworksConfiguration, Wilfried Steiner, Marina Gutierrez, Zoltan Matyas, Fran-cisco Pozo, Guillermo Rodriguez-Navas. In Proceedings of the 34th

IEEE/AIAA Digital Avionics Systems Conference (DASC), Prague,Czech Republic, September 2015.

3. Period-Aware Segmented Synthesis of Schedules for Multi-Hop Time-Triggered Networks, Francisco Pozo, Guillermo Rodriguez-Navas, Wil-fried Steiner, and Hans Hansson. In Proceedings of the 22nd IEEE Inter-national Conference on Embedded Real-Time Computing Systems andApplications (RTCSA), Daegu, South Korea, August, 2016.

4. Next Generation Real-Time Networks Based on IT Technologies, Wil-fried Steiner, Pablo Gutierrez Peon, Marina Gutierrez, Ayhan Mehmed,Guillermo Rodriguez-Navas, Elena Lisova, Francisco Pozo. In Pro-ceedings of the 21st IEEE International Conference on Emergin Tech-nologies and Factory (ETFA), Berlin, Germany, September 2016.

5. Cognitive Radio for Improved Reliability in a Real-Time WirelessMAC Protocol based on TDMA, Pablo Peon Gutierrez, Pedro ManuelRodrıguez, Zaloa Fernandez, Francisco Pozo, Elisabeth Uhlemann,Inaki Val, and Wilfried Steiner. In Proceedings of the 22th IEEE Confer-ence on Emerging Technologies and Factory Automation (ETFA), Ly-massol, Cyprus, September, 2017.

6. Work-in-Progress: A Hot-Patching Protocol for Repairing Time-Triggered Network Schedules, Francisco Pozo, Guillermo Rodriguez-Navas and Hans Hansson. In Proceedings of the 24th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS),Porto, Portugal, April, 2018.

15

xii

7. A Semi-Distributed Self-Healing Protocol for Run-Time Repairs of Time-Triggered Schedules, Francisco Pozo and Guillermo Rodriguez-Navas.In Proceedings of the 24th IEEE Conference on Emerging Technologiesand Factory Automation (ETFA), Zaragoza, Spain, September, 2019.

xii

7. A Semi-Distributed Self-Healing Protocol for Run-Time Repairs of Time-Triggered Schedules, Francisco Pozo and Guillermo Rodriguez-Navas.In Proceedings of the 24th IEEE Conference on Emerging Technologiesand Factory Automation (ETFA), Zaragoza, Spain, September, 2019.

16

Contents

I Thesis 1

1 Introduction 31.1 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Background 72.1 Time-Triggered Communication Paradigm . . . . . . . . . . . 7

2.1.1 Standards and Protocols for Time-Triggered Networks 92.1.2 Time-Triggered Switched Network: System Model . . 11

2.2 Time-Triggered Scheduling . . . . . . . . . . . . . . . . . . . 132.2.1 Scheduling Tools . . . . . . . . . . . . . . . . . . . . 142.2.2 Time-Triggered Scheduling Constraint Definition . . . 15

2.3 Discussion on the System Model Selected . . . . . . . . . . . 19

3 Related Work 213.1 Scheduling Scalability . . . . . . . . . . . . . . . . . . . . . 21

3.1.1 Scheduling for Time-Triggered Networks . . . . . . . 213.1.2 Divide-and-Conquer Approaches . . . . . . . . . . . 24

3.2 Adaptive Time-Triggered Networks . . . . . . . . . . . . . . 25

4 Problem Formulation 274.1 Research Problem . . . . . . . . . . . . . . . . . . . . . . . . 274.2 Research Goals . . . . . . . . . . . . . . . . . . . . . . . . . 284.3 Research Methodology . . . . . . . . . . . . . . . . . . . . . 29

5 Thesis Contributions 335.1 Contribution TC1: Synthesis of Extremely Large Time-

Triggered Networks Schedules . . . . . . . . . . . . . . . . . 33

xiii

Contents

I Thesis 1

1 Introduction 31.1 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Background 72.1 Time-Triggered Communication Paradigm . . . . . . . . . . . 7

2.1.1 Standards and Protocols for Time-Triggered Networks 92.1.2 Time-Triggered Switched Network: System Model . . 11

2.2 Time-Triggered Scheduling . . . . . . . . . . . . . . . . . . . 132.2.1 Scheduling Tools . . . . . . . . . . . . . . . . . . . . 142.2.2 Time-Triggered Scheduling Constraint Definition . . . 15

2.3 Discussion on the System Model Selected . . . . . . . . . . . 19

3 Related Work 213.1 Scheduling Scalability . . . . . . . . . . . . . . . . . . . . . 21

3.1.1 Scheduling for Time-Triggered Networks . . . . . . . 213.1.2 Divide-and-Conquer Approaches . . . . . . . . . . . 24

3.2 Adaptive Time-Triggered Networks . . . . . . . . . . . . . . 25

4 Problem Formulation 274.1 Research Problem . . . . . . . . . . . . . . . . . . . . . . . . 274.2 Research Goals . . . . . . . . . . . . . . . . . . . . . . . . . 284.3 Research Methodology . . . . . . . . . . . . . . . . . . . . . 29

5 Thesis Contributions 335.1 Contribution TC1: Synthesis of Extremely Large Time-

Triggered Networks Schedules . . . . . . . . . . . . . . . . . 33

xiii

17

xiv Contents

5.2 Contribution TC2: Increasing Success Rate of Healing Time-Triggered Network Schedules . . . . . . . . . . . . . . . . . 36

5.3 Contribution TC3: Distributed Online Protocol to Increase theFault-Tolerance Capabilities of Time-Triggered Networks . . . 38

6 Overview of the Included Papers 416.1 Paper A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416.2 Paper B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426.3 Paper C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446.4 Paper D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.5 Paper E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

7 Conclusions and Future Work 497.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 497.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Bibliography 55

II Included Papers 67

8 Paper A:SMT-based Synthesis of TTEthernet Schedules:a Performance Study 698.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 718.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

8.2.1 Network Definition . . . . . . . . . . . . . . . . . . . 728.2.2 Scheduling Rationale . . . . . . . . . . . . . . . . . . 738.2.3 Constraints Definition . . . . . . . . . . . . . . . . . 74

8.3 SMT-Based Synthesis of Schedules . . . . . . . . . . . . . . 748.4 Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

8.4.1 Yices 2 Background Theories . . . . . . . . . . . . . 778.4.2 Stepsize . . . . . . . . . . . . . . . . . . . . . . . . . 80

8.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 81Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

9 Paper B:Methods for Large-Scale Time-Triggered Network Scheduling 859.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 879.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . 89

xiv Contents

5.2 Contribution TC2: Increasing Success Rate of Healing Time-Triggered Network Schedules . . . . . . . . . . . . . . . . . 36

5.3 Contribution TC3: Distributed Online Protocol to Increase theFault-Tolerance Capabilities of Time-Triggered Networks . . . 38

6 Overview of the Included Papers 416.1 Paper A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416.2 Paper B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426.3 Paper C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446.4 Paper D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.5 Paper E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

7 Conclusions and Future Work 497.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 497.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Bibliography 55

II Included Papers 67

8 Paper A:SMT-based Synthesis of TTEthernet Schedules:a Performance Study 698.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 718.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

8.2.1 Network Definition . . . . . . . . . . . . . . . . . . . 728.2.2 Scheduling Rationale . . . . . . . . . . . . . . . . . . 738.2.3 Constraints Definition . . . . . . . . . . . . . . . . . 74

8.3 SMT-Based Synthesis of Schedules . . . . . . . . . . . . . . 748.4 Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

8.4.1 Yices 2 Background Theories . . . . . . . . . . . . . 778.4.2 Stepsize . . . . . . . . . . . . . . . . . . . . . . . . . 80

8.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 81Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

9 Paper B:Methods for Large-Scale Time-Triggered Network Scheduling 859.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 879.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . 89

18

Contents xv

9.2.1 Network Architecture . . . . . . . . . . . . . . . . . . 899.2.2 Traffic Model . . . . . . . . . . . . . . . . . . . . . . 909.2.3 Schedule Synthesis Problem . . . . . . . . . . . . . . 91

9.3 SMT Schedule Synthesis . . . . . . . . . . . . . . . . . . . . 929.4 Four enhancements to the Segmented

Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 969.4.1 Incremental Approach . . . . . . . . . . . . . . . . . 979.4.2 Frame Preprocessing . . . . . . . . . . . . . . . . . . 999.4.3 Segment Preprocessing . . . . . . . . . . . . . . . . . 1009.4.4 Segment Efficiency . . . . . . . . . . . . . . . . . . . 101

9.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1049.5.1 Baseline . . . . . . . . . . . . . . . . . . . . . . . . . 1049.5.2 Case Study . . . . . . . . . . . . . . . . . . . . . . . 1059.5.3 Time Performance . . . . . . . . . . . . . . . . . . . 1079.5.4 Segmentation Performance . . . . . . . . . . . . . . . 109

9.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . 1129.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 113Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

10 Paper C:Schedule Reparability: Enhancing Time-Triggered NetworkRecovery upon Link Failures 11910.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 12110.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . 12310.3 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . 124

10.3.1 Time-Triggered Networks . . . . . . . . . . . . . . . 12410.3.2 Scheduling Problem . . . . . . . . . . . . . . . . . . 12610.3.3 Fault Model . . . . . . . . . . . . . . . . . . . . . . . 126

10.4 Scheduling Constraints . . . . . . . . . . . . . . . . . . . . . 12710.5 Reparability . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

10.5.1 Reparability Concept . . . . . . . . . . . . . . . . . . 12910.5.2 Low-Cost Repair Algorithm . . . . . . . . . . . . . . 13010.5.3 Initial Schedule . . . . . . . . . . . . . . . . . . . . . 132

10.6 Schedule Optimization . . . . . . . . . . . . . . . . . . . . . 13710.7 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

10.7.1 Description . . . . . . . . . . . . . . . . . . . . . . . 13810.7.2 Scheduling Results . . . . . . . . . . . . . . . . . . . 14010.7.3 Reparability Results . . . . . . . . . . . . . . . . . . 142

10.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

Contents xv

9.2.1 Network Architecture . . . . . . . . . . . . . . . . . . 899.2.2 Traffic Model . . . . . . . . . . . . . . . . . . . . . . 909.2.3 Schedule Synthesis Problem . . . . . . . . . . . . . . 91

9.3 SMT Schedule Synthesis . . . . . . . . . . . . . . . . . . . . 929.4 Four enhancements to the Segmented

Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 969.4.1 Incremental Approach . . . . . . . . . . . . . . . . . 979.4.2 Frame Preprocessing . . . . . . . . . . . . . . . . . . 999.4.3 Segment Preprocessing . . . . . . . . . . . . . . . . . 1009.4.4 Segment Efficiency . . . . . . . . . . . . . . . . . . . 101

9.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1049.5.1 Baseline . . . . . . . . . . . . . . . . . . . . . . . . . 1049.5.2 Case Study . . . . . . . . . . . . . . . . . . . . . . . 1059.5.3 Time Performance . . . . . . . . . . . . . . . . . . . 1079.5.4 Segmentation Performance . . . . . . . . . . . . . . . 109


10 Paper C:Schedule Reparability: Enhancing Time-Triggered NetworkRecovery upon Link Failures 11910.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 12110.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . 12310.3 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . 124

10.3.1 Time-Triggered Networks . . . . . . . . . . . . . . . 12410.3.2 Scheduling Problem . . . . . . . . . . . . . . . . . . 12610.3.3 Fault Model . . . . . . . . . . . . . . . . . . . . . . . 126

10.4 Scheduling Constraints . . . . . . . . . . . . . . . . . . . . . 12710.5 Reparability . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

10.5.1 Reparability Concept . . . . . . . . . . . . . . . . . . 12910.5.2 Low-Cost Repair Algorithm . . . . . . . . . . . . . . 13010.5.3 Initial Schedule . . . . . . . . . . . . . . . . . . . . . 132

10.6 Schedule Optimization . . . . . . . . . . . . . . . . . . . . . 13710.7 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

10.7.1 Description . . . . . . . . . . . . . . . . . . . . . . . 13810.7.2 Scheduling Results . . . . . . . . . . . . . . . . . . . 14010.7.3 Reparability Results . . . . . . . . . . . . . . . . . . 142

10.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

19

xvi Contents

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

11 Paper D:Self-Healing Protocol: Repairing Schedules Onlineafter Link Failures in Time-Triggered Networks 15111.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 15311.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . 154

11.2.1 System Model . . . . . . . . . . . . . . . . . . . . . 15411.2.2 Time-Triggered Scheduling . . . . . . . . . . . . . . 156

11.3 Our rationale for Self-Healing . . . . . . . . . . . . . . . . . 15711.4 SHP Notification and Preparation . . . . . . . . . . . . . . . . 159

11.4.1 Notification . . . . . . . . . . . . . . . . . . . . . . . 15911.4.2 Membership . . . . . . . . . . . . . . . . . . . . . . 163

11.5 SHP Schedule Update . . . . . . . . . . . . . . . . . . . . . . 16411.5.1 Patching . . . . . . . . . . . . . . . . . . . . . . . . . 16411.5.2 Update . . . . . . . . . . . . . . . . . . . . . . . . . 16511.5.3 Optimization . . . . . . . . . . . . . . . . . . . . . . 16611.5.4 Multiple Protocol Activations . . . . . . . . . . . . . 167

11.6 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16711.6.1 Reparability Results . . . . . . . . . . . . . . . . . . 16911.6.2 Performance Results . . . . . . . . . . . . . . . . . . 170


12 Paper E:Semi-Distributed Self-Healing Protocol for Online Schedule Repairafter Network Failures 18112.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 18312.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . 185

12.2.1 Time-Triggered Networks . . . . . . . . . . . . . . . 18512.2.2 Scheduling Problem . . . . . . . . . . . . . . . . . . 18512.2.3 Fault Model . . . . . . . . . . . . . . . . . . . . . . . 18612.2.4 Schedule repairs and self-healing . . . . . . . . . . . 186

12.3 Semi-Distributed Self-Healing Protocol . . . . . . . . . . . . 18712.4 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

12.4.1 Preemptive Healing . . . . . . . . . . . . . . . . . . . 19012.4.2 Path Selection . . . . . . . . . . . . . . . . . . . . . . 19112.4.3 Regular Switch Healing . . . . . . . . . . . . . . . . 193

xvi Contents

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

11 Paper D:Self-Healing Protocol: Repairing Schedules Onlineafter Link Failures in Time-Triggered Networks 15111.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 15311.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . 154

11.2.1 System Model . . . . . . . . . . . . . . . . . . . . . 15411.2.2 Time-Triggered Scheduling . . . . . . . . . . . . . . 156

11.3 Our rationale for Self-Healing . . . . . . . . . . . . . . . . . 15711.4 SHP Notification and Preparation . . . . . . . . . . . . . . . . 159

11.4.1 Notification . . . . . . . . . . . . . . . . . . . . . . . 15911.4.2 Membership . . . . . . . . . . . . . . . . . . . . . . 163

11.5 SHP Schedule Update . . . . . . . . . . . . . . . . . . . . . . 16411.5.1 Patching . . . . . . . . . . . . . . . . . . . . . . . . . 16411.5.2 Update . . . . . . . . . . . . . . . . . . . . . . . . . 16511.5.3 Optimization . . . . . . . . . . . . . . . . . . . . . . 16611.5.4 Multiple Protocol Activations . . . . . . . . . . . . . 167



12 Paper E:Semi-Distributed Self-Healing Protocol for Online Schedule Repairafter Network Failures 18112.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 18312.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . 185

12.2.1 Time-Triggered Networks . . . . . . . . . . . . . . . 18512.2.2 Scheduling Problem . . . . . . . . . . . . . . . . . . 18512.2.3 Fault Model . . . . . . . . . . . . . . . . . . . . . . . 18612.2.4 Schedule repairs and self-healing . . . . . . . . . . . 186

12.3 Semi-Distributed Self-Healing Protocol . . . . . . . . . . . . 18712.4 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

12.4.1 Preemptive Healing . . . . . . . . . . . . . . . . . . . 19012.4.2 Path Selection . . . . . . . . . . . . . . . . . . . . . . 19112.4.3 Regular Switch Healing . . . . . . . . . . . . . . . . 193

20

Contents xvii


12.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . 19912.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 204Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

Contents xvii


12.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . 19912.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 204Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

21

I

Thesis

1

I

Thesis

1

23

Chapter 1

Introduction

Embedded systems are a combination of hardware and software designed tofulfil a specific function within larger systems. Over the years, many domainssuch as automotive, aerospace or factory automation have implemented them.In many embedded systems, the exchange of information is carried out overa network that connects the involving computing nodes. The communicationtiming properties are crucial for the correct behaviour of the whole system. Insome cases, these timing requirements are rigorous, with strict requirementson low latency and bounded end-to-end delays. The time-triggered paradigm[1] can satisfy such strict timing constraints thanks to a static communicationschedule executed according to a shared timebase for the whole network pro-vided by a clock-synchronization protocol. The schedule states the transmis-sion times over the network of all the time-triggered traffic. However, this isachieved at the price of a limited flexibility as no modifications can be per-formed during run-time without also updating the schedule. Moreover, obtain-ing the schedule is a known NP-complete problem [2] with complexity drivenby the network and traffic size.

The constant grow in size and complexity of real-time embedded systemspresents some challenges for the real-time network implementation. The real-time industry has considered Ethernet as a solution to its requirements. Ether-net enables the design of more extensive networks using an intermediary node,called switch, which relays information over other switches until reaching itsdestination. However, Ethernet was designed for best-effort traffic, and it doesnot support real-time communication. Time-Triggered Ethernet [3] and IEEE802.1Qbv or Time Sensitive Networks (TSN) [4], among others, incorporate

3

Chapter 1

Introduction

Embedded systems are a combination of hardware and software designed tofulfil a specific function within larger systems. Over the years, many domainssuch as automotive, aerospace or factory automation have implemented them.In many embedded systems, the exchange of information is carried out overa network that connects the involving computing nodes. The communicationtiming properties are crucial for the correct behaviour of the whole system. Insome cases, these timing requirements are rigorous, with strict requirementson low latency and bounded end-to-end delays. The time-triggered paradigm[1] can satisfy such strict timing constraints thanks to a static communicationschedule executed according to a shared timebase for the whole network pro-vided by a clock-synchronization protocol. The schedule states the transmis-sion times over the network of all the time-triggered traffic. However, this isachieved at the price of a limited flexibility as no modifications can be per-formed during run-time without also updating the schedule. Moreover, obtain-ing the schedule is a known NP-complete problem [2] with complexity drivenby the network and traffic size.

The constant grow in size and complexity of real-time embedded systemspresents some challenges for the real-time network implementation. The real-time industry has considered Ethernet as a solution to its requirements. Ether-net enables the design of more extensive networks using an intermediary node,called switch, which relays information over other switches until reaching itsdestination. However, Ethernet was designed for best-effort traffic, and it doesnot support real-time communication. Time-Triggered Ethernet [3] and IEEE802.1Qbv or Time Sensitive Networks (TSN) [4], among others, incorporate

3

25

4 Chapter 1. Introduction

enhancements to Ethernet to support soft and hard real-time requirements. Theintroduction of real-time switched Ethernet networks accentuated the obstaclesto implement the time-triggered paradigm: the complexity to obtain the sched-ule and the lack of flexibility. Current state-of-the-art schedulers present scal-ability issues related to the synthesizes of industrial-size networks. Moreover,the mechanisms to achieve flexibility, mostly for fault-tolerance, are becomingexcessively costly and inefficient in the use of resources.

Ethernet-based real-time protocols have the potential for implementation ofmuch larger networks compared with current industry-size ones, which containtens of nodes and hundreds of frames. E.g., the ongoing conceptualization offog computing [5], bringing cloud computing closer to the nodes to satisfy la-tency requirements, has opened the notion of larger applications with real-timerequirements, such as massive factory automation [6] or vehicular networks[7]. We can perceive that this constant growth might spark the implementationof large-scale real-time applications that will also require what we call next-generation networks. We project that the next-generation might increase thecurrent network size by one order of magnitude, to hundreds of nodes, and thetraffic by two orders of magnitude, to tens of thousands of frames. Currentscheduling techniques cannot handle such a massive increase in complexity.Additionally, as the number of network components increases, the probabilityof failures also increases. The notion of a static network where no changesare performed after deployment is hard to motivate for the next-generation net-works. Different components would require to enter and leave the system reg-ularly, e.g., vehicles in vehicular fog computing. This need for adaptabilitycoupled with the new challenges to tolerate failures manifests a new problemas networks, and therefore schedules, would require to be changed during run-time.

Moreover, next-generation networks might require to support hybrid wired-wireless communication. Wireless can reduce costs and provide easy deploy-ment to reach places where wired can not. E.g., in the vehicular fog comput-ing, a backbone wired network would require wireless to communicate withdifferent cars. However, time-triggered wireless presents some crucial chal-lenges. The reliability is significantly smaller as frame losses are more promi-nent due to external or internal interference provoked by the environment orby other multiple frame transmissions at the same time, respectively. Never-theless, some protocols have been successfully implemented for time-triggeredwireless, such as the WirelessHART protocol [8]. The introduction of wirelesscommunication further increases the complexity of schedule synthesis as in-terference need to be accounted for in the schedule, due to techniques such as


enhancements to Ethernet to support soft and hard real-time requirements. Theintroduction of real-time switched Ethernet networks accentuated the obstaclesto implement the time-triggered paradigm: the complexity to obtain the sched-ule and the lack of flexibility. Current state-of-the-art schedulers present scal-ability issues related to the synthesizes of industrial-size networks. Moreover,the mechanisms to achieve flexibility, mostly for fault-tolerance, are becomingexcessively costly and inefficient in the use of resources.

Ethernet-based real-time protocols have the potential for implementation ofmuch larger networks compared with current industry-size ones, which containtens of nodes and hundreds of frames. E.g., the ongoing conceptualization offog computing [5], bringing cloud computing closer to the nodes to satisfy la-tency requirements, has opened the notion of larger applications with real-timerequirements, such as massive factory automation [6] or vehicular networks[7]. We can perceive that this constant growth might spark the implementationof large-scale real-time applications that will also require what we call next-generation networks. We project that the next-generation might increase thecurrent network size by one order of magnitude, to hundreds of nodes, and thetraffic by two orders of magnitude, to tens of thousands of frames. Currentscheduling techniques cannot handle such a massive increase in complexity.Additionally, as the number of network components increases, the probabilityof failures also increases. The notion of a static network where no changesare performed after deployment is hard to motivate for the next-generation net-works. Different components would require to enter and leave the system reg-ularly, e.g., vehicles in vehicular fog computing. This need for adaptabilitycoupled with the new challenges to tolerate failures manifests a new problemas networks, and therefore schedules, would require to be changed during run-time.

Moreover, next-generation networks might require to support hybrid wired-wireless communication. Wireless can reduce costs and provide easy deploy-ment to reach places where wired can not. E.g., in the vehicular fog comput-ing, a backbone wired network would require wireless to communicate withdifferent cars. However, time-triggered wireless presents some crucial chal-lenges. The reliability is significantly smaller as frame losses are more promi-nent due to external or internal interference provoked by the environment orby other multiple frame transmissions at the same time, respectively. Never-theless, some protocols have been successfully implemented for time-triggeredwireless, such as the WirelessHART protocol [8]. The introduction of wirelesscommunication further increases the complexity of schedule synthesis as in-terference need to be accounted for in the schedule, due to techniques such as

26

1.1 Thesis Outline 5

frame replication and a more complex collision scheme.In summary, next-generation large-scale, real-time networks present signif-

icant challenges to be implemented, especially with regards to scheduling. Themain goal of this thesis is to design mechanisms that allow scheduling of thenext-generation real-time networks and advances the state-of-the-art towardsan adaptive time-triggered paradigm.

Next-generation network schedules can be obtained by applying divide-and-conquer approaches. State-of-the-art schedulers, such as these imple-mented with Satisfiability Modulo Theories (SMT) solvers, present scalabilityissues when synthesizing the whole schedule. Instead, we propose to dividethe schedule into adequately small segments, susceptible to synthesis with astandard solver. The final schedule is then retrieved from the union of all thesolved segments. Segmented scheduling can synthesize networks in the hours’range and present little scalability issues with regards to network and trafficsize. In this thesis, we study different segmentation schemes and evaluate theirperformance.

An adaptive time-triggered paradigm would require to modify schedulesonline in a short amount of time to adapt to unexpected changes. This thesisproposes a path towards such realization focusing on fault-tolerance of linksand switches. We present the concept of self-healing, algorithms that can per-form minimal schedule modification into the affected network area to recoverall frame transmissions after links or switches failures. The application of suchalgorithms is performed by online protocols that heal the schedule during run-time in the milliseconds’ range. Furthermore, the success of self-healing isclosely related to a schedule property we call reparability. We can increasethe schedule reparability by maximizing the distances between frame trans-missions, so more schedule modifications are possible. For example, whenthe self-healing requires a longer path after a link failure, frames must be suf-ficiently separated to allow allocating of the longer path in the gaps betweenframes. We propose to obtain high reparability schedules shifting our solverinto an Integer Linear Programming (ILP) solver, which, contrary to SMTsolvers, implements optimizing capabilities required to maximize frame dis-tances.

1.1 Thesis Outline

This thesis is divided into two parts: a comprehensive summary and a collec-tion of papers that constitutes the main contributions. Chapter 2 of the sum-

1.1 Thesis Outline 5

frame replication and a more complex collision scheme.In summary, next-generation large-scale, real-time networks present signif-

icant challenges to be implemented, especially with regards to scheduling. Themain goal of this thesis is to design mechanisms that allow scheduling of thenext-generation real-time networks and advances the state-of-the-art towardsan adaptive time-triggered paradigm.

Next-generation network schedules can be obtained by applying divide-and-conquer approaches. State-of-the-art schedulers, such as these imple-mented with Satisfiability Modulo Theories (SMT) solvers, present scalabilityissues when synthesizing the whole schedule. Instead, we propose to dividethe schedule into adequately small segments, susceptible to synthesis with astandard solver. The final schedule is then retrieved from the union of all thesolved segments. Segmented scheduling can synthesize networks in the hours’range and present little scalability issues with regards to network and trafficsize. In this thesis, we study different segmentation schemes and evaluate theirperformance.

An adaptive time-triggered paradigm would require to modify schedulesonline in a short amount of time to adapt to unexpected changes. This thesisproposes a path towards such realization focusing on fault-tolerance of linksand switches. We present the concept of self-healing, algorithms that can per-form minimal schedule modification into the affected network area to recoverall frame transmissions after links or switches failures. The application of suchalgorithms is performed by online protocols that heal the schedule during run-time in the milliseconds’ range. Furthermore, the success of self-healing isclosely related to a schedule property we call reparability. We can increasethe schedule reparability by maximizing the distances between frame trans-missions, so more schedule modifications are possible. For example, whenthe self-healing requires a longer path after a link failure, frames must be suf-ficiently separated to allow allocating of the longer path in the gaps betweenframes. We propose to obtain high reparability schedules shifting our solverinto an Integer Linear Programming (ILP) solver, which, contrary to SMTsolvers, implements optimizing capabilities required to maximize frame dis-tances.

1.1 Thesis Outline

This thesis is divided into two parts: a comprehensive summary and a collec-tion of papers that constitutes the main contributions. Chapter 2 of the sum-

27


mary introduces the background of the thesis. Chapter 3 presents related workand Chapter 4 the problem formulation. After that, the thesis contributionsand the overview of the included papers are given in Chapter 5 and Chapter6, respectively. The conclusions and future work of the comprehensive sum-mary are presented in Chapter 7. Finally, the appended papers are included inChapters 8 to 12.


mary introduces the background of the thesis. Chapter 3 presents related workand Chapter 4 the problem formulation. After that, the thesis contributionsand the overview of the included papers are given in Chapter 5 and Chapter6, respectively. The conclusions and future work of the comprehensive sum-mary are presented in Chapter 7. Finally, the appended papers are included inChapters 8 to 12.

28

Chapter 2

Background

2.1 Time-Triggered Communication ParadigmIn real-time embedded systems, the information correctness and timing are cru-cial for correct functionality of the whole system. Real-time communicationcan be classified into two different categories regarding the frame triggering:event-triggered or time-triggered [9]. In event-triggered, an event at any pointof time, such as a signal interruption, triggers the frame transmission. An on-line scheduler implements certain arbitration or contention techniques to con-trol the frame transmissions over the network, so with the right configuration(e.g., proper priority assignments), the timing requirements are satisfied. Theevent implementation introduces flexibility in the paradigm at the cost of thecomplete certainty that no transmission will miss their deadlines. There mightexist some spontaneous execution cases in which the timing requirements aremomentarily missed. These deadline missed can be tolerated in soft real-timesystems, where they only reduce the quality of the service. Frames in time-triggered communication are triggered following a scheme that is periodicallyrepeated, called the schedule. The schedule is synthesized at design time stat-ing the transmission times of all the frames to satisfy the requirements. Theschedule is upload to the network and executed cyclically. The Time-triggeredcommunication underlying assumption is that all nodes implement a clock-synchronization protocol that guarantees they share a common perception oftime, such that the schedule is consistently executed. The timed implemen-tation guarantees by design that the timing requirements are always satisfied,but no flexibility can be provided. This tradeoff is assumed in hard real-time

7

Chapter 2

Background

2.1 Time-Triggered Communication ParadigmIn real-time embedded systems, the information correctness and timing are cru-cial for correct functionality of the whole system. Real-time communicationcan be classified into two different categories regarding the frame triggering:event-triggered or time-triggered [9]. In event-triggered, an event at any pointof time, such as a signal interruption, triggers the frame transmission. An on-line scheduler implements certain arbitration or contention techniques to con-trol the frame transmissions over the network, so with the right configuration(e.g., proper priority assignments), the timing requirements are satisfied. Theevent implementation introduces flexibility in the paradigm at the cost of thecomplete certainty that no transmission will miss their deadlines. There mightexist some spontaneous execution cases in which the timing requirements aremomentarily missed. These deadline missed can be tolerated in soft real-timesystems, where they only reduce the quality of the service. Frames in time-triggered communication are triggered following a scheme that is periodicallyrepeated, called the schedule. The schedule is synthesized at design time stat-ing the transmission times of all the frames to satisfy the requirements. Theschedule is upload to the network and executed cyclically. The Time-triggeredcommunication underlying assumption is that all nodes implement a clock-synchronization protocol that guarantees they share a common perception oftime, such that the schedule is consistently executed. The timed implemen-tation guarantees by design that the timing requirements are always satisfied,but no flexibility can be provided. This tradeoff is assumed in hard real-time

7

29

8 Chapter 2. Background

systems, where a missed deadline might affect the service of the whole system.Time-triggered communication was first implemented into bus-based pro-

tocols, such as TTP [10], TTCAN [11], and FlexRay [12]. Bus-based protocolsshare a single transmission medium, typically with a bandwidth of few Mbps,in which collisions are avoided by allowing only one node to transmit at a time.The increase in size and complexity of real-time embedded systems challengedthe scalability capabilities of bus-based protocols. E.g., the inclusion of cam-eras for vehicle guidance requires real-time transmission from multiple nodecameras to processing nodes that make decisions. Recently, time-triggeredcommunication has been implemented over switched Ethernet networks dueto their high scalability, configurability and bandwidth. In switched Ethernet,a frame sent by an end system (sender) travels over links through differentswitches that store and forward the frame until it reaches its destination, an-other end system (receiver). As seen in Figure 2.1, switches receive framesover input ports and keep them in the receive buffer. Later on, a frame handlerthat posses the schedule information selects the FIFO Queue the frame is re-layed over. The dispatcher, also containing the schedule information, decidesfrom which queue a frame should be moved to the output port for transmission.

ReceiveBuffer 1

ReceiveBuffer n

FrameHandler

FIFO Queues 1

FIFO Queues n

Frame

Dispatchern

Input Port 1 Output Port 1

Output Port nInput Port n

... ...

Frame

Dispatchern

Figure 2.1: Example of the internal structure of a Ethernet switch

Wireless Time-Triggered Communication

Opposite to wired communication, wireless communication does not requireany physical wire that connects different nodes to transmit information. How-ever, the absence of a guided physical medium makes it more susceptible topath loss, shadowing, multi-path fading, and collisions, which increases the er-


systems, where a missed deadline might affect the service of the whole system.Time-triggered communication was first implemented into bus-based pro-

tocols, such as TTP [10], TTCAN [11], and FlexRay [12]. Bus-based protocolsshare a single transmission medium, typically with a bandwidth of few Mbps,in which collisions are avoided by allowing only one node to transmit at a time.The increase in size and complexity of real-time embedded systems challengedthe scalability capabilities of bus-based protocols. E.g., the inclusion of cam-eras for vehicle guidance requires real-time transmission from multiple nodecameras to processing nodes that make decisions. Recently, time-triggeredcommunication has been implemented over switched Ethernet networks dueto their high scalability, configurability and bandwidth. In switched Ethernet,a frame sent by an end system (sender) travels over links through differentswitches that store and forward the frame until it reaches its destination, an-other end system (receiver). As seen in Figure 2.1, switches receive framesover input ports and keep them in the receive buffer. Later on, a frame handlerthat posses the schedule information selects the FIFO Queue the frame is re-layed over. The dispatcher, also containing the schedule information, decidesfrom which queue a frame should be moved to the output port for transmission.

ReceiveBuffer 1

ReceiveBuffer n

FrameHandler

FIFO Queues 1

FIFO Queues n

Frame

Dispatchern



... ...

Frame

Dispatchern

Figure 2.1: Example of the internal structure of a Ethernet switch

Wireless Time-Triggered Communication

Opposite to wired communication, wireless communication does not requireany physical wire that connects different nodes to transmit information. How-ever, the absence of a guided physical medium makes it more susceptible topath loss, shadowing, multi-path fading, and collisions, which increases the er-

30

2.1 Time-Triggered Communication Paradigm 9

ror rate compared to wired communication. These are inherent to the wirelessmedium and complicate achieving reliability requirements, particularly for themost safety-critical applications. Two techniques are commonly applied to in-crease the reliability of wireless communication: Automatic-Repeat Request(ARQ) and Forward Error Correction (FEC). In ARQ, frames are retransmit-ted at different times (temporal diversity) or different paths (spatial diversity),while in FEC, extra information is added to the frame to recover when errorsoccur. In this thesis, we will only focus in ARQ as is the technique that requiresmodifications in the scheduling problem. This will be expanded in Chapter 3.

Collisions are avoided by transmitting frames in different mediums. Themost applied accesses are Time Division Medium Access (TDMA), FrequencyDivision Medium Access (FDMA) and Code Division Medium Access (CDMA).Following the philosophy of time-triggered, this thesis focus only on TDMA.At every area of transmission where collisions are possible, called collisiondomain, only one transmission is allowed at the same time. A schedule, sameas in wired communication, states the frame transmission times such as nocollision occur, and the timing requirements are satisfied. Note that FDMAcan also be implemented distinguishing different collision domains at differentfrequencies. However, this is left for future work.

2.1.1 Standards and Protocols for Time-Triggered Networks

TTEhernet

Time-Triggered Ethernet [3] was designed to integrate into a single mixed-critical switched Ethernet networks all the requirements present in differ-ent network implementations. TTEthernet combines time-triggered, event-triggered and non-real-time communication on a single network. Time-triggered is performed over Time-Triggered (TT) frames that follow a staticschedule. TT frames have reserved the highest priority FIFO queues in theswitch. Event-triggered is performed over Rate-Constrained (RC) frames inwhich a traffic shaping function ensures that there exists a minimum inter-frame gap. The lowest priority FIFO queues are reserved for non-real-timecommunication carried over Best Effort (BE) frames. TTEthernet does notprovide timing or transmission guarantees for BE frames. To assure that thehighest priority frames are always transmitted first, TTEthernet implements atimely block where no lower priority transmission can start if it cannot fin-ish before a scheduled higher priority transmission. TTEthernet enforces aglobal shared notion of time implemented by a two-steps global synchroniza-


ror rate compared to wired communication. These are inherent to the wirelessmedium and complicate achieving reliability requirements, particularly for themost safety-critical applications. Two techniques are commonly applied to in-crease the reliability of wireless communication: Automatic-Repeat Request(ARQ) and Forward Error Correction (FEC). In ARQ, frames are retransmit-ted at different times (temporal diversity) or different paths (spatial diversity),while in FEC, extra information is added to the frame to recover when errorsoccur. In this thesis, we will only focus in ARQ as is the technique that requiresmodifications in the scheduling problem. This will be expanded in Chapter 3.

Collisions are avoided by transmitting frames in different mediums. Themost applied accesses are Time Division Medium Access (TDMA), FrequencyDivision Medium Access (FDMA) and Code Division Medium Access (CDMA).Following the philosophy of time-triggered, this thesis focus only on TDMA.At every area of transmission where collisions are possible, called collisiondomain, only one transmission is allowed at the same time. A schedule, sameas in wired communication, states the frame transmission times such as nocollision occur, and the timing requirements are satisfied. Note that FDMAcan also be implemented distinguishing different collision domains at differentfrequencies. However, this is left for future work.

2.1.1 Standards and Protocols for Time-Triggered Networks

TTEhernet

Time-Triggered Ethernet [3] was designed to integrate into a single mixed-critical switched Ethernet networks all the requirements present in differ-ent network implementations. TTEthernet combines time-triggered, event-triggered and non-real-time communication on a single network. Time-triggered is performed over Time-Triggered (TT) frames that follow a staticschedule. TT frames have reserved the highest priority FIFO queues in theswitch. Event-triggered is performed over Rate-Constrained (RC) frames inwhich a traffic shaping function ensures that there exists a minimum inter-frame gap. The lowest priority FIFO queues are reserved for non-real-timecommunication carried over Best Effort (BE) frames. TTEthernet does notprovide timing or transmission guarantees for BE frames. To assure that thehighest priority frames are always transmitted first, TTEthernet implements atimely block where no lower priority transmission can start if it cannot fin-ish before a scheduled higher priority transmission. TTEthernet enforces aglobal shared notion of time implemented by a two-steps global synchroniza-

31


tion protocol. In the first protocol step, Synchronization Masters (SMs) nodes,typically end systems, send a Protocol Control Frame (PCF) with a timestampto all the Compression Masters (CMs), usually switches. In the second step,CMs send back PCFs to the SMs and the Synchronization Clients (SCs), whichcomputes a new reference point of time from the timestamps received.

IEEE 802.1Qbv

The sub-standard IEEE 802.1Qbv, also known as Time-Sensitive Network(TSN) [4], was developed to enable time-triggered communication over theIEEE 802 standard. This sub-standard is one of the many designed to pro-vide functionalities required by real-time applications, such as network-wideclock synchronization or frame preemption. TSN differs from the TTEthernetor other time-triggered protocols by the control of the frames dispatching. Asseen in Figure 2.2, instead of implementing a global dispatcher for every out-put port, it defines single gate dispatchers for every FIFO queue. This shifts thescheduling of frame transmissions to a schedule of the gate dispatchers, alsonamed Gate List Scheduling.

ReceiveBuffer 1

ReceiveBuffer n

FrameHandler

FIFO Queues 1

FIFO Queues n



... ...Gate List 1

Gate List n

Figure 2.2: Example of the internal structure of a TSN switch

WirelessHART

WirelessHART [8] is a wireless time-triggered protocol based on the IEEE802.15.5 standard [13] designed for applications where sensors and actuatorsrequire real-time requirements and high reliability. In WirelessHART, all nodescan transmit and relay information to the neighbouring nodes, creating a mesh


tion protocol. In the first protocol step, Synchronization Masters (SMs) nodes,typically end systems, send a Protocol Control Frame (PCF) with a timestampto all the Compression Masters (CMs), usually switches. In the second step,CMs send back PCFs to the SMs and the Synchronization Clients (SCs), whichcomputes a new reference point of time from the timestamps received.

IEEE 802.1Qbv

The sub-standard IEEE 802.1Qbv, also known as Time-Sensitive Network(TSN) [4], was developed to enable time-triggered communication over theIEEE 802 standard. This sub-standard is one of the many designed to pro-vide functionalities required by real-time applications, such as network-wideclock synchronization or frame preemption. TSN differs from the TTEthernetor other time-triggered protocols by the control of the frames dispatching. Asseen in Figure 2.2, instead of implementing a global dispatcher for every out-put port, it defines single gate dispatchers for every FIFO queue. This shifts thescheduling of frame transmissions to a schedule of the gate dispatchers, alsonamed Gate List Scheduling.

ReceiveBuffer 1

ReceiveBuffer n

FrameHandler

FIFO Queues 1

FIFO Queues n



... ...Gate List 1

Gate List n

Figure 2.2: Example of the internal structure of a TSN switch

WirelessHART

WirelessHART [8] is a wireless time-triggered protocol based on the IEEE802.15.5 standard [13] designed for applications where sensors and actuatorsrequire real-time requirements and high reliability. In WirelessHART, all nodescan transmit and relay information to the neighbouring nodes, creating a mesh

32


topology. A central network manager coordinates the paths of the frame trans-mission over the mesh and designs a TDMA time-triggered schedule. A pri-mary difference of the WirelessHART schedule is that time slots are strictlyequal to 10ms where superframes are transmitted. To increase the bandwidth,it implements Frequency Hopping Spread Spectrum (FHSS) with 15 channelsof different frequency. Transmission at different frequencies do not collide andcan be transmitted at the same time. To reduce frame loss caused by externalinterference, the protocol implements alternative-path retransmissions if theframe does not reach its destination [8].

2.1.2 Time-Triggered Switched Network: System ModelIn this subsection, we present the formalized model of next-generation time-triggered networks. Note that, since this conceptualization evolved over thethesis process, some early publications include a slightly different model.However, they are all in general compatible with the one presented here.

We define a multi-hop network as an directed graph G = (V,E), wherethe vertices V represent switches and end systems, and the edges E representdirectional connections between vertices. Data is exchanged between verticesthrough frames, with F denoting the set of all the frames. Particularly, twovertices vx, vy ∈ V can exchange frames if they are connected to an edgeand only if at least one of the vertices is a switch. In contrast, switches arepermitted to connect to multiple switches and end systems. We denote the setof all links as L. E.g., the link (vx, vy) ∈ L designates the connection fromvertex x to vertex y. Each link l ∈ L has an associated capacity Cl, measuredin Bps (bytes per second).

Additionally, a communication link can be either wired or wireless. A linkis considered to be wired if there exists a physical connection between thevertices. In wireless links, a Wireless Access Points (WAP), located in thecorresponding switches and end systems, transmits the frames without the needfor a physical connection. We denote the set of all wired links and the set ofall wireless links Lw and Lx, respectively. Note that L = Lx ∪ Lw, and Lx ∩Lw = ∅. A distinctiveness of wireless links, in contrast to wired links, is thattransmissions can collide with other wireless transmissions in the proximity, asthey share the same medium, as seen in Subsection 2.1. To avoid frame lossesdue to such collisions among them, we define a collision domain CD as:

CD = {(vx1, vy1

), (vx2, vy2

), ..., (vxn, vyn

)} ⊆ L (2.1)

A CD includes n (wireless) links that are not allowed to transmit at the


topology. A central network manager coordinates the paths of the frame trans-mission over the mesh and designs a TDMA time-triggered schedule. A pri-mary difference of the WirelessHART schedule is that time slots are strictlyequal to 10ms where superframes are transmitted. To increase the bandwidth,it implements Frequency Hopping Spread Spectrum (FHSS) with 15 channelsof different frequency. Transmission at different frequencies do not collide andcan be transmitted at the same time. To reduce frame loss caused by externalinterference, the protocol implements alternative-path retransmissions if theframe does not reach its destination [8].

2.1.2 Time-Triggered Switched Network: System ModelIn this subsection, we present the formalized model of next-generation time-triggered networks. Note that, since this conceptualization evolved over thethesis process, some early publications include a slightly different model.However, they are all in general compatible with the one presented here.

We define a multi-hop network as an directed graph G = (V,E), wherethe vertices V represent switches and end systems, and the edges E representdirectional connections between vertices. Data is exchanged between verticesthrough frames, with F denoting the set of all the frames. Particularly, twovertices vx, vy ∈ V can exchange frames if they are connected to an edgeand only if at least one of the vertices is a switch. In contrast, switches arepermitted to connect to multiple switches and end systems. We denote the setof all links as L. E.g., the link (vx, vy) ∈ L designates the connection fromvertex x to vertex y. Each link l ∈ L has an associated capacity Cl, measuredin Bps (bytes per second).

Additionally, a communication link can be either wired or wireless. A linkis considered to be wired if there exists a physical connection between thevertices. In wireless links, a Wireless Access Points (WAP), located in thecorresponding switches and end systems, transmits the frames without the needfor a physical connection. We denote the set of all wired links and the set ofall wireless links Lw and Lx, respectively. Note that L = Lx ∪ Lw, and Lx ∩Lw = ∅. A distinctiveness of wireless links, in contrast to wired links, is thattransmissions can collide with other wireless transmissions in the proximity, asthey share the same medium, as seen in Subsection 2.1. To avoid frame lossesdue to such collisions among them, we define a collision domain CD as:

CD = {(vx1, vy1

), (vx2, vy2

), ..., (vxn, vyn

)} ⊆ L (2.1)

A CD includes n (wireless) links that are not allowed to transmit at the

33


Figure 2.3: Example of hybrid wired-wireless next-generation network

same time due to the occurrence of a collision. Moreover, a wireless link canbelong to multiple collision domains. In the case of wired links, we assumethat each forms its collision domain of a single link.

As discussed in Subsection 2.1, one of the main drawbacks of wirelesscommunication is its low reliability caused by external interferences. Temporalredundancy is a technique often applied to increase reliability where severalreplicas of a frame are transmitted, each one separated by the so-called InterTransmission Time (ITI). In our model, we implement this redundancy andassume that the total number of replicas is fixed (= K) for all the wirelesslinks. Figure 2.3 shows a hybrid network with 7 end systems, 3 switches, 12wired links and 6 wireless links belonging to the same collision domain.

An end system sender can transmit information over frames to one or mul-tiple end systems receivers. The sequence of links between the sender and areceiver defines the data flow path p:

p = [(vs, vs+1), ..., (vr−1, vr)] (2.2)

where vs is the sender and vr is the receiver. We define the tree path of aframe f , denoted as TPf, as the union of all data flow paths from the sender off to each one of its receivers.

Formally, a frame f is defined by the tuple:


Figure 2.3: Example of hybrid wired-wireless next-generation network

same time due to the occurrence of a collision. Moreover, a wireless link canbelong to multiple collision domains. In the case of wired links, we assumethat each forms its collision domain of a single link.

As discussed in Subsection 2.1, one of the main drawbacks of wirelesscommunication is its low reliability caused by external interferences. Temporalredundancy is a technique often applied to increase reliability where severalreplicas of a frame are transmitted, each one separated by the so-called InterTransmission Time (ITI). In our model, we implement this redundancy andassume that the total number of replicas is fixed (= K) for all the wirelesslinks. Figure 2.3 shows a hybrid network with 7 end systems, 3 switches, 12wired links and 6 wireless links belonging to the same collision domain.

An end system sender can transmit information over frames to one or mul-tiple end systems receivers. The sequence of links between the sender and areceiver defines the data flow path p:

p = [(vs, vs+1), ..., (vr−1, vr)] (2.2)

where vs is the sender and vr is the receiver. We define the tree path of aframe f , denoted as TPf, as the union of all data flow paths from the sender off to each one of its receivers.

Formally, a frame f is defined by the tuple:

34

2.2 Time-Triggered Scheduling 13

f = 〈T f,Df,Lf,TPf〉, (2.3)

where T f is the frame period, Df is the frame deadline, Lf is the size of theframe (measured in bytes), and TPf is the tree path as defined above.

Given a set of frames F , they determine the schedule hyper-period (TF) asthe least common multiple of the periods of the frames: TF = LCM(∀T f, f ∈F ). Within the hyper-period, a frame might need to be transmitted more thanonce: the number of frame instances of frame f ∈ F (denoted N f) within thehyper-period is calculated as N f = TF

T f.

We will denote as δlf the time needed to transmit frame f ∈ F over link

l ∈ L, calculated as δlf =Lf

Cl.

2.2 Time-Triggered Scheduling

The synthesis of a time-triggered schedule that satisfies all traffic and net-work constraints is a well known NP-complete problem by reduction to a bin-packing problem [2]. If an objective, or optimization function, is added to thesynthesis, it becomes an NP-hard scheduling problem. The schedule is sup-posed to be executed for an undefined amount of time. As frames are periodic,it is only needed to calculate the schedule length for the least common multipleof all frame periods, called the schedule hyper-period. In the case the periodsare not harmonic, many techniques can harmonize them without affecting thereal-time requirements [14].

A widely used approach to reduce the problem complexity is to introduceschedule granularity of time slots [15]. Different implementations apply dif-ferent time slots sizes, e.g., as long as the maximum time to transmit any frameover any link [16], or as shorter as one nanosecond [17]. Note that smaller timeslots increases the scheduling complexity but provide better utilization of thebandwidth. There exist many different techniques to obtain these schedules.In this thesis, we model the network and traffic constraints into linear integerformula that can be satisfied by a Satisfiability Modulo Theory (SMT) solver tosynthesize a schedule, or by an Integer Linear Programming (ILP) solver whenan optimization function is required. We show the process in Figure 2.4.


f = 〈T f,Df,Lf,TPf〉, (2.3)

where T f is the frame period, Df is the frame deadline, Lf is the size of theframe (measured in bytes), and TPf is the tree path as defined above.

Given a set of frames F , they determine the schedule hyper-period (TF) asthe least common multiple of the periods of the frames: TF = LCM(∀T f, f ∈F ). Within the hyper-period, a frame might need to be transmitted more thanonce: the number of frame instances of frame f ∈ F (denoted N f) within thehyper-period is calculated as N f = TF

T f.

We will denote as δlf the time needed to transmit frame f ∈ F over link

l ∈ L, calculated as δlf =Lf

Cl.

2.2 Time-Triggered Scheduling

The synthesis of a time-triggered schedule that satisfies all traffic and net-work constraints is a well known NP-complete problem by reduction to a bin-packing problem [2]. If an objective, or optimization function, is added to thesynthesis, it becomes an NP-hard scheduling problem. The schedule is sup-posed to be executed for an undefined amount of time. As frames are periodic,it is only needed to calculate the schedule length for the least common multipleof all frame periods, called the schedule hyper-period. In the case the periodsare not harmonic, many techniques can harmonize them without affecting thereal-time requirements [14].

A widely used approach to reduce the problem complexity is to introduceschedule granularity of time slots [15]. Different implementations apply dif-ferent time slots sizes, e.g., as long as the maximum time to transmit any frameover any link [16], or as shorter as one nanosecond [17]. Note that smaller timeslots increases the scheduling complexity but provide better utilization of thebandwidth. There exist many different techniques to obtain these schedules.In this thesis, we model the network and traffic constraints into linear integerformula that can be satisfied by a Satisfiability Modulo Theory (SMT) solver tosynthesize a schedule, or by an Integer Linear Programming (ILP) solver whenan optimization function is required. We show the process in Figure 2.4.

35


NetworkConstrains

Scheduler

Schedule

Figure 2.4: Process to obtain schedules with SMT or ILP solvers

2.2.1 Scheduling Tools

Satisfiability Modulo Theories

Satisfiability (SAT) solvers are tools that determine the satisfiability of booleanformulas applying algorithms such as DPLL [18]. In other words, it proves ifthere exists a solution for the given boolean formula constraints. As describ-ing a problem with only boolean formulas can be challenging, SatisfiabilityModulo Theories (SMT) solvers extend SAT solvers implementing First OrderLogic (FOL), such as integer or array theory, to ease the problem description.The SMT solver acts as a translator of the FOL constraints to the SAT solver,which finds the satisfiability of the problem. If the set of constraints is satisfi-able, it returns a model as a proof of such satisfiability. In case the constraintsare not satisfiable, the solver tells this and provides a counter-example.

SMT solvers are usually applied for verification problems [19], but the sig-nificant increase of performance of SAT and SMT solvers in the last yearshas allowed for applying them to more complex problems such as scheduling.There exist many different implementations of SMT solvers. In this thesis, weimplement the scheduler with two state-of-the-art solvers, Z3 [20] and Yices


NetworkConstrains

Scheduler

Schedule

Figure 2.4: Process to obtain schedules with SMT or ILP solvers

2.2.1 Scheduling Tools

Satisfiability Modulo Theories

Satisfiability (SAT) solvers are tools that determine the satisfiability of booleanformulas applying algorithms such as DPLL [18]. In other words, it proves ifthere exists a solution for the given boolean formula constraints. As describ-ing a problem with only boolean formulas can be challenging, SatisfiabilityModulo Theories (SMT) solvers extend SAT solvers implementing First OrderLogic (FOL), such as integer or array theory, to ease the problem description.The SMT solver acts as a translator of the FOL constraints to the SAT solver,which finds the satisfiability of the problem. If the set of constraints is satisfi-able, it returns a model as a proof of such satisfiability. In case the constraintsare not satisfiable, the solver tells this and provides a counter-example.

SMT solvers are usually applied for verification problems [19], but the sig-nificant increase of performance of SAT and SMT solvers in the last yearshas allowed for applying them to more complex problems such as scheduling.There exist many different implementations of SMT solvers. In this thesis, weimplement the scheduler with two state-of-the-art solvers, Z3 [20] and Yices

36


2 [21]. Z3 supports multiple language API while Yices 2 only supports C andSMT-lib 2.0 [22], a standard language to specify a model for SMT solvers.Still, we mostly employ Yices 2 because it provides better performance for ourspecific scheduling cases.

Integer Linear Programming

Linear Programming (LP) is a framework for solving linear problems whilealso obtaining an optimized solution given an objective. There exist variants ofLP tailored to problems at hand. For example, Integer LP (ILP) involves onlyinteger variables, while if extended with float variables, it is called Mixed-Integer LP (MILP). The main difference from SMT solvers is the definitionof an objective function to find a specific solution from a range of valid solu-tions. But if no objective is defined, it finds a satisfiable solution just like SMT.However, the internal algorithms implemented are very different; ILP solversmainly apply the simplex method [23] and the integer-point method [24].

ILP solvers have been widely used in many different areas, notably in plan-ning and scheduling [25]. There exist many different solver implementations,both free and commercial, but their performance vary widely, up to two ordersof magnitude in some cases. In this thesis, we exclusively use the Gurobi ILPSolver [26] under its academic license.

2.2.2 Time-Triggered Scheduling Constraint Definition

In this subsection, we present the formulation of all the scheduling constraintsand objective functions. Slight variations can be found in the publications.

Objective Function The objective function defines the sum of distances be-tween frame transmissions, which is maximized by the solver. Two new vari-ables classes are defined to account for frame distances: frame and link inter-missions. The frame intermission, represented by one variable for each frame,expresses the distance between the same frame transmissions over subsequentlinks of path. While the link intermission, one variable for each link, representsthe distance between frames transmitted on the same link. A more detailed ex-planation can be found in Section 5.2. The objective function is to maximizethe summation of all the frames and links intermissions variables:


2 [21]. Z3 supports multiple language API while Yices 2 only supports C andSMT-lib 2.0 [22], a standard language to specify a model for SMT solvers.Still, we mostly employ Yices 2 because it provides better performance for ourspecific scheduling cases.

Integer Linear Programming

Linear Programming (LP) is a framework for solving linear problems whilealso obtaining an optimized solution given an objective. There exist variants ofLP tailored to problems at hand. For example, Integer LP (ILP) involves onlyinteger variables, while if extended with float variables, it is called Mixed-Integer LP (MILP). The main difference from SMT solvers is the definitionof an objective function to find a specific solution from a range of valid solu-tions. But if no objective is defined, it finds a satisfiable solution just like SMT.However, the internal algorithms implemented are very different; ILP solversmainly apply the simplex method [23] and the integer-point method [24].

ILP solvers have been widely used in many different areas, notably in plan-ning and scheduling [25]. There exist many different solver implementations,both free and commercial, but their performance vary widely, up to two ordersof magnitude in some cases. In this thesis, we exclusively use the Gurobi ILPSolver [26] under its academic license.

2.2.2 Time-Triggered Scheduling Constraint Definition

In this subsection, we present the formulation of all the scheduling constraintsand objective functions. Slight variations can be found in the publications.

Objective Function The objective function defines the sum of distances be-tween frame transmissions, which is maximized by the solver. Two new vari-ables classes are defined to account for frame distances: frame and link inter-missions. The frame intermission, represented by one variable for each frame,expresses the distance between the same frame transmissions over subsequentlinks of path. While the link intermission, one variable for each link, representsthe distance between frames transmitted on the same link. A more detailed ex-planation can be found in Section 5.2. The objective function is to maximizethe summation of all the frames and links intermissions variables:

37


maximize|F |∑f=1

(wf ∗ If ) +

|L|∑l=1

(wl ∗ Il) (2.4)

where wf is a user-defined weight influencing all the frame intermissionsand wl all the link intermissions. Different values of such weights impact thepreference to maximize frames or link intermissions. Note that Il denotes alink intermission and If denotes a frame intermission.

Frame Period Constraints:

All frame instances allocations should be transmitted periodically. We mustensure that all frame instances are within their specific period range:

∀f ∈ F,∀i ∈ Nf ,∀k ∈ K,∀l ∈ L :

(0 ∗ i) < Φf (i, l, k) + δlf ≤ (Tf ∗ i) (2.5)

Frame Deadline Constraint:

To provide a frame deadline shorter than the period (Df < Tf ), we need tolimit the range of the first instance period constraint:

∀f ∈ F,∀i ∈ Nf ,∀k ∈ K,∀l ∈ L :

(0 ∗ i) < Φf (f, l, k) + δlf ≤ (Tf ∗ (i− 1) +Df ) (2.6)

Replica constraints:

These constraints are applied to the wireless links, where temporal redundancyis required. As described in Subsection 2.1.2 ITIsize defines the distance ofthe ITI between different replicas:

∀f ∈ F,∀i ∈ Nf ,∀k ∈ K,∀l ∈ L :

Φf (i, l, k) = Φf (i, l, 1) + (k − 1)× ITIsize (2.7)


maximize|F |∑f=1

(wf ∗ If ) +

|L|∑l=1

(wl ∗ Il) (2.4)

where wf is a user-defined weight influencing all the frame intermissionsand wl all the link intermissions. Different values of such weights impact thepreference to maximize frames or link intermissions. Note that Il denotes alink intermission and If denotes a frame intermission.

Frame Period Constraints:

All frame instances allocations should be transmitted periodically. We mustensure that all frame instances are within their specific period range:

∀f ∈ F,∀i ∈ Nf ,∀k ∈ K,∀l ∈ L :

(0 ∗ i) < Φf (i, l, k) + δlf ≤ (Tf ∗ i) (2.5)

Frame Deadline Constraint:

To provide a frame deadline shorter than the period (Df < Tf ), we need tolimit the range of the first instance period constraint:

∀f ∈ F,∀i ∈ Nf ,∀k ∈ K,∀l ∈ L :

(0 ∗ i) < Φf (f, l, k) + δlf ≤ (Tf ∗ (i− 1) +Df ) (2.6)

Replica constraints:

These constraints are applied to the wireless links, where temporal redundancyis required. As described in Subsection 2.1.2 ITIsize defines the distance ofthe ITI between different replicas:

∀f ∈ F,∀i ∈ Nf ,∀k ∈ K,∀l ∈ L :

Φf (i, l, k) = Φf (i, l, 1) + (k − 1)× ITIsize (2.7)

38


Avoid-Collision Constraints:

In a collision domain, only one frame can be transmitted at the same time. It isindispensable to avoid collision between frames not to lose any transmission.This constraint ensures that no frame starts its transmission when another frameis being transmitted in the same collision domain. Moreover, it also ensuresthat no transmission is allowed if it cannot finish before another scheduledtransmission starts:

∀f ∈ F,∀i ∈ Nf ,∀k ∈ K,∀lf ∈ TPf ,

∀g ∈ F,∀j ∈ Nf ,∀h ∈ K,∀lg ∈ TPg,

f 6= g,∀cd ∈ CD, lf ∧ lg ∈ cd :

a = 1→ Φf (i, lf , k) + δlff + Ilf ≤ Φg(j, lg, h)

b = 1→ Φg(j, lg, h) + δlgg + Ilg ≤ Φf (i, lf , k)

a+ b = 1 (2.8)

where a = 1 indicates that the frame instance of f is being transmittedbefore g. On the other hand, if b = 1 then the frame instance of g is transmittedbefore f . Last, we ensure that either one the cases is active with a+ b = 1.

Ensure-Causality Constraints:

Frame paths need to follow a sequenced order of links from the sender to thereceiver. We can model it by not allowing a switch to relay a frame that it hasnot previously received.

∀f ∈ F,∀i ∈ Nf ,∀(vx, vy), (vy, vz) ∈ TPf :

Φf (i, (vy, vz), 1)− Φf (i, (vx, vy), 1) ≥ hopdelay + If (2.9)

where hopdelay is the processing time for a switch to relay the frame. Notethat once this constraint is defined for the first replica, the subsequent replicasalso fulfil the property transitively, due to the Replica constraints.

Avoid-Buffer-Overflow Constraints:

The schedule also needs to take into account the memory limitations of theintermediate switches. Modelling a maximum memory size is very challenging


Avoid-Collision Constraints:

In a collision domain, only one frame can be transmitted at the same time. It isindispensable to avoid collision between frames not to lose any transmission.This constraint ensures that no frame starts its transmission when another frameis being transmitted in the same collision domain. Moreover, it also ensuresthat no transmission is allowed if it cannot finish before another scheduledtransmission starts:

∀f ∈ F,∀i ∈ Nf ,∀k ∈ K,∀lf ∈ TPf ,

∀g ∈ F,∀j ∈ Nf ,∀h ∈ K,∀lg ∈ TPg,

f 6= g,∀cd ∈ CD, lf ∧ lg ∈ cd :

a = 1→ Φf (i, lf , k) + δlff + Ilf ≤ Φg(j, lg, h)

b = 1→ Φg(j, lg, h) + δlgg + Ilg ≤ Φf (i, lf , k)

a+ b = 1 (2.8)

where a = 1 indicates that the frame instance of f is being transmittedbefore g. On the other hand, if b = 1 then the frame instance of g is transmittedbefore f . Last, we ensure that either one the cases is active with a+ b = 1.

Ensure-Causality Constraints:

Frame paths need to follow a sequenced order of links from the sender to thereceiver. We can model it by not allowing a switch to relay a frame that it hasnot previously received.


Φf (i, (vy, vz), 1)− Φf (i, (vx, vy), 1) ≥ hopdelay + If (2.9)

where hopdelay is the processing time for a switch to relay the frame. Notethat once this constraint is defined for the first replica, the subsequent replicasalso fulfil the property transitively, due to the Replica constraints.

Avoid-Buffer-Overflow Constraints:

The schedule also needs to take into account the memory limitations of theintermediate switches. Modelling a maximum memory size is very challenging

39


and increases the complexity significantly. It would add a new dimension ofvariables to keep stored the state of the memory utilization of each switch ateach instant of time. Instead, we limit the time a frame can stay in the switch(memory):


Φf (i, (vy, vz), 1)− Φf (i, (vx, vy), 1) ≤ memory (2.10)

Simultaneous-Relay Constraints:

Some protocol implementations require the switch to relay a multicast framethrough multiple links at the same time. In particular, this happens whenevertwo links in the same tree path share the same predecessor:

∀f ∈ F,∀i ∈ Nf ,∀(vx, vy), (vx, vz) ∈ TPf :

Φf (i, (vx, vy), 1) = Φf (i, (vx, vz), 1) (2.11)

End-to-End Latency Constraints:

It is essential to limit the time elapsed between the start of the frame transmis-sion until when it arrives at all the receivers (endtime):

∀f ∈ F,∀i ∈ Nf ,∀(vs, vx), (vy, vr) ∈ TPf :

Φf (i, (vy, vr), 1)− Φf (i, (vs, vx), 1) < endtime (2.12)

Application Constraints:

Application constraints model a generic class of dependencies between frames.Many different sources might require these dependencies. For example, it iscommon that the task and network schedule are obtained separately; first, thetask schedule, and afterwards, the network schedule. In the task schedule, tasksmight have dependencies between them, such as waiting for a task executionto be requested by another task. The frames also inherit these dependencies.We can model them with the definition of application constraints such that aframe f ∈ F can only be received exactly after some time (indicated by theparameter app time) has passed since the reception of another frame g ∈ F .


and increases the complexity significantly. It would add a new dimension ofvariables to keep stored the state of the memory utilization of each switch ateach instant of time. Instead, we limit the time a frame can stay in the switch(memory):


Φf (i, (vy, vz), 1)− Φf (i, (vx, vy), 1) ≤ memory (2.10)

Simultaneous-Relay Constraints:

Some protocol implementations require the switch to relay a multicast framethrough multiple links at the same time. In particular, this happens whenevertwo links in the same tree path share the same predecessor:

∀f ∈ F,∀i ∈ Nf ,∀(vx, vy), (vx, vz) ∈ TPf :

Φf (i, (vx, vy), 1) = Φf (i, (vx, vz), 1) (2.11)

End-to-End Latency Constraints:

It is essential to limit the time elapsed between the start of the frame transmis-sion until when it arrives at all the receivers (endtime):

∀f ∈ F,∀i ∈ Nf ,∀(vs, vx), (vy, vr) ∈ TPf :

Φf (i, (vy, vr), 1)− Φf (i, (vs, vx), 1) < endtime (2.12)

Application Constraints:

Application constraints model a generic class of dependencies between frames.Many different sources might require these dependencies. For example, it iscommon that the task and network schedule are obtained separately; first, thetask schedule, and afterwards, the network schedule. In the task schedule, tasksmight have dependencies between them, such as waiting for a task executionto be requested by another task. The frames also inherit these dependencies.We can model them with the definition of application constraints such that aframe f ∈ F can only be received exactly after some time (indicated by theparameter app time) has passed since the reception of another frame g ∈ F .

40

2.3 Discussion on the System Model Selected 19

∀f ∈ F,∀i ∈ Nf ,∀g ∈ F,∀j ∈ Ng,

(vx, vr) ∈ TPf , (v′x, v′r) ∈ TPg :

Φf (i, (vx, vr), 1) = Φg(j, (v′x, v′r), 1) + app time (2.13)

Note that to map applications to frame transmissions, the constraint is de-fined only for the last link of the tree path, since all the predecessor verticesfulfil it transitively.

2.3 Discussion on the System Model SelectedWe selected a system model closely tied to TTEthernet. At the start of thisthesis, TTEthernet was one of the potential leading protocols for switched Eth-ernet, implemented on safety-critical systems such as NASA Orion [27] andBoeing 787 Dreamliner [28]. Moreover, the project that funded the first threeyears of this research was based on scheduling TTEhernet next-generation net-works. The standard IEEE 802.1Qbv (or TSN) was later published in 2015,and the first publication implementing a TSN scheduler appeared in 2016 [29].The main difference, as explained in 2.2, is the dispatch of frames by open-ing and closing control gates at the queue level instead of dispatching specificframes. In terms of the scheduling problem, the only difference is the inclusionof a new constraint, the frame isolation constraint [29], which ensures that atthe time the gates opens, only the frame to be transmitted is in that queue.

We decided to continue our research on a generalized time-triggeredswitched Ethernet implementation closer to TTEthernet instead of switchingto TSN. We wanted to keep a more generalized and basic system model thatcould be extended to other protocols such as TSN. The scheduling of next-generation networks can be extended to TSN by adding the frame isolationconstraint. The results obtained should be similar, as the new constraints addonly some extra complexity. In regards to schedule reparability, it can be im-plemented if the TSN switches have similar capabilities that were assumed toTTEthernet switches. In the TTEthernet scheduling literature, it is typically as-sumed that the switch frame handler can store and place frames into the FIFOqueues in the required sequence. However, in the first proposed TSN sched-ulers the frame handler does not store any frame, instead it directly forwardsthe frame to the specified FIFO queue. Moreover, only one frame can be inone FIFO queue at a time. Such an assumption is very restrictive and reduces

2.3 Discussion on the System Model Selected 19

∀f ∈ F,∀i ∈ Nf ,∀g ∈ F,∀j ∈ Ng,

(vx, vr) ∈ TPf , (v′x, v′r) ∈ TPg :

Φf (i, (vx, vr), 1) = Φg(j, (v′x, v′r), 1) + app time (2.13)

Note that to map applications to frame transmissions, the constraint is de-fined only for the last link of the tree path, since all the predecessor verticesfulfil it transitively.

2.3 Discussion on the System Model SelectedWe selected a system model closely tied to TTEthernet. At the start of thisthesis, TTEthernet was one of the potential leading protocols for switched Eth-ernet, implemented on safety-critical systems such as NASA Orion [27] andBoeing 787 Dreamliner [28]. Moreover, the project that funded the first threeyears of this research was based on scheduling TTEhernet next-generation net-works. The standard IEEE 802.1Qbv (or TSN) was later published in 2015,and the first publication implementing a TSN scheduler appeared in 2016 [29].The main difference, as explained in 2.2, is the dispatch of frames by open-ing and closing control gates at the queue level instead of dispatching specificframes. In terms of the scheduling problem, the only difference is the inclusionof a new constraint, the frame isolation constraint [29], which ensures that atthe time the gates opens, only the frame to be transmitted is in that queue.

We decided to continue our research on a generalized time-triggeredswitched Ethernet implementation closer to TTEthernet instead of switchingto TSN. We wanted to keep a more generalized and basic system model thatcould be extended to other protocols such as TSN. The scheduling of next-generation networks can be extended to TSN by adding the frame isolationconstraint. The results obtained should be similar, as the new constraints addonly some extra complexity. In regards to schedule reparability, it can be im-plemented if the TSN switches have similar capabilities that were assumed toTTEthernet switches. In the TTEthernet scheduling literature, it is typically as-sumed that the switch frame handler can store and place frames into the FIFOqueues in the required sequence. However, in the first proposed TSN sched-ulers the frame handler does not store any frame, instead it directly forwardsthe frame to the specified FIFO queue. Moreover, only one frame can be inone FIFO queue at a time. Such an assumption is very restrictive and reduces

41


the schedule reparability as frames are transmitted as a burst, minimizing theframe distances. But, if the TSN switch posses the same storing capabilitiesassumed to TTEthernet, high reparability schedules can also be obtained usingthe same objective function.


the schedule reparability as frames are transmitted as a burst, minimizing theframe distances. But, if the TSN switch posses the same storing capabilitiesassumed to TTEthernet, high reparability schedules can also be obtained usingthe same objective function.

42

Chapter 3

Related Work

In this chapter, we discuss work related to the primary two challenges for ap-plying the time-triggered paradigm in scheduling of next-generation networks:complexity and adaptability. We first overview the complexity of differentscheduling techniques applied to time-triggered protocols. Later, we extendthe study to decomposition approaches from areas where scheduling has beeninvestigated, and mature and scalable techniques exist. Finally, we study lit-erature that advances time-triggered towards more adaptive schedules, withparticular interest to fault-tolerance strategies.

3.1 Scheduling Scalability

3.1.1 Scheduling for Time-Triggered NetworksWe present the current techniques and approaches being applied to synthesizeschedules for time-triggered communication in bus-based networks, networks-on-chip, and switched networks.

Bus-Based Protocols

Obtaining time-triggered schedules for a single bus is a relatively small prob-lem. Researchers have focused on jointly designing the network and taskschedules, emphasizing an incremental design that allow the designer to addnew functionalities to the system quickly. Zheng et al. have applied a math-ematical constraint solver to synthesize of such incremental schedules [30].

21

Chapter 3

Related Work

In this chapter, we discuss work related to the primary two challenges for ap-plying the time-triggered paradigm in scheduling of next-generation networks:complexity and adaptability. We first overview the complexity of differentscheduling techniques applied to time-triggered protocols. Later, we extendthe study to decomposition approaches from areas where scheduling has beeninvestigated, and mature and scalable techniques exist. Finally, we study lit-erature that advances time-triggered towards more adaptive schedules, withparticular interest to fault-tolerance strategies.

3.1 Scheduling Scalability

3.1.1 Scheduling for Time-Triggered NetworksWe present the current techniques and approaches being applied to synthesizeschedules for time-triggered communication in bus-based networks, networks-on-chip, and switched networks.

Bus-Based Protocols

Obtaining time-triggered schedules for a single bus is a relatively small prob-lem. Researchers have focused on jointly designing the network and taskschedules, emphasizing an incremental design that allow the designer to addnew functionalities to the system quickly. Zheng et al. have applied a math-ematical constraint solver to synthesize of such incremental schedules [30].

21

43

22 Chapter 3. Related Work

Later literature has focused on optimizing different systems parameters suchas timing, control stability, and performance, applying ILP solvers [31, 32].When FlexRay allowed the development of larger and more complex systems,simple approaches could not synthesize schedules in a reasonable amount oftime. Lukasiweycz et al. [33] have reduced the synthesis time applying a hi-erarchical approach which solves the schedule in steps from single computingnodes to the whole system to reduce the synthesis time. Researchers have alsoinvestigated techniques to optimize different schedule metrics, such as reducedjitter [34], reduced utilization [35], or increasing extensibility [36]. Zeng etal. [37] have computed schedules with Mixed-Integer Linear Programming(MILP). They exhibit higher scalability in the number of nodes compared tothe previously mentioned work, and the possibility of obtaining optimal sched-ules for the cases studied taken from actual implemented systems.

Network-on-Chip Protocols

Multi-Processor-System-on-a-Chip implements a Network-on-a-Chip to com-municate between the different nodes [38]. For many hard real-time appli-cations, the network implements time-triggered communication to satisfy thetiming requirements. The physical topology in these networks is typically agrid connection between nodes in which the designer has to produce a vir-tual topology where the time-triggered messages are going to be transmitted.Researchers have also implemented network-on-a-chip schedulers with ILPsolvers [39][40]. Scholer et al. [41] have performed a comparison betweenusing MILP and Satisfiability Boolean (SAT) solvers to synthesize schedules.SAT solvers, in contrast to ILP, do not have any built-in optimization capabil-ity, they only search for a valid (satisfiable) solution for boolean constraints,and are proven to be much faster if no optimization is needed. Biewer et al.[42] have applied an SMT solver to synthesize the schedule, but they have alsoincluded the design of the virtual topology combining the SMT solver with anAnswer Set Solver. Andres et al. have investigated an improved version inwhich they look into the heuristic inside both solvers to find a better fit of theconfiguration parameters for the synthesis problem [43]. Huang et al. haveresearched the combination with heuristics to improve the scalability of pureSMT-based approaches applying an incremental approach [44].


Later literature has focused on optimizing different systems parameters suchas timing, control stability, and performance, applying ILP solvers [31, 32].When FlexRay allowed the development of larger and more complex systems,simple approaches could not synthesize schedules in a reasonable amount oftime. Lukasiweycz et al. [33] have reduced the synthesis time applying a hi-erarchical approach which solves the schedule in steps from single computingnodes to the whole system to reduce the synthesis time. Researchers have alsoinvestigated techniques to optimize different schedule metrics, such as reducedjitter [34], reduced utilization [35], or increasing extensibility [36]. Zeng etal. [37] have computed schedules with Mixed-Integer Linear Programming(MILP). They exhibit higher scalability in the number of nodes compared tothe previously mentioned work, and the possibility of obtaining optimal sched-ules for the cases studied taken from actual implemented systems.

Network-on-Chip Protocols

Multi-Processor-System-on-a-Chip implements a Network-on-a-Chip to com-municate between the different nodes [38]. For many hard real-time appli-cations, the network implements time-triggered communication to satisfy thetiming requirements. The physical topology in these networks is typically agrid connection between nodes in which the designer has to produce a vir-tual topology where the time-triggered messages are going to be transmitted.Researchers have also implemented network-on-a-chip schedulers with ILPsolvers [39][40]. Scholer et al. [41] have performed a comparison betweenusing MILP and Satisfiability Boolean (SAT) solvers to synthesize schedules.SAT solvers, in contrast to ILP, do not have any built-in optimization capabil-ity, they only search for a valid (satisfiable) solution for boolean constraints,and are proven to be much faster if no optimization is needed. Biewer et al.[42] have applied an SMT solver to synthesize the schedule, but they have alsoincluded the design of the virtual topology combining the SMT solver with anAnswer Set Solver. Andres et al. have investigated an improved version inwhich they look into the heuristic inside both solvers to find a better fit of theconfiguration parameters for the synthesis problem [43]. Huang et al. haveresearched the combination with heuristics to improve the scalability of pureSMT-based approaches applying an incremental approach [44].

44

3.1 Scheduling Scalability 23

Switched-Ethernet Protocols

Time-Triggered Switched Ethernet networks were designed to cope with theincrease in size and bandwidth requirements of modern applications. This in-crease of complexity has required for application of different meta-heuristicsto obtain optimized schedules, such as Neighborhood Search [45] and TabuSearch [46]. Steiner has also applied SMT solvers when the schedule did notdemand any optimization parameter. Moreover, Steiner has implemented anincremental approach that was able to solve the scalability issues of single-shot approaches [16]. This approach builds the schedule incrementally stepby step, in which the allocation of a few frames is done in every step. Theperformance increase is notable, allowing to find schedules of networks withone order of magnitude more frames (thousand frames) in less time. Otherresearches have also applied similar approaches into networks following theTSN. They have applied Tabu Search [47], SMT solvers [48] and OptimizationModulo Theories (OMT) Solvers [29], an augmented SMT solver with opti-mization capabilities. Due to the increase of performance of the incrementalapproach, Craciunas et al. and Zhang et al. have investigated how to increasethe schedulability co-designing the network and task schedules together apply-ing Mixed Integer Programming (MIP) [49] and SMT solvers [50]. Craciunaset al. [51] have also implemented a two-step approach, in which (1) most con-strained frames and task are first scheduled, and (2) the remaining frames andtasks are scheduled with an incremental approach. This approach has goodperformance for large networks (thousands of frames). The authors made acritical reflection on the scalability of schedule synthesis, where they observethat scheduling extremely large networks cannot be done with similar toolsand approaches as the problem becomes intractable. They suggest that a com-bination of heuristics and solvers is required. This thesis came to the sameconclusion when exploring the limitations of state-of-the-art schedulers [52]and different heuristics to divide the problem into easily solvable segments bySMT and ILP solvers are proposed.

Wireless Protocols

A new trend, especially in factory networks, is the integration of wirelessin some network segments. Scheduling wireless communication is differentthan wired; one of the extra considerations being the possibility of losingframes. One way to solve this is to plan such losses and schedule with re-transmission [53]. To the best of the author’s knowledge, no schedule syn-thesis of hybrid wired-wireless networks has been studied before, but some

3.1 Scheduling Scalability 23

Switched-Ethernet Protocols

Time-Triggered Switched Ethernet networks were designed to cope with theincrease in size and bandwidth requirements of modern applications. This in-crease of complexity has required for application of different meta-heuristicsto obtain optimized schedules, such as Neighborhood Search [45] and TabuSearch [46]. Steiner has also applied SMT solvers when the schedule did notdemand any optimization parameter. Moreover, Steiner has implemented anincremental approach that was able to solve the scalability issues of single-shot approaches [16]. This approach builds the schedule incrementally stepby step, in which the allocation of a few frames is done in every step. Theperformance increase is notable, allowing to find schedules of networks withone order of magnitude more frames (thousand frames) in less time. Otherresearches have also applied similar approaches into networks following theTSN. They have applied Tabu Search [47], SMT solvers [48] and OptimizationModulo Theories (OMT) Solvers [29], an augmented SMT solver with opti-mization capabilities. Due to the increase of performance of the incrementalapproach, Craciunas et al. and Zhang et al. have investigated how to increasethe schedulability co-designing the network and task schedules together apply-ing Mixed Integer Programming (MIP) [49] and SMT solvers [50]. Craciunaset al. [51] have also implemented a two-step approach, in which (1) most con-strained frames and task are first scheduled, and (2) the remaining frames andtasks are scheduled with an incremental approach. This approach has goodperformance for large networks (thousands of frames). The authors made acritical reflection on the scalability of schedule synthesis, where they observethat scheduling extremely large networks cannot be done with similar toolsand approaches as the problem becomes intractable. They suggest that a com-bination of heuristics and solvers is required. This thesis came to the sameconclusion when exploring the limitations of state-of-the-art schedulers [52]and different heuristics to divide the problem into easily solvable segments bySMT and ILP solvers are proposed.

Wireless Protocols

A new trend, especially in factory networks, is the integration of wirelessin some network segments. Scheduling wireless communication is differentthan wired; one of the extra considerations being the possibility of losingframes. One way to solve this is to plan such losses and schedule with re-transmission [53]. To the best of the author’s knowledge, no schedule syn-thesis of hybrid wired-wireless networks has been studied before, but some

45


research has been done on fully wireless time-triggered networks. Kirton etal. [54] have performed scheduling of Wireless Sensor Networks optimizingsecurity-aware characteristics by applying genetic algorithms. Saifullah et al.[55] have obtained WirelessHART schedules with a heuristic-based algorithmcalled Conflict-aware Least Laxity First (C-LLF), a modification of the LeastLaxity First (LLF) algorithm used in many wired scheduling algorithms. Theapproach considers the collisions that appear in wireless communication toavoid conflicts between different frames. In this thesis, the integration of wiredand wireless mixed networks is studied, and the possibility of multiple colli-sion domains are defined where collisions between wireless transmissions areavoided.

3.1.2 Divide-and-Conquer Approaches

Even though the time-triggered schedule synthesis has not yet faced severescalability problems in industry applications, other communities have alreadyencountered them and proposed different divide-and-conquer approaches as aremedy. Railway scheduling is very similar to switched network scheduling,because trains, just as frames, travel from an origin to a destination passingintermediate stations, corresponding to sender, receiver and switches. Manydifferent decomposition techniques have been proposed, Caimi et al. [56] havesegmented the railway network into condensation zones where substantial traf-fic is expected (urban centres) and compensation zones which usually connectscondensation zones and have less traffic with considerable room for manage-ment. They solved each segmented scheduling problem by applying an ILPsolver. Juette et al. [57] have proposed another decomposition approach, thatfocuses on larger railway scheduling problems, decomposing the whole net-work in weakly overlapping regions that are later solved with an ILP solver.

Job Shop scheduling is also very similar to switched network scheduling.Jobs have to pass different machines in a specific order to be completed. Asignificant difference with other scheduling problems is that jobs are usuallyhighly dependent among them, similar to large networks containing a highamount of application constraints. Dependencies are the main challenge toapply divide-and-conquer approaches in time-triggered scheduling. Zhang etal. [58] have applied simulated annealing to extract one subproblem of thewhole problem and to schedule it applying another metaheuristic, a genetic al-gorithm. The same authors also have developed another decomposition policy,simulated annealing decomposing, but based on upper and lower bounds onthe time to finish the jobs [59]. Later, they have also applied a particle swarm


research has been done on fully wireless time-triggered networks. Kirton etal. [54] have performed scheduling of Wireless Sensor Networks optimizingsecurity-aware characteristics by applying genetic algorithms. Saifullah et al.[55] have obtained WirelessHART schedules with a heuristic-based algorithmcalled Conflict-aware Least Laxity First (C-LLF), a modification of the LeastLaxity First (LLF) algorithm used in many wired scheduling algorithms. Theapproach considers the collisions that appear in wireless communication toavoid conflicts between different frames. In this thesis, the integration of wiredand wireless mixed networks is studied, and the possibility of multiple colli-sion domains are defined where collisions between wireless transmissions areavoided.

3.1.2 Divide-and-Conquer Approaches

Even though the time-triggered schedule synthesis has not yet faced severescalability problems in industry applications, other communities have alreadyencountered them and proposed different divide-and-conquer approaches as aremedy. Railway scheduling is very similar to switched network scheduling,because trains, just as frames, travel from an origin to a destination passingintermediate stations, corresponding to sender, receiver and switches. Manydifferent decomposition techniques have been proposed, Caimi et al. [56] havesegmented the railway network into condensation zones where substantial traf-fic is expected (urban centres) and compensation zones which usually connectscondensation zones and have less traffic with considerable room for manage-ment. They solved each segmented scheduling problem by applying an ILPsolver. Juette et al. [57] have proposed another decomposition approach, thatfocuses on larger railway scheduling problems, decomposing the whole net-work in weakly overlapping regions that are later solved with an ILP solver.

Job Shop scheduling is also very similar to switched network scheduling.Jobs have to pass different machines in a specific order to be completed. Asignificant difference with other scheduling problems is that jobs are usuallyhighly dependent among them, similar to large networks containing a highamount of application constraints. Dependencies are the main challenge toapply divide-and-conquer approaches in time-triggered scheduling. Zhang etal. [58] have applied simulated annealing to extract one subproblem of thewhole problem and to schedule it applying another metaheuristic, a genetic al-gorithm. The same authors also have developed another decomposition policy,simulated annealing decomposing, but based on upper and lower bounds onthe time to finish the jobs [59]. Later, they have also applied a particle swarm

46

3.2 Adaptive Time-Triggered Networks 25

metaheuristic to solve each problem, which is shown to provide high-qualitysolutions [58]. However, the literature does not focus only on metaheuristics.For a specific job shop scheduling problem, steel plant production, Harjunkoskiet al. [60] have scheduled it implementing the decomposition approach withMILP. As it is a specific case, a smarter decomposition can be performed usingspecific parameters found in steel plant production. It is crucial for applyingthe decomposition approach to find the specific parameters of the schedulingproblem. The segmented approach presented in this thesis has been inspired inthese more mature areas to adapt a specific decomposition for time-triggeredschedules in segments divided by time.

3.2 Adaptive Time-Triggered Networks

Researches have widely investigated the topic of increasing fault-tolerance oftime-triggered networks at design time. These strategies increase the robust-ness of the networks to potential failures. They can be classified into two types:active replication and passive replication. In active replication, schedules areobtained with replicated frames to increase the guarantees of frame arrivalseven after failures are present in the network. Pop et al. [61] have proposedsuch re-execution of frames to deal with transient failures. Later, Wisniewski etal. [62][63] have suggested using disjoints paths for the replicated frames. Re-searches have looked into including the topology synthesis into the problem toproduce more robust and disjoints paths for the replicated frames [64][65][66].The disadvantage of active robustness is that the replicated frames occupybandwidth that others frame could use. Passive replication does not exhibitthis problem, since it applies dormant or quasi-static schedules [67]. A setof quasi-static schedules to cope with potential failures is calculated at designtime. These schedules can be switched at run time, to recover to a correctschedule, in case of failure. Novak et al. [68] have mixed these with F-shapedframes, where replicated frames are only transmitted when they are required.Both active and latent methods provide an almost instantaneous reaction time,but they cover only a limited number of cases. If one or multiple failures oc-cur that were not taken into account, the network is not able to tolerate themand will fail to transmit frames. Moreover, these techniques are not scalable tolarge networks. As the number of failures is proportional to the network size,it becomes harder to cover the majority of cases.

A strategy to cope with unpredicted network changes is to recalculate thewhole network schedule during run-time. Re-scheduling always returns a valid

3.2 Adaptive Time-Triggered Networks 25

metaheuristic to solve each problem, which is shown to provide high-qualitysolutions [58]. However, the literature does not focus only on metaheuristics.For a specific job shop scheduling problem, steel plant production, Harjunkoskiet al. [60] have scheduled it implementing the decomposition approach withMILP. As it is a specific case, a smarter decomposition can be performed usingspecific parameters found in steel plant production. It is crucial for applyingthe decomposition approach to find the specific parameters of the schedulingproblem. The segmented approach presented in this thesis has been inspired inthese more mature areas to adapt a specific decomposition for time-triggeredschedules in segments divided by time.

3.2 Adaptive Time-Triggered Networks

Researches have widely investigated the topic of increasing fault-tolerance oftime-triggered networks at design time. These strategies increase the robust-ness of the networks to potential failures. They can be classified into two types:active replication and passive replication. In active replication, schedules areobtained with replicated frames to increase the guarantees of frame arrivalseven after failures are present in the network. Pop et al. [61] have proposedsuch re-execution of frames to deal with transient failures. Later, Wisniewski etal. [62][63] have suggested using disjoints paths for the replicated frames. Re-searches have looked into including the topology synthesis into the problem toproduce more robust and disjoints paths for the replicated frames [64][65][66].The disadvantage of active robustness is that the replicated frames occupybandwidth that others frame could use. Passive replication does not exhibitthis problem, since it applies dormant or quasi-static schedules [67]. A setof quasi-static schedules to cope with potential failures is calculated at designtime. These schedules can be switched at run time, to recover to a correctschedule, in case of failure. Novak et al. [68] have mixed these with F-shapedframes, where replicated frames are only transmitted when they are required.Both active and latent methods provide an almost instantaneous reaction time,but they cover only a limited number of cases. If one or multiple failures oc-cur that were not taken into account, the network is not able to tolerate themand will fail to transmit frames. Moreover, these techniques are not scalable tolarge networks. As the number of failures is proportional to the network size,it becomes harder to cover the majority of cases.

A strategy to cope with unpredicted network changes is to recalculate thewhole network schedule during run-time. Re-scheduling always returns a valid

47


schedule if a solution exists after the component failure. It also can be appliedwhen new components or traffic are required to be introduced to the networkduring run-time. Zhang et al. [69] have proposed a central node or cloud-basedsolving to find new schedules after changes and upload it to the network. How-ever, scheduling a new network can take several minutes, and it is not scalableto large networks. To cope with the complexity, Nayak et al. [70] have pro-posed a simplification of the scheduling problem to reduce the synthesis time toseconds. The obtained schedule was then distributed thanks to the implemen-tation of time-triggered software-defined networks. The main drawback of thisapproach is the limitations of only seven hops in the network and maximumutilization of 13%.

Some literature investigates on modifying only a part of the schedule, ratherthan a complete rescheduling, to reduce the response time of reacting to net-work changes during run-time. Avni et al. [71] have implemented a combi-nation of offline and online techniques to tolerate k failures. At design time,the schedule is prepared to react to a link failure calling the online algorithmto modify the schedule following some guidelines. However, their algorithmspresent scalability issues [72]. Kandasamay et al. [73] have implemented an-other strategy where slacks in the schedule allow online policies to make smalladjustments of the frame timing. In the train time-triggered network litera-ture, researchers have applied a similar concept to add new network segments,where the traffic was changed at the cluster level [74][75]. Raagaard et al.[76] have opted for applying a quick list scheduling heuristic to re-allocateonly the affected frames after a failure. Such heuristic struggles to schedulehigh utilization networks but is fast in obtaining schedules. However, it stillshows an exponential increase in synthesis time when the traffic or networksize is substantial. In their evaluated networks, far from the requirements ofnext-generation networks, the synthesis time increases from an acceptable mil-liseconds’ range to the unacceptable seconds’ range. This thesis work seeksto reduce the scalability compared to the mentioned approaches by further re-ducing the scheduling problem (or healing) by enclosing the schedule modifi-cations when a failure occurs during runtime. The healing encapsulation helpsreducing scalability when the number of nodes is large as it is independent ofnetwork size.


schedule if a solution exists after the component failure. It also can be appliedwhen new components or traffic are required to be introduced to the networkduring run-time. Zhang et al. [69] have proposed a central node or cloud-basedsolving to find new schedules after changes and upload it to the network. How-ever, scheduling a new network can take several minutes, and it is not scalableto large networks. To cope with the complexity, Nayak et al. [70] have pro-posed a simplification of the scheduling problem to reduce the synthesis time toseconds. The obtained schedule was then distributed thanks to the implemen-tation of time-triggered software-defined networks. The main drawback of thisapproach is the limitations of only seven hops in the network and maximumutilization of 13%.

Some literature investigates on modifying only a part of the schedule, ratherthan a complete rescheduling, to reduce the response time of reacting to net-work changes during run-time. Avni et al. [71] have implemented a combi-nation of offline and online techniques to tolerate k failures. At design time,the schedule is prepared to react to a link failure calling the online algorithmto modify the schedule following some guidelines. However, their algorithmspresent scalability issues [72]. Kandasamay et al. [73] have implemented an-other strategy where slacks in the schedule allow online policies to make smalladjustments of the frame timing. In the train time-triggered network litera-ture, researchers have applied a similar concept to add new network segments,where the traffic was changed at the cluster level [74][75]. Raagaard et al.[76] have opted for applying a quick list scheduling heuristic to re-allocateonly the affected frames after a failure. Such heuristic struggles to schedulehigh utilization networks but is fast in obtaining schedules. However, it stillshows an exponential increase in synthesis time when the traffic or networksize is substantial. In their evaluated networks, far from the requirements ofnext-generation networks, the synthesis time increases from an acceptable mil-liseconds’ range to the unacceptable seconds’ range. This thesis work seeksto reduce the scalability compared to the mentioned approaches by further re-ducing the scheduling problem (or healing) by enclosing the schedule modifi-cations when a failure occurs during runtime. The healing encapsulation helpsreducing scalability when the number of nodes is large as it is independent ofnetwork size.

48

Chapter 4

Problem Formulation

In this chapter, we introduce the research problem in Section 4.1, and the over-all goal, together with the decomposition in sub-goals, in Section 4.2. Wefinish the chapter, in Section 4.3, presenting the methodology used to achievethe goals in this thesis.

4.1 Research Problem

The time-triggered paradigm has been widely applied in safety-critical sys-tems to satisfy their most strict real-time requirements, low latency and relia-bility among many others. Current industrial networks have been successfullyscheduled using meta-heuristics or different solvers, and several replicationtechniques bring fault-tolerance capabilities to achieve high reliability. How-ever, the size and complexity of next-generation networks accentuate a severedeficiency of current scheduling tools and fault-tolerant capabilities, their poorscalability. Scheduling times increase exponentially with the size of the net-work and traffic. Existing state-of-the-art schedulers are on the verge of beingunable to solve contemporary time-triggered network requirements. Moreover,the costs of full replication are increasing together with the size of networksin a market where safety requirements are demanded continuously on mass-produced systems for which a low-cost is essential.

27

Chapter 4

Problem Formulation

In this chapter, we introduce the research problem in Section 4.1, and the over-all goal, together with the decomposition in sub-goals, in Section 4.2. Wefinish the chapter, in Section 4.3, presenting the methodology used to achievethe goals in this thesis.

4.1 Research Problem

The time-triggered paradigm has been widely applied in safety-critical sys-tems to satisfy their most strict real-time requirements, low latency and relia-bility among many others. Current industrial networks have been successfullyscheduled using meta-heuristics or different solvers, and several replicationtechniques bring fault-tolerance capabilities to achieve high reliability. How-ever, the size and complexity of next-generation networks accentuate a severedeficiency of current scheduling tools and fault-tolerant capabilities, their poorscalability. Scheduling times increase exponentially with the size of the net-work and traffic. Existing state-of-the-art schedulers are on the verge of beingunable to solve contemporary time-triggered network requirements. Moreover,the costs of full replication are increasing together with the size of networksin a market where safety requirements are demanded continuously on mass-produced systems for which a low-cost is essential.

27

49

28 Chapter 4. Problem Formulation

4.2 Research GoalsBased on the Research Problem described in the previous section. The overallresearch goal is as follows:

Overall RG: Overcome the scalability issues that current state-of-the-art schedulers encounter when scheduling next-generation time-triggered networks. Moreover, advance the time-triggered paradigmfor the adaptive needs of fault-tolerance and constant evolution of next-generation networks.

The overall goal of this thesis is to advance the time-triggered paradigmmethods to the requirements of next-generation networks. Time-triggered net-works are increasing in size and complexity, as do the demands on the genera-tion of schedules, specifically in the time-triggered traffic. Current state-of-the-art solvers are already encountering scalability issues. Thus, our first researchgoal is:

Research Goal 1: Enable scheduling of next-generation time-triggerednetworks with the development of a divide-and-conquer approachscheduler.

By developing scalable schedulers, the primary impediment to adopt time-triggered in such large networks is addressed. However, the condition that suchnetworks are static over their life cycle is fading. Large networks might needto be adaptive to unpredicted changes as they present a higher rate of failuresdue to the increase of network components. Even though the first researchgoal enables scheduling, such schedulers are not fast enough to re-schedulethe whole network after every change without affecting the functionality of thesystem. In the second research goal, we want to investigate if it is possible toonly modify small schedule segments in a fast manner to avoid full re-schedule:

Research Goal 2: Defining the characteristics of the schedule that en-able small adjustments and avoid full re-schedule when undesiredchanges occur.

By addressing the second research goal, we maximize the success ratewhen healing the schedule after a component failure. However, it presumes


4.2 Research GoalsBased on the Research Problem described in the previous section. The overallresearch goal is as follows:

Overall RG: Overcome the scalability issues that current state-of-the-art schedulers encounter when scheduling next-generation time-triggered networks. Moreover, advance the time-triggered paradigmfor the adaptive needs of fault-tolerance and constant evolution of next-generation networks.

The overall goal of this thesis is to advance the time-triggered paradigmmethods to the requirements of next-generation networks. Time-triggered net-works are increasing in size and complexity, as do the demands on the genera-tion of schedules, specifically in the time-triggered traffic. Current state-of-the-art solvers are already encountering scalability issues. Thus, our first researchgoal is:

Research Goal 1: Enable scheduling of next-generation time-triggerednetworks with the development of a divide-and-conquer approachscheduler.

By developing scalable schedulers, the primary impediment to adopt time-triggered in such large networks is addressed. However, the condition that suchnetworks are static over their life cycle is fading. Large networks might needto be adaptive to unpredicted changes as they present a higher rate of failuresdue to the increase of network components. Even though the first researchgoal enables scheduling, such schedulers are not fast enough to re-schedulethe whole network after every change without affecting the functionality of thesystem. In the second research goal, we want to investigate if it is possible toonly modify small schedule segments in a fast manner to avoid full re-schedule:

Research Goal 2: Defining the characteristics of the schedule that en-able small adjustments and avoid full re-schedule when undesiredchanges occur.

By addressing the second research goal, we maximize the success ratewhen healing the schedule after a component failure. However, it presumes

50

4.3 Research Methodology 29

global knowledge of the schedule and a central solving component. Such as-sumptions are not realistic in large-scale networks and go against the philos-ophy of localized healing. By the third research goal, we want to study andimplement a distributed solution that can heal the network locally and achievea fast response time:

Research Goal 3: Implement distributed online protocols that heal theschedule during run-time after component failures.

4.3 Research MethodologyThe essential progress of our deductive research methodology [77] is shown inFigure 4.1. We start by establishing and defining the research problem that webelieve is crucial to apply time-triggered in next-generation networks. Our nextstep is to perform an intensive literature review of scientific papers. We beginby researching the state-of-the-art and practice in the research area of the prob-lem to solve. In the case of publications A and B, we study different schedul-ing tools used in the time-triggered literature and evaluate their performance.In publications C, D and E, the study was focused on different techniques thathave been implemented to tolerate failures in time-triggered networks. As weare trying to advance time-triggered to needs that are usually not considered,we also investigated the state-of-the-art in other more mature research areas.E.g., for publication B, divide-and-conquer techniques were intensively stud-ied in areas such as job shop or train scheduling. At this step, we can asses thesuitability of the current approach and propose a new approach to solve the re-search problem. The proposed solution is discussed with supervisors to assessits feasibility, and the expected results are written down for later comparison.

To test the validity of the proposed solution, we implement a software pro-totype of our solution. In the case of publication B, the segmented approachand its enhancements were coded in a scheduler. The scheduling of segmentswas obtained applying a state-of-the-art scheduler using SMT solvers that weremodified to better fit the segments solving requirements. Over the prototype de-velopment process, tests were executed to grasp the complexity of the software.In our implementations, we continuously encounter the need for coding com-plex algorithms that might affect the outcome of the scheduler considerably.Consistent refinements of our prototypes were required, with multiple imple-mentations of different schedulers to achieve better performance. For eachpublication, a new scheduler was coded to improve further its performance,


global knowledge of the schedule and a central solving component. Such as-sumptions are not realistic in large-scale networks and go against the philos-ophy of localized healing. By the third research goal, we want to study andimplement a distributed solution that can heal the network locally and achievea fast response time:

Research Goal 3: Implement distributed online protocols that heal theschedule during run-time after component failures.

4.3 Research MethodologyThe essential progress of our deductive research methodology [77] is shown inFigure 4.1. We start by establishing and defining the research problem that webelieve is crucial to apply time-triggered in next-generation networks. Our nextstep is to perform an intensive literature review of scientific papers. We beginby researching the state-of-the-art and practice in the research area of the prob-lem to solve. In the case of publications A and B, we study different schedul-ing tools used in the time-triggered literature and evaluate their performance.In publications C, D and E, the study was focused on different techniques thathave been implemented to tolerate failures in time-triggered networks. As weare trying to advance time-triggered to needs that are usually not considered,we also investigated the state-of-the-art in other more mature research areas.E.g., for publication B, divide-and-conquer techniques were intensively stud-ied in areas such as job shop or train scheduling. At this step, we can asses thesuitability of the current approach and propose a new approach to solve the re-search problem. The proposed solution is discussed with supervisors to assessits feasibility, and the expected results are written down for later comparison.

To test the validity of the proposed solution, we implement a software pro-totype of our solution. In the case of publication B, the segmented approachand its enhancements were coded in a scheduler. The scheduling of segmentswas obtained applying a state-of-the-art scheduler using SMT solvers that weremodified to better fit the segments solving requirements. Over the prototype de-velopment process, tests were executed to grasp the complexity of the software.In our implementations, we continuously encounter the need for coding com-plex algorithms that might affect the outcome of the scheduler considerably.Consistent refinements of our prototypes were required, with multiple imple-mentations of different schedulers to achieve better performance. For eachpublication, a new scheduler was coded to improve further its performance,

51


Figure 4.1: Research method cycle

applying the knowledge gained from previous deployments. In the case of pub-lication C, we had to shift from SMT to ILP solvers due to the introduction ofan objective function. SMT solvers were not conceived to introduce optimiza-tion capabilities, and ILP solvers significantly outperforms them. However, thepossibility of applying OMT solvers to obtain high reparable schedules weretested but failed as inspected in publication C.

Once we finished the prototype, we evaluated the solution. As there is alack of information about future next-generation networks requirements, wedeveloped a software that can generate synthetic networks and traffic. In thecase of publication B, we adapted an already existing extensive network used


Figure 4.1: Research method cycle

applying the knowledge gained from previous deployments. In the case of pub-lication C, we had to shift from SMT to ILP solvers due to the introduction ofan objective function. SMT solvers were not conceived to introduce optimiza-tion capabilities, and ILP solvers significantly outperforms them. However, thepossibility of applying OMT solvers to obtain high reparable schedules weretested but failed as inspected in publication C.

Once we finished the prototype, we evaluated the solution. As there is alack of information about future next-generation networks requirements, wedeveloped a software that can generate synthetic networks and traffic. In thecase of publication B, we adapted an already existing extensive network used

52


in a non-real-time application (MallorcaWIFI1), but that we believe can be apossible topology of next-generation networks. For publications D and E, wealso developed an event simulator to inject failures to the network and simu-late the protocol interactions to obtain the response times of the healing of theschedule. Once the results were as expected or better, a publication was con-sidered to be written. When the results were better than expected, we searchedfor an explanation. E.g., publication E patching response time was much bet-ter than publication D, even though they instinctively should be similar. Wediscovered that the language C implementation of publication E significantlyoutperformed the Python implementation of publication D. If the results wereunsatisfactory, we returned to the literature review and rethought the solution.The solution for publication C suffered from unsatisfactory results when ap-plying the healing algorithm to a schedule with low reparability. The furtherstudy of the problem made us realize the need for a schedule with differentproperties.

1The Cisco press release about this case study can be found here:https://meraki.cisco.com/customers/hospitality-and-tourism/city-of-palma-de-mallorca


in a non-real-time application (MallorcaWIFI1), but that we believe can be apossible topology of next-generation networks. For publications D and E, wealso developed an event simulator to inject failures to the network and simu-late the protocol interactions to obtain the response times of the healing of theschedule. Once the results were as expected or better, a publication was con-sidered to be written. When the results were better than expected, we searchedfor an explanation. E.g., publication E patching response time was much bet-ter than publication D, even though they instinctively should be similar. Wediscovered that the language C implementation of publication E significantlyoutperformed the Python implementation of publication D. If the results wereunsatisfactory, we returned to the literature review and rethought the solution.The solution for publication C suffered from unsatisfactory results when ap-plying the healing algorithm to a schedule with low reparability. The furtherstudy of the problem made us realize the need for a schedule with differentproperties.

1The Cisco press release about this case study can be found here:https://meraki.cisco.com/customers/hospitality-and-tourism/city-of-palma-de-mallorca

53

Chapter 5

Thesis Contributions

5.1 Contribution TC1: Synthesis of ExtremelyLarge Time-Triggered Networks Schedules

The first thesis contribution is the study of different varieties of divide-and-conquer approaches to overcome the scalability issues of synthesizing sched-ules for the next-generation time-triggered networks. Schedule synthesis isan NP-complete problem with complexity driven by the network size and theamount of traffic. State-of-the-art approaches, such as a combination of anincremental approach with AMT Solvers, can synthesize schedules of currentindustrial networks. However, we expect next-generation networks to have upto one order of magnitude more nodes and up to two orders of magnitude moretraffic.

To overcome the scalability issues, we considered segmenting the schedulesynthesis in small enough problems solvable with the coupling of incrementalapproach and SMT solvers. The motivation is to reset the SMT solver beforethe problem becomes too complex to handle. Over the length of this thesiswork, we proposed three different segmentation approaches, as seen in Figure5.1. The distinction between the three approaches arises on how the segmenta-tion is applied: amount of frames, number of time slots, and feedback from theSMT solver.

In the first segmentation, a user-defined number of frames is selected tobe allocated by the incremental approach. The obtained solution, a sched-ule segment, is then reallocated to the start of the schedule and locked so no

33

Chapter 5

Thesis Contributions

5.1 Contribution TC1: Synthesis of ExtremelyLarge Time-Triggered Networks Schedules

The first thesis contribution is the study of different varieties of divide-and-conquer approaches to overcome the scalability issues of synthesizing sched-ules for the next-generation time-triggered networks. Schedule synthesis isan NP-complete problem with complexity driven by the network size and theamount of traffic. State-of-the-art approaches, such as a combination of anincremental approach with AMT Solvers, can synthesize schedules of currentindustrial networks. However, we expect next-generation networks to have upto one order of magnitude more nodes and up to two orders of magnitude moretraffic.

To overcome the scalability issues, we considered segmenting the schedulesynthesis in small enough problems solvable with the coupling of incrementalapproach and SMT solvers. The motivation is to reset the SMT solver beforethe problem becomes too complex to handle. Over the length of this thesiswork, we proposed three different segmentation approaches, as seen in Figure5.1. The distinction between the three approaches arises on how the segmenta-tion is applied: amount of frames, number of time slots, and feedback from theSMT solver.

In the first segmentation, a user-defined number of frames is selected tobe allocated by the incremental approach. The obtained solution, a sched-ule segment, is then reallocated to the start of the schedule and locked so no

33

55

34 Chapter 5. Thesis Contributions

more frames can be allocated in it. Then, iteratively, a new group of framesis selected, scheduled, locked, and reallocated on top of the previously lockedsegment. This process is repeated until all frames have been allocated, and theresult is the whole schedule. The second segmentation approach seeks to re-duce the number of unused slots in a locked segment. A user-defined segmentsize, in time slots, is selected. As many frames as possible are allocated in thesegment until the incremental approach returns that it cannot schedule anotherframe, at which point the segment is locked. We found that this strategy leavesless unused slots in each segment, allowing to schedule higher utilization net-works. The last segmentation proposed pursues two improvements: to reducethe number of unused slots further and to avoid user-defined configuration pa-rameters. It follows a segmentation by solver feedback. If the SMT solverstarts presenting scalability issues, it automatically reduces the segment size.On the contrary, if it solves the segment fast, it increments the segment size.

There exist some challenges in applying the previously described ap-proaches. When resetting the SMT solver, all the constraints of the segment areforgotten, which might constitute a breach of constraints among different seg-ments. We proposed two solutions to assure that all the constraints concerningdifferent segments (inter-segment constraints) are satisfied. One solution intro-duces segment-specific constraints to the segment scheduling problem whilethe other solution selects the frames to be scheduled in a specific sequence toavoid such inter-segment constraints. An example of inter-segment constraintssolved by frame sequencing are application constraints. We implemented aframe pre-processing that sorts them in a specific order. A particular sequenc-ing when allocating frames can avoid some of these constraints to be violatedwithout requiring the involvement of the solver, which also reduces the com-plexity of the problem. An example of inter-segment constraints solved intro-ducing segment-specific constraints is the frame period constraints. We imple-mented a segment pre-processing, in which these constraints are added at thestart of each segment scheduling process. E.g., as seen in Figure 5.2, frames1, 5, 7, and 8 need to be re-transmitted to satisfy its period constraints. Weexamine its previous allocations and add constraints to ensure that no period-constraint is breached.

Next-generation networks might also contain wireless communications,which introduces a more complex collision domain of frames compared towired frames, that can only collide in a single link. This adds an extra layerof complexity to the problem. However, if we apply segmentation with SMTsolver feedback, the segment sizes automatically adapt so the complexity canbe handled.


more frames can be allocated in it. Then, iteratively, a new group of framesis selected, scheduled, locked, and reallocated on top of the previously lockedsegment. This process is repeated until all frames have been allocated, and theresult is the whole schedule. The second segmentation approach seeks to re-duce the number of unused slots in a locked segment. A user-defined segmentsize, in time slots, is selected. As many frames as possible are allocated in thesegment until the incremental approach returns that it cannot schedule anotherframe, at which point the segment is locked. We found that this strategy leavesless unused slots in each segment, allowing to schedule higher utilization net-works. The last segmentation proposed pursues two improvements: to reducethe number of unused slots further and to avoid user-defined configuration pa-rameters. It follows a segmentation by solver feedback. If the SMT solverstarts presenting scalability issues, it automatically reduces the segment size.On the contrary, if it solves the segment fast, it increments the segment size.

There exist some challenges in applying the previously described ap-proaches. When resetting the SMT solver, all the constraints of the segment areforgotten, which might constitute a breach of constraints among different seg-ments. We proposed two solutions to assure that all the constraints concerningdifferent segments (inter-segment constraints) are satisfied. One solution intro-duces segment-specific constraints to the segment scheduling problem whilethe other solution selects the frames to be scheduled in a specific sequence toavoid such inter-segment constraints. An example of inter-segment constraintssolved by frame sequencing are application constraints. We implemented aframe pre-processing that sorts them in a specific order. A particular sequenc-ing when allocating frames can avoid some of these constraints to be violatedwithout requiring the involvement of the solver, which also reduces the com-plexity of the problem. An example of inter-segment constraints solved intro-ducing segment-specific constraints is the frame period constraints. We imple-mented a segment pre-processing, in which these constraints are added at thestart of each segment scheduling process. E.g., as seen in Figure 5.2, frames1, 5, 7, and 8 need to be re-transmitted to satisfy its period constraints. Weexamine its previous allocations and add constraints to ensure that no period-constraint is breached.

Next-generation networks might also contain wireless communications,which introduces a more complex collision domain of frames compared towired frames, that can only collide in a single link. This adds an extra layerof complexity to the problem. However, if we apply segmentation with SMTsolver feedback, the segment sizes automatically adapt so the complexity canbe handled.

56

5.1 Contribution TC1: Synthesis of Extremely Large Time-TriggeredNetworks Schedules 35

Links

Off

sets

(tim

e)

1000 Frames

((a)) Incremental Approach

Links

Off

sets

(tim

e)

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

0

Y1

Y2

Y3

Y4

Y5

Y6

Y7

Y8

Y9

Y10

((b)) Segments by number offrames

Links

Off

sets

(tim

e)

X1 Frames

X2 Frames

X3 Frames

X4 Frames

X5 Frames

X6 Frames

X7 Frames

X8 Frames

X9 Frames

X10 Frames

0

100

200

300

400

500

600

700

800

900

1000

((c)) Segments by number of slots

Links

Off

sets

(tim

e)

X1 Frames

X2 Frames

X3 Frames

X4 Frames

X5 Frames

X6 Frames

X7 Frames

X8 Frames

X9 Frames

X10 Frames

0

Y1

Y2

Y3

Y4

Y5

Y6

Y7

Y8

Y9

Y10

((d)) Segments by Solver feedback

Figure 5.1: Comparison of schedule segments and 1000 frames allocationin the Incremental approach and the three different Divide-and-Conquer ap-proaches

5.1 Contribution TC1: Synthesis of Extremely Large Time-TriggeredNetworks Schedules 35

Links

Off

sets

(tim

e)

1000 Frames

((a)) Incremental Approach

Links

Off

sets

(tim

e)

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

100 Frames

0

Y1

Y2

Y3

Y4

Y5

Y6

Y7

Y8

Y9

Y10

((b)) Segments by number offrames

Links

Off

sets

(tim

e)

X1 Frames

X2 Frames

X3 Frames

X4 Frames

X5 Frames

X6 Frames

X7 Frames

X8 Frames

X9 Frames

X10 Frames

0

100

200

300

400

500

600

700

800

900

1000

((c)) Segments by number of slots

Links

Off

sets

(tim

e)

X1 Frames

X2 Frames

X3 Frames

X4 Frames

X5 Frames

X6 Frames

X7 Frames

X8 Frames

X9 Frames

X10 Frames

0

Y1

Y2

Y3

Y4

Y5

Y6

Y7

Y8

Y9

Y10

((d)) Segments by Solver feedback

Figure 5.1: Comparison of schedule segments and 1000 frames allocationin the Incremental approach and the three different Divide-and-Conquer ap-proaches

57


TimeSlot

Link 1

Link 2

Link 3

Link 4

1 2 3 4 5 6 7 8 9 10

Segment 1 Segment 2

F8

F1

F7

F8

F5

F1

F2

F7

F12

F5

F10

F2

F14

F12

F10

F14

F1

F1

F5

F5

F8

F8

F7

F7

Figure 5.2: Pre-processing of a segment to include constraints that fix framesneeded to be allocated in segment 2 by their inter-segment strictly periodicityconstraint

Conclusions: applying divide-and-conquer approaches makes the synthe-sis of next-generation network schedules possible without presenting scalabil-ity issues. We also proposed different techniques to satisfy all the networkconstraints, even when segmenting the scheduling problem.

5.2 Contribution TC2: Increasing Success Rate ofHealing Time-Triggered Network Schedules

In this contribution, we wanted to address the lack of adaptive capabilitiesof the time-triggered paradigm. As the amount of components increases, itis more probable that changes, expected or unexpected, increase in the net-work during run-time. Even though in the last contribution we obtained next-generation network schedules, the time needed (a few hours) is unreasonablyhigh if the schedule needs to be acquired during run-time. Current approachesonly address expected changes or calculate alternative schedules of a set ofpredicted unexpected changes to shift the schedule during run-time. However,as the network size grows, it is more likely that an unpredicted change wasnot considered which could provoke a network failure. We propose to reactto changes by solving them during run-time in the least time possible. Insteadof re-scheduling the whole network, we suggest a healing algorithm that mod-ifies the affected schedule segment to avoid frame losses. Unfortunately, we


TimeSlot

Link 1

Link 2

Link 3

Link 4

1 2 3 4 5 6 7 8 9 10

Segment 1 Segment 2

F8

F1

F7

F8

F5

F1

F2

F7

F12

F5

F10

F2

F14

F12

F10

F14

F1

F1

F5

F5

F8

F8

F7

F7

Figure 5.2: Pre-processing of a segment to include constraints that fix framesneeded to be allocated in segment 2 by their inter-segment strictly periodicityconstraint

Conclusions: applying divide-and-conquer approaches makes the synthe-sis of next-generation network schedules possible without presenting scalabil-ity issues. We also proposed different techniques to satisfy all the networkconstraints, even when segmenting the scheduling problem.

5.2 Contribution TC2: Increasing Success Rate ofHealing Time-Triggered Network Schedules

In this contribution, we wanted to address the lack of adaptive capabilitiesof the time-triggered paradigm. As the amount of components increases, itis more probable that changes, expected or unexpected, increase in the net-work during run-time. Even though in the last contribution we obtained next-generation network schedules, the time needed (a few hours) is unreasonablyhigh if the schedule needs to be acquired during run-time. Current approachesonly address expected changes or calculate alternative schedules of a set ofpredicted unexpected changes to shift the schedule during run-time. However,as the network size grows, it is more likely that an unpredicted change wasnot considered which could provoke a network failure. We propose to reactto changes by solving them during run-time in the least time possible. Insteadof re-scheduling the whole network, we suggest a healing algorithm that mod-ifies the affected schedule segment to avoid frame losses. Unfortunately, we

58

5.2 Contribution TC2: Increasing Success Rate of HealingTime-Triggered Network Schedules 37

N1

N2

N3

N4

N5 N6

1

2

3

4

5 6

7

8

9

10

11

125

9

Figure 5.3: Network with a failure in link 7 that has to redirect the frames tothe path with links 5 and 9

found that our healing algorithm failed in most cases. Schedules are usuallycompressed to reduce latency to the minimum. However, if we want to heala network, we need some empty slots to redistribute the affected traffic whilestill satisfying all the constraints.

For this reason, we propose to obtain schedules that increase the frame dis-tances as much as possible, within the latency constraints, to produce emptyslots in-between frames. To do so, we replaced the solvers we were using. ILPsolvers are slower than SMT solvers for scheduling, but they possess capabil-ities to define an objective. Our objective is to maximize the frame distancesover the whole schedule, a characteristic we call schedule reparability. An ex-ample can be seen in Figure 5.4, in which the failure of link 7 of Figure 5.3activates the healing algorithm. Our healing algorithm tries to find an alterna-tive path that connects both nodes connected to the faulty link, nodes 3 and 5,and redistribute the affected frames over the new path. If the schedule is com-pressed, as seen in the left schedule, there are no empty slots to redistribute thetraffic. However, if the schedule has a high reparability, the healing algorithmcan find free slots and modify the schedule.

Conclusions: when the initial schedule has a high reparability, the healingalgorithms show a considerably high success rate to heal link failures. As thealgorithm only modifies a small part of the schedule, its response time is fastand scalable.

5.2 Contribution TC2: Increasing Success Rate of HealingTime-Triggered Network Schedules 37

N1

N2

N3

N4

N5 N6

1

2

3

4

5 6

7

8

9

10

11

125

9

Figure 5.3: Network with a failure in link 7 that has to redirect the frames tothe path with links 5 and 9

found that our healing algorithm failed in most cases. Schedules are usuallycompressed to reduce latency to the minimum. However, if we want to heala network, we need some empty slots to redistribute the affected traffic whilestill satisfying all the constraints.

For this reason, we propose to obtain schedules that increase the frame dis-tances as much as possible, within the latency constraints, to produce emptyslots in-between frames. To do so, we replaced the solvers we were using. ILPsolvers are slower than SMT solvers for scheduling, but they possess capabil-ities to define an objective. Our objective is to maximize the frame distancesover the whole schedule, a characteristic we call schedule reparability. An ex-ample can be seen in Figure 5.4, in which the failure of link 7 of Figure 5.3activates the healing algorithm. Our healing algorithm tries to find an alterna-tive path that connects both nodes connected to the faulty link, nodes 3 and 5,and redistribute the affected frames over the new path. If the schedule is com-pressed, as seen in the left schedule, there are no empty slots to redistribute thetraffic. However, if the schedule has a high reparability, the healing algorithmcan find free slots and modify the schedule.

Conclusions: when the initial schedule has a high reparability, the healingalgorithms show a considerably high success rate to heal link failures. As thealgorithm only modifies a small part of the schedule, its response time is fastand scalable.

59


TimeSlotLink 1Link 3Link 5Link 7Link 9

Link 11

1 2 3 4 5 6 7 8F1

F1

F1

F2

F2

F2

F3

F3F1

F1 F2

F3

F4

F4F2

F4

((a)) Compressed Schedule


Link 11

1 2 3 4 5 6 7 8F1

F1

F1

F1

F2

F2

F2

F1 F2

F2

F3

F3

F3

F4

F4

F4

((b)) High Reparable Schedule

Figure 5.4: Result of applying the repair algorithm to a normal obtained sched-ule or a high reparable schedule

5.3 Contribution TC3: Distributed Online Proto-col to Increase the Fault-Tolerance Capabili-ties of Time-Triggered Networks

To apply healing algorithms to the next-generation networks, we provide anonline distributed protocol, as a centralized approach would present scalabil-ity issues on large networks. We proposed a Distributed Self-Healing Proto-col (SHP-D), where nodes collaborate and are responsible for healing its localschedule after a link failure. The SHP-D is divided into two phases: notifica-tion and preparation, and schedule update. In the first phase, the receiver nodeof the link notifies the sender node of the failure. Then, surrounding nodes,lead by the sender node, start to collaborate to find a new path and recollectthe needed information for the nodes to perform its local schedule healing. Inthe second phase, nodes execute the healing with an optimization algorithm



Link 11

1 2 3 4 5 6 7 8F1

F1

F1

F2

F2

F2

F3

F3F1

F1 F2

F3

F4

F4F2

F4

((a)) Compressed Schedule


Link 11

1 2 3 4 5 6 7 8F1

F1

F1

F1

F2

F2

F2

F1 F2

F2

F3

F3

F3

F4

F4

F4

((b)) High Reparable Schedule

Figure 5.4: Result of applying the repair algorithm to a normal obtained sched-ule or a high reparable schedule

5.3 Contribution TC3: Distributed Online Proto-col to Increase the Fault-Tolerance Capabili-ties of Time-Triggered Networks

To apply healing algorithms to the next-generation networks, we provide anonline distributed protocol, as a centralized approach would present scalabil-ity issues on large networks. We proposed a Distributed Self-Healing Proto-col (SHP-D), where nodes collaborate and are responsible for healing its localschedule after a link failure. The SHP-D is divided into two phases: notifica-tion and preparation, and schedule update. In the first phase, the receiver nodeof the link notifies the sender node of the failure. Then, surrounding nodes,lead by the sender node, start to collaborate to find a new path and recollectthe needed information for the nodes to perform its local schedule healing. Inthe second phase, nodes execute the healing with an optimization algorithm

60

5.3 Contribution TC3: Distributed Online Protocol to Increase theFault-Tolerance Capabilities of Time-Triggered Networks 39

that maintains the schedule’s overall high reparability. However, the responsetime (in seconds) is still too high. We solved this problem by executing firsta fast algorithm, the patching, that achieves the healing of the local schedulein milliseconds. Nevertheless, the optimization is still needed as the patchingalgorithm does not maintain the high reparability.

Although the success rate of the SHP-D is high, it is lower than full re-scheduling. Most of these cases are caused because a sub-optimal path ischosen when performing the healing. The lack of information compared toa centralized approach combined with the difficulties to collaborate and recol-lect information between nodes makes it difficult to select better paths. As aremedy, we propose a Semi-Distributed Self-Healing Protocol (SHP-SD) thatseeks to combine the advantages of distribution and centralization. The SHP-SD utilizes a series of High-Performance Switches (HPC) to act as healers of anetwork segment. HPCs recollect information of their surrounding to performmore advanced path selections and increase the success rate of healing link fail-ures close to re-scheduling. Moreover, thanks to the learnt information, it canenhance the protocol with capabilities such as switch healing and pre-emptivescheduling. In the case of pre-emptive scheduling, the HPCs classify the diffi-culty to heal failures using a Support Vector Machine classification. When theyare idle, they can simulate the most time-consuming links failures to save therequired modifications in case of such failure. In the case such a componentfails, the HPC already has everything calculated.

Conclusions: The SHP-D achieves healing of link failures in the millisec-ond range while requiring some extra seconds to return the schedule to highschedulability. The improved SHP-SD further reduces the response time, es-pecially for the most time-consuming cases, while increasing the success ratecloser to full re-scheduling thanks to a more advanced path selection. Further-more, the SHP-SD extends the previous protocol with the capability of healingswitches.

5.3 Contribution TC3: Distributed Online Protocol to Increase theFault-Tolerance Capabilities of Time-Triggered Networks 39

that maintains the schedule’s overall high reparability. However, the responsetime (in seconds) is still too high. We solved this problem by executing firsta fast algorithm, the patching, that achieves the healing of the local schedulein milliseconds. Nevertheless, the optimization is still needed as the patchingalgorithm does not maintain the high reparability.

Although the success rate of the SHP-D is high, it is lower than full re-scheduling. Most of these cases are caused because a sub-optimal path ischosen when performing the healing. The lack of information compared toa centralized approach combined with the difficulties to collaborate and recol-lect information between nodes makes it difficult to select better paths. As aremedy, we propose a Semi-Distributed Self-Healing Protocol (SHP-SD) thatseeks to combine the advantages of distribution and centralization. The SHP-SD utilizes a series of High-Performance Switches (HPC) to act as healers of anetwork segment. HPCs recollect information of their surrounding to performmore advanced path selections and increase the success rate of healing link fail-ures close to re-scheduling. Moreover, thanks to the learnt information, it canenhance the protocol with capabilities such as switch healing and pre-emptivescheduling. In the case of pre-emptive scheduling, the HPCs classify the diffi-culty to heal failures using a Support Vector Machine classification. When theyare idle, they can simulate the most time-consuming links failures to save therequired modifications in case of such failure. In the case such a componentfails, the HPC already has everything calculated.

Conclusions: The SHP-D achieves healing of link failures in the millisec-ond range while requiring some extra seconds to return the schedule to highschedulability. The improved SHP-SD further reduces the response time, es-pecially for the most time-consuming cases, while increasing the success ratecloser to full re-scheduling thanks to a more advanced path selection. Further-more, the SHP-SD extends the previous protocol with the capability of healingswitches.

61

Chapter 6

Overview of the IncludedPapers

6.1 Paper A

Title:SMT-based Synthesis of TTEthernet Schedules: A Performance Study.

Authors:Francisco Pozo, Guillermo Rodriguez-Navas, Hans Hansson, and Wilfried

Steiner.Summary:

The time-triggered paradigm requires a predefined schedule to guaranteelow communication latency and minimal jitter. However, synthesizing suchschedules is known to be an NP-complete problem by reduction to a bin-packing problem. Many different approaches and tools have been applied fortime-triggered schedule synthesis, but the ever-increasing size and complexityof the network applications have been forcing researchers to develop better ap-proaches and tools. Recently, the rapid performance increase of SatisfiabilityModulo Theory (SMT) solvers have gained the attention of the time-triggeredscheduling community. Schedulers implemented with SMT solvers have beenevaluated and shown capable of synthesizing schedules for small networks in afew minutes. In the case of industry-size networks, an implementation combin-ing an incremental approach with an SMT solver could synthesize schedules ofsuch network sizes in less than one hour. The goal of this paper is to perform a

41

Chapter 6

Overview of the IncludedPapers

6.1 Paper A

Title:SMT-based Synthesis of TTEthernet Schedules: A Performance Study.

Authors:Francisco Pozo, Guillermo Rodriguez-Navas, Hans Hansson, and Wilfried

Steiner.Summary:

The time-triggered paradigm requires a predefined schedule to guaranteelow communication latency and minimal jitter. However, synthesizing suchschedules is known to be an NP-complete problem by reduction to a bin-packing problem. Many different approaches and tools have been applied fortime-triggered schedule synthesis, but the ever-increasing size and complexityof the network applications have been forcing researchers to develop better ap-proaches and tools. Recently, the rapid performance increase of SatisfiabilityModulo Theory (SMT) solvers have gained the attention of the time-triggeredscheduling community. Schedulers implemented with SMT solvers have beenevaluated and shown capable of synthesizing schedules for small networks in afew minutes. In the case of industry-size networks, an implementation combin-ing an incremental approach with an SMT solver could synthesize schedules ofsuch network sizes in less than one hour. The goal of this paper is to perform a

41

63

42 Chapter 6. Overview of the Included Papers

study to select the best performance parameters both in the SMT solver and theincremental approach and consider its feasibility for more extensive networks.We found that for the studied cases, we could reduce the synthesis time by 75%compared to the de-facto configuration. However, we found that the scalabilityissues from the basic configuration are still present. Slightly larger networksizes or traffic would timeout or yield out-of-memory errors. We concludedthat if we want to schedule next-generation extremely large and complex net-works, other approaches with scalability on mind needed to be investigated.Contributions:

In this paper, an adequate configuration for schedule synthesis was foundfor both the incremental approach and the SMT Solver, reducing up to 75% thesynthesis time. This knowledge was applied for all the publications involvingeither SMT solvers or the incremental approach. Moreover, we gained com-prehension in the cases when the incremental approach and the SMT solverpresents limitations both in computing time and memory resources. The ex-pertise in identifying when scalability issues might occur helped the design ofthe other approaches. This paper addresses contribution TC1.Author’s Contributions:

I was the primary driver of the paper and wrote most of the text. Theco-author, Wilfried Steiner, provided the scheduler implementation from hisprevious work. I designed the experimental studies to determine the fastestconfiguration of Steiner’s scheduler and find the scalability limitations of theincremental approach. All the remaining co-authors provided feedback on thestudy and the text, and had the role of supervisors.Status:

Published in Proceedings of the 10th IEEE International Symposium onIndustrial Embedded Systems (SIES), Siegen, Germany, 2015.

6.2 Paper BTitle:

Methods for Large-Scale Time-Triggered Network Scheduling.Authors:

Francisco Pozo, Guillermo Rodriguez-Navas, and Hans Hansson.Summary:

In this paper, we considered the characteristics of next-generation time-triggered networks. Compared to industry-size networks, these networks mighthold one orders of magnitude more nodes, and two orders of magnitude more


study to select the best performance parameters both in the SMT solver and theincremental approach and consider its feasibility for more extensive networks.We found that for the studied cases, we could reduce the synthesis time by 75%compared to the de-facto configuration. However, we found that the scalabilityissues from the basic configuration are still present. Slightly larger networksizes or traffic would timeout or yield out-of-memory errors. We concludedthat if we want to schedule next-generation extremely large and complex net-works, other approaches with scalability on mind needed to be investigated.Contributions:

In this paper, an adequate configuration for schedule synthesis was foundfor both the incremental approach and the SMT Solver, reducing up to 75% thesynthesis time. This knowledge was applied for all the publications involvingeither SMT solvers or the incremental approach. Moreover, we gained com-prehension in the cases when the incremental approach and the SMT solverpresents limitations both in computing time and memory resources. The ex-pertise in identifying when scalability issues might occur helped the design ofthe other approaches. This paper addresses contribution TC1.Author’s Contributions:

I was the primary driver of the paper and wrote most of the text. Theco-author, Wilfried Steiner, provided the scheduler implementation from hisprevious work. I designed the experimental studies to determine the fastestconfiguration of Steiner’s scheduler and find the scalability limitations of theincremental approach. All the remaining co-authors provided feedback on thestudy and the text, and had the role of supervisors.Status:

Published in Proceedings of the 10th IEEE International Symposium onIndustrial Embedded Systems (SIES), Siegen, Germany, 2015.

6.2 Paper BTitle:

Methods for Large-Scale Time-Triggered Network Scheduling.Authors:


In this paper, we considered the characteristics of next-generation time-triggered networks. Compared to industry-size networks, these networks mighthold one orders of magnitude more nodes, and two orders of magnitude more

64

6.2 Paper B 43

traffic. Inspired by the requirements of emerging fog computing, we also in-troduced wired-wireless mixed communication. We developed the segmentedapproach to solve the scalability issues when scheduling such massive net-works. The segmented approach divides the schedule into small enough seg-ments that can be solved by current state-of-the-art schedules. One of the sig-nificant challenges was to keep the segments independent between each otherto be scheduled separately. The assertion of tailored constraints at the start ofevery segment allowed to keep segments independent in the presence of inter-segment constraints. We studied static segment division by frames or by time.However, the modelling of wireless communication introduced a considerabledisparity in transmission time between wired and wireless. Such difference,together with a static segment division, required the segmented approach tochoose segment sizes large enough that presented scalability issues. We imple-mented a dynamic segment size that reacts to solver feedback, which increasesor decreases the segment size (by time) depending on the solver response time.On top of that, we attended the low utilization of segment scheduling inher-ent to our approach by the introduction of relaxed constraints. When a framecannot fit a single segment, we allow such frame to be allocated in both thecurrent segment and the next segment. Following frames are permitted to con-tinue populating the current segment, therefore avoiding locking the segmentat the first frame that does not fit. The evaluation showed that the segmentedapproach could schedule next-generation time-triggered networks in under 4hours without presenting scalability issues. Relaxing constraints enabled thesegmented approach to schedule very-high utilization networks, reaching be-tween 90% and 95% maximum link utilization and even 99% in some studiedcases.Contributions:

A framework was developed to schedule next-generation time-triggerednetworks — the segmented approach provided the means to overcome scalabil-ity issues. We achieved to schedule such networks in under 4 hours. Moreover,different enhancements were introduced to overcome some of the limitationsinherent to segmentation, allowing to synthesize schedules of high-utilizationnetworks. This paper addresses and completes the contribution TC1.Author’s Contributions:

I was the primary driver of the paper and wrote most of the text. I designedand implemented all the presented divide-and-conquer approaches. Moreover,I developed a program to generate synthesized traffics and networks that is em-ployed in all the evaluations of the subsequent publications. I also performedall the experimental studies. All the remaining co-authors provided feedback,

6.2 Paper B 43

traffic. Inspired by the requirements of emerging fog computing, we also in-troduced wired-wireless mixed communication. We developed the segmentedapproach to solve the scalability issues when scheduling such massive net-works. The segmented approach divides the schedule into small enough seg-ments that can be solved by current state-of-the-art schedules. One of the sig-nificant challenges was to keep the segments independent between each otherto be scheduled separately. The assertion of tailored constraints at the start ofevery segment allowed to keep segments independent in the presence of inter-segment constraints. We studied static segment division by frames or by time.However, the modelling of wireless communication introduced a considerabledisparity in transmission time between wired and wireless. Such difference,together with a static segment division, required the segmented approach tochoose segment sizes large enough that presented scalability issues. We imple-mented a dynamic segment size that reacts to solver feedback, which increasesor decreases the segment size (by time) depending on the solver response time.On top of that, we attended the low utilization of segment scheduling inher-ent to our approach by the introduction of relaxed constraints. When a framecannot fit a single segment, we allow such frame to be allocated in both thecurrent segment and the next segment. Following frames are permitted to con-tinue populating the current segment, therefore avoiding locking the segmentat the first frame that does not fit. The evaluation showed that the segmentedapproach could schedule next-generation time-triggered networks in under 4hours without presenting scalability issues. Relaxing constraints enabled thesegmented approach to schedule very-high utilization networks, reaching be-tween 90% and 95% maximum link utilization and even 99% in some studiedcases.Contributions:

A framework was developed to schedule next-generation time-triggerednetworks — the segmented approach provided the means to overcome scalabil-ity issues. We achieved to schedule such networks in under 4 hours. Moreover,different enhancements were introduced to overcome some of the limitationsinherent to segmentation, allowing to synthesize schedules of high-utilizationnetworks. This paper addresses and completes the contribution TC1.Author’s Contributions:

I was the primary driver of the paper and wrote most of the text. I designedand implemented all the presented divide-and-conquer approaches. Moreover,I developed a program to generate synthesized traffics and networks that is em-ployed in all the evaluations of the subsequent publications. I also performedall the experimental studies. All the remaining co-authors provided feedback,

65


helped with the formulation of the problem, and had the role of supervisors.Status:

Published in MDPI Electronics, June 2019.

6.3 Paper CTitle:

Schedule Reparability: Enhancing Time-Triggered Network Recoveryupon Link Failures.Authors:


Considering that in previous publications we successfully advanced thedevelopment of a scalable framework for the next-generation time-triggerednetworks, we decided to shift our research into other issues that such large net-works could encounter. With the ever-increasing size of systems, it is inevitablethat the number of failures also grows. Nonetheless, the primary disadvantageof the time-triggered paradigm is its lack of flexibility. E.g., if a link fails, theschedule is not valid anymore, and many frames are lost until the schedule isrecalculated. Current approaches that handle component failures are suited forsmall networks, but they are not scalable for next-generation networks. Theyare too expensive to implement, too slow to react, or take into account only afew potential failure scenarios, neglecting the rest. In this paper, we investigatea healing algorithm that reacts after a link failure is detected in the network.It can isolate a small segment of the network schedule and heal it rapidly toa valid state where all frames are transmitted again. The healing takes intoits benefit alternative paths between both nodes connected to the faulty link toredirect and re-schedule the affected frames. However, we found that currentschedules are not suited for healing, causing our algorithm to fail in most cases.We noticed that this was caused by how current schedules are obtained, whereall the communications are compressed, generating a very constrained problemwhen trying to modify and redirect frames. We proposed to, opposite to currentapproaches, maximize the distances between frames in the schedule while stillsatisfying all the communication constraints, which we called schedule repara-bility. Our evaluation confirmed that maximizing frame distances increasesthe success rate of the healing algorithm considerably, superior to 90% for thestudied cases. An exciting discovery was that the success rate was better forlarger networks, and therefore suited for next-generation networks.


helped with the formulation of the problem, and had the role of supervisors.Status:

Published in MDPI Electronics, June 2019.

6.3 Paper CTitle:

Schedule Reparability: Enhancing Time-Triggered Network Recoveryupon Link Failures.Authors:


Considering that in previous publications we successfully advanced thedevelopment of a scalable framework for the next-generation time-triggerednetworks, we decided to shift our research into other issues that such large net-works could encounter. With the ever-increasing size of systems, it is inevitablethat the number of failures also grows. Nonetheless, the primary disadvantageof the time-triggered paradigm is its lack of flexibility. E.g., if a link fails, theschedule is not valid anymore, and many frames are lost until the schedule isrecalculated. Current approaches that handle component failures are suited forsmall networks, but they are not scalable for next-generation networks. Theyare too expensive to implement, too slow to react, or take into account only afew potential failure scenarios, neglecting the rest. In this paper, we investigatea healing algorithm that reacts after a link failure is detected in the network.It can isolate a small segment of the network schedule and heal it rapidly toa valid state where all frames are transmitted again. The healing takes intoits benefit alternative paths between both nodes connected to the faulty link toredirect and re-schedule the affected frames. However, we found that currentschedules are not suited for healing, causing our algorithm to fail in most cases.We noticed that this was caused by how current schedules are obtained, whereall the communications are compressed, generating a very constrained problemwhen trying to modify and redirect frames. We proposed to, opposite to currentapproaches, maximize the distances between frames in the schedule while stillsatisfying all the communication constraints, which we called schedule repara-bility. Our evaluation confirmed that maximizing frame distances increasesthe success rate of the healing algorithm considerably, superior to 90% for thestudied cases. An exciting discovery was that the success rate was better forlarger networks, and therefore suited for next-generation networks.

66

6.4 Paper D 45

Contributions:The adjustments of the scheduling problem from SMT solvers to ILP

solvers was developed to introduce the objective of obtaining high reparabil-ity schedules. A formalization of the concept of schedule reparability wasperformed, and a cost function that maximizes the distances between frameswas modelled. A healing algorithm was also introduced to redirect and re-schedule frames affected by a link failure. The usefulness of high-reparabilitywas demonstrated and the possibility to heal small segments of the networkwith minimal adjustments instead of full re-scheduling. This last contributionallowed for the design of run-time online protocols for schedule alterations orhealings. This paper addresses contribution TC2.Author’s Contributions:

I was the primary driver of the paper and wrote most of the text. The dis-cussion with the co-author Guillermo Rodriguez-Navas inspired the strategy ofhealing time-triggered schedules that opened the line of research for the cur-rent and subsequent publications. I designed the cost function to obtain highlyreparable schedules and implemented it. Moreover, I developed a healing al-gorithm capable of repairing time-triggered schedules and performed all theexperimental studies. All the co-authors provided feedback and had the role ofsupervisors.Status:

Published in Proceedings of the 24th IEEE International Conference on Em-bedded and Real-Time Computing Systems and Applications (RTCSA), Hako-date, Japan, August 2018.

6.4 Paper DTitle:

Self-Healing Protocol: Repairing Schedules Online after Link Failures inTime-Triggered Networks.Authors:


The healing algorithm coupled with an initial highly-reparable scheduledemonstrated to be successful to recover from link failures. Nevertheless, apractical implementation proves to be troublesome for next-generation net-works, as we require full knowledge of the network together with a central en-tity that heals and distributes schedules during runtime. An unrealistic scenario

6.4 Paper D 45

Contributions:The adjustments of the scheduling problem from SMT solvers to ILP

solvers was developed to introduce the objective of obtaining high reparabil-ity schedules. A formalization of the concept of schedule reparability wasperformed, and a cost function that maximizes the distances between frameswas modelled. A healing algorithm was also introduced to redirect and re-schedule frames affected by a link failure. The usefulness of high-reparabilitywas demonstrated and the possibility to heal small segments of the networkwith minimal adjustments instead of full re-scheduling. This last contributionallowed for the design of run-time online protocols for schedule alterations orhealings. This paper addresses contribution TC2.Author’s Contributions:

I was the primary driver of the paper and wrote most of the text. The dis-cussion with the co-author Guillermo Rodriguez-Navas inspired the strategy ofhealing time-triggered schedules that opened the line of research for the cur-rent and subsequent publications. I designed the cost function to obtain highlyreparable schedules and implemented it. Moreover, I developed a healing al-gorithm capable of repairing time-triggered schedules and performed all theexperimental studies. All the co-authors provided feedback and had the role ofsupervisors.Status:

Published in Proceedings of the 24th IEEE International Conference on Em-bedded and Real-Time Computing Systems and Applications (RTCSA), Hako-date, Japan, August 2018.

6.4 Paper DTitle:

Self-Healing Protocol: Repairing Schedules Online after Link Failures inTime-Triggered Networks.Authors:


The healing algorithm coupled with an initial highly-reparable scheduledemonstrated to be successful to recover from link failures. Nevertheless, apractical implementation proves to be troublesome for next-generation net-works, as we require full knowledge of the network together with a central en-tity that heals and distributes schedules during runtime. An unrealistic scenario

67


which would also cause overhead from detecting a failure to heal it. Besides,these requirements are not synchronized with the healing schedule philosophyof enclosing the modifications to a small network segment. We proposed toinvestigate a fully-distributed approach, the Self-Healing Protocol Distributed(SHP-D), in which nodes without any knowledge of their surroundings, col-laborate to identify a link failure and heal their local schedules during runtime.However, the healing (optimization) algorithm from the previous publicationdoes not provide the expected fast response time, requiring up to two secondsfor the studied cases. In the conception of the protocol, we aimed for a fewmilliseconds healing time. To reduce the time frames are being lost, we alsoimplemented a fast-heuristic patching algorithm that heals the network neglect-ing the solution reparability. Once the patch recovers the transmission of theaffected frames, we can apply the optimization algorithm which heals the net-work returning to a high reparability. We confirmed in our evaluations thatwithout losing success rate compared with the centralized approach of the pre-vious paper, we could patch the studied network in milliseconds, and optimizein less than two seconds. The performance of our protocol also increased withlarger networks, as the traffic is more distributed.Contributions:

The formalization and development of the SHP-D were proposed. The pro-tocol introduced a collaboration framework between nodes to detect and find analternative path to heal the network. Moreover, a patching algorithm was pro-posed to heal the network in the aimed millisecond range, while the optimiza-tion algorithm maintained the high reparability of the schedule. The SHP-Dproved to achieve a high success rate but was still lower than full re-schedule.Besides, fully-distribution showed some limitations in the collaborations ofnodes. This paper addresses contribution TC3.Author’s Contributions:

I was the primary driver of the paper and wrote most of the text. I conceivedthe idea of the fully-distributed protocol and overcame the excessive responsetime of the healing (optimization) algorithm implementing a fast-responsepatching algorithm. The co-author Guillermo Rodriguez-Navas helped withthe formalization of the protocol and inspired the implementation of the patch-ing algorithm. Moreover, I developed an event simulator to run the experimen-tal evaluations. All the co-authors provided feedback on the text and had therole of supervisors.Status:

Published as MRTC Report, MDH-MRTC-327/2019-1-SE, MalardalenReal-Time Research Centre, Malardalen University, September, 2019. Sub-


which would also cause overhead from detecting a failure to heal it. Besides,these requirements are not synchronized with the healing schedule philosophyof enclosing the modifications to a small network segment. We proposed toinvestigate a fully-distributed approach, the Self-Healing Protocol Distributed(SHP-D), in which nodes without any knowledge of their surroundings, col-laborate to identify a link failure and heal their local schedules during runtime.However, the healing (optimization) algorithm from the previous publicationdoes not provide the expected fast response time, requiring up to two secondsfor the studied cases. In the conception of the protocol, we aimed for a fewmilliseconds healing time. To reduce the time frames are being lost, we alsoimplemented a fast-heuristic patching algorithm that heals the network neglect-ing the solution reparability. Once the patch recovers the transmission of theaffected frames, we can apply the optimization algorithm which heals the net-work returning to a high reparability. We confirmed in our evaluations thatwithout losing success rate compared with the centralized approach of the pre-vious paper, we could patch the studied network in milliseconds, and optimizein less than two seconds. The performance of our protocol also increased withlarger networks, as the traffic is more distributed.Contributions:

The formalization and development of the SHP-D were proposed. The pro-tocol introduced a collaboration framework between nodes to detect and find analternative path to heal the network. Moreover, a patching algorithm was pro-posed to heal the network in the aimed millisecond range, while the optimiza-tion algorithm maintained the high reparability of the schedule. The SHP-Dproved to achieve a high success rate but was still lower than full re-schedule.Besides, fully-distribution showed some limitations in the collaborations ofnodes. This paper addresses contribution TC3.Author’s Contributions:

I was the primary driver of the paper and wrote most of the text. I conceivedthe idea of the fully-distributed protocol and overcame the excessive responsetime of the healing (optimization) algorithm implementing a fast-responsepatching algorithm. The co-author Guillermo Rodriguez-Navas helped withthe formalization of the protocol and inspired the implementation of the patch-ing algorithm. Moreover, I developed an event simulator to run the experimen-tal evaluations. All the co-authors provided feedback on the text and had therole of supervisors.Status:

Published as MRTC Report, MDH-MRTC-327/2019-1-SE, MalardalenReal-Time Research Centre, Malardalen University, September, 2019. Sub-

68

6.5 Paper E 47

mitted to IEEE Transactions in Industrial Informatics, September, 2019, thirdreview round.

6.5 Paper ETitle:

Semi-Distributed Self-Healing Protocol for Online Schedule Repair afterNetwork Failures.Authors:


The SHP-D opened a wide range of different research directions. Eventhough it proved to be insufficient for the most safety-critical systems, it can beimplemented on top of current approaches to enhance the network adaptability,while also providing some fault-tolerant capabilities for less critical communi-cations without considerable extra cost. In this paper, we continue to work onthe practicality of the protocol and solve some of the limitations of SHP-D. Aprobably unrealistic requirement of our protocol is that all nodes should havesolving capabilities, which requires all switches to have considerable comput-ing and memory resources. Moreover, fully-distribution limits the extent ofcooperation among nodes and the knowledge learnt about the network and itsstate. As a remedy, we investigate a hybrid between fully-distribution andcentralized, the Semi-Distributed Self-Healing Protocol (SHP-SD), that takesadvantage of both paradigms to reduce their limitations. Large networks usu-ally contain a set of High-Performance Switches (HPC) that possess higherresources. Analogue to the centralized approach, but on a smaller scale, wewould like several HPCs to maintain and recollect information about their sur-rounding nodes and take the solving role when healing is needed. The knowl-edge owned by HPCs allows for a series of enhancements that were not pos-sible with the SHP-D. Some of the upgrades were to predict and pre-computehealings for the potentially more complex failures, creating more advancedhealing paths or extending the fault model to heal switch failures. In this publi-cation, we further increased the success rate and reduced the average responsetime compared to the SHP-D, especially for the most time-consuming failures,thanks to reactively computing healing adjustments during protocol idle times.Finally, we studied the feasibility of a more extensive fault model with nodefailures that needed more significant adjustments in the schedule.Contributions:

6.5 Paper E 47

mitted to IEEE Transactions in Industrial Informatics, September, 2019, thirdreview round.

6.5 Paper ETitle:

Semi-Distributed Self-Healing Protocol for Online Schedule Repair afterNetwork Failures.Authors:


The SHP-D opened a wide range of different research directions. Eventhough it proved to be insufficient for the most safety-critical systems, it can beimplemented on top of current approaches to enhance the network adaptability,while also providing some fault-tolerant capabilities for less critical communi-cations without considerable extra cost. In this paper, we continue to work onthe practicality of the protocol and solve some of the limitations of SHP-D. Aprobably unrealistic requirement of our protocol is that all nodes should havesolving capabilities, which requires all switches to have considerable comput-ing and memory resources. Moreover, fully-distribution limits the extent ofcooperation among nodes and the knowledge learnt about the network and itsstate. As a remedy, we investigate a hybrid between fully-distribution andcentralized, the Semi-Distributed Self-Healing Protocol (SHP-SD), that takesadvantage of both paradigms to reduce their limitations. Large networks usu-ally contain a set of High-Performance Switches (HPC) that possess higherresources. Analogue to the centralized approach, but on a smaller scale, wewould like several HPCs to maintain and recollect information about their sur-rounding nodes and take the solving role when healing is needed. The knowl-edge owned by HPCs allows for a series of enhancements that were not pos-sible with the SHP-D. Some of the upgrades were to predict and pre-computehealings for the potentially more complex failures, creating more advancedhealing paths or extending the fault model to heal switch failures. In this publi-cation, we further increased the success rate and reduced the average responsetime compared to the SHP-D, especially for the most time-consuming failures,thanks to reactively computing healing adjustments during protocol idle times.Finally, we studied the feasibility of a more extensive fault model with nodefailures that needed more significant adjustments in the schedule.Contributions:

69


The SHP-D was adjusted to the semi-distributed approach. As a result, theSHP-SD was formalized and developed. A Machine Learning approach usingSupport Vector Machines was applied to predict the most time-consuming fail-ures that HPCs could reactively heal. An advanced path healing algorithm wasintroduced to increase further the success rate of healing link failures using theknowledge learnt by the HPCs. Lastly, the formalization of node failures as aset of link failures was proposed, and its performance evaluated.Author’s Contributions:

I was the primary driver of the paper and wrote most of the text. I designedthe semi-distributed protocol and its enhancements to overcome the limitationsof a fully distributed approach with almost all the benefits of a centralized ap-proach. Moreover, I adjusted the event simulator from the previous publicationto support the new protocol. The details and feasibility of the Machine Learn-ing algorithm derived from discussions with the AI/Machine Learning researchgroup at RISE SICS Vasteras. All the co-authors provided feedback on the textand had the role of supervisors.Status:

Published as MRTC Report, MDH-MRTC-326/2019-1-SE, MalardalenReal-Time Research Centre, Malardalen University, September, 2019. Sub-mitted to IEEE Transactions in Industrial Informatics, September 2019.


The SHP-D was adjusted to the semi-distributed approach. As a result, theSHP-SD was formalized and developed. A Machine Learning approach usingSupport Vector Machines was applied to predict the most time-consuming fail-ures that HPCs could reactively heal. An advanced path healing algorithm wasintroduced to increase further the success rate of healing link failures using theknowledge learnt by the HPCs. Lastly, the formalization of node failures as aset of link failures was proposed, and its performance evaluated.Author’s Contributions:

I was the primary driver of the paper and wrote most of the text. I designedthe semi-distributed protocol and its enhancements to overcome the limitationsof a fully distributed approach with almost all the benefits of a centralized ap-proach. Moreover, I adjusted the event simulator from the previous publicationto support the new protocol. The details and feasibility of the Machine Learn-ing algorithm derived from discussions with the AI/Machine Learning researchgroup at RISE SICS Vasteras. All the co-authors provided feedback on the textand had the role of supervisors.Status:

Published as MRTC Report, MDH-MRTC-326/2019-1-SE, MalardalenReal-Time Research Centre, Malardalen University, September, 2019. Sub-mitted to IEEE Transactions in Industrial Informatics, September 2019.

70

Chapter 7

Conclusions and FutureWork

In this thesis, we have focused on investigating some of the main challengesthe time-triggered paradigm might encounter in needs for the requirements offuture next-generation networks. In particular, we have researched the scal-ability issues in obtaining large-scale network schedules and the adaptabilityrequirements of a system that suffers from continuous unpredicted changes.

7.1 ConclusionsSchedule Synthesis of Next-Generation Time-Triggered Schedules.Time-Triggered networks have continuously been increasing in size and com-plexity. We can envision a future where next-generation networks will com-prise hundreds of nodes and thousands of frames. Moreover, such networksmight embrace a mix of different technologies, such as wired and wireless com-munications. Time-triggered networks employ a schedule calculated at designtime that states the transmission times of all the time-triggered traffic. How-ever, synthesizing this schedule is a well-known NP-complete problem. Thestate-of-the-art solver, an SMT solver coupled with an incremental approach,was starting to present scalability issues for current industrial size networks.We noticed that applying a configuration to the SMT solver and the incremen-tal approach specifically adapted to synthesis of schedules we could reducethe synthesis time with up to 70%, obtaining schedules of 1000 frames in less

49

Chapter 7

Conclusions and FutureWork

In this thesis, we have focused on investigating some of the main challengesthe time-triggered paradigm might encounter in needs for the requirements offuture next-generation networks. In particular, we have researched the scal-ability issues in obtaining large-scale network schedules and the adaptabilityrequirements of a system that suffers from continuous unpredicted changes.

7.1 ConclusionsSchedule Synthesis of Next-Generation Time-Triggered Schedules.Time-Triggered networks have continuously been increasing in size and com-plexity. We can envision a future where next-generation networks will com-prise hundreds of nodes and thousands of frames. Moreover, such networksmight embrace a mix of different technologies, such as wired and wireless com-munications. Time-triggered networks employ a schedule calculated at designtime that states the transmission times of all the time-triggered traffic. How-ever, synthesizing this schedule is a well-known NP-complete problem. Thestate-of-the-art solver, an SMT solver coupled with an incremental approach,was starting to present scalability issues for current industrial size networks.We noticed that applying a configuration to the SMT solver and the incremen-tal approach specifically adapted to synthesis of schedules we could reducethe synthesis time with up to 70%, obtaining schedules of 1000 frames in less

49

71

50 Chapter 7. Conclusions and Future Work

than 30 minutes. But scalability issues were still present and we could not ob-tain schedules when we surpass the 2000 frames threshold, a rudimentary casecompared with next-generation networks.

We focused on how to avoid the scalability issues applying a divide-and-conquer approach. We implemented the segmented approach that divides thewhole schedule into segments that are easily solvable with state-of-the-artsolvers. In the segmented approach, frames are allocated into a segment sched-ule until the segment is full. Then, the next segment is selected and scheduleduntil no frames are left to be scheduled. We showed that this approach couldschedule the envisioned next-generation networks in a few hours. Moreover,some enhancements were developed to improve the segmented approach, e.g.,handling wireless communications and scheduling high utilization networks,up to 99% maximum link utilization in some cases. We showed that the seg-mented approach does not present scalability issues even in the extreme casewhen scheduling up to a million frames.

Adaptability in Time-Triggered Networks. After obtaining next-generation network schedules, we wanted to bring focus to the lack ofadaptability of the time-triggered paradigm. As the size and complexityof networks increases, it is reasonable to think that the network will notstay static. Components might enter and leave the network regularly, whichcompletely contradicts the philosophy of time-triggered. Nonetheless, even ifthe network might remain static, as the number of components increase, thenumber of failures will also increase. Techniques to deal with such failures areusually based on replication, tending to costs that become prohibitive for largenetworks. However, even though the most safety-critical network segmentsmight be replicated, it will still be desirable to develop fault-tolerance tech-niques for lower safety-critical network segments that do not add much extracosts but that can tolerate some failures.

We proposed to quickly heal schedules during run-time when a failure oc-curs. We developed a healing algorithm that, after a link failure, finds an al-ternative new path that connects both nodes connected to the faulty link andadjust only the minimum parts of the schedule. However, we found that ouralgorithm did not succeed as much as we expected. This is due to the waycurrent schedules are obtained. Schedulers try to compress the time-triggeredcommunication as much as possible, leaving no space for adjustments. In-stead, we want a schedule that has maximum distances between transmission,a characteristic we call reparability. Applying our healing algorithm to highreparability schedules increased the success rate (to more than 90%) and en-


than 30 minutes. But scalability issues were still present and we could not ob-tain schedules when we surpass the 2000 frames threshold, a rudimentary casecompared with next-generation networks.

We focused on how to avoid the scalability issues applying a divide-and-conquer approach. We implemented the segmented approach that divides thewhole schedule into segments that are easily solvable with state-of-the-artsolvers. In the segmented approach, frames are allocated into a segment sched-ule until the segment is full. Then, the next segment is selected and scheduleduntil no frames are left to be scheduled. We showed that this approach couldschedule the envisioned next-generation networks in a few hours. Moreover,some enhancements were developed to improve the segmented approach, e.g.,handling wireless communications and scheduling high utilization networks,up to 99% maximum link utilization in some cases. We showed that the seg-mented approach does not present scalability issues even in the extreme casewhen scheduling up to a million frames.

Adaptability in Time-Triggered Networks. After obtaining next-generation network schedules, we wanted to bring focus to the lack ofadaptability of the time-triggered paradigm. As the size and complexityof networks increases, it is reasonable to think that the network will notstay static. Components might enter and leave the network regularly, whichcompletely contradicts the philosophy of time-triggered. Nonetheless, even ifthe network might remain static, as the number of components increase, thenumber of failures will also increase. Techniques to deal with such failures areusually based on replication, tending to costs that become prohibitive for largenetworks. However, even though the most safety-critical network segmentsmight be replicated, it will still be desirable to develop fault-tolerance tech-niques for lower safety-critical network segments that do not add much extracosts but that can tolerate some failures.

We proposed to quickly heal schedules during run-time when a failure oc-curs. We developed a healing algorithm that, after a link failure, finds an al-ternative new path that connects both nodes connected to the faulty link andadjust only the minimum parts of the schedule. However, we found that ouralgorithm did not succeed as much as we expected. This is due to the waycurrent schedules are obtained. Schedulers try to compress the time-triggeredcommunication as much as possible, leaving no space for adjustments. In-stead, we want a schedule that has maximum distances between transmission,a characteristic we call reparability. Applying our healing algorithm to highreparability schedules increased the success rate (to more than 90%) and en-

72

7.2 Future Work 51

abled recovery to a correct schedule state similar to full re-schedule. Eventhough it is not reasonable to only apply our healing into highly safety-criticalapplications, it can be employed as an extra measure to increase reliability ontop of replication or increase the reliability for the less safety-critical segmentswithout additional cost.

In order to apply the healing algorithm, we developed a fully-distributedSelf-Healing Protocol. In our protocol, network nodes collaborate to detect alink failure and modify their local schedules using a healing algorithm duringrun-time, called optimization algorithm. We found that we could heal link fail-ures in a few seconds in the worse case. The response time of the optimizationalgorithm was unacceptably high, but it was needed as it maintained a highschedule reparability. To reduce the response time, we first executed anotheralgorithm before the optimization, the patching algorithm, that finds a solutionas fast as possible. The patching algorithm could heal link failures in millisec-onds. We further improved the protocol with a semi-distributed Self HealingProtocol that takes advantage of the existence of high-performance switches assolving and decision devices. These switches could recollect surrounding in-formation and apply more advanced techniques, such as finding better healingpaths or preemptively calculate healing of potential failures at idle times. Thesemi-distributed protocol achieved a success rate very close to full re-schedulewhile further reducing the average healing time.

7.2 Future Work

Parallel Segmented Approach. In today’s computing, multi-core processorsare easily accessible and widespread, even for the consumer market and embed-ded systems. Executing the scheduler in parallel would significantly decreasethe schedule synthesis time and could lead to a reduction to less than one hourfor next-generation networks schedule synthesis. However, parallel solving hasproven very challenging. Even though SMT and ILP solvers provide some par-allelization capabilities, by offering a portfolio of different configurations thatare run in parallel [78][79], portfolio parallelization only provides a minimalreduction of synthesis time, as it can only be as advantageous as the best ofits configurations. True parallelization with collaboration between cores is stillin early development, and results are far from the performance improvementexpected [80][81][82].

In contrast, the segmented approach can be parallelized thanks to its divide-and-conquer approach. In this thesis, segments have been solved sequentially

7.2 Future Work 51

abled recovery to a correct schedule state similar to full re-schedule. Eventhough it is not reasonable to only apply our healing into highly safety-criticalapplications, it can be employed as an extra measure to increase reliability ontop of replication or increase the reliability for the less safety-critical segmentswithout additional cost.

In order to apply the healing algorithm, we developed a fully-distributedSelf-Healing Protocol. In our protocol, network nodes collaborate to detect alink failure and modify their local schedules using a healing algorithm duringrun-time, called optimization algorithm. We found that we could heal link fail-ures in a few seconds in the worse case. The response time of the optimizationalgorithm was unacceptably high, but it was needed as it maintained a highschedule reparability. To reduce the response time, we first executed anotheralgorithm before the optimization, the patching algorithm, that finds a solutionas fast as possible. The patching algorithm could heal link failures in millisec-onds. We further improved the protocol with a semi-distributed Self HealingProtocol that takes advantage of the existence of high-performance switches assolving and decision devices. These switches could recollect surrounding in-formation and apply more advanced techniques, such as finding better healingpaths or preemptively calculate healing of potential failures at idle times. Thesemi-distributed protocol achieved a success rate very close to full re-schedulewhile further reducing the average healing time.

7.2 Future Work

Parallel Segmented Approach. In today’s computing, multi-core processorsare easily accessible and widespread, even for the consumer market and embed-ded systems. Executing the scheduler in parallel would significantly decreasethe schedule synthesis time and could lead to a reduction to less than one hourfor next-generation networks schedule synthesis. However, parallel solving hasproven very challenging. Even though SMT and ILP solvers provide some par-allelization capabilities, by offering a portfolio of different configurations thatare run in parallel [78][79], portfolio parallelization only provides a minimalreduction of synthesis time, as it can only be as advantageous as the best ofits configurations. True parallelization with collaboration between cores is stillin early development, and results are far from the performance improvementexpected [80][81][82].

In contrast, the segmented approach can be parallelized thanks to its divide-and-conquer approach. In this thesis, segments have been solved sequentially

73


to satisfy inter-segment constraints. However, in our experiments, not all thesegments contain such constraints. These segments could be solved in parallelin different solver instances using the full capabilities of the multi-core proces-sors. The main challenges for this parallelization are on the selection of whichframes and which segments to be scheduled in parallel and when sequentialscheduling is needed. Parallel divide-and-conquer approaches are extensivein the literature [83][84], including application in some areas such as processplanning [85].

Moreover, horizontal segmentation, i.e., segmentation by links, might fur-ther reduce the synthesis time and allow for parallelization. This segmentationmight only be possible to apply in hierarchical networks, but most of the largenetworks applications envisioned are organized hierarchically [86]. Two dif-ferent methods might accomplish horizontal parallel segmentation. The firstmethod applies a collaboration framework between parallel solver instancesthat satisfies constraints in different segments. However, this solution mightprove to be challenging to implement successfully and only employed in highlyhierarchical networks. A second method would be to divide the network be-tween the backbone and the different sub-networks. In this way, first the back-bone network can be scheduled. Subsequently, the sub-networks can be syn-thesized in parallel, accommodating to the backbone schedule.

A Heuristic to Synthesize Schedule Segments. In this thesis, we havesolved schedule segments using SMT Solvers and ILP Solvers. They haveboth proven to have excellent performance. Having said that, the choice ofthese solvers was mostly motivated by interest to apply general-purpose toolsto implement prototype schedulers quickly. Moreover, as we only need to as-sert the mathematical formulas into the solver, we have a higher assurance thatour implementation is bug-free, based on the extensive use of these solversin a broad range of applications. However, there might be other approachesthat provide a much better performance, even though the schedulability mightsuffer and the development might be more complex and prone to bugs. Fastmeta-heuristics such as list scheduling in the automotive domain [87] and em-bedded system [88] yield very promising results when the scheduling problemis not demanding, i.e., the network utilization is not very hard.

One of the main disadvantages of SMT and ILP solvers for segmentscheduling is that they need to be called multiple times regularly, which pro-duces an overhead as they apply many preprocessing techniques intended tosolve more complex problems [89]. A specific scheduling algorithm that suitsthe needs of segment scheduling with a fast set up for repeated invocation could


to satisfy inter-segment constraints. However, in our experiments, not all thesegments contain such constraints. These segments could be solved in parallelin different solver instances using the full capabilities of the multi-core proces-sors. The main challenges for this parallelization are on the selection of whichframes and which segments to be scheduled in parallel and when sequentialscheduling is needed. Parallel divide-and-conquer approaches are extensivein the literature [83][84], including application in some areas such as processplanning [85].

Moreover, horizontal segmentation, i.e., segmentation by links, might fur-ther reduce the synthesis time and allow for parallelization. This segmentationmight only be possible to apply in hierarchical networks, but most of the largenetworks applications envisioned are organized hierarchically [86]. Two dif-ferent methods might accomplish horizontal parallel segmentation. The firstmethod applies a collaboration framework between parallel solver instancesthat satisfies constraints in different segments. However, this solution mightprove to be challenging to implement successfully and only employed in highlyhierarchical networks. A second method would be to divide the network be-tween the backbone and the different sub-networks. In this way, first the back-bone network can be scheduled. Subsequently, the sub-networks can be syn-thesized in parallel, accommodating to the backbone schedule.

A Heuristic to Synthesize Schedule Segments. In this thesis, we havesolved schedule segments using SMT Solvers and ILP Solvers. They haveboth proven to have excellent performance. Having said that, the choice ofthese solvers was mostly motivated by interest to apply general-purpose toolsto implement prototype schedulers quickly. Moreover, as we only need to as-sert the mathematical formulas into the solver, we have a higher assurance thatour implementation is bug-free, based on the extensive use of these solversin a broad range of applications. However, there might be other approachesthat provide a much better performance, even though the schedulability mightsuffer and the development might be more complex and prone to bugs. Fastmeta-heuristics such as list scheduling in the automotive domain [87] and em-bedded system [88] yield very promising results when the scheduling problemis not demanding, i.e., the network utilization is not very hard.

One of the main disadvantages of SMT and ILP solvers for segmentscheduling is that they need to be called multiple times regularly, which pro-duces an overhead as they apply many preprocessing techniques intended tosolve more complex problems [89]. A specific scheduling algorithm that suitsthe needs of segment scheduling with a fast set up for repeated invocation could

74

7.2 Future Work 53

significantly reduce the synthesis time. We propose to extend this research di-rection with algorithms similar to the patching algorithm [90] that has provento solve simple problems in milliseconds.

A Segmented Approach to Obtain High Reparability Schedules. Whenobtaining highly reparable schedules, we have been applying a one-shot ap-proach for small networks and a modification of the original incremental ap-proach [16] for larger networks when scalability issues appeared. However,obtaining high reparability schedules for next-generation networks leads to anew series of challenges. The one-shot and incremental approach are solvedwithout resseting the constraints in the ILP solver. This means that the solvercan maximize the frame distances when we assert the cost function. But in thesegmented approach, multiple instances of the solver need to be called, one foreach segment. Therefore, we can obtain highly reparable individual segments,but the relation between segments does not possess a high reparability as thecost function is reset at the start of every segment.

A strategy to keep high reparability among segments needs to be devel-oped. One of the possible solutions could be to avoid solving the segmentssequentially; instead, select specific segments to schedule strategically to ob-tain an overall highly reparable schedule. Another possible solution would beto shift to a more specific segment oriented algorithm that would not solve asingle segment at the same time, instead, it would schedule some strategic seg-ments concurrently and placing frames such that distances between differentsegments are maximized.

Extending the Fault Model. In this thesis, we started studying the feasi-bility of schedule healing by applying the SHP-D after a permanent or longenough transient link failure. The development of SHP-SD takes advantage ofHPCs and their broader knowledge of the network to extend the fault modelto consider also switch failures. A single HPC could handle those two kindsof failures as they occurred in its range of knowledge. However, it remains tostudy how to heal failures that affect an HPC, or jeopardize a large networksegment that requires the collaboration of two or more HPCs.

Healing a HPC failure requires the intervention of another neighboringHPC that needs to hold a channel of communication and posses the same up-dated knowledge. First, to detect an HPC failure, we need another neighboringHPC to ping and check its status. Second, this neighboring HPC has to per-form the healing. Lastly, neighbouring HPCs needs to agree on which HPC that

7.2 Future Work 53

significantly reduce the synthesis time. We propose to extend this research di-rection with algorithms similar to the patching algorithm [90] that has provento solve simple problems in milliseconds.

A Segmented Approach to Obtain High Reparability Schedules. Whenobtaining highly reparable schedules, we have been applying a one-shot ap-proach for small networks and a modification of the original incremental ap-proach [16] for larger networks when scalability issues appeared. However,obtaining high reparability schedules for next-generation networks leads to anew series of challenges. The one-shot and incremental approach are solvedwithout resseting the constraints in the ILP solver. This means that the solvercan maximize the frame distances when we assert the cost function. But in thesegmented approach, multiple instances of the solver need to be called, one foreach segment. Therefore, we can obtain highly reparable individual segments,but the relation between segments does not possess a high reparability as thecost function is reset at the start of every segment.

A strategy to keep high reparability among segments needs to be devel-oped. One of the possible solutions could be to avoid solving the segmentssequentially; instead, select specific segments to schedule strategically to ob-tain an overall highly reparable schedule. Another possible solution would beto shift to a more specific segment oriented algorithm that would not solve asingle segment at the same time, instead, it would schedule some strategic seg-ments concurrently and placing frames such that distances between differentsegments are maximized.

Extending the Fault Model. In this thesis, we started studying the feasi-bility of schedule healing by applying the SHP-D after a permanent or longenough transient link failure. The development of SHP-SD takes advantage ofHPCs and their broader knowledge of the network to extend the fault modelto consider also switch failures. A single HPC could handle those two kindsof failures as they occurred in its range of knowledge. However, it remains tostudy how to heal failures that affect an HPC, or jeopardize a large networksegment that requires the collaboration of two or more HPCs.

Healing a HPC failure requires the intervention of another neighboringHPC that needs to hold a channel of communication and posses the same up-dated knowledge. First, to detect an HPC failure, we need another neighboringHPC to ping and check its status. Second, this neighboring HPC has to per-form the healing. Lastly, neighbouring HPCs needs to agree on which HPC that

75


should perform the heal and how to replicate all the information to maintain aconsistent schedule knowledge. This also holds for network segment failures,where different HPCs need to communicate and collaborate to perform the re-quired healing.

Plug-and-Play Capabilities for the Semi-Distributed Self-Healing Proto-col. We have only considered the usability of schedule healing as a fault-tolerance measure. However, the fact that when performing healing it allocatesmore transmissions gives us a hint that it can also be applied to aggregate newtraffic to the network. The introduction of networks like fog computing that re-quire low latency and real-time requirements, while providing capabilities forcomponents and traffic to enter and leave the network, is an excellent motiva-tion to research plug-and-play schedules [91]. However, for instance, adding anew frame during run-time comprises a new series of challenges.

If we follow the example of adding a new frame, some frames will likelyneed to be transmitted over a large part of the network. Many small localschedule changes might be required. This relates directly to extending the faultmodel, where multiple HPCs are expected to perform adjustments. To keepconsistency communication between HPCs is essential for making decisionson the path of the new traffic, its allocation and the schedule updates. Thesame holds true when adding components instead of traffic.


should perform the heal and how to replicate all the information to maintain aconsistent schedule knowledge. This also holds for network segment failures,where different HPCs need to communicate and collaborate to perform the re-quired healing.

Plug-and-Play Capabilities for the Semi-Distributed Self-Healing Proto-col. We have only considered the usability of schedule healing as a fault-tolerance measure. However, the fact that when performing healing it allocatesmore transmissions gives us a hint that it can also be applied to aggregate newtraffic to the network. The introduction of networks like fog computing that re-quire low latency and real-time requirements, while providing capabilities forcomponents and traffic to enter and leave the network, is an excellent motiva-tion to research plug-and-play schedules [91]. However, for instance, adding anew frame during run-time comprises a new series of challenges.

If we follow the example of adding a new frame, some frames will likelyneed to be transmitted over a large part of the network. Many small localschedule changes might be required. This relates directly to extending the faultmodel, where multiple HPCs are expected to perform adjustments. To keepconsistency communication between HPCs is essential for making decisionson the path of the new traffic, its allocation and the schedule updates. Thesame holds true when adding components instead of traffic.

76

Bibliography

[1] Hermann Kopetz and Gunther Bauer. The Time-Triggered Architecture.Proceedings of the IEEE, 91(1):112–126, 2003.

[2] Michael R Garey. A Guide to the Theory of NP-Completeness. Comput-ers and Intractability, 1979.

[3] Wilfried Steiner. TTEthernet Specification. TTTech Computertechnik AG,Nov, 39:40, 2008.

[4] Intstitute of Electrical and Electronics Engineers,Inc. 802.1Qbv - Enhancements for Scheduled Traffic.https://standards.ieee.org/standard/802 1Qbv-2015.html. Accessed:2019-06-10.

[5] Flavio Bonomi, Rodolfo Milito, Jiang Zhu, and Sateesh Addepalli. FogComputing and its Role in the Internet of Things. In Proceedings of the1st Workshop on Mobile Cloud Computing (MCC), pages 13–16, NewYork, NY, USA, 2012. ACM.

[6] M. Aazam, S. Zeadally, and K. A. Harras. Deploying Fog Computing inIndustrial Internet of Things and Industry 4.0. Transactions on IndustrialInformatics, 14(10):4674–4682, Oct 2018.

[7] C. Huang, R. Lu, and K. R. Choo. Vehicular Fog Computing: Architec-ture, Use Case, and Security and Forensic Challenges. CommunicationsMagazine, 55(11):105–111, Nov 2017.

[8] J. Song, S. Han, A. Mok, D. Chen, M. Lucas, M. Nixon, and W. Pratt.Wirelesshart: Applying Wireless Technology in Real-Time IndustrialProcess Control. In Proceedings of the 14th Real-Time and Embedded

55

Bibliography

[1] Hermann Kopetz and Gunther Bauer. The Time-Triggered Architecture.Proceedings of the IEEE, 91(1):112–126, 2003.

[2] Michael R Garey. A Guide to the Theory of NP-Completeness. Comput-ers and Intractability, 1979.

[3] Wilfried Steiner. TTEthernet Specification. TTTech Computertechnik AG,Nov, 39:40, 2008.

[4] Intstitute of Electrical and Electronics Engineers,Inc. 802.1Qbv - Enhancements for Scheduled Traffic.https://standards.ieee.org/standard/802 1Qbv-2015.html. Accessed:2019-06-10.

[5] Flavio Bonomi, Rodolfo Milito, Jiang Zhu, and Sateesh Addepalli. FogComputing and its Role in the Internet of Things. In Proceedings of the1st Workshop on Mobile Cloud Computing (MCC), pages 13–16, NewYork, NY, USA, 2012. ACM.

[6] M. Aazam, S. Zeadally, and K. A. Harras. Deploying Fog Computing inIndustrial Internet of Things and Industry 4.0. Transactions on IndustrialInformatics, 14(10):4674–4682, Oct 2018.

[7] C. Huang, R. Lu, and K. R. Choo. Vehicular Fog Computing: Architec-ture, Use Case, and Security and Forensic Challenges. CommunicationsMagazine, 55(11):105–111, Nov 2017.

[8] J. Song, S. Han, A. Mok, D. Chen, M. Lucas, M. Nixon, and W. Pratt.Wirelesshart: Applying Wireless Technology in Real-Time IndustrialProcess Control. In Proceedings of the 14th Real-Time and Embedded

55

77

56 Bibliography

Technology and Applications Symposium (RTAS), pages 377–386. IEEE,2008.

[9] Hermann Kopetz. Real-time systems: design principles for distributedembedded applications. Springer Science & Business Media, 2011.

[10] H. Kopetz and G. Grunsteidl. TTP - A Time-Triggered Protocol for Fault-Tolerant Real-Time Systems. In Proceedings of the 23rd InternationalSymposium on Fault-Tolerant Computing (FTCS), pages 524–533. IEEE,1993.

[11] G. Leen and D. Heffernan. TTCAN: a new Time-Triggered ControllerArea Network. Microprocessors and Microsystems, 26(2):77–94, 2002.

[12] R. Makowitz and C. Temple. Flexray - A Communication Network forAutomotive Control Systems. In Proceedings of the 6th InternationalWorkshop on Factory Communication Systems (WFCS), pages 207–212.IEEE, 2006.

[13] M. J. Lee, R. Zhang, J. Zheng, G. Ahn, C. Zhu, T. R. Park, S. R. Cho,C. S. Shin, and J. S. Ryu. IEEE 802.15.5 WPAN Mesh Standard-LowRate Part: Meshing the Wireless Sensor Networks. Journal on SelectedAreas in Communications, 28(7):973–983, Sep. 2010.

[14] M. Nasri, G. Fohler, and M. Kargahi. A Framework to Construct Cus-tomized Harmonic Periods for Real-Time Systems. In Proceedings ofthe 26th Euromicro Conference on Real-Time Systems (ECRTS), pages211–220. IEEE, July 2014.

[15] A. K. Mok and Wang Weirong. Window-Constrained Real-Time Peri-odic Task Scheduling. In Proceedings of the 22nd Real-Time SystemsSymposium (RTSS), pages 15–24. IEEE, 2001.

[16] Wilfried Steiner. An Evaluation of SMT-based Schedule Synthesis forTime-Triggered Multi-hop Networks. In Proceedings of the 31st Interna-tional Confrence Real-Time Systems Symposium (RTSS), pages 375–384.IEEE, 2010.

[17] Francisco Pozo, Guillermo Rodriguez-Navas, and Hans Hansson. Meth-ods for Large-Scale Time-Triggered Network Scheduling. Electronics,8(7), 2019.

56 Bibliography

Technology and Applications Symposium (RTAS), pages 377–386. IEEE,2008.

[9] Hermann Kopetz. Real-time systems: design principles for distributedembedded applications. Springer Science & Business Media, 2011.

[10] H. Kopetz and G. Grunsteidl. TTP - A Time-Triggered Protocol for Fault-Tolerant Real-Time Systems. In Proceedings of the 23rd InternationalSymposium on Fault-Tolerant Computing (FTCS), pages 524–533. IEEE,1993.

[11] G. Leen and D. Heffernan. TTCAN: a new Time-Triggered ControllerArea Network. Microprocessors and Microsystems, 26(2):77–94, 2002.

[12] R. Makowitz and C. Temple. Flexray - A Communication Network forAutomotive Control Systems. In Proceedings of the 6th InternationalWorkshop on Factory Communication Systems (WFCS), pages 207–212.IEEE, 2006.

[13] M. J. Lee, R. Zhang, J. Zheng, G. Ahn, C. Zhu, T. R. Park, S. R. Cho,C. S. Shin, and J. S. Ryu. IEEE 802.15.5 WPAN Mesh Standard-LowRate Part: Meshing the Wireless Sensor Networks. Journal on SelectedAreas in Communications, 28(7):973–983, Sep. 2010.

[14] M. Nasri, G. Fohler, and M. Kargahi. A Framework to Construct Cus-tomized Harmonic Periods for Real-Time Systems. In Proceedings ofthe 26th Euromicro Conference on Real-Time Systems (ECRTS), pages211–220. IEEE, July 2014.

[15] A. K. Mok and Wang Weirong. Window-Constrained Real-Time Peri-odic Task Scheduling. In Proceedings of the 22nd Real-Time SystemsSymposium (RTSS), pages 15–24. IEEE, 2001.

[16] Wilfried Steiner. An Evaluation of SMT-based Schedule Synthesis forTime-Triggered Multi-hop Networks. In Proceedings of the 31st Interna-tional Confrence Real-Time Systems Symposium (RTSS), pages 375–384.IEEE, 2010.

[17] Francisco Pozo, Guillermo Rodriguez-Navas, and Hans Hansson. Meth-ods for Large-Scale Time-Triggered Network Scheduling. Electronics,8(7), 2019.

78

Bibliography 57

[18] Bruno Dutertre and Leonardo de Moura. A Fast Linear-Arithmetic Solverfor DPLL(T). In Thomas Ball and Robert B. Jones, editors, Com-puter Aided Verification, pages 81–94, Berlin, Heidelberg, 2006. SpringerBerlin Heidelberg.

[19] Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J.Kochenderfer. Reluplex: An Efficient SMT Solver for Verifying DeepNeural Networks. In Rupak Majumdar and Viktor Kuncak, editors, Com-puter Aided Verification, pages 97–117, Cham, 2017. Springer Interna-tional Publishing.

[20] Leonardo de Moura and Nikolaj Bjørner. Z3: An Efficient SMT Solver.In C. R. Ramakrishnan and Jakob Rehof, editors, Tools and Algorithmsfor the Construction and Analysis of Systems, pages 337–340, Berlin,Heidelberg, 2008. Springer Berlin Heidelberg.

[21] Bruno Dutertre. Yices 2.2. In Computer Aided Verification, pages 737–744. Springer, 2014.

[22] Clark Barrett, Aaron Stump, Cesare Tinelli, et al. The smt-lib standard:Version 2.0. In Proceedings of the 8th International Workshop on Satisfi-ability Modulo Theories (SMT), volume 13, page 14, 2010.

[23] J. A. Nelder and R. Mead. A Simplex Method for Function Minimization.The Computer Journal, 7(4):308–313, 01 1965.

[24] N. Karmarkar. A New Polynomial-time Algorithm for Linear Program-ming. In Proceedings of the 16th Annual Symposium on Theory of Com-puting (STC), STOC ’84, pages 302–311, New York, NY, USA, 1984.ACM.

[25] A. Richards and J. P. How. Aircraft Trajectory Planning with CollisionAvoidance using Mixed Integer Linear Programming. In Proceedings ofthe American Control Conference (IEEE Cat. No.CH37301), volume 3,pages 1936–1941 vol.3. IEEE, May 2002.

[26] Gurobi Optimization. Inc.,“Gurobi Optimizer Reference Manual,”, 2015.

[27] Clint Baggerman, Mary McCabe, and Dinesh Verma. Avionics systemarchitecture for the nasa orion vehicle. Technical report, SAE TechnicalPaper, 2009.

Bibliography 57

[18] Bruno Dutertre and Leonardo de Moura. A Fast Linear-Arithmetic Solverfor DPLL(T). In Thomas Ball and Robert B. Jones, editors, Com-puter Aided Verification, pages 81–94, Berlin, Heidelberg, 2006. SpringerBerlin Heidelberg.

[19] Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J.Kochenderfer. Reluplex: An Efficient SMT Solver for Verifying DeepNeural Networks. In Rupak Majumdar and Viktor Kuncak, editors, Com-puter Aided Verification, pages 97–117, Cham, 2017. Springer Interna-tional Publishing.

[20] Leonardo de Moura and Nikolaj Bjørner. Z3: An Efficient SMT Solver.In C. R. Ramakrishnan and Jakob Rehof, editors, Tools and Algorithmsfor the Construction and Analysis of Systems, pages 337–340, Berlin,Heidelberg, 2008. Springer Berlin Heidelberg.

[21] Bruno Dutertre. Yices 2.2. In Computer Aided Verification, pages 737–744. Springer, 2014.

[22] Clark Barrett, Aaron Stump, Cesare Tinelli, et al. The smt-lib standard:Version 2.0. In Proceedings of the 8th International Workshop on Satisfi-ability Modulo Theories (SMT), volume 13, page 14, 2010.

[23] J. A. Nelder and R. Mead. A Simplex Method for Function Minimization.The Computer Journal, 7(4):308–313, 01 1965.

[24] N. Karmarkar. A New Polynomial-time Algorithm for Linear Program-ming. In Proceedings of the 16th Annual Symposium on Theory of Com-puting (STC), STOC ’84, pages 302–311, New York, NY, USA, 1984.ACM.

[25] A. Richards and J. P. How. Aircraft Trajectory Planning with CollisionAvoidance using Mixed Integer Linear Programming. In Proceedings ofthe American Control Conference (IEEE Cat. No.CH37301), volume 3,pages 1936–1941 vol.3. IEEE, May 2002.

[26] Gurobi Optimization. Inc.,“Gurobi Optimizer Reference Manual,”, 2015.

[27] Clint Baggerman, Mary McCabe, and Dinesh Verma. Avionics systemarchitecture for the nasa orion vehicle. Technical report, SAE TechnicalPaper, 2009.

79

58 Bibliography

[28] Hermann Kopetz, Christian El Salloum, Bernhard Huber, Roman Ober-maisser, and Christian Paukovits. Composability in the Time-TriggeredSystem-on-Chip Architecture. In Proceedings of the 6th Conference onSystems-on-Chip, pages 87–90. IEEE, 2008.

[29] Silviu S Craciunas, Ramon Serna Oliver, Martin Chmelık, and WilfriedSteiner. Scheduling Real-Time Communication in IEEE 802.1 Qbv TimeSensitive Networks. In Proceedings of the 24th International Confer-ence on Real-Time Networks and Systems (RTNS), pages 183–192. ACM,2016.

[30] Wei Zheng, Jike Chong, Claudio Pinello, Sri Kanajan, and AlbertoSangiovanni-Vincentelli. Extensible and Scalable Time TriggeredScheduling. In Proceedings of the 5th International Conference on Appli-cation of Concurrency to System Design (ACSD), pages 132–141. IEEE,2005.

[31] Dip Goswami, Martin Lukasiewycz, Reinhard Schneider, and SamarjitChakraborty. Time-Triggered Implementations of Mixed-Criticality Au-tomotive Software. In Proceedings of the Conference on Design, Automa-tion and Test in Europe (DATE), pages 1227–1232. EDA Consortium,2012.

[32] Klaus Schmidt and Ece Guran Schmidt. Message Scheduling for theFlexRay Protocol: The Static Segment. IEEE Transactions on Vehicu-lar Technology, 58(5):2170–2179, 2009.

[33] Martin Lukasiewycz and Samarjit Chakraborty. Concurrent Architec-ture and Schedule Optimization of Time-Triggered Automotive Systems.In Proceedings of the 8th International Conference on Hardware/Soft-ware Codesign and System Synthesis (CODE+ISSS), pages 383–392.IEEE/ACM/IFIP, 2012.

[34] K. Schmidt and E. G. Schmidt. Optimal Message Scheduling for theStatic Segment of FlexRay. In Proceedings of the 72nd Vehicular Tech-nology Conference - Fall, pages 1–5. IEEE, Sep. 2010.

[35] Minkoo Kang, K. Park, and Bongjun Kim. A Static Message Schedul-ing Algorithm for Reducing FlexRay Network Utilization. In Interna-tional Symposium on Industrial Electronics, pages 1287–1291. IEEE,July 2009.

58 Bibliography

[28] Hermann Kopetz, Christian El Salloum, Bernhard Huber, Roman Ober-maisser, and Christian Paukovits. Composability in the Time-TriggeredSystem-on-Chip Architecture. In Proceedings of the 6th Conference onSystems-on-Chip, pages 87–90. IEEE, 2008.

[29] Silviu S Craciunas, Ramon Serna Oliver, Martin Chmelık, and WilfriedSteiner. Scheduling Real-Time Communication in IEEE 802.1 Qbv TimeSensitive Networks. In Proceedings of the 24th International Confer-ence on Real-Time Networks and Systems (RTNS), pages 183–192. ACM,2016.

[30] Wei Zheng, Jike Chong, Claudio Pinello, Sri Kanajan, and AlbertoSangiovanni-Vincentelli. Extensible and Scalable Time TriggeredScheduling. In Proceedings of the 5th International Conference on Appli-cation of Concurrency to System Design (ACSD), pages 132–141. IEEE,2005.

[31] Dip Goswami, Martin Lukasiewycz, Reinhard Schneider, and SamarjitChakraborty. Time-Triggered Implementations of Mixed-Criticality Au-tomotive Software. In Proceedings of the Conference on Design, Automa-tion and Test in Europe (DATE), pages 1227–1232. EDA Consortium,2012.

[32] Klaus Schmidt and Ece Guran Schmidt. Message Scheduling for theFlexRay Protocol: The Static Segment. IEEE Transactions on Vehicu-lar Technology, 58(5):2170–2179, 2009.

[33] Martin Lukasiewycz and Samarjit Chakraborty. Concurrent Architec-ture and Schedule Optimization of Time-Triggered Automotive Systems.In Proceedings of the 8th International Conference on Hardware/Soft-ware Codesign and System Synthesis (CODE+ISSS), pages 383–392.IEEE/ACM/IFIP, 2012.

[34] K. Schmidt and E. G. Schmidt. Optimal Message Scheduling for theStatic Segment of FlexRay. In Proceedings of the 72nd Vehicular Tech-nology Conference - Fall, pages 1–5. IEEE, Sep. 2010.

[35] Minkoo Kang, K. Park, and Bongjun Kim. A Static Message Schedul-ing Algorithm for Reducing FlexRay Network Utilization. In Interna-tional Symposium on Industrial Electronics, pages 1287–1291. IEEE,July 2009.

80

Bibliography 59

[36] Haibo Zeng, Wei Zheng, M. Di Natale, A. Ghosal, P. Giusto, andA. Sangiovanni-Vincentelli. Scheduling the FlexRay Bus using Opti-mization Techniques. In Proceedings of the 46th Design AutomationConference (DAC, pages 874–877. ACM/IEEE, July 2009.

[37] Haibo Zeng, Marco Di Natale, Arkadeb Ghosal, and AlbertoSangiovanni-Vincentelli. Schedule Optimization of Time-Triggered Sys-tems Communicating over the Flexray Static Segment. Transactions onIndustrial Informatics, 7(1):1–17, 2011.

[38] Christian Paukovits and Hermann Kopetz. Concepts of Switching in theTime-Triggered Network-on-Chip. In Proceedings of the 14th Interna-tional Conference on Embedded and Real-Time Computing Systems andApplications (RTCSA, pages 120–129. IEEE, 2008.

[39] Martin Schoeberl, Florian Brandner, Jens Sparsø, and Evangelia Kas-apaki. A Statically Scheduled Time-Division-Multiplexed Network-on-Chip for Real-Time Systems. In Proceedings of the 6th InternationalSymposium on Networks on Chip (NoCS), pages 152–160. IEEE, 2012.

[40] L. Yang, W. Liu, W. Jiang, M. Li, J. Yi, and E. H. Sha. ApplicationMapping and Scheduling for Network-on-Chip-Based MultiprocessorSystem-on-Chip With Fine-Grain Communication Optimization. Trans-actions on Very Large Scale Integration (VLSI) Systems, 24(10):3027–3040, Oct 2016.

[41] Christian Scholer, Rene Krenz-Baath, Ayman Murshed, and Ro-man Obermaisser. Optimal SAT-based Scheduler for Time-TriggeredNetworks-on-a-Chip. In 10th IEEE International Symposium on Indus-trial Embedded Systems (SIES), pages 1–6. IEEE, 2015.

[42] Alexander Biewer, Benjamin Andres, Jens Gladigau, Torsten Schaub, andChristian Haubelt. A Symbolic System Synthesis Approach for HardReal-Time Systems based on Coordinated SMT-solving. In Proocedingsof the 19th Design, Automation & Test in Europe Conference & Exhibi-tion (DATE), pages 357–362. EDA Consortium, 2015.

[43] Benjamin Andres, Alexander Biewer, Javier Romero, Christian Haubelt,and Torsten Schaub. Improving Coordinated SMT-based System Synthe-sis by Utilizing Domain-Specific Heuristics. In Logic Programming andNonmonotonic Reasoning, pages 55–68. Springer, 2015.

Bibliography 59

[36] Haibo Zeng, Wei Zheng, M. Di Natale, A. Ghosal, P. Giusto, andA. Sangiovanni-Vincentelli. Scheduling the FlexRay Bus using Opti-mization Techniques. In Proceedings of the 46th Design AutomationConference (DAC, pages 874–877. ACM/IEEE, July 2009.

[37] Haibo Zeng, Marco Di Natale, Arkadeb Ghosal, and AlbertoSangiovanni-Vincentelli. Schedule Optimization of Time-Triggered Sys-tems Communicating over the Flexray Static Segment. Transactions onIndustrial Informatics, 7(1):1–17, 2011.

[38] Christian Paukovits and Hermann Kopetz. Concepts of Switching in theTime-Triggered Network-on-Chip. In Proceedings of the 14th Interna-tional Conference on Embedded and Real-Time Computing Systems andApplications (RTCSA, pages 120–129. IEEE, 2008.

[39] Martin Schoeberl, Florian Brandner, Jens Sparsø, and Evangelia Kas-apaki. A Statically Scheduled Time-Division-Multiplexed Network-on-Chip for Real-Time Systems. In Proceedings of the 6th InternationalSymposium on Networks on Chip (NoCS), pages 152–160. IEEE, 2012.

[40] L. Yang, W. Liu, W. Jiang, M. Li, J. Yi, and E. H. Sha. ApplicationMapping and Scheduling for Network-on-Chip-Based MultiprocessorSystem-on-Chip With Fine-Grain Communication Optimization. Trans-actions on Very Large Scale Integration (VLSI) Systems, 24(10):3027–3040, Oct 2016.

[41] Christian Scholer, Rene Krenz-Baath, Ayman Murshed, and Ro-man Obermaisser. Optimal SAT-based Scheduler for Time-TriggeredNetworks-on-a-Chip. In 10th IEEE International Symposium on Indus-trial Embedded Systems (SIES), pages 1–6. IEEE, 2015.

[42] Alexander Biewer, Benjamin Andres, Jens Gladigau, Torsten Schaub, andChristian Haubelt. A Symbolic System Synthesis Approach for HardReal-Time Systems based on Coordinated SMT-solving. In Proocedingsof the 19th Design, Automation & Test in Europe Conference & Exhibi-tion (DATE), pages 357–362. EDA Consortium, 2015.

[43] Benjamin Andres, Alexander Biewer, Javier Romero, Christian Haubelt,and Torsten Schaub. Improving Coordinated SMT-based System Synthe-sis by Utilizing Domain-Specific Heuristics. In Logic Programming andNonmonotonic Reasoning, pages 55–68. Springer, 2015.

81

60 Bibliography

[44] J. Huang, J. O. Blech, A. Raabe, C. Buckl, and A. Knoll. Static Schedul-ing of a Time-Triggered Network-on-Chip based on SMT Solving. InProceedings of the Conference Design, Automation and Test in Europe(DATE), pages 509–514. IEEE, 2012.

[45] Mohammed Abuteir and Roman Obermaisser. Scheduling of Rate-Constrained and Time-Triggered Traffic in Multi-cluster TTEthernet Sys-tems. In Proceedings of the 13th International Conference on IndustrialInformatics (INDIN), pages 239–245. IEEE, 2015.

[46] Domitian Tamas-Selicean, Paul Pop, and Wilfried Steiner. Design Op-timization of TTEthernet-based Distributed Real-Time Systems. Real-Time Systems, 51(1):1–35, 2015.

[47] V. Gavrilut and P. Pop. Scheduling in Time Sensitive Networks (TSN)for Mixed-Criticality Industrial Applications. In Proceedings of the 14thInternational Workshop on Factory Communication Systems (WFCS),pages 1–4. IEEE, June 2018.

[48] R. Serna Oliver, S. S. Craciunas, and W. Steiner. Ieee 802.1Qbv GateControl List Synthesis using Array Theory Encoding. In Proceedings ofthe 24th Conference Real-Time and Embedded Technology and Applica-tions Symposium (RTAS), pages 13–24. IEEE, 2018.

[49] Licong Zhang, Debkalpa Goswami, Reinhard Schneider, and Shi-ladri Chakraborty. Task and Network-Level Schedule Co-Synthesis ofEthernet-based Time-Triggered Systems. In Proceedings of the 19th Asiaand South Pacific Conference on Design Automation Conference (ASP-DAC), pages 119–124. IEEE, 2014.

[50] Silviu S Craciunas and Ramon Serna Oliver. Smt-based Task andNetwork-Level Static Schedule Generation for Time-Triggered Net-worked Systems. In Proceedings of the 22nd International Conferenceon Real-Time Networks and Systems (RTNS), page 45. ACM, 2014.

[51] Silviu S Craciunas and Ramon Serna Oliver. Combined Task-andNetwork-level Scheduling for Distributed Time-Triggered Systems.Real-Time Systems, pages 1–40, 2015.

[52] Francisco Pozo, Hansson Hans Rodriguez-Navas, Guillermo, and Wil-fried Steiner. SMT-based Synthesis of TTEthernet Schedules: a Perfor-mance Study. In 10th International Symposium on Industrial EmbeddedSystems (SIES), pages 162–165. IEEE, 2015.

60 Bibliography

[44] J. Huang, J. O. Blech, A. Raabe, C. Buckl, and A. Knoll. Static Schedul-ing of a Time-Triggered Network-on-Chip based on SMT Solving. InProceedings of the Conference Design, Automation and Test in Europe(DATE), pages 509–514. IEEE, 2012.

[45] Mohammed Abuteir and Roman Obermaisser. Scheduling of Rate-Constrained and Time-Triggered Traffic in Multi-cluster TTEthernet Sys-tems. In Proceedings of the 13th International Conference on IndustrialInformatics (INDIN), pages 239–245. IEEE, 2015.

[46] Domitian Tamas-Selicean, Paul Pop, and Wilfried Steiner. Design Op-timization of TTEthernet-based Distributed Real-Time Systems. Real-Time Systems, 51(1):1–35, 2015.

[47] V. Gavrilut and P. Pop. Scheduling in Time Sensitive Networks (TSN)for Mixed-Criticality Industrial Applications. In Proceedings of the 14thInternational Workshop on Factory Communication Systems (WFCS),pages 1–4. IEEE, June 2018.

[48] R. Serna Oliver, S. S. Craciunas, and W. Steiner. Ieee 802.1Qbv GateControl List Synthesis using Array Theory Encoding. In Proceedings ofthe 24th Conference Real-Time and Embedded Technology and Applica-tions Symposium (RTAS), pages 13–24. IEEE, 2018.

[49] Licong Zhang, Debkalpa Goswami, Reinhard Schneider, and Shi-ladri Chakraborty. Task and Network-Level Schedule Co-Synthesis ofEthernet-based Time-Triggered Systems. In Proceedings of the 19th Asiaand South Pacific Conference on Design Automation Conference (ASP-DAC), pages 119–124. IEEE, 2014.

[50] Silviu S Craciunas and Ramon Serna Oliver. Smt-based Task andNetwork-Level Static Schedule Generation for Time-Triggered Net-worked Systems. In Proceedings of the 22nd International Conferenceon Real-Time Networks and Systems (RTNS), page 45. ACM, 2014.

[51] Silviu S Craciunas and Ramon Serna Oliver. Combined Task-andNetwork-level Scheduling for Distributed Time-Triggered Systems.Real-Time Systems, pages 1–40, 2015.

[52] Francisco Pozo, Hansson Hans Rodriguez-Navas, Guillermo, and Wil-fried Steiner. SMT-based Synthesis of TTEthernet Schedules: a Perfor-mance Study. In 10th International Symposium on Industrial EmbeddedSystems (SIES), pages 162–165. IEEE, 2015.

82

Bibliography 61

[53] J. W. Ro, P. Roop, and A. Malik. Schedule Synthesis for Time-TriggeredMulti-hop Wireless Networks with Retransmissions. In Proceedings ofthe 18th International Symposium on Real-Time Distributed Computing(ISORC), pages 94–101. IEEE, 2015.

[54] Jack Kirton, Matthew Bradbury, and Arshad Jhumka. Towards OptimalSource Location Privacy-Aware TDMA Schedules in Wireless SensorNetworks. Computer Networks, 146:125 – 137, 2018.

[55] Abusayeed Saifullah, You Xu, Chenyang Lu, and Yixin Chen. Real-Time Scheduling for WirelessHART Networks. In Proceedings of the31st Real-Time Systems Symposium (RTSS), pages 150–159. IEEE, 2010.

[56] Gabrio Caimi, Dan Burkolter, Thomas Herrmann, Fabian Chudak, andMarco Laumanns. Design of a Railway Scheduling Model for DenseServices. Networks and Spatial Economics, 9(1):25–46, 2009.

[57] Silke Juette and Ulrich W Thonemann. Divide-and-Price: A Decompo-sition Algorithm for Solving Large Railway Crew Scheduling Problems.European Journal of Operational Research, 219(2):214–223, 2012.

[58] Rui Zhang and Cheng Wu. A Hybrid Approach to Large-Scale Job ShopScheduling. Applied Intelligence, 32(1):47–59, 2010.

[59] Rui Zhang and Cheng Wu. A Divide-and-Conquer Strategy with ParticleSwarm Optimization for the Job Shop Scheduling Problem. EngineeringOptimization, 42(7):641–670, 2010.

[60] Iiro Harjunkoski and Ignacio E Grossmann. A Decomposition Approachfor the Scheduling of a Steel Plant Production. Computers & ChemicalEngineering, 25(11):1647–1660, 2001.

[61] Paul Pop, Kare Harbo Poulsen, Viacheslav Izosimov, and Petru Eles.Scheduling and Voltage Scaling for Energy/Reliability Trade-offs inFault-Tolerant Time-Triggered Embedded Systems. In Proceedings of the5th IEEE/ACM international conference on Hardware/software codesignand system synthesis, pages 233–238. ACM, 2007.

[62] L. Wisniewski, V. Wendt, J. Jasperneite, and C. Diedrich. Schedulingof Profinet IRT Communication in Redundant Network Topologies. InProceedings of the 16th World Conference on Factory CommunicationSystems (WFCS), pages 1–4. IEEE, 2016.

Bibliography 61

[53] J. W. Ro, P. Roop, and A. Malik. Schedule Synthesis for Time-TriggeredMulti-hop Wireless Networks with Retransmissions. In Proceedings ofthe 18th International Symposium on Real-Time Distributed Computing(ISORC), pages 94–101. IEEE, 2015.

[54] Jack Kirton, Matthew Bradbury, and Arshad Jhumka. Towards OptimalSource Location Privacy-Aware TDMA Schedules in Wireless SensorNetworks. Computer Networks, 146:125 – 137, 2018.

[55] Abusayeed Saifullah, You Xu, Chenyang Lu, and Yixin Chen. Real-Time Scheduling for WirelessHART Networks. In Proceedings of the31st Real-Time Systems Symposium (RTSS), pages 150–159. IEEE, 2010.

[56] Gabrio Caimi, Dan Burkolter, Thomas Herrmann, Fabian Chudak, andMarco Laumanns. Design of a Railway Scheduling Model for DenseServices. Networks and Spatial Economics, 9(1):25–46, 2009.

[57] Silke Juette and Ulrich W Thonemann. Divide-and-Price: A Decompo-sition Algorithm for Solving Large Railway Crew Scheduling Problems.European Journal of Operational Research, 219(2):214–223, 2012.

[58] Rui Zhang and Cheng Wu. A Hybrid Approach to Large-Scale Job ShopScheduling. Applied Intelligence, 32(1):47–59, 2010.

[59] Rui Zhang and Cheng Wu. A Divide-and-Conquer Strategy with ParticleSwarm Optimization for the Job Shop Scheduling Problem. EngineeringOptimization, 42(7):641–670, 2010.

[60] Iiro Harjunkoski and Ignacio E Grossmann. A Decomposition Approachfor the Scheduling of a Steel Plant Production. Computers & ChemicalEngineering, 25(11):1647–1660, 2001.

[61] Paul Pop, Kare Harbo Poulsen, Viacheslav Izosimov, and Petru Eles.Scheduling and Voltage Scaling for Energy/Reliability Trade-offs inFault-Tolerant Time-Triggered Embedded Systems. In Proceedings of the5th IEEE/ACM international conference on Hardware/software codesignand system synthesis, pages 233–238. ACM, 2007.

[62] L. Wisniewski, V. Wendt, J. Jasperneite, and C. Diedrich. Schedulingof Profinet IRT Communication in Redundant Network Topologies. InProceedings of the 16th World Conference on Factory CommunicationSystems (WFCS), pages 1–4. IEEE, 2016.

83

62 Bibliography

[63] Lukasz Wisniewski, Markus Schumacher, Juergen Jasperneite, and Chris-tian Diedrich. Increasing Flexibility of Time Triggered Ethernet basedSystems by Optimal Greedy Scheduling Approach. In 20th IEEE Confer-ence on Emerging Technologies and Factory Automation (ETFA), pages1–6. IEEE, 2015.

[64] Voica Gavrilut, Domitian Tamas-Selicean, and Paul Pop. Fault-tolerantTopology Selection for TTEthernet Networks. In Proceedings of the 1stSafety and Reliability of Complex Engineered Systems Conference (ES-REL), pages 4001–4009. Citeseer, 2015.

[65] A. A. Atallah, G. B. Hamad, and O. A. Mohamed. Fault-resilient Topol-ogy Planning and Traffic Configuration for IEEE 802.1 Qbv TSN Net-works. In Proceedings of the 24th International Symposium on On-LineTesting And Robust System Design (IOLTS), pages 151–156. IEEE, 2018.

[66] Voica Gavrilut, Bahram Zarrin, Paul Pop, and Soheil Samii. Fault-tolerantTopology and Routing Synthesis for IEEE Time-sensitive Networking. InProceedings of the 25th International Conference on Real-Time Networksand Systems (RTNS), pages 267–276, New York, NY, USA, 2017. ACM.

[67] Viacheslav Izosimov, Paul Pop, Petru Eles, and Zebo Peng. Schedulingof Fault-Tolerant Embedded Systems with Soft and Hard Timing Con-straints. In Proceedings of the 11th conference on Design, Automationand Test in Europe (DATE), pages 915–920. ACM, 2008.

[68] A. Novak, Z. Hanzalek, and P. Sucha. Scheduling of safety-critical time-constrained traffic with f-shaped messages. In Proceedings of the 13thInternational Workshop on Factory Communication Systems (WFCS),pages 1–9. IEEE, May 2017.

[69] L. Zhang, D. Roy, P. Mundhenk, and S. Chakraborty. Schedule Manage-ment Framework for Cloud-Based Future Automotive Software Systems.In Proceedings of the 22nd International Conference on Embedded andReal-Time Computing Systems and Applications (RTCSA), pages 12–21,2016.

[70] N. G. Nayak, F. Durr, and K. Rothermel. Incremental Flow Schedulingand Routing in Time-Sensitive Software-Defined Networks. Transactionson Industrial Informatics, 14(5):2066–2075, 2018.

62 Bibliography

[63] Lukasz Wisniewski, Markus Schumacher, Juergen Jasperneite, and Chris-tian Diedrich. Increasing Flexibility of Time Triggered Ethernet basedSystems by Optimal Greedy Scheduling Approach. In 20th IEEE Confer-ence on Emerging Technologies and Factory Automation (ETFA), pages1–6. IEEE, 2015.

[64] Voica Gavrilut, Domitian Tamas-Selicean, and Paul Pop. Fault-tolerantTopology Selection for TTEthernet Networks. In Proceedings of the 1stSafety and Reliability of Complex Engineered Systems Conference (ES-REL), pages 4001–4009. Citeseer, 2015.

[65] A. A. Atallah, G. B. Hamad, and O. A. Mohamed. Fault-resilient Topol-ogy Planning and Traffic Configuration for IEEE 802.1 Qbv TSN Net-works. In Proceedings of the 24th International Symposium on On-LineTesting And Robust System Design (IOLTS), pages 151–156. IEEE, 2018.

[66] Voica Gavrilut, Bahram Zarrin, Paul Pop, and Soheil Samii. Fault-tolerantTopology and Routing Synthesis for IEEE Time-sensitive Networking. InProceedings of the 25th International Conference on Real-Time Networksand Systems (RTNS), pages 267–276, New York, NY, USA, 2017. ACM.

[67] Viacheslav Izosimov, Paul Pop, Petru Eles, and Zebo Peng. Schedulingof Fault-Tolerant Embedded Systems with Soft and Hard Timing Con-straints. In Proceedings of the 11th conference on Design, Automationand Test in Europe (DATE), pages 915–920. ACM, 2008.

[68] A. Novak, Z. Hanzalek, and P. Sucha. Scheduling of safety-critical time-constrained traffic with f-shaped messages. In Proceedings of the 13thInternational Workshop on Factory Communication Systems (WFCS),pages 1–9. IEEE, May 2017.

[69] L. Zhang, D. Roy, P. Mundhenk, and S. Chakraborty. Schedule Manage-ment Framework for Cloud-Based Future Automotive Software Systems.In Proceedings of the 22nd International Conference on Embedded andReal-Time Computing Systems and Applications (RTCSA), pages 12–21,2016.

[70] N. G. Nayak, F. Durr, and K. Rothermel. Incremental Flow Schedulingand Routing in Time-Sensitive Software-Defined Networks. Transactionson Industrial Informatics, 14(5):2066–2075, 2018.

84

Bibliography 63

[71] Guy Avni, Shibashis Guha, and Guillermo Rodriguez-Navas. Synthesiz-ing Time-Triggered Schedules for Switched Networks with Faulty Links.In International Conference on Embedded Software (EMSOFT), pages1–10. IEEE, 2015.

[72] Guy Avni, Shubham Goel, Thomas A. Henzinger, and GuillermoRodriguez-Navas. Computing Scores of Forwarding Schemes inSwitched Networks with Probabilistic Faults. Tools and Algorithms forthe Construction and Analysis of Systems, pages 169–187, 2017.

[73] N. Kandasamy, J. P. Hayes, and B. T. Murray. Transparent Recovery fromIntermittent Faults in Time-Triggered Distributed Systems. Transactionson Computers, 52(2):113–125, 2003.

[74] N. Wang, Q. Yu, H. Wan, X. Song, and X. Zhao. Adaptive Scheduling forMulti-cluster Time-Triggered Train Communication Networks. Transac-tions on Industrial Informatics, 15(2):1120–1130, 2019.

[75] Q. Yu, T. Wang, X. Zhao, H. Wang, Y. Gao, C. Lu, and M. Gu. FastReal-Time Scheduling for Ethernet-Based Train Control Networks. InProceedings of the 16th International Conference on Parallel and Dis-tributed Processing with Applications, Ubiquitous Computing and Com-munications, Big Data and Cloud Computing, Social Computing and Net-working, Sustainable Computing and Communications (ISPA/IUCC/BD-Cloud/SocialCom/SustainCom), pages 533–540. IEEE, 2018.

[76] M. L. Raagaard, P. Pop, M. Gutierrez, and W. Steiner. Runtime Recon-figuration of Time-Sensitive Networking (TSN) Schedules for Fog Com-puting. In Proocedings of the Fog World Congress (FWC), pages 1–6,2017.

[77] Michael J Baker. Selecting a Research Methodology. The marketingreview, 1(3):373–397, 2000.

[78] Christoph M. Wintersteiger, Youssef Hamadi, and Leonardo de Moura.A Concurrent Portfolio Approach to SMT Solving. In Computer AidedVerification, pages 715–720, Berlin, Heidelberg, 2009. Springer BerlinHeidelberg.

[79] M. Lindauer, H. Hoos, and F. Hutter. From Sequential Algorithm Selec-tion to Parallel Portfolio Selection. In Learning and Intelligent Optimiza-tion, pages 1–16, Cham, 2015. Springer International Publishing.

Bibliography 63

[71] Guy Avni, Shibashis Guha, and Guillermo Rodriguez-Navas. Synthesiz-ing Time-Triggered Schedules for Switched Networks with Faulty Links.In International Conference on Embedded Software (EMSOFT), pages1–10. IEEE, 2015.

[72] Guy Avni, Shubham Goel, Thomas A. Henzinger, and GuillermoRodriguez-Navas. Computing Scores of Forwarding Schemes inSwitched Networks with Probabilistic Faults. Tools and Algorithms forthe Construction and Analysis of Systems, pages 169–187, 2017.

[73] N. Kandasamy, J. P. Hayes, and B. T. Murray. Transparent Recovery fromIntermittent Faults in Time-Triggered Distributed Systems. Transactionson Computers, 52(2):113–125, 2003.

[74] N. Wang, Q. Yu, H. Wan, X. Song, and X. Zhao. Adaptive Scheduling forMulti-cluster Time-Triggered Train Communication Networks. Transac-tions on Industrial Informatics, 15(2):1120–1130, 2019.

[75] Q. Yu, T. Wang, X. Zhao, H. Wang, Y. Gao, C. Lu, and M. Gu. FastReal-Time Scheduling for Ethernet-Based Train Control Networks. InProceedings of the 16th International Conference on Parallel and Dis-tributed Processing with Applications, Ubiquitous Computing and Com-munications, Big Data and Cloud Computing, Social Computing and Net-working, Sustainable Computing and Communications (ISPA/IUCC/BD-Cloud/SocialCom/SustainCom), pages 533–540. IEEE, 2018.

[76] M. L. Raagaard, P. Pop, M. Gutierrez, and W. Steiner. Runtime Recon-figuration of Time-Sensitive Networking (TSN) Schedules for Fog Com-puting. In Proocedings of the Fog World Congress (FWC), pages 1–6,2017.

[77] Michael J Baker. Selecting a Research Methodology. The marketingreview, 1(3):373–397, 2000.

[78] Christoph M. Wintersteiger, Youssef Hamadi, and Leonardo de Moura.A Concurrent Portfolio Approach to SMT Solving. In Computer AidedVerification, pages 715–720, Berlin, Heidelberg, 2009. Springer BerlinHeidelberg.

[79] M. Lindauer, H. Hoos, and F. Hutter. From Sequential Algorithm Selec-tion to Parallel Portfolio Selection. In Learning and Intelligent Optimiza-tion, pages 1–16, Cham, 2015. Springer International Publishing.

85

64 Bibliography

[80] Sylvain Conchon, Amit Goel, Sava Krstic, Alain Mebsout, and FatihaZaıdi. Cubicle: A Parallel SMT-Based Model Checker for Parameter-ized Systems. In Computer Aided Verification, pages 718–724, Berlin,Heidelberg, 2012. Springer Berlin Heidelberg.

[81] Florian Corzilius, Gereon Kremer, Sebastian Junges, Stefan Schupp, andErika Abraham. SMT-RAT: An Open Source C++ Toolbox for Strategicand Parallel SMT Solving. In Theory and Applications of SatisfiabilityTesting (SAT), pages 360–368, Cham, 2015. Springer International Pub-lishing.

[82] TK Ralphs. Parallel Branch and Cut. Parallel Combinatorial Optimiza-tion, 58:53, 2006.

[83] S. Goil, S. Alaru, and S. Ranka. Concatenated Parallelism: a Techniquefor Efficient Parallel Divide and Conquer. In Proceedings of the 8th Sym-posium on Parallel and Distributed Processing (PDP), pages 488–495.IEEE, Oct 1996.

[84] Bernd Freisleben and Thilo Kielmann. Automated Transformation of Se-quential Divide-and-Conquer Algorithms into Parallel Programs. Com-puters and Artificial Intelligence, 14:579–596, 1995.

[85] Philip Husbands, Frank Mill, and Stephen Warrington. Genetic Al-gorithms, Production Plan Optimisation and Scheduling. In ParallelProblem Solving from Nature, pages 80–84, Berlin, Heidelberg, 1991.Springer Berlin Heidelberg.

[86] Seung Jun Baek, Gustavo de Veciana, and Xun Su. Minimizing EnergyConsumption in Large-Scale Sensor Networks Through Distributed DataCompression and Hierarchical Aggregation. Journal on Selected Areasin Communications, 22(6):1130–1140, Aug 2004.

[87] M. Hu, J. Luo, Y. Wang, M. Lukasiewycz, and Z. Zeng. Holistic Schedul-ing of Real-Time Applications in Time-Triggered In-Vehicle Networks.Transactions on Industrial Informatics, 10(3):1817–1828, Aug 2014.

[88] P. Pop, P. Eles, and Zebo Peng. An Improved Scheduling Techniquefor Time-Triggered Embedded Systems. In Proceedings of the 25th EU-ROMICRO Conference. Informatics: Theory and Practice for the NewMillennium, volume 1, pages 303–310 vol.1, Sep. 1999.

64 Bibliography

[80] Sylvain Conchon, Amit Goel, Sava Krstic, Alain Mebsout, and FatihaZaıdi. Cubicle: A Parallel SMT-Based Model Checker for Parameter-ized Systems. In Computer Aided Verification, pages 718–724, Berlin,Heidelberg, 2012. Springer Berlin Heidelberg.

[81] Florian Corzilius, Gereon Kremer, Sebastian Junges, Stefan Schupp, andErika Abraham. SMT-RAT: An Open Source C++ Toolbox for Strategicand Parallel SMT Solving. In Theory and Applications of SatisfiabilityTesting (SAT), pages 360–368, Cham, 2015. Springer International Pub-lishing.

[82] TK Ralphs. Parallel Branch and Cut. Parallel Combinatorial Optimiza-tion, 58:53, 2006.

[83] S. Goil, S. Alaru, and S. Ranka. Concatenated Parallelism: a Techniquefor Efficient Parallel Divide and Conquer. In Proceedings of the 8th Sym-posium on Parallel and Distributed Processing (PDP), pages 488–495.IEEE, Oct 1996.

[84] Bernd Freisleben and Thilo Kielmann. Automated Transformation of Se-quential Divide-and-Conquer Algorithms into Parallel Programs. Com-puters and Artificial Intelligence, 14:579–596, 1995.

[85] Philip Husbands, Frank Mill, and Stephen Warrington. Genetic Al-gorithms, Production Plan Optimisation and Scheduling. In ParallelProblem Solving from Nature, pages 80–84, Berlin, Heidelberg, 1991.Springer Berlin Heidelberg.

[86] Seung Jun Baek, Gustavo de Veciana, and Xun Su. Minimizing EnergyConsumption in Large-Scale Sensor Networks Through Distributed DataCompression and Hierarchical Aggregation. Journal on Selected Areasin Communications, 22(6):1130–1140, Aug 2004.

[87] M. Hu, J. Luo, Y. Wang, M. Lukasiewycz, and Z. Zeng. Holistic Schedul-ing of Real-Time Applications in Time-Triggered In-Vehicle Networks.Transactions on Industrial Informatics, 10(3):1817–1828, Aug 2014.

[88] P. Pop, P. Eles, and Zebo Peng. An Improved Scheduling Techniquefor Time-Triggered Embedded Systems. In Proceedings of the 25th EU-ROMICRO Conference. Informatics: Theory and Practice for the NewMillennium, volume 1, pages 303–310 vol.1, Sep. 1999.

86

[89] Leonardo De Moura and Nikolaj Bjørner. Satisfiability Modulo Theories:Introduction and Applications. Communications of the ACM, 54(9):69–77, 2011.

[90] F. Pozo and G. Rodriguez-Navas. A Semi-Distributed Self-Healing Pro-tocol for Run-Time Repairs of Time-Triggered Schedules. In Proceedingsof the 24th Conference on Emerging Technologies and Factory Automa-tion (ETFA), pages 1–4. IEEE, 2019.

[91] V. Gazis, A. Leonardi, K. Mathioudakis, K. Sasloglou, P. Kikiras, andR. Sudhaakar. Components of Fog Computing in an Industrial Internet ofThings Context. In Proceedings of the 12th Annual International Confer-ence on Sensing, Communication, and Networking - Workshops (SECONWorkshops), pages 1–6. IEEE, June 2015.

[89] Leonardo De Moura and Nikolaj Bjørner. Satisfiability Modulo Theories:Introduction and Applications. Communications of the ACM, 54(9):69–77, 2011.

[90] F. Pozo and G. Rodriguez-Navas. A Semi-Distributed Self-Healing Pro-tocol for Run-Time Repairs of Time-Triggered Schedules. In Proceedingsof the 24th Conference on Emerging Technologies and Factory Automa-tion (ETFA), pages 1–4. IEEE, 2019.

[91] V. Gazis, A. Leonardi, K. Mathioudakis, K. Sasloglou, P. Kikiras, andR. Sudhaakar. Components of Fog Computing in an Industrial Internet ofThings Context. In Proceedings of the 12th Annual International Confer-ence on Sensing, Communication, and Networking - Workshops (SECONWorkshops), pages 1–6. IEEE, June 2015.

87

methods for efficient and adaptive scheduling of...

Documents