how linux rt preempt works - elinux.org · how linux rt_preempt works ... this presentation...
TRANSCRIPT
![Page 1: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/1.jpg)
![Page 2: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/2.jpg)
How Linux RT_PREEMPT WorksA common observation about real time systems isthat the cost of the increased determinism of realtime is decreased throughput and increasedaverage latency. Does this hold true for LinuxPREEMPT_RT_FULL?
This presentation enumerates some of the designchoices and implementation that enable LinuxPREEMPT_RT_FULL real time and the resultingperformance implications.
Frank Rowand, Sony Network Entertainment October 28, 2011 111028_0434
![Page 3: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/3.jpg)
The Question
The cost of the increased determinism of real time
- Increased average latency
- Decreased throughput
Is this true for Linux PREEMPT_RT_FULL?
![Page 4: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/4.jpg)
Some Random Data
Compare the latency of an application onkernels built with:
1) CONFIG_PREEMPT_NONE
2) CONFIG_PREEMPT_RT_FULL
![Page 5: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/5.jpg)
Some Random Data
Test System:
- ARM11 MPCore development system
- 4 cpus
- 210 Mhz processor clock
- L1 cache 64 Kbyte per processor
- L2 cache 1 Mbyte unified
- Linux 3.0.6-rt17
![Page 6: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/6.jpg)
![Page 7: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/7.jpg)
Latency (Response Time)
Kernel without RT patchset:
+ smaller average + smaller minimum - larger maximum
PREEMPT RT enabled:
- larger average - larger minimum + smaller maximum + more consistent
![Page 8: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/8.jpg)
Statistics
Min Avg Max ---- ---- ----PREEMPT_NONE 29 38 9186
PREEMPT_RT_FULL 35 41 95
![Page 9: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/9.jpg)
Latency (Response Time)
Next graph shows an old kernel, circa 2009
Hardware configuration: unknown
![Page 10: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/10.jpg)
Red Hat Enterprise LinuxRed Hat MRG tuned
source: Red Hat
Messaging Workload
![Page 11: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/11.jpg)
The Answer
The cost of the increased determinism of real time
- Increased average latency
Is this true for Linux PREEMPT_RT_FULL?
YES
![Page 12: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/12.jpg)
Some Random Data
Compare the throughput of an application onkernels built with:
1) CONFIG_PREEMPT_NONE
2) CONFIG_PREEMPT_RT_FULL
![Page 13: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/13.jpg)
Some Random Data
The workload I used for the throughput results
- is not realistic
- is not reasonable
- violates real time application design rules
- is stupid!
- but was easy to implement...
![Page 14: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/14.jpg)
Some Random Data
Test System (same as first test system):
- ARM11 MPCore development system
- 4 cpus
- 210 Mhz processor clock
- L1 cache 64 Kbyte per processor
- L2 cache 1 Mbyte unified
- Linux 3.0.6-rt17
![Page 15: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/15.jpg)
Test Variables
UP vs SMP
SMP, maxcpus=4 vs SMP maxcpus=1
workload: SCHED_FIFO vs SCHED_NORMAL
1, 2, or 4 instances of the workload
Permutations of variables results in 10 tests
![Page 16: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/16.jpg)
![Page 17: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/17.jpg)
![Page 18: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/18.jpg)
![Page 19: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/19.jpg)
![Page 20: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/20.jpg)
![Page 21: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/21.jpg)
![Page 22: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/22.jpg)
![Page 23: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/23.jpg)
![Page 24: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/24.jpg)
![Page 25: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/25.jpg)
![Page 26: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/26.jpg)
The Answer
The cost of the increased determinism of real time
- Increased average latency
- Decreased throughput
Is this true for Linux PREEMPT_RT_FULL?
YES
![Page 27: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/27.jpg)
Part 2
This presentation enumerates some of the designchoices and implementation that enable LinuxPREEMPT_RT_FULL real time and the resultingperformance implications.
![Page 28: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/28.jpg)
Enabling real-time Linux
- preemptible kernel
- locking
- threaded interrupt handlers
- threaded softirq
![Page 29: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/29.jpg)
Non-Preemptible Kernel
When a task invokes a system call, thesystem call must complete (or sleep dueto blocking on a resource) before anothertask can be scheduled.
Preemption can not occur during theexecution of the system call.
![Page 30: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/30.jpg)
Non-Preemptible Kernel
Preemption can not occur during theexecution of the system call.
Scheduling may occur on:
- completion of system call
- system call sleeping
![Page 31: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/31.jpg)
Non-Preemptible Kernel
Problems of typical non-preemptible kernel:
- kernel path lengths non-deterministic
- longest kernel path has long duration
- large variance in kernel path length
![Page 32: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/32.jpg)
Non-Preemptible Kernel
Next slide illustrates non-preemptible kernel.
![Page 33: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/33.jpg)
syscall external syscall event completes
RTtask
Normaltask
syscall
IRQhandler
wake RT task RT task runs
![Page 34: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/34.jpg)
Non-Preemptible Kernel
Next slide illustrates non-preemptible kernel.
Adding some complexity:
- 2 external events occur
- lock (critical section) during syscall
![Page 35: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/35.jpg)
syscall external syscall events completes
RTtask
Normaltask
syscall
IRQ 1handler
IRQ 2handler
wake RT task RT task runs
critical section
![Page 36: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/36.jpg)
Preemptible Kernel
Mainline 2.6 and 3.0 kernel
CONFIG_PREEMPT_NONE
No forced kernel preemption
CONFIG_PREEMPT_VOLUNTARY
Explicit preemption points in kernel
CONFIG_PREEMPT
All kernel code (not in critical section) preemptible
![Page 37: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/37.jpg)
Preemptible Kernel
RT_PREEMPT patch renames config option:
Vanilla 2.6 kernel
CONFIG_PREEMPT
RT_PREEMPT 2.6 kernel
CONFIG_PREEMPT_DESKTOP
![Page 38: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/38.jpg)
Preemptible Kernel
RT_PREEMPT patch renames config option:
Vanilla 3.0 kernel
CONFIG_PREEMPT
RT_PREEMPT 3.0 kernel
CONFIG_PREEMPT_LL
![Page 39: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/39.jpg)
Preemptible Kernel
Mainline 2.6 and 3.0 kernel
CONFIG_PREEMPT “fully preemptible”
- except when preemption is explicitly disabled
- except when interrupts are explicitly disabled
- except when a lock is held (“in a critical section”)
![Page 40: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/40.jpg)
Preemptible Kernel
Next slide illustrates preemptible kernel.
![Page 41: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/41.jpg)
syscall external syscall events completes
RTtask
Normaltask
syscall
IRQ 1handler
IRQ 2handler
wake RT task RT task runs
critical section
![Page 42: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/42.jpg)
Score
- added 0 schedule with context switch
+ shorter wakeup latency
![Page 43: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/43.jpg)
Preemptible Kernel
RT_PREEMPT 2.6 kernel
CONFIG_PREEMPT_RT
“fully preemptible”
- except when preemption is explicitly disabled
- except when interrupts are explicitly disabled
- except when a raw spinlock is held
![Page 44: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/44.jpg)
Preemptible Kernel
RT_PREEMPT 3.0 kernel
CONFIG_PREEMPT_RT_FULL
“fully preemptible”
- except when preemption is explicitly disabled
- except when interrupts are explicitly disabled
- except when a raw spinlock is held
![Page 45: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/45.jpg)
Preemptible Kernel
CONFIG_PREEMPT_RTCONFIG_PREEMPT_RT_FULL
Most kernel locks are converted to preemptible priority inheritance mutex.
Some kernel locks are converted to non-preemptible raw spinlock.
![Page 46: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/46.jpg)
Preemptible Kernel
Next slide illustrates preemptible kernelwith spinlocks converted to mutexes.
![Page 47: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/47.jpg)
syscall external syscall events completes
RTtask
Normaltask
syscall
IRQ 1handler
IRQ 2handler
wake RT task RT task runs
critical section (mutex)
![Page 48: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/48.jpg)
Score
- added 0 schedule with context switch
+ shorter wakeup latency
![Page 49: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/49.jpg)
Priority Inheritance Mutex
- May result in more schedule events.
- Avoids priority inversion.
- Reader-Writer lock limited to one concurrent reader to minimize PI complexity.
- Limits scalability of multiple readers.
![Page 50: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/50.jpg)
non-PI Mutex – Priority Inversionhigher number is higher priority
p = 1A
p = 2B
p = 3C
A gets lock B wakes C wakes C blocks on lock
![Page 51: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/51.jpg)
PI Mutexhigher number is higher priority
p = 1 p = 3 p = 1A
p = 2B
p = 3C
A gets lock B wakes C wakes C blocks on lock, A boosted to 3A releases lock, unboosted back to 1, C gets lock
![Page 52: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/52.jpg)
Threaded Interrupt Handler
![Page 53: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/53.jpg)
Overview Of Interrupt handling algorithm
- Save context - Handle “highest priority” interrupt Interrupt handler executes in interrupt mode irq_exit() may process softirq or wake softirqd - Iterate over active interrupts (arch dependent) - Schedule - Restore context returning either to previous process or to newly scheduled process
![Page 54: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/54.jpg)
Overview Of Threaded Interrupt handling algorithm
- Save context - Handle “highest priority” interrupt Wake Interrupt handler thread. irq_exit() may process softirq or wake softirqd - Iterate over active interrupts (arch dependent) - Schedule - Restore context returning either to previous process or to newly scheduled process
Interrupt handler thread executes when scheduled.
![Page 55: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/55.jpg)
Threaded Interrupt Handler
RT_PREEMPT patchset converts almost alldrivers to threaded model. (Timer handlerexecutes in interrupt context.)
2.6.xx mainline does not convert drivers tothreaded model. Each driver must beexplicitly converted.
![Page 56: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/56.jpg)
Threaded Interrupt Handler
Priorities must be properly set for:
- interrupt handler threads
- softirq threads
- other kernel threads
- real time application processes / threads
Do not expect default priorities to be proper.
![Page 57: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/57.jpg)
Preemptible Kernel
Next slide illustrates preemptible kernelwith interrupt threads.
![Page 58: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/58.jpg)
syscall external syscall events completes
RTtask
Normaltask
syscall
IRQ 1handler
IRQ 2handler
wake RT task RT task runs
critical section (mutex)
![Page 59: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/59.jpg)
Score
- added 2 schedule with context switch
+ shorter wakeup latency
![Page 60: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/60.jpg)
Other Interrupt Overhead
![Page 61: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/61.jpg)
Other Interrupt Overhead
CONFIG_PREEMPT_RT andCONFIG_PREEMPT_RT_FULL changes:
irq_exit() may process softirq or wake ksoftirqd thread
to:
irq_exit() may wake ksoftirqd thread
![Page 62: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/62.jpg)
syscall external syscall events completes
RTtask
Normaltask
syscall
IRQ 1
IRQ 2
softirq
![Page 63: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/63.jpg)
syscall external syscall events completes
RTtask
Normaltask
syscall
IRQ 1
IRQ 2
softirq
![Page 64: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/64.jpg)
Score
- added 1 schedule with context switch
+ shorter wakeup latency
![Page 65: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/65.jpg)
Other Interrupt Overhead
raise_softirq(), (softirq trigger) typically calledfrom:
irq context (timer softirq)
interrupt thread
but can be called from anywhere in the kernel.
Previous slides show trigger from irq context.
![Page 66: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/66.jpg)
Other Locking Overhead
CONFIG_TREE_PREEMPT_RCU
Evolving in the early 3.0 RT patches...
Not analyzed in this presentation.
![Page 67: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/67.jpg)
Other Locking Overhead
local_lock()
Uses migrate_disable() instead of preempt_disable().
Evolving in the early 3.0 RT patches...
Not analyzed in this presentation.
![Page 68: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/68.jpg)
Real Life
Real systems are much more complicatedthan the previous diagrams.
Other scenarios can generate differentperformance improvements or penalties.
![Page 69: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/69.jpg)
Recap: Enabling real-time Linux
- preemptible kernel
- locking
- interrupt handlers
- threaded softirq
![Page 70: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/70.jpg)
Impact of Real-Time Features
+ Variance of real-time task latency decreased
+ Maximum real-time task latency decreased
- Average real-time task latency may be increased
- Throughput decreased
![Page 71: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/71.jpg)
Recap: The Answer
The cost of the increased determinism of real time
- Increased average latency
- Decreased throughput
Is this true for Linux PREEMPT_RT_FULL?
YES
![Page 72: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/72.jpg)
Questions?
![Page 74: How Linux RT PREEMPT Works - eLinux.org · How Linux RT_PREEMPT Works ... This presentation enumerates some of the design choices and implementation that enable Linux PREEMPT_RT_FULL](https://reader031.vdocument.in/reader031/viewer/2022020114/5b2d42bd7f8b9ace6e8bb5c3/html5/thumbnails/74.jpg)