Lab 4: If you liked it, then you should have put a“lock” on it
Advanced Operating Systems
Zubair Nabi
February 20, 2013
Concurrency within the OS
1 Multiple CPUs share kernel data structures• Can interfere with each other leading to inconsistencies
2 Even on uniprocessors, interrupt handlers can interfere withnon-interrupt code
3 xv6 uses locks for both situations
Concurrency within the OS
1 Multiple CPUs share kernel data structures• Can interfere with each other leading to inconsistencies
2 Even on uniprocessors, interrupt handlers can interfere withnon-interrupt code
3 xv6 uses locks for both situations
Concurrency within the OS
1 Multiple CPUs share kernel data structures• Can interfere with each other leading to inconsistencies
2 Even on uniprocessors, interrupt handlers can interfere withnon-interrupt code
3 xv6 uses locks for both situations
Race conditions: Example
• Several processors share a single disk
• Disk driver has a single linked list idequeue of outstanding diskrequests
• Processors concurrently make a call to iderw which adds arequest to the list
Race conditions: Example
• Several processors share a single disk
• Disk driver has a single linked list idequeue of outstanding diskrequests
• Processors concurrently make a call to iderw which adds arequest to the list
Race conditions: Example
• Several processors share a single disk
• Disk driver has a single linked list idequeue of outstanding diskrequests
• Processors concurrently make a call to iderw which adds arequest to the list
Race conditions (2): Example
1 struct list {2 int data;3 struct list ∗next;4 };56 struct list ∗list = 0;78 void insert(int data)9 {
10 struct list ∗l;11 l = malloc(sizeof ∗l);12 l−>data = data;13 l−>next = list;14 list = l;15 }
Race conditions (3): Problem
• Possible race condition on line 13 and 14
• In case of two concurrent insertions, the first one will be lost
Race conditions (3): Problem
• Possible race condition on line 13 and 14
• In case of two concurrent insertions, the first one will be lost
Race conditions (4): Solution
1 struct list ∗list = 0;2 struct lock listlock;34 void insert(int data)5 {6 struct list ∗l;7 acquire(&listlock);8 l = malloc(sizeof ∗l);9 l−>data = data;
10 l−>next = list;11 list = l;12 release(&listlock);13 }
Lock representation
1 struct spinlock {2 uint locked;3 };
Acquire lock
1 void acquire(struct spinlock ∗lk)2 {3 for(;;) {4 if(!lk−>locked) {5 lk−>locked = 1;6 break;7 }8 }9 }
Acquiring lock (2): Problem
• Possible race condition on line 4 and 5
• So the solution itself causes a race condition!
Acquiring lock (2): Problem
• Possible race condition on line 4 and 5
• So the solution itself causes a race condition!
Hardware support
1 static inline uint2 xchg(volatile uint ∗addr, uint newval)3 {4 uint result;56 asm volatile("lock; xchgl %0, %1" :7 "+m" (∗addr), "=a" (result) :8 "1" (newval) :9 "cc");
10 }
Atomic acquire lock
1 void acquire(struct spinlock ∗lk)2 {3 pushcli(); // disable interrupts.45 if(holding(lk))6 panic("acquire");78 while(xchg(&lk−>locked, 1) != 0)9 ;
10 }
Atomic release lock
1 void release(struct spinlock ∗lk)2 {3 if(!holding(lk))4 panic("release");56 xchg(&lk−>locked, 0);78 popcli();9 }
Recursive
• What happens if a callee tries to acquire a lock held by its caller?
• Solution: recursive locks
• But they do not ensure mutual exclusion between the caller andthe callee
• Pass the buck to the programmer
Recursive
• What happens if a callee tries to acquire a lock held by its caller?
• Solution: recursive locks
• But they do not ensure mutual exclusion between the caller andthe callee
• Pass the buck to the programmer
Recursive
• What happens if a callee tries to acquire a lock held by its caller?
• Solution: recursive locks
• But they do not ensure mutual exclusion between the caller andthe callee
• Pass the buck to the programmer
Recursive
• What happens if a callee tries to acquire a lock held by its caller?
• Solution: recursive locks
• But they do not ensure mutual exclusion between the caller andthe callee
• Pass the buck to the programmer
Lock usage example
1 void iderw(struct buf ∗b)2 {3 struct buf ∗∗pp;4 acquire(&idelock);5 b−>qnext = 0;6 for(pp=&idequeue; ∗pp; pp=&(∗pp)−>qnext)7 ;8 ∗pp = b;9 // Wait for request to finish.
10 while((b−>flags & (B_VALID|B_DIRTY))11 != B_VALID){12 sleep(b, &idelock);13 }14 release(&idelock);15 }
When to use locks
• To use:1 A variable can concurrently be written to by multiple CPUs2 An invariant spans multiple data structures
• Possible to protect the kernel through a “giant kernel lock”• Problem(s)?• Reminiscent of the GIL in Python
• Possible to have fine-grained locks
When to use locks
• To use:1 A variable can concurrently be written to by multiple CPUs2 An invariant spans multiple data structures
• Possible to protect the kernel through a “giant kernel lock”• Problem(s)?• Reminiscent of the GIL in Python
• Possible to have fine-grained locks
When to use locks
• To use:1 A variable can concurrently be written to by multiple CPUs2 An invariant spans multiple data structures
• Possible to protect the kernel through a “giant kernel lock”• Problem(s)?• Reminiscent of the GIL in Python
• Possible to have fine-grained locks
When to use locks
• To use:1 A variable can concurrently be written to by multiple CPUs2 An invariant spans multiple data structures
• Possible to protect the kernel through a “giant kernel lock”• Problem(s)?• Reminiscent of the GIL in Python
• Possible to have fine-grained locks
When to use locks
• To use:1 A variable can concurrently be written to by multiple CPUs2 An invariant spans multiple data structures
• Possible to protect the kernel through a “giant kernel lock”• Problem(s)?• Reminiscent of the GIL in Python
• Possible to have fine-grained locks
When to use locks
• To use:1 A variable can concurrently be written to by multiple CPUs2 An invariant spans multiple data structures
• Possible to protect the kernel through a “giant kernel lock”• Problem(s)?• Reminiscent of the GIL in Python
• Possible to have fine-grained locks
When to use locks
• To use:1 A variable can concurrently be written to by multiple CPUs2 An invariant spans multiple data structures
• Possible to protect the kernel through a “giant kernel lock”• Problem(s)?• Reminiscent of the GIL in Python
• Possible to have fine-grained locks
Lock ordering
• If code path x needs locks in the order A and B while y needsthem in order B and A, could there be a problem?
• Need to ensure that all code paths acquire locks in the sameorder
Lock ordering
• If code path x needs locks in the order A and B while y needsthem in order B and A, could there be a problem?
• Need to ensure that all code paths acquire locks in the sameorder
Interrupt handlers
• Locks are also used to synchronize access between interrupthandlers and non-interrupt code
Interrupt handlers (2): Example
1 T_IRQ0 + IRQ_TIMER:2 if(cpu−>id == 0){3 acquire(&tickslock);4 ticks++;5 wakeup(&ticks);6 release(&tickslock);7 }8 lapiceoi();9 break;
Interrupt handlers (3): Example
1 void sys_sleep(void){2 int n;3 uint ticks0;4 if(argint(0, &n) < 0)5 return −1;6 acquire(&tickslock);7 ticks0 = ticks;8 while(ticks − ticks0 < n){9 if(proc−>killed){
10 release(&tickslock);11 return −1;12 }13 sleep(&ticks, &tickslock);14 }15 release(&tickslock);16 }
Interrupt handlers
• Interrupts lead to concurrency problems on uniprocessors too
• Disable interrupts before acquiring a lock: pushcli()
• Re-enable on release: popcli()
• Avoids recursive locks
Interrupt handlers
• Interrupts lead to concurrency problems on uniprocessors too
• Disable interrupts before acquiring a lock: pushcli()
• Re-enable on release: popcli()
• Avoids recursive locks
Interrupt handlers
• Interrupts lead to concurrency problems on uniprocessors too
• Disable interrupts before acquiring a lock: pushcli()
• Re-enable on release: popcli()
• Avoids recursive locks
Interrupt handlers
• Interrupts lead to concurrency problems on uniprocessors too
• Disable interrupts before acquiring a lock: pushcli()
• Re-enable on release: popcli()
• Avoids recursive locks
Today’s Task
• xv6 processes only have a single thread so processes do notshare any memory
• But it is definitely possible to add multithreading
• Imagine that you have been provided a library with methods tocreate (xv6_thread_create()) and block(xv6_thread_join()) threads
• xv6 already provides you with a spinlock with methods toacquire and release that lock.
• In addition, it also has methods to:1 Put a thread to sleep (sleep(void *chan, structspinlock *lk)) on a variable (*chan) (it releases lk beforesleeping and then reacquires it after waking up)
2 Wake up (wakeup(void *chan)) threads sleeping on avariable (*chan)
Today’s Task
• xv6 processes only have a single thread so processes do notshare any memory
• But it is definitely possible to add multithreading
• Imagine that you have been provided a library with methods tocreate (xv6_thread_create()) and block(xv6_thread_join()) threads
• xv6 already provides you with a spinlock with methods toacquire and release that lock.
• In addition, it also has methods to:1 Put a thread to sleep (sleep(void *chan, structspinlock *lk)) on a variable (*chan) (it releases lk beforesleeping and then reacquires it after waking up)
2 Wake up (wakeup(void *chan)) threads sleeping on avariable (*chan)
Today’s Task
• xv6 processes only have a single thread so processes do notshare any memory
• But it is definitely possible to add multithreading
• Imagine that you have been provided a library with methods tocreate (xv6_thread_create()) and block(xv6_thread_join()) threads
• xv6 already provides you with a spinlock with methods toacquire and release that lock.
• In addition, it also has methods to:1 Put a thread to sleep (sleep(void *chan, structspinlock *lk)) on a variable (*chan) (it releases lk beforesleeping and then reacquires it after waking up)
2 Wake up (wakeup(void *chan)) threads sleeping on avariable (*chan)
Today’s Task
• xv6 processes only have a single thread so processes do notshare any memory
• But it is definitely possible to add multithreading
• Imagine that you have been provided a library with methods tocreate (xv6_thread_create()) and block(xv6_thread_join()) threads
• xv6 already provides you with a spinlock with methods toacquire and release that lock.
• In addition, it also has methods to:1 Put a thread to sleep (sleep(void *chan, structspinlock *lk)) on a variable (*chan) (it releases lk beforesleeping and then reacquires it after waking up)
2 Wake up (wakeup(void *chan)) threads sleeping on avariable (*chan)
Today’s Task
• xv6 processes only have a single thread so processes do notshare any memory
• But it is definitely possible to add multithreading
• Imagine that you have been provided a library with methods tocreate (xv6_thread_create()) and block(xv6_thread_join()) threads
• xv6 already provides you with a spinlock with methods toacquire and release that lock.
• In addition, it also has methods to:1 Put a thread to sleep (sleep(void *chan, structspinlock *lk)) on a variable (*chan) (it releases lk beforesleeping and then reacquires it after waking up)
2 Wake up (wakeup(void *chan)) threads sleeping on avariable (*chan)
Today’s Task
• xv6 processes only have a single thread so processes do notshare any memory
• But it is definitely possible to add multithreading
• Imagine that you have been provided a library with methods tocreate (xv6_thread_create()) and block(xv6_thread_join()) threads
• xv6 already provides you with a spinlock with methods toacquire and release that lock.
• In addition, it also has methods to:1 Put a thread to sleep (sleep(void *chan, structspinlock *lk)) on a variable (*chan) (it releases lk beforesleeping and then reacquires it after waking up)
2 Wake up (wakeup(void *chan)) threads sleeping on avariable (*chan)
Today’s Task
• xv6 processes only have a single thread so processes do notshare any memory
• But it is definitely possible to add multithreading
• Imagine that you have been provided a library with methods tocreate (xv6_thread_create()) and block(xv6_thread_join()) threads
• xv6 already provides you with a spinlock with methods toacquire and release that lock.
• In addition, it also has methods to:1 Put a thread to sleep (sleep(void *chan, structspinlock *lk)) on a variable (*chan) (it releases lk beforesleeping and then reacquires it after waking up)
2 Wake up (wakeup(void *chan)) threads sleeping on avariable (*chan)
Today’s Task (2)
• Design a thread-safe library (in pseudo code) for xv6 that usesthese existing primitives, to implement semaphores (binary andcounting) and conditional variables and their various methods
• Also, add commenting to describe why your solution isthread-safe
Reading
• Chapter 4 from “xv6: a simple, Unix-like teaching operatingsystem”
• Timothy L. Harris. 2001. A Pragmatic Implementation ofNon-blocking Linked-Lists. In Proceedings of the 15thInternational Conference on Distributed Computing (DISC ’01),Jennifer L. Welch (Ed.). Springer-Verlag, London, UK, 300-314.Online: http://www.timharris.co.uk/papers/2001-disc.pdf