1

Chapter 4 Threads

2

4.1 Overview

• This is what a thread has of its own:– A thread id– A program counter– A register set– A stack

• This is what it shares with other threads belonging to the same process:– A code section– A data section– Other resources, such as open files

3

• A traditional process with only one thread of control may be known as a heavyweight process

• In a system that supports multi-threaded execution of common code, the threads may be known as lightweight processes.

4

• Motivation– Multi-threading is a way of avoiding the overhead

of creating full new processes– It is a way of allowing multi-tasking within a single

process

5

• Examples– A word processor might support concurrent text

entry and spell checking by having each run as a separate thread

– A Web server may receive many requests for service—all essentially the same. Rather than creating a separate process for each, it may spawn threads of the same request handling code

– O/S kernels may also be multi-threaded. Solaris does interrupt handling with threads

6

• Benefits– By definition, threads allow resource sharing– Threads decrease the overhead of process creation

—the creation of threads is less computationally demanding

– Threads introduce a form of concurrency, promoting efficient use of resources and system responsiveness

– Threads may be used in multi-processing where separate threads run on each processor, rather than separate processes

7

4.2 Multi-threading Models

• User and kernel threads• Threads can be supported at the user level in the layered

diagram of system software• Threads can also be supported directly in the kernel of the

O/S• In either case, context switching between threads is

necessary. This less expensive than switching between processes

• Whether implemented at the user or kernel level, there has to be a mapping between user level threads and kernel level threads or processes

8

The Many-to-One Model

• Many user level threads share (are concurrently executed by) a single kernel level thread

• Characteristics of this model:– Efficiency: Thread management is handled in user space– Blocking: Any one blocking call by a user thread will

cause all of the threads to block– Parallel processing: This model is not applicable in this

environment. There is only one underlying kernel thread• Solaris Green threads and GNU portable threads

implement this model

9

The One-to-One Model

• Each user level thread is mapped to a single kernel level thread

• Characteristics of this model:– Efficiency: Each user thread has the overhead of kernel

thread creation. Most systems put an upper bound on the number of threads allowed

– Blocking: One thread can’t block the others– Parallel processing: This model fits the requirements

• Linux and Windows 95/98/NT/2000/XP implement this model

10

The Many-to-Many Model

• Multiple user level threads map to a set, possibly smaller, of kernel threads

• The system interactively mps user to kernel threads when scheduling

• This is more flexible than the other models, but also more complex

11

• Characteristics of this model:– Efficiency: The system can allocated as many kernel threads as is

practical or optimal– Blocking: There is no blocking– Parallel processing: This supports parallelizing a number of

processors up to the number of kernel threads• The term “two level model” describes a system with m-to-n

mapping plus the ability to map one user thread to a single kernel thread

• The two level model is the most general model• IRIX, HP-UX, and Tru64 Unix implement the two level model

12

4.3 Thread Libraries

• A thread library provides an API for creating and using threads

• There are two main non-virtual machine thread libraries– The POSIX (Unix) Pthread library supports either user or kernel

threads (the details of the subsection on this will not be covered)– The Win32 threads library supports kernel threads (the details of

the subsection on this will not be covered)• Linux also supports the thread concept, although it uses

different terminology• Java threads are implemented on top of whatever O/S

thread environment the JVM is installed on

13

4.4 Java Threads

• Threads are fundamental to the Java model• A single program runs as a single thread if it

doesn’t explicitly create threads• Syntax exists to create separate threads from a

program• The idea is that if the machine is virtual, why not

support virtual processes?• This allows the use of multi-programming/multi-

tasking when writing code at the user level

14

• Java threads don’t fit the user level vs. kernel level thread API library distinction very well

• In Java, threads are supported in the Java language API• They are actually implemented in the JVM, which relies

on the underlying system• One of the biggest challenges to the use of Java threads

is understanding that their behavior and scheduling depends in part on the behavior and scheduling of threads as defined in the underlying system

15

Java Thread Creation

• This section is mostly about syntax• The syntax itself is important if you want to

use Java threads• In the long run it’s also helpful because it

makes it possible to write an example program• The vague discussion of what threads are may

become clear if you can understand what an actual threaded program is and how it works

16

First Approach to Writing a Threaded Application

• Write a class that extends the Thread class in the Java API• Override the run() method in that class• It is the run() method which contains the program logic which

is to be threaded• In a program, construct an instance of that class• Call the start() method on that object• The start() method allocates memory and initializes a new

thread in the JVM• It then calls the object’s run() method• The user program shouldn’t call the run() method directly

because the initialization and allocation steps would be missed

17

• This first approach works fine in simple cases• Observe that since a class can only extend one other class, if

your class extends the Thread class, it can’t be a subclass of any other class

• For example, an applet is created by extending the JApplet class. This approach won’t allow you to make a threaded applet

• This suggests that there has to be an interface based solution to the problem—That will be the second approach

• There is a conceptual explanation for why the simple approach is not the most general approach: Theoretically, it is not necessarily the best idea to be extending a class when the subclass you are creating isn’t a “kind of” the superclass

18

The Second Approach to Writing a Threaded Application

• Write a class that implements the Runnable interface• That interface specifies a run() method and nothing more• Implement a run() method in your class• You can create a thread based on your class using this

mechanism:• Construct an instance of your class• Construct an instance of the Thread class, passing it the

instance of your class as the construction parameter• Call the start() method on this instance of the Thread

class

19

• For what it’s worth, note that in the Java API, the Thread class implements the Runnable interface

• The Thread class has a run() method, and therefore meets the requirements for being Runnable

• When using the second approach, clearly, a special constructor in the Thread class is being used—one that takes a runnable object as a parameter, and “wraps” it into a thread

• There are no special requirements for the constructors of the runnable class. A default constructor may work; constructors that take parameters may be needed. It’s application dependent

20

An Example Threaded Program

• The book’s example program is based on the idea of summing the first n integers

• It will be covered in this way– By explaining the general idea behind the

implementation– By pointing out some of the syntactical details– And then by looking at the code overall

21

• Recall the previous example on producers and consumers• The authors wanted to illustrate a concept using Java code• However, in order to do so, it would have been necessary

to use syntax that hadn’t been explained yet• This example suffers from the same shortsightedness• In this case the full code will be examined and all of the

necessary syntax will also be explained, even though it’s extraneous to the fundamental concept which the example is supposed to illustrate

22

• The overall point of the program is to find the sum of the integers less than or equal to some given upper limit

• The overall structure of the program is a driver, which does input and output, and a run() method in another class which does the summing

• The driver and the run() method run as separate threads

23

• Because the run() method in the Runnable interface specification is void, it is not possible for it to return a computed value

• This can be overcome by passing a reference to an object where the result of the computation is stored in that object

• The authors choose to name the class which holds the result of the computation MutableInteger

• This name indicates that it would not be possible to pass an instance of the system supplied Integer class, since objects of that class are immutable

24

• Although, in theory, a principal advantage of threading is that the threads run concurrently, this introduces the potential for synchronization problems

• These problems have not been addressed yet, so the authors avoid them in this way: They use syntax which requires that after starting the summation thread, the thread of the driver has to wait for it to complete

25

• From the point of view of clarity of the example, this has two disadvantages:

• It’s necessary to introduce the syntax for making one thread wait on another

• Conceptually, it results in a threaded program whose behavior could more easily have been accomplished by non-threaded code

• In any case, the example does result in code which is threaded and does not have lurking synchronization issues

26

• The name of the method that causes one thread to depend on another is join()

• If the main() method constructs and starts a thread, a call to join() on that thread will cause the main() method to depend on it

• The call to join() has to occur in a try block because it can throw an exception

27

• The authors introduce one more thread concept which isn’t directly relevant to the example

• There are daemon threads and non-daemon threads

• For all practical purposes, you can consider user threads to be non-daemon threads

• There is no need to worry about daemon threads or the syntax for making a thread a daemon thread

28

• class MutableInteger• {• private int value;

• public int get() {• return value;• }

• public void set(int sum) {• this.value = sum;• }• }

29

• class Summation implements Runnable• {• private int upper;• private MutableInteger sumValue;

• public Summation(int upper, MutableInteger sumValue) {• if (upper < 0)• throw new IllegalArgumentException();

• this.upper = upper;• this.sumValue = sumValue;• }

• public void run() {• int sum = 0;

• for (int i = 0; i <= upper; i++)• sum += i;

• sumValue.set(sum);• }• }

30

• public class Driver• {• public static void main(String[] args) {• if (args.length != 1) {• System.err.println("Usage Driver <integer>");• System.exit(0);• }

• MutableInteger sumObject = new MutableInteger();• int upper = Integer.parseInt(args[0]);•

• Thread worker = new Thread(new Summation(upper, sumObject));

• worker.start();• try {• worker.join();• } catch (InterruptedException ie) { }• System.out.println("The value of " + upper + " is " +

sumObject.get());• }• }

31

Java Thread States

• Note the parallel with processes– Threads are like processes at the user level– Processes have states– Threads also have a life cycle that can be

described with states

32

• Java states– New: Results from construction call to new()– Runnable:• Calling start() allocates memory for a thread object• When run() is called, a thread enters the runnable state• Java doesn’t distinguish between runnable and running.

A running thread is in the runnable state. Other threads may be in the runnable state but not currently running

33

– Blocked:• This happens if a thread issues a command that causes

blocking• The classic example is I/O• There are also thread methods that explicitly cause

blocking, such as a call to sleep()

34

– Dead: A thread enters the dead state when execution reaches the end of the run() method

35

36

• Note that it may be difficult or impossible for the programmer/user to determine the relationship between a Java thread and the native environment thread or process that it is running on.

• There are Java API methods that make it possible to check the status of a Java thread:– isAlive(): returns true if the thread has been started and

hasn’t reached the dead state– getState(): returns “state”. This is not the same as the

simple states in the diagram. More explanation will come later

37

• The JVM and the host O/S• The Java specification does not say how Java threads are to be

mapped to system threads• This is up to whoever does the implementation for Java for a

given environment (note that in theory there could be more than one)

• Windows XP, for example, does one-to-one• Unix type systems may do many-to-one or many-to-many• There are no thread scheduling requirements (other than

correctness) in the Java specifications. Threaded Java applications can vary in their behavior in different environments

38

• A multi-threaded solution to the producer-consumer problem

• This takes the message passing example of the last chapter one step closer to reality

• It uses this code, already given:• The Channel interface• The MessageQueue class. Remember that this

contained an instance of a vector and implemented an unbounded buffer

39

• The example program overall contains four other classes

• It makes use of messages that contain dates that are instances of the Java API Date class

• The four classes and their contents are outlined below

40

• Factory– Create the mailbox– Create the producer thread, passing it a reference

to the mailbox– Create the consumer thread, passing it a reference

to the mailbox– Start both threads

41

• Producer– Run in a loop– Sleep a while– Create a message– Put it in the mailbox– Print a message saying so

42

• Consumer– Run in a loop– Sleep a while– Retrieve a message or null if there isn’t one (this is

non-blocking)– Print a message saying so

43

• The SleepUtilities class– Sets a given sleeping time– Calls the Thread class sleep method– Note that try/catch blocks are needed for various

calls. They are brought together here

44

• Final notes on the new code• This example is more nearly complete than

given in the last chapter• However, it is still not entirely complete. Once

two threads share (a reference to) an object, there is a concurrency control or synchronization issue

• This code does not deal with that issue explicitly. The issue will be discussed in detail later

45

• public interface Channel• {• /**• * Send a message to the channel.• * It is possible that this method may or may not

block.• */• public abstract void send(Object message);• • /**• * Receive a message from the channel• * It is possible that this method may or may not

block.• */• public abstract Object receive();• }

46

• import java.util.Vector;• • public class MessageQueue implements Channel• {• private Vector queue;

• public MessageQueue() {• queue = new Vector();• }• • /*• * This implements a non-blocking send• */• public void send(Object item) {• queue.addElement(item);• }• • /*• * This implements a non-blocking receive• */• • public Object receive() {• if (queue.size() == 0)• return null;• else • return queue.remove(0);• }• }

47

• public class Factory • { • public Factory()• {• // first create the message buffer • Channel mailBox = new MessageQueue();• • // now create the producer and consumer threads• Thread producerThread = new Thread(new Producer(mailBox));• Thread consumerThread = new Thread(new Consumer(mailBox));• • producerThread.start();• consumerThread.start(); • }

• public static void main(String args[]) { • Factory server = new Factory();• }• }

48

• import java.util.*;

• class Consumer implements Runnable• {• public Consumer(Channel m) { • mbox = m;• }• • public void run() {• Date message;• • while (true)• {• SleepUtilities.nap();

• // consume an item from the buffer• System.out.println("Consumer wants to consume.");• message = (Date)mbox.receive();• if (message != null)• System.out.println("Consumer consumed " + message);• }• }• • private Channel mbox;• }

49

• /**• * Utilities for causing a thread to sleep.• * Note, we should be handling interrupted exceptions• * but choose not to do so for code clarity.• */

• public class SleepUtilities• {• /**• * Nap between zero and NAP_TIME seconds.• */• public static void nap() {• nap(NAP_TIME);• }

• /**• * Nap between zero and duration seconds.• */• public static void nap(int duration) {• int sleeptime = (int) (duration * Math.random() );• try { Thread.sleep(sleeptime*1000); }• catch (InterruptedException e) {}• }

• private static final int NAP_TIME = 5;• }

50

4.5 Threading Issues

• fork() and exec()—What happens when you have threads on top of processes?

• Q: In a multi-threaded program, if one thread calls exec(), should the whole process, all of the threads, be replaced, or just the calling thread?

• A: exec() typically replaces the whole process

51

• In a multi-threaded program, if one thread calls fork(), should all of the threads be duplicate, or just the calling thread?

52

• Q: In a multi-threaded program, if one thread calls fork(), should all of the threads be duplicate, or just the calling thread?

• A: The answer to this question depends on whether or not you plan on using exec() in the forked code

• It’s more expensive to duplicate all threads rather than just one thread

• Accordingly, some Unix systems have two versions of fork()

53

• If you’re writing code where you’re going to call exec() after fork(), use the fork() that just duplicates the one thread because they’re all going to get wiped out anyway.

• If you don’t call exec(), use the fork() that duplicates all of the threads. If you’re not replacing the running process, you want duplicates of all of its components

54

• Thread cancellation• This term refers to making a call that will

terminate a running thread before it has naturally come to the end of run()

• You have one piece of code, or thread, which contains a reference to another thread, the target thread

• The one piece of code can cancel the target thread by making a call on it

55

• Two models of cancellation• Asynchronous cancellation: The one thread

immediately terminates the target• Deferred cancellation:• The one thread sets a parameter or makes a call

signaling that the target should be cancelled• The target is coded to periodically check its status

and to terminate itself if it has been signaled to cancel

56

• Asynchronous cancellation can lead to concurrency problems

• If the target shares resources with another thread or if it holds resources allocated by the system, asynchronous/immediate cancellation may:– Leave the resources in an inconsistent (undesirable) state– Make it impossible for the system to reclaim the thread’s

resources• In Java, the stop() call implements asynchronous

cancellation. It is deprecated due to these synchronization problems

57

• The logic of deferred cancellation:• The target thread checks to see if it has been

signaled to terminate• If so, it runs housekeeping code which leaves

resources in a consistent state• Then it exits• In Java, in the canceling thread, deferred

cancellation is triggered by this call: targetThread.interrupt();

58

• In the target code there is a choice between two different calls:

• me.interrupted()• This returns true or false and clears the

interrupted status• me.isInterrupted()• This returns true or false and doesn’t clear the

interrupted status

59

• Code for an Interrupter class and an InterruptibleThread class follow

• The code for the InterruptibleThread class has been modified slightly from the book’s example

• Note that the InterruptibleThread class implements Runnable rather than extending Thread. This means you have to call the static method Thread.currentThread() in order to get a reference to yourself

60

• public class InterruptibleThread implements Runnable• {• /**• * This thread will continue to run as long• * as it is not interrupted.• */• private boolean keepOnRunning = true;• public void run()• {• while (keepOnRunning)• {• /**• * do some work for awhile• */

• if (Thread.currentThread().isInterrupted())• {• System.out.println("I'm interrupted!");• keepOnRunning = false;• }• }• // clean up and terminate (housekeeping)• }• }

61

• public class Interrupter• {• public static void main(String[] args)• {• Thread worker = new Thread (new InterruptibleThread());• worker.start(); • // now wait 3 seconds before interrupting it• try• { • Thread.sleep(3000);• }• catch (InterruptedException ie)• {• }• worker.interrupt();• }• }

62

• Another logical puzzle typical of operating systems• A blocked thread can’t run, so it can’t check its

status• A thread blocked for I/O, for example, won’t check

its status until the I/O finishes• If it is desirable to interrupt blocked processes in

Java, suitable methods can be found in the API under java.nio

• The standard methods in java.io won’t work

63

• Signal handling• Like the discussion of fork() and exec(), this

section is Unix related• In Java, you get to understand threading at the

user level• In Unix, it is possible to work directly with

processes and system threads• It is useful to consider the interactions that

could occur in that environment

64

• In Unix, the occurrence of events is signaled like a software interrupt

• When a signal is received, it has to be handled• The question is, what happens when a multi-

threaded process is signaled? Which of the threads is signaled?

65

• There are two kinds of signals, synchronous and asynchronous– Synchronous: If a running process is the immediate

cause of the signal generation, the signal is sent to that process. Examples are division by 0 and illegal memory access

– Asynchronous: These are signals generated by events external to the running process. Examples are the expiration of a timer or pressing CTRL+C

• The four examples of signals would cause termination. This doesn’t have to be the case

66

• Any event may be handled by one of two possible types of handlers– A default signal handler—system code run by the

kernel– A user-defined signal handler may be provided

• In a single-threaded process, signal delivery is clear: The single is delivered to the one process in question

67

• In a multi-threaded environment, four choices exist:• 1. Deliver the signal to the one thread to which it

applies (see the concrete example below)• 2. Deliver the signal to every thread in the process

(see the concrete example below)• 3. Deliver the signal to certain threads in the process• 4. Deliver the thread to a thread which has been

designated to receive all of the signals for the process

68

• 1. For synchronous threads, it makes sense to deliver the signal to the thread which caused the signal to be generated. If one thread of many divided by 0, that threat should be signaled

• 2. CTRL+C in Unix is interpreted to mean “stop the process.” Thus, in a multi-threaded application, this should generate a signal to all threads to stop

69

• The general plan of action in Unix:• The Unix command kill() is used to send signals. Think of kill()

as meaning interrupt, not terminate• Individual threads can be set to block or not block signals of

different kinds• Signals are sent to/received by the first available thread that is

not blocking that kind of signal• Sending a signal to one (appropriate) thread is sufficient since

signals only have to be handled once• Windows doesn’t use signals, but it has a facility that can

accomplish the same thing. This fact and the details of the facility are not important

70

• Thread pools• If a system is trying to run too many

processes, performance can suffer• Process creation can be expensive• One of the motivations for threads was to save

on the expense of full-scale process creation

71

• The same arguments apply to threads• Running too many threads can affect

performance• In the long run, the expense of thread creation

is also pure overhead cost

72

• The overall problem is one of continuously creating things, destroying them, and creating new ones again

• A thread pool is a solution to this problem• The idea is to create a collection of threads in

advance• When a task arrives in the system which can be

handled by a thread, one from the pool is assigned to that task

73

• If no free thread is available, the task has to wait• When the task is finished, the thread is returned to the

pool, not destroyed• This is both a reasonably simple and reasonably clever

idea for managing system resources• It would take some effort to come up with a good

algorithm to match the thread pool size to the workload on a system at any given time

• The book gives a bunch of details on thread pools. None of those details are of importance at this time.

74

• Thread specific data• The default condition for threads is that they share

data. This is one of the reasons they are advantageous. This means that by default, simple threads don’t have data of their own

• It may be desirable for threads to keep track of data items of their own. An example might be assigning a thread the id of a task it’s been assigned to do (like when giving a thread in a pool a task)

75

• It is possible to give threads their own data items using known syntax

• For example, the thread class could be extended with an instance variable added

• Or a class that implements the Runnable interface could have an instance variable and get() and set() methods, and the run() method might affect the value of the variable

76

• The previous solutions don’t work in the thread pool scenario, where the user doesn’t create the thread that does the task

• Java has a ThreadLocal class that can be used to declare data items that will be distinguished by the system according to which thread is running them/which thread they belong to

• There is no reason to pursue the details of this. The book gives an example, but it doesn’t show the solution to the pool problem. It illustrates the new syntax in order to solve a problem more easily solved using the techniques mentioned earlier

77

• Scheduler activations• This is a discussion of issues involved in the

many-to-many model of mapping user threads to kernel threads

• Here is a layered diagram– User thread– Lightweight process– Kernel threads– Hardware

78

• The lightweight process is what the threading system presents to the user level thread library

• It can be thought of like a virtual processor that a user level thread can be scheduled on

• Communication between the kernel and user level thread library is done by means of upcalls and upcall handlers

79

• An upcall is like a signal (or interrupt) to the thread library• An upcall handler is like an interrupt handler. In this case it can

be thought of scheduler code• The kernel sends an upcall when a lightweight process has

become available for a thread to be scheduled on• At the beginning of a system run, many lightweight processes

may be free• After a system has begun running, the classic cause of an upcall

is that a thread which has already be scheduled on a lightweight process has issued a blocking call. At that point the thread enters a waiting state and the lightweight process becomes available for scheduling

80

4.6 Operating System Examples

• Skip

81

The End