c on trol in p rin tin g s ystem s: mo du lar r ec on gu...
Post on 07-Feb-2018
215 Views
Preview:
TRANSCRIPT
Control in Printing Systems: Modular Reconfigurable Media Paths
Haitham Hindi and Lara S. Crawford
Palo Alto Research Center (PARC)
Palo Alto, California, USA
{hhindi,lcrawford}@parc.com
Abstract— The remarkable drop in the cost of embeddedcomputing, sensing, and actuation is creating a correspondingexplosion in the use of these technologies in new systems. Inorder to reduce complexity and contain costs, manufacturersare moving toward modularizing their systems and buildingreconfigurable products from simpler but smarter components.Of particular interest have recently been highly reconfigurablesystems, i.e., systems that can be customized, repaired, andupgraded at a fine level of granularity throughout their lifetime.
High reconfigurability puts new demands on the dynamiccalibration, control, and coordination software in the system.There is much promise in existing software approaches, butcurrent techniques face a number of new challenges beforethey can be embedded in the kind of real-time, distributed, anddynamic environment found in highly reconfigurable systems.In this tutorial paper, we will discuss challenges, solutions, andlessons learned in the context of a long-term project at PARCto bring such techniques to a highly reconfigurable paper pathsystem.
I. INTRODUCTION
This paper presents several aspects of the control of a
high-speed, highly modular distributed printing system. Such
a system presents several challenges from a control point of
view. The high-speed sheet transport and distributed embed-
ded nature of the controller necessitate control algorithms
that are efficient in terms of tracking performance, processing
time, and communication overhead. The highly distributed
nature of the system also introduces a coordination challenge,
as several actuators are acting on a sheet of paper at any given
time. This coordination must be achieved under conditions
of finite bandwidth and network delays.
First we will describe the physical system and control
architecture for the tightly coupled, distributed system. We
will then present an overview of the control and coordination
algorithms developed for this system. Next we will describe
the software architecture and the interactions among the
software components. Finally, we will show some results
from the real system. This paper draws heavily from [16],
[10], but also conttains previously unpublished material.
II. HIGH-SPEED PARALLEL PRINTING TESTBED
The highly modular printing system is shown in Figs. 1
and 2. It contains four black and white 55-page-per-minute
print engines that do the printing, two paper feeders, and
three outputs, all linked together with custom-designed paper
path modules. There are three high-speed sheet “highways”
in the system, which run sheets at 1.25 m/s, with “off-ramps”
and “on-ramps” sending paper to and from the marking
Fig. 1. The hypermodular high-speed printer prototype.
engines (which run at a fixed slower speed, 0.257 m/s). Many
cross-connections link the highways. At full capacity, the
printing system can produce 220 sheets per minute and can
have up to 15 sheets being processed simultaneously.
There is an independently controlled roller actuator (called
a “nip”) module between each section of bidirectional paper
path. The nips are driven with stepper motors. There is one
infrared sheet-edge-detection sensor on either side of each
nip (not shown in the diagram). Each director element (three-
way paper path module) also has its own controller, to control
the solenoid flipper actuators that direct the paper on different
paths. Each of the “straight-through” paper path modules also
has a processor, though it is not currently used for control
purposes. Finally, the feeders and print engines each have a
processor as well. There are over 150 independent processors
in the system, along with 90 nip motors and 180 sensors.
The modules communicate via multiple CAN buses. Multiple
buses were required for reasons of loading, and also proved
useful for bandwidth reasons. The prototype is designed to
be highly modular and reconfigurable, in both hardware and
software.
III. CONTROL ARCHITECTURE
In our printing system, the low-level control (module level)
is distributed for reasons of modularity and flexibility, but
maybe just as importantly, sample rate and bandwidth/sensor
2009 American Control ConferenceHyatt Regency Riverfront, St. Louis, MO, USAJune 10-12, 2009
WeA02.1
978-1-4244-4524-0/09/$25.00 ©2009 AACC 38
Fig. 2. Diagram of the hypermodular printer prototype. Green lines indicatepaper path. Gray boxes indicate paper path module boundaries. Blue doublecircles represent nip rollers. The thick red line shows a typical sheet path.
rate considerations. A tight level of coordination is required
among these distributed actuators, however. As a sheet pro-
gresses through the machine, a different group of actuators
will be in contact with it at different points in time. These
actuators form a dynamic control group, and must all act
precisely in synchrony to avoid damage to the sheet and
potential paper jams. For example, a nominal sheet speed
might be 1 m/s. To support a variety of sheet sizes, the
spacing between nips might be on the order of 0.1 m. An
8.5 x 11 inch sheet of paper moving through the system
long edge first would be in contact with an individual nip
for approximately 0.2 s. Given two nips spaced 0.1 m apart,
it would be in contact with those two nips simultaneously for
0.2−0.1 = 0.1m of travel, or 0.1 s. In that amount of time,
a 10% velocity discrepancy between the two nips translates
into 1 cm of position differential, assuming no slippage. If
the trailing nip were the one rotating faster, this differential
would probably result in buckling the sheet, and possibly a
paper jam. If the lead nip is moving faster, the sheet risks
surface damage or simply poor control tracking.
Since the sensory feedback is event-based and sporadic
in this system, a significant potential source of velocity dis-
crepancy among nips is different controllers applying sensory
feedback at different times. If, for example, a sensor signal
were obtained at a local controller, acted on immediately, and
also passed to other controllers in the control group, those
controllers would act on the information with some delay.
The delay depends on the networking architecture and the
update rate of the controllers. During the delay, a control
deviation will only produce a small position discrepancy,
but such errors corrupt the synchronization of the controller
states, and the discrepancy can grow over time. In our sys-
tem, there are two sensors for every nip, so sensor crossings
are quite frequent. It is therefore important to ensure that the
actuators do not apply sensory feedback at different times,
and do act in complete synchrony.
These tight coordination requirements between the mod-
ules dictate the existence of some kind of coordination
entity. This entity, the sheet controller, serves as a kind of
proxy for a given sheet from the point of view of higher
u1 u2 u3 u4
MC−2 MC−3 MC−4MC−1
x1 x2 x3 x4
Mod−1 Mod−2 Mod−3 Mod−4
Planner
Sheet
Controller
sheet
<< local area network>>
<<controller area network>>
u
module1
module2
module3
t
t
module2
module3
module1
x
<module1, operation2, 3.2 seconds>
<module2, operation1, 3.7 seconds>
<module3, operation1, 4.2 seconds>
Fig. 3. Hierarchical networked control system. The thin lines showcommunication pathways. A specific example is shown in thicker lines:a sensor (small square) generates information that is passed up to the sheetcontroller, which then relays it to all modules that need the feedback. Theright part of the figure shows plan translation and distribution by the sheetcontroller. The planner sends a plan in the form of a list of tuples to the sheetcontroller, which converts it into trajectories for the modules to track. Duringtimes of overlap, the trajectories are identical. The module controllers trackthe identical trajectories, producing synchronized actuator outputs.
levels of control, and is responsible for the sheet’s safe
passage through the machine. The sheet controller monitors
the progress of the sheet and serves as an information
broker between the module controllers by relaying sensor
information to appropriate module controllers via a publish-
subscribe protocol over the network.
Based on these considerations, we arrived at the hier-
archical control architecture shown in Fig. 3. The plan-
ner/scheduler specifies a series of way points which are
desired arrival times and locations for the leading edge of
the sheet of paper. This information is then passed (over
a network) to a sheet controller which processes it and
passes trajectory commands to the relevant module con-
trollers (again over a network). There is one sheet controller
per sheet – a new controller is created when instructions
for a new sheet are received from the planner. Since the
exact arrival time of the sheet is not known in advance
(due to noise disturbances), the sensing is asynchronous. The
module controllers feed the sensor information to a control
algorithm (PTOS algorithm [11]) and then apply the resulting
control signals to the rollers (nips), which move the paper.
The module controller algorithm is shown in Fig.4, and will
be described in more detail later. The nips are actuated by
stepper motors with extremely fast dynamics compared to
the sheet system. The paper is in contact with more than
one nip at any given time. This example system is discussed
in more detail in Fromherz, et al. [12]. Further control issues
in printing systems can be found in Hamby and Gross [14],
Li, et al. [20], Chen and Chiu [7], and Krucinski, et al. [18].
The control goals of the system are to deliver tight control,
which appears to the paper as good as centralized control,
in the presence of random network delays and limited
bandwidth. The desired behavior can be seen in Figs. 5 and
6. Fig. 5 shows an example of a nominal sheet trajectory
and the times the different nips are active. The trajectory is
39
Fig. 4. Block diagram of the module controller algorithm. A time varyingKalman filter handles the irregular sensing rate; proximate time optimalcontroller PTOS handles the actuator saturation; delay equalizer handles thebounded network delay; and synchronization statemachine enables controllerhandoff for distributed implementation.
0 0.5 1 1.5 2 2.5 30
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
Nip−0
Nip−1
Nip−2
Nip−3
time (s)
positio
n (
m)
Fig. 5. An example of a nominal sheet trajectory obtained by directinterpolation of way points (asterisks). Leading and trailing edge trajectoriesare both shown. The dark dotted lines are the module boundaries. Thesquares on the y-axis are the positions of the edge sensors. The x-es andcircles on the x-axis show the nominal leading edge and trailing edge,respectively, sensor crossing times. Also shown are the nominal states ofthe FSM: dotted=Synch, dashed=Comp, solid=Drive.
given by the way points, computed by the sheet controller
and shown as asterisks, and marks the time the leading edge
of the sheet should nominally arrive at a point. The time the
trailing edge of the sheet should leave a point is also shown.
The dark bands connecting these two trajectories show the
times that different nips should be active. The sheet controller
uses these trajectories to compute the activation times for the
nips. The desired control behavior for the system can be seen
in the simulation results shown in Fig. 6. The active control
signals for four different actuators are shown as solid lines
on the four plots; the dashed trace is the same on all plots,
and is the control output that would be used in a centralized
system (the desired control output from all the distributed
controllers). The actuators activate and then deactivate in
sequence. The states of the controllers, also shown in Figs. 5
and 6, will be discussed in Section VI.
One of the key design decisions in our networked system
architecture was the practical question of how to implement
the control algorithm mentioned above in this distributed
0 0.5 1 1.5 2 2.5 3
00.10.20.3
Nip
−0
time (sec)
0 0.5 1 1.5 2 2.5 3
00.10.20.3
Nip
−1
0 0.5 1 1.5 2 2.5 3
00.10.20.3
Nip
−2
0 0.5 1 1.5 2 2.5 3
00.10.20.3
Distributed Nip Control
Nip
−3
Fig. 6. Overall nip speed control signal (dashed) along with the drivesignals (solid) superimposed for the first four nips. Along the x-axis, theline styles show the states of the FSMs (dotted=Synch, dashed=Comp,solid=Drive).
actuation and sensing environment. There are at least two
alternatives. One reasonable method might be to implement
the controller centrally in the sheet controller, and simply
communicate the control signals to the appropriate nips via
the module controllers. However, in our application, the
control updates happen much more frequently than sensor
messages, so a lot of network bandwidth could be saved by
transmitting only the sensor messages.
We have instead, therefore, chosen a more distributed
implementation, at the cost of redundancy. In this distributed
implementation, each module controller runs its own local
replica of the control algorithm, i.e., the same controller
is used in all the nips in contact with the sheet. When
properly initialized, if all these controllers receive the same
sensor information, they will apply the same control signal
to the sheet. As the sheet moves through the system, new
nips will constantly be joining and leaving the actuation
of the paper. Hence there is a need for a synchronization
method, which can be viewed as a mechanism for controller
hand-off. As we shall show, this synchronization can be
effected by embedding the control algorithm in a finite state
machine (FSM), which takes care of the synchronization (see
Fig. 14 ). Thus the sheet moves as if being controlled by a
single centralized controller running the algorithm in Fig.4.
Consequently, the sheet controller only needs to send very
infrequent messages to the module controllers consisting of
high level commands telling the nips when to switch on and
off and from handling one sheet to another and relays of the
relatively infrequent sensor information. Network bandwidth
is thus used more efficiently.
IV. DYNAMICAL CONTROLLER
In this section, we show how classical closed form time
optimal control [19], [6], [2], [11], which exploits the special
40
structure of double integrator systems (i.e. ones with point-
mass Newton’s Law dynamics) can be used to design a
hybrid controller for waypoint tracking in the highly modular
printing system, for example. While double integrator dy-
namics are very simple, they are extremely prevalent in many
domains, beyond manufacturing systems, e.g., transportation,
disk drives, robotics, and aerospace. Our approach is based
on appropriate setting (switching) of the state of a reference
generator with the same point mass dynamics in a two degree
of freedom controller topology [19], [6], [2], [28], [11]. It
can be viewed as a special case of the more sophisticated
modern methods available today, see for example [4], [8],
[3], [5]. However, the techniques we present admit very
compact implementations, suitable for use in low cost micro
controllers and DSP chips used in modular reconfigurable
embedded systems applications.
The goal of the control is to drive paper along the paper
paths shown, while meeting waypoint tracking requirements.
The waypoints for each sheet of paper, along with the routing
and scheduling, are determined by the planner/scheduler, as
discussed above.
A. PROBLEM STATEMENT
We will be concerned mainly with the double integrator
system:s = v
v = u(1)
where s and v are the position (m) and speed (m/s), respec-
tively; u is the acceleration control signal (m/s/s). If we define
the plant state xP = (s,v), then (1) can be written in state
space form as
xP(t) = AxP(t)+Bu(t), xP(t0) = xP0
sP(t) = Cxp(t)(2)
where
A =
[
0 1
0 0
]
, B =
[
0
1
]
, C =[
0 1]
.
Note that [A,B] is controllable, and the system has an
invertible time evolution matrix
Φ(t0, t f ) = eA(t f −t0) =
[
1 (t f − t0)0 1
]
.
In many modern control systems, such as the one in Fig. 2,
an abstract planner handles task ordering and scheduling, and
then passes a set of waypoints to the physical controller.
These are a set of specific times, locations, and speeds,
{(t0,s0,v0), . . . ,(tN ,sN ,vN)} for the system to achieve at
certain times. Suppose we view the way points as sam-
ples at instants {t0, . . . , tN} of a fictitious reference state
trajectory xR(t), whose values at those times are xR(t0) =(s0,v0), . . . ,xR(tN) = (sN ,vN). Then we may state the way-
point tracking problem in its most skeletal form as
minu
∑ti∈{t0,...,tN}
||xP(ti)− xR(ti)||p
s.t. xP(t) = AxP(t)+Bu(t); xP(t0) = (s(t0),v(t0))||u||∞ ≤ amax
module
amax
(t0, s0, v0) (tf, sf, vf)
sf−s0
Fig. 7. Single module waypoint tracking
This captures our primary objective, namely that of hitting
the waypoints without exceeding our motor peak acceleration
constraints.
As systems become more modular, and as the modules
themselves become more autonomous, eventually, one arrives
at the following single module waypoint tracking problem,
which will be the focus of this paper. The multiple waypoint
tracking problem above can be decomposed into a sequence
of single module problems, possibly by using tricks such as
fictitious modules, to handle nonuniform spacing, etc.
Problem Statement: [single module waypoint tracking]
Given a module with double integrator dynamics (2), find a
control u : R2 → R that will take the state from xP0 = (s0,v0)at t0 to xP f = (s f ,v f ) at t f , while not exceeding the peak
acceleration limit amax. If such a control u can be found
from a certain class of control laws, then we say that the
pair {(t0,xP0),(t f ,xP f )} is amax-feasible under that class of
control laws. See Fig. 7.
Note that the above only captures the most critical con-
straints. In practice, one would like to have all the following:
1) pass through the waypoints (or as close as possible),
2) don’t exceed acceleration limits
3) easy to check that {(t0,xP0),(t f ,xP f )} is amax-feasible
4) feedback implementation to reject disturbances
5) low memory and computation costs for DSP imple-
mentation
6) compact parametrization to fit network bandwidth lim-
its
7) minimal transit time through overall path
8) smooth machine-friendly position and velocity trajec-
tories
9) speed should always be positive (no moving back-
wards)
B. 2DOF CONTROL TOPOLOGY
We now introduce the basic idea that we use to solve
the waypoint tracking problem. Specifically, we will make
use of the two degree of freedom controller (2DOF) topol-
ogy, Fig. 8, which uses a reference generator to provide
feedforward control, and a state feedback for stabilization
and disturbance rejection. This is a classical architecture,
discussed in many standard references [19], [6], [2], [28],
[11]. The equations for the overall system are now
xR = AxR, xR(t0) = xR0
xP = AxP +Bu, xP(t0) = xP0
e = Cxp −CxR,
u = f (xP − xR).
(3)
In this section, we assume all states directly available and
we focus on the designing a full-state feedback/feedforward
41
xR0 R
Pf (·) ux
xP
xR
sP
sR
e
Fig. 8. Two degree of freedom controller topology. The controller consistsof R and f (·); takes 2 inputs: initial condition xR0 and plant state xP; andproduces the control output u = f (x).
control law; observer implementation is discussed in the next
section.
In the hypermodular printer fixture in Fig.2, each stepper
motor is controlled independently by its own DSP processor,
which runs such a 2DOF controller.
Proposition 1: Consider the problem of tracking the state
of a linear reference generator, as in (3), whose state is fully
observable, with the same dynamics A-matrix as the plant.
Let u : Rn → R be a stabilizing (possibly nonlinear) state
feedback for the plant. Let x := xP − xR. Then the control
u(x) will stabilize the tracking error x and drive it to zero
with the same closed loop dynamics as it has on the plant
by itself.
Proof : Trivially we observe that subtracting the two state
equations in (3) gives
˙x = Ax+Bu(x)
which has exactly the same dynamics as the original system
in closed loop with the stabilizing state feedback u.�
Proposition 2: Assume that Φ(t0, t f ) = eA(t f −t0) is invert-
ible and that [A,B] is controllable. Set xR(t0) = eA(t0−t f )xP f .
Then if u is a control law which takes the state x := xP −xR
to zero in finite time t∗ ∈ [t0, t f ], then u(x) solves the single
module waypoint trajectory control problem.
Proof : Since Φ is invertible, we can propagate xR back
in time from t f to t0. Now from time t∗ onwards, we
have xP(t∗) ≡ xR(t∗). By controllability, this means that u ≡0,∀t ≥ t∗. Thus xP and xR will both evolve homogeneously
according to xP(t) = xR(t) = eA(t−t∗)xR(t∗),∀t ≥ t∗. Since the
reference generator was evolving homogeneously all along,
we have xR(t∗) = eA(t∗−t0)xR(t0) = eA(t∗−t0)eA(t0−t f )xP f =eA(t∗−t f )xP f . Substituting into the expression for xP(t) and
evaluating at t = t f gives xP f . �
Thus given any finite time stabilizing control law u for the
double integrator system, we have a paradigm for solving the
single module waypoint tracking problem as follows:
1) check points are amax-feasible
2) set xR(t0) = eA(t0−t f )xP f
3) apply control law u(x) from t0 to t f
C. TIME OPTIMAL CONTROL BASED TRACKING
We will now review some classical results on time optimal
control for double integrator systems, which will supply
us with a finite time stabilizing control law, that can be
used in the propositions above to design waypoint tracking
controllers, see [19], [6], [2], [11].
The time optimal control problem is
minu
T =∫ t0+T
t0
1 dt
s.t. x(t) = Ax(t)+Bu(t); x(0) = xo
x(T ) = 0
‖u‖∞ ≤ amax
The solution to the time optimal control problem for the dou-
ble integrator system is a nonlinear state-dependent control
law
u(t) = fTOC(x(t)) (4)
= −amaxsgn(
sgn(s(t))√
2amax|s(t)|+ v(t))
where recall that the state x(t) = (s(t),v(t)).At this point one might be tempted to apply u(t) =
fTOC(x(t)) right away to the plant and reference system to
solve the problem. But first we need to be able to check
amax-feasibility of a given pair of waypoints under the control
law fTOC(x(t)). Fortunately, once again thanks to the special
structure of the double integrator system, there is a closed
form expression for the time optimal transfer time: For the
double integrator system (2), the time taken to bring the state
from x0 to the origin under the time optimal control law (4)
is given by
tTOC(x(t0))=
v(t0)+2
√
s(t0)+v(t0)2
2amax;s(t0) > −
v(t0)|v(t0)|2amax
−v(t0)+2
√
−s(t0)+v(t0)2
2amax;s(t0) < −
v(t0)|v(t0)|2amax
|v(t0)| ;s(t0) = −v(t0)|v(t0)|
2amax
(5)
Using these facts together with Prop1 and Prop2, we
can propose the following controller for the single module
waypoint tracking problem:
Proposition 3: Consider the double integrator system,
with acceleration limit amax, together with the reference
generator system as in (3). Let x := xP − xR, and suppose
we want to move from xP0 = (s0,v0) at t0 to xP f = (s f ,v f )at t f . Then by taking xR(t0) = eA(t0−t f )xP f , the control
u(x) = fTOC(x) (6)
will take the state from (t0,xP0) to (t f ,xP f ) while not
exceeding the peak acceleration limit amax, if and only if
tTOC(x(t0)) ≤ t f − t0. (7)
where x(t0) = xP(t0)− xR(t0).Remark: Note that here evaluating xR(t0) = eA(t0−t f )xP f is
merely setting
xR(t0) =
[
1 (t0 − t f )0 1
][
s f
v f
]
=
[
s f − v f (t f − t0)v f
]
.
In other words, we simply extrapolate back the desired
output trajectory, start the reference generator on that desired
42
trajectory, and then apply the control that will move the
system from its input trajectory to the desired one in minimal
time.
Unfortunately, this control approach results in excessive
switching which can be harsh on the mechanics, and can also
produce trajectories where the sheet moves in reverse, for
short periods of time, thus violating some of the secondary
design considerations above. The method we describe next
reduces the switching and eliminates the backwards motion.
�
D. INFLECTION FREE CONTROL BASED TRACKING
In this section we will describe another waypoint tracking
control method which has smoother profiles, at the expense
of some restriction on the feasible set of state transfers.
Similar ideas can be found in standard references, eg: [9].
However, we have not seen elsewhere the closed loop im-
plementation via time optimal control, presented here. The
resulting controller contains switching not only due to the
time optimal control law, but also due to deliberate switching
of the state of the reference generator. It can be viewed as
a practical example of a simple hybrid systems switching
controller.
Specifically, in some applications, it is known ahead of
time, or by design, that the overall function of a certain mod-
ule is either to slow down or to speed up. In this case, one
can design inflection free trajectories. These are trajectories
where the acceleration does not flip sign and hence are either
convex or concave. We will design controllers for a specific
subclass of inflection free trajectories.
Assumption: For the rest of this section, we will assume
that it is required to keep the controlled objects moving in
the forward direction, ie, we will not allow the velocity to
reverse.
Definition 1: A constant acceleration inflection free
(CAIF) trajectory is one which is generated by applying
a constant acceleration amax for some subinterval of time
[t ′0, t′f ] ⊆ [t0, t f ], see Fig.9.
By integrating the area under the curves, one can see that
the trajectory will have the following form:
s(t)=
s0 + v0(t − t0) ; t0 ≤ t < t ′0s0 + v0(t − t0)+ 1
2 amax(t − t ′0)2 ; t ′0 ≤ t < t ′f
s0 + v0(t′f − t0)+ 1
2 amax(t′f − t ′0)
2 + v f (t − t ′f ) ; t ′f ≤ t < t f
(8)
The next question is: what pairs of waypoints are CAIF
amax-feasible? Since (8) gives us the explicit expression for
CAIF amax-feasible points, then we can check the feasibility
of a given waypoint pair ((t0,s0,v0),(t f ,s f ,v f )) by checking
if they are consistent with (8). Specifically, if we evaluate
(8) at t = t f , we obtain the expression
s f = s0 + v0(t′f − t0)+
1
2amax(t
′f − t ′0)
2 + v f (t f − t ′f ).
Using this together with the observation (see Fig.9)
amax =v f − v0
t ′f − t ′0⇒ t ′f = t ′0 +
v f − v0
amax(9)
t t ’ t ’ t
v
v
s
s
0 0 f f
o
f
o
f
a
v
v
o
f
Fig. 9. Constant acceleration inflection free trajectory.
t0 t ’0 tf
t ’f tf
t ’f
t0 t ’0
v
v
0
f
v
v
0
f
=
=
amax
amax
Fig. 10. Extreme cases: given a fixed amax, the top figure shows the minimalpossible change in distance (smallest possible area under the velocity curvefor amax); the bottom figure shows the maximal possible distance change(maximum possible area).
we can solve for the switching times as functions of((t0,s0,v0),(t f ,s f ,v f )):
t ′0 =
(
1
v f − v0
){
v f t f − v0 t0 − (s f − s0)−1
2amax(v f − v0)
2
}
(10)
t ′f =
(
1
v f − v0
){
v f t f − v0 t0 − (s f − s0)+1
2amax(v f − v0)
2
}
.
Therefore, the waypoint pair ((t0,s0,v0),(t f ,s f ,v f )) being
feasible and consistent with (8), is equivalent to plugging it
into (10) and obtaining valid switching times which satisfy:
t0 ≤ t ′0 and t ′f ≤ t f . (11)
Equivalent consistency conditions can be obtained by in-
specting the extreme cases in Fig.10. Noting that the distance
covered is the area under those curves, one obtains the
following conditions, which turn out to be exactly equivalent
to (10) and (11):
v0∆t +δ ≤ ∆s ≤ v f ∆t −δ ;v0 ≤ v f
v f ∆t +δ ≤ ∆s ≤ v0∆t −δ ;v0 > v f(12)
43
0 0.5 1 1.5 20
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
v0 (m/s)
vf
(m/s
)
(v0, vf) Pairs Achievable with aMax = 10 m/s/s, dx = 0.1 m
dt=0.06
dt=0.08
dt=0.14
Fig. 11. As we sweep ∆t from 0 to 0.15, for each value, (12) producesa pair of “lemon” shaped regions, corresponding to the set of (v0,v f ) pairsthat are amax CAIF feasible for that choice of ∆t and the fixed ∆s and amax.Also shown for comparison in the cyan dotted lines are the regions of thecontrol law in the previous section.
where ∆v = v f −v0, ∆t = t f − t0, ∆s = s f −s0, and δ = ∆v2
2amax.
Fig.11 shows the feasible input and output speeds for a
module of fixed length and a given amax, computed using
(12), which is easy to check in a low cost processor. Pairs
that are outside the boundaries are infeasible. Furthermore,
if we ignore the waypoint timing constraint and simply ask:
what pairs ((s0,v0),(s f ,v f )) are CAIF amax-feasible for some
∆t = t f − t0, one obtains the condition:
1
2amax(v2
f − v20) ≤ (x f − x0)
which is shown as the solid boundary lines in Fig.11. This
follows from (9) and Fig.10 & Fig.9, which show that
any CAIF curve must contain at least the trapezoidal area
(distance) between t ′0 and t ′f , which is 12amax
(v2f − v2
0). Also
(9) gives us the minumum ∆t required to achieve ∆v using
amax.
Proposition 4: Given amax, the waypoint pair
((t0,s0,v0),(t f ,s f ,v f )) is CAIF amax-feasible if and
only if (11) or (12) hold. Then the double integrator system
can be made to track the CAIF trajectory using the 2DOF
architecture with the time optimal control law by setting
xR(t0) = xP(t0) = (s0,v0) and applying the control law
u(x) = fTOC(x); then at time t = t ′0 given by (10), the
reference generator state is switched to track the desired
output trajectory, xR(t ′0) = eA(t ′0−t f )x f = (s f −v f (t f − t ′0),v f ),while continuing to apply u(x) = fTOC(x) all along.
Proof : By construction, the amax CAIF curve is the result of
applying amax to the double integrator on the interval [t ′0, t′f ].
Since the TOC control will attempt to drive x to zero in
minimum time, it must also apply amax for that same duration
- it cannot do anything else to reduce the error any faster. �
Thus if we use (6) for f (·) in the 2DOF architecture of
Fig.8, with the reference generator switching as described
above, the resulting controller now satisfies all of our desired
practical criteria above, see Fig.12. Of course the satisfaction
of all of the constraints comes at a cost: the inflection free
property of the trajectories reduces the set feasible end-
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18
−0.1
−0.05
0
0.05
0.1
sR
,sP
t
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18
0
0.5
1
v
t
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18−20
−10
0
10
20
a=
u
t
Fig. 12. Tracking using the CAIF control of the waypoints((0.0,0.0,0.25),(0.18,0.10,1.0)). (Top) position plant (red) and reference(blue). (Mid) plant speed. (Bot) acceleration control.
to-end transfers, see Fig.11; the reference switching adds
(minimal) complexity to the controller; and there is a small
additional messaging bandwidth cost: once feasibility has
been established, instead of sending x at t0, we must now
send xR(t ′0) and also include t ′0. Nevertheless, we have found
these tradeoffs quite acceptable, in order to achieve the specs
of smoother trajectories and no backwards motion.
Remark: Note that in the absence of noise, the optimal
control will be zero on the initial interval [t0, t′0] since both the
reference and the plant have the same dynamics and initial
conditions, and hence the tracking state error x will be zero.
The control will be nonzero on the interval [t ′0, t′f ], during
which the tracking error will be reduced to zero. Then again
after t ′f , the control will again be zero. See Fig.12. Of course,
the benefit of having the closed loop control implementation
is that, in practice, the system will always be subject to noise
and disturbances, and thus there is the opportunity for the
control to correct for these perturbations.
Remark: Our presentation has been in continuous time;
however, in practice, we have implemented our systems in
discrete time. There are various subtleties in approximating
continuous time optimal control with discrete time-optimal
control. However there are solutions too. We refer the reader
to [11] for a discussion of these issues. We have found the
proximate time optimal servo (PTOS) described there to be
quite satisfactory in addressing the main issues in discrete
time implementation.
Remark: Note that we have derived the conditions for
switching in terms of times t ′0 and t ′f . One could just as
well compute corresponding transition conditions in terms of
44
Fig. 13. Delay equalizer removing variations in round-trip message delay,as applied to sensor messages. Messages to the control group are routedthrough the sheet controller. The delay equalizer at each module controllerholds a received message until an apply time ta specified by the sheetcontroller. ta is the sensor trip time plus a maximum round trip networkand processing delay equal to d controller time steps.
the state xP. It would be interesting to compare the hybrid
robustness properties of both approaches, along the lines of
[25].
V. DISTRIBUTED NETWORKED
IMPLEMENTATION
As described above, these module controllers commu-
nicate to the sheet controllers over a network, and must
therefore be synchronized in the presence of network delay.
We assume that there is a worst case network and processing
delay describing the time it takes for sensor or controller state
information to be passed up from the module controller to
the sheet controller and back down over the multiple parallel
CAN buses. Putting the sheet controller in the loop increases
the delay, but enables the modules to be ignorant of the
machine configuration. The delays can then be made constant
for each message type through the use of a delay equalizer
mechanism [12], [23], [22], [21]; see Fig. 13. In the modular
printing system, the state information has an equalized delay
of D control samples, D ∗ Ts, and the sensor data has an
equalized delay of d ∗ Ts, where d ≤ D due to the relative
message sizes. These delays are significant in terms of the
control sample time, and affect controller synchronization as
well as the controller design. The module controller’s delay
equalizer simply accepts new incoming messages and delays
them further until the apply time, ta, when the total delay
since their origination time ts is d or D time steps.
The synchronization problem we are solving is to deter-
mine what information to send over the network and when
to send it in order to initiate and maintain synchronization,
given the network delays. Ideally, we want to send the
minimum amount of information that is sufficient for syn-
chronization. The synchronization protocol is quite general
and does not depend on the specific tasks of the controllers.
Consider, then, a set of processes all following a recursion:
x(k +1) = f (x(k),y(k−d),k) ; x(0) = x0
u(k) = g(x(k),y(k−d),k)(13)
for any functions f and g of x, y and k. We refer to such
a set of processes, where the x(k) and u(k) are identical
for all time, as synchronized. For the control group to be
synchronized, it is sufficient that each controller:
• has a synchronized clock,
• runs an identical algorithm, as above,
• is properly initialized when joining the control group,
so that its state matches the state of the control group,
• receives identical commands, and
• receives identical sensory information.
Clock synchronization is assumed here, and is discussed in
[10] as well as voluminously elsewhere (see, for example,
[13], [27]). The sheet controller ensures that the processes
(module controllers) receive identical commands. It also
ensures that the control group receives the same sensory
information, and uses it at the same time, using a delay
equalizer (Fig. 13) as described above. This section describes
the process of synchronization initialization.
In the following we apply the general synchronization
technique from [17] to an observer-based controller. Consider
a controller with an observer-based structure, which has a
state of the form x(k1|I (k2)), the best-estimate state at
time k1 given information available up to and including time
k2. As a shorthand, we will write x(k1|I (k2)) as simply
x(k1|k2). Note that x(k1|k2) will include sensor information
generated only up to time k2 − d, due to the sensor delay.
Assume that the estimation update operation, performed
when new data y(k− d) is received at time k, acts on the
state estimate from d time steps ago:
x(k−d|k) = fu(x(k−d|k−1),y(k−d),k) (14)
This update can be a Kalman filter or some other type of
update function. In the printing system the sensors were ac-
curate enough that using the sensor position data directly was
sufficient, while a Kalman filter added a lot of computational
overhead for not much improvement. After the estimation
update, in order to bring the improved state estimate forward
to the present time the controller must propagate x(k−d|k)forward using the one-step time update function ftu:
x(k−d +1|k) = ftu(x(k−d|k),u(k−d),k−d)
... (15)
x(k|k) = ftu(x(k−1|k),u(k−1),k−1)
These operations, fu followed by d iterations of ftu, together
comprise the measurement update, fmu:
x(k|k) = fmu(x(k−d|k−1),y(k−d), (16)
u(k−d), . . . ,u(k−1),k)
Note that if no measurement y(k − d) arrives in a given
time step k (as the measurements are asynchronous), the
measurement update step can be skipped. Next the controller
must calculate u. Assume its calculation is of the form:
u(k) = gu(x(k|k),k) (17)
Finally, the controller needs to perform one further time
update to obtain the new current state:
x(k +1|k) = ftu(x(k|k),u(k),k) (18)
In the hypermodular printing system, we implemented these
calculations using history buffers for x and u.
45
Now we will consider how to synchronize controllers of
this type.
Proposition 5 (Augmented State): Given a con-
troller of the observer-based form described above,
representing its state in the augmented form:
x(k) = (x(k−d|k−1),u(k−d), . . . ,u(k−1))
yields state updates of the form of the recursion in
(13).
The proof follows from analyzing the dependencies of the
update functions in (14) – (15), and is given in more detail
in [10].
Now, given the sensor message delay of d steps and
the state message delay of D steps, we have the following
conditions required for synchronization [17]:
Proposition 6 (Synchronization): Let
{p0, . . . , pn−1} be a set of processes running
(13). A new process pn, which knows f and
g, can be synchronized with the given set
from time k′ onwards, and for any functions
f (x(k),y(k− d),k) and g(x(k),y(k− d),k), if and
only if the following conditions hold:
A. pn receives the same input measurements
{y(k−d) | k ≥ k′} and
B. k′ ≥ D and, at time k′, pn has access to
x(k′ − D) and {y(k′ − D − d), . . . ,y(k′ − 1 −d)}, for synchronization via forward propa-
gation. Forward propagation consists of per-
forming D iterations of the state recursion
(13) to obtain x(k′) from x(k′ −D) and the
input measurements {y(k′−D−d), . . . ,y(k′−1−d)}.
The proof follows from performing the forward propaga-
tion, and is given in [17].
In the modular printing system, in order to ensure property
A in Proposition 6, that the controllers receive identical
sensory input, all sensory information is routed through the
sheet controller and equalized to d time steps of delay, as
described above (Fig. 13). The sensor input is based on sheet
edge crossings, and is thus asynchronous (sensor readings do
not arrive at every time step).
Property B in Proposition 6 addresses the means for
initializing synchronization for a new controller joining a
control group. In the printing system, it is not sufficient to
synchronize all controllers at the beginning of operation, or
even perform synchronization updates at regular intervals,
as there are multiple different ephemeral control tasks that
arise, are executed, and end, and each task involves a dy-
namically evolving group of controllers with dynamic state.
Furthermore, it is not sufficient to synchronize all controllers
that will be involved in a given task at the beginning of that
task, because the embedded processors are not capable of
running the number of concurrent controllers required. Such
synchronization at the beginning of the task would also be
wasted effort should the task change in midstream, such as
due to a replanning event in response to a system exception.
In order to perform the initialization, therefore, a new
controller must dynamically obtain a copy of the state of the
control group, as described in property B in Proposition 6. As
described in this property, for an observer-based controller,
the state that must be sent is not x, but rather x; x is insuf-
ficient for initializing the new controller. It must also listen
to sensor information for an interval before synchronization
through forward propagation can occur.
In the printing system, then, to synchronize a new con-
troller i to a control group j at time k, the following steps
are needed:
1) At time k−D controller i begins listening for sensor
information y
2) At time k−D a controller in the control group sends
x j(k−D) (Proposition 5) to i
3) At time k controller i sets its own variables equal to
the received state information and performs forward
propagation to construct the current state xi(k) as well
as xi(k|k−1)4) Controller i is then synchronized and begins to perform
the regular control updates.
At time k, then, the synchronization information is re-
ceived by controller i, at which time it also has available
the sensor information y(k − D − d), . . . ,y(k − 1 − d) for
which it has been listening. Note that the history state
x j(k−D− d|k−D− 1) will have all the sensor corrections
up to y(k−D− d − 1) already applied to it, because these
will all be available at k−D.
Controller i sets its internal variables xi(k−D−d|k−D−1) and ui(k−D− d), . . . ,ui(k−D− 1) equal to the equiva-
lents sent from j. Then it can perform forward propagation
to get the current state xi(k|k−1):
xi(k−D|k−D) = fmu(xi(k−D−d|k−D−1),
y(k−D−d),
ui(k−D−d), . . . ,
ui(k−D−1),k−D)
ui(k−D) = gu(xi(k−D|k−D),k−D)
xi(k−D+1|k−D) = ftu(xi(k−D|k−D),
ui(k−D),k−D)
... (19)
xi(k−1|k−1) = fmu(xi(k−1−d|k−2),
y(k−1−d), (20)
ui(k−1−d), . . . ,
ui(k−2),k−1)
ui(k−1) = gu(xi(k−1|k−1),k−1)
xi(k|k−1) = ftu(xi(k−1|k−1),
ui(k−1),k−1)
Note that as part of the last stage (20) of the forward
46
Fig. 14. Simplified statechart of the synchronization state machine (FSM).
propagation, xi(k−d|k−1) is calculated:
xi(k−1−d|k−1) =
fu(xi(k−1−d|k−2),y(k−1−d),k−1)
xi(k−d|k−1) = (21)
ftu(xi(k−1−d|k−1),ui(k−1−d),k−1−d)
As the control history ui(k − d), . . . ,ui(k − 1) is also con-
structed during the propagation, this algorithm produces all
the elements of xi(k) = x j(k), at which point it will be
synchronized with the control group.
To implement this synchronization algorithm, the module
controller is embedded in a finite state machine (FSM; see
Fig. 14) with an explicit synchronization state. When the
module controller is turned on (at a particular time), it enters
the Synch state from the Off state. During this state, the
controller waits until the synchronization time k, when it
has the control group state and sensor information described
above. It can then perform the synchronization calculations.
The FSM its state chart implementation are explained in
more detail in [17]. Once synchronization has been achieved,
the controller can progress to the Compute state, where it
performs normal control updates every Ts. From there it
advances to the Drive state when it is time for it to take
control of the actuator. Finally, at a specified time the FSM
turns itself off. This process is diagrammed in Fig. 14. The
state activation is also shown in Figs. 5 and 6.
VI. REAL-TIME SOFTWARE ISSUES
This section describes some of the networked, real-time
software developed to deploy the control and coordination
algorithms discussed above on the highly modular printer
prototype. The distributed nature of the system can be seen
in the deployment diagram in Fig. 15. The planner/scheduler
has its own processor, as does the sheet controller factory
(an entity that generates sheet controllers). Each module has
its own processor. In general, the software is designed to
support modules containing any number of nips or sensors.
The modules and the sheet controllers are connected via a
CAN-bus (Controller Area Network), and the sheet controller
<< control area network >>
MCFactoryMCFactoryMCFactory
SCFactory
<< local area network >>
Planner
Fig. 15. Deployment diagram showing the distributed control systemimplementation.
Fig. 16. Flow of sensor data through the system.
factory (and sheet controllers) communicate with the planner
over Ethernet (TCP). As an example, the complete flow of
sensor data in the system is diagrammed in Fig. 16.
We will describe here the real-time portions of the soft-
ware: the sheet controller factory and sheet controllers, and
the module controller factory and module controllers. A
simplified class diagram representing the software structure
is shown in Fig. 18. The diagram, while not containing
all the details of the software, is designed to capture the
key elements and portray the philosophy behind the design.
The sheet controller factory’s main role is to receive new
sheet itineraries from the planner and create new sheet con-
trollers corresponding to these itineraries. It also maintains
a system model of the machine configuration for the sheet
47
controllers, and passes changes in the model (received from
the modules, for example) along to the planner. The sheet
controllers themselves are responsible for sending commands
to the modules, maintaining a model of the sheet’s progress,
and subscribing to, processing, and distributing sensor and
controller state data related to the sheet.
The module controller factory resides in the module
processor. Its main job is to create module controllers in
response to new tracking commands sent by a sheet con-
troller. Multiple module controllers can exist at the same
time, though only one will be actively engaging any given
nip – the others will be preparing for upcoming sheets, for
example. Separate module controllers can also be spawned
for different nips within a module, in the generic case.
In addition, the module controller factory is responsible
for monitoring any sensors in the module and maintaining
subscriptions to those sensors (sending sensor events to a
subscribed sheet controller). It also passes along controller
state data to subscribed sheet controllers.
The roles of the sheet controller factory and the module
controller factory are diagrammed in Fig. 17. Each new sheet
itinerary from the planner causes a new sheet controller to be
created by the sheet controller factory. The sheet controller
remains active until the sheet has left the machine. It in turn
sends appropriate command messages to the relevant module
controller factories, which instantiate module controllers to
handle the sheet while it is in contact with their local nips.
The module controllers remain alive for the duration of time
that the paper is in the module. This timing information is
supplied in the command message.
A more detailed view of the software components is
portrayed in the class diagram (Fig. 18). When a new sheet
itinerary reaches the sheet controller factory, then, a new
sheet controller is spawned through the processNewPlan()
function. At each time step, the sheet controller factory runs
a mainControlLoop(), in which it cycles through all existing
sheet controllers and allows them to update their internal
model of the sheet (sheetUpdate()), send new commands
(including sensor subscriptions) to modules that are about
to engage the sheet (generateModuleMessages()), process
incoming sensor messages from the modules (processSen-
sorMsg()), send relevant sensor information to the modules
actively engaged on the sheet (createSensorUpdates()), and
forward on controller state information received (forward-
StateMsg()). The sheet controller factory maintains configu-
ration data about the system, which includes information on
different types of modules, the modules in the system, and
their connectivity. This configuration information allows the
sheet controller to correctly segment trajectory information
for the different modules and nips.
At each control cycle, the module controller factory exe-
cutes its mainControlLoop(). Within this loop it checks for
messages from sheet controllers (processMessage()), checks
its sensors and publishes any sensor events (checkSensors()
and maintainSensorSubscriptions()), notifies any active mod-
ule controller that needs to send its state out for synchroniza-
tion purposes (sendStateMsg()), allows each active module
Fig. 19. Functional groups in the sheet controller.
controller to update its state (updateController()), and outputs
any controls to its actuators (outputControls()).
The message types used by the sheet and module con-
trollers are also shown in Fig. 18. Each message has a
message type flag and a data payload. The main classes
of messages are command messages, which also include
sensor subscriptions, delete command messages, sensor event
messages (from a module controller to a sheet controller),
state announce messages (from a module controller to a sheet
controller), sensor update messages (from a sheet controller
to a module controller), state update messages (from a sheet
controller to a module controller), and configuration change
messages.
The sheet controller and module controller can be seen
to have parallel functionality, although they may be imple-
mented somewhat differently in practice: each has a sensor
and state data monitoring and processing functionality and
an observation and control functionality. The sheet controller
actions, for example, can be aggregated into two functional
groupings, as shown in Fig. 19. The sheet controller can
be viewed as having a Sensor/State Manager component
and a Sheet Observer component. The Sensor/State Manager
component receives sensor event messages from the modules,
processes them, and sends out sensor update messages to the
current coordinated control group. It also forwards control
group state data that it receives to new modules needing
to synchronize with the group. The Sheet Observer tracks
the progress of the sheet and, when a new module is being
brought into the control group, sends appropriate command
and sensor subscription messages.
The module controller processes can also be aggregated
into functional groups, depicted in Fig. 20. The Sensor/State
Manager is responsible for monitoring the module’s sensors,
detecting whether there has been a change (edge crossing),
and reporting any sensor events to sheet controllers that have
subscribed to the relevant sensor. It is also responsible for
sending a copy of an active controller’s state data to the sheet
controller when required for synchronization. The Delay
Equalizer is responsible for accepting incoming sensor and
state update messages from the sheet controller and delaying
them until the appropriate apply times. It then supplies the
sensor or state data to the relevant Nip Control element. The
third module controller functionality is, of course, running
the control algorithm and its synchronization finite state
machine. It is responsible for maintaining the correct control
state and calculating the control signals.
48
new
scFactory
sc1
sc2
planner mcFactory1 mcFactory2
mc12
mc21
mc22
mc11
new
new
cmdMsg
cmdMsg
sh2 itin
sh1 itin
new
new
newcmdMsg
cmdMsg
Fig. 17. Sequence diagram showing the instantiation of sheet controllers and module controllers by their respective factories.
Fig. 20. Functional groups in the module controller.
Statecharts (see, for example, Harel [15], Selic et al.
[26], and Samek [24]) are a useful technique for visualizing
dynamic and concurrent aspects of real-time software. In
Fig. 21, we have attempted to capture the different concurrent
tasks running on the sheet controller and module controller
along with their event-driven, message-based interaction.
Note that in our implementation these are not actually
separate threads, but are conceptually separate processes that
are activated in different ways (time-triggered or event-based,
for example).
A sheet controller starts off in the “Running” state, which
contains two orthogonal regions (two concurrent threads or
conceptual processes), separated by the dashed line. The
“SCSensor/StateMgr” process is triggered by the “sensEv-
entMsg” or “stateAnnounceMsg,” which are messages gen-
erated by the sensor/state manager in the module controller.
The “SCSensor/StateMgr” process processes a sensor mes-
sage by applying a transformation to the sensor data, gener-
ating an appropriate apply time, and then relaying the sensor
data to the nips that are currently acting on the sheet by gen-
erating a sheet controller sensor message “sensUpdateMsg,”
which is sent to the module controller over the network. This
is indicated by the “ˆ” notation in the figure. Upon receipt
of a state announcement message, the sensor/state manager
simply forwards the state information to the relevant module
controller via a “stateUpdateMsg.” The “SheetObserver”
process runs concurrently with “SCSensor/StateMgr.” It is
triggered synchronously, every control time sample, as shown
by the self-transition labeled by the event “Ts.” At each time
step, its role is to maintain an updated estimate of the sheet
position, via “sheetUpdate(),” to generate an “ERROR” if a
sensor measurement is too deviant from the expected sheet
position, and to wake up the appropriate module controllers
when the sheet position approaches their location by gener-
ating the command message “cmdMsg” event, which is sent
to the module controllers over the network. The command
messages include various data and parameters related to the
itinerary of the particular sheet being handled, which the
module controller uses for computing the control signals that
are sent to the nips. Command messages can also be sensor
subscription messages.
A module controller starts in the “OFF” state, and transi-
tions to the “ON” state upon receipt of a trajectory-specifying
“cmdMsg.” The “ON” state contains three orthogonal regions
which run simultaneously. The “MCSensor/StateManager”
process synchronously polls the sensors every control time
step and generates the event “sensEventMsg” when an edge
is detected. This message contains data which includes the
time of detection (time-stamp) and sensor identification,
and is sent to the sheet controller over the network. The
“MCSensor/StateMgr” also sends a “stateAnnounceMsg”
when required for control group synchronization purposes.
The “Delay Equalizer” process is triggered by the arrival
of a new sheet controller sensor or state update message.
49
Fig. 18. Simplified class diagram for the high-end printing system control. SC = sheet controller, MC = module controller.
It stores the message in a list. It then prepares messages
from the list whose apply time matches the current time
and identifies them as ready, thereby making it appear to
the “NipControl” process that all arriving messages have a
fixed delay (though the two different message types may have
different fixed delays). The “NipControl” process is triggered
synchronously every control time step “Ts”. It dispatches any
ready sensor or state messages to the synchronization state
machine and the PTOS control algorithm which computes
the control signals. Once the sheet is about to arrive at that
nip, the control is actually applied to the nips.
VII. RESULTS AND CONCLUSIONS
A. Experimental Results
The controller synchronization method described here has
been successfully implemented in the hypermodular high-
speed printer prototype. In the results shown here, the
controller sample time was 2ms and d and D were 5 and
6 controller time steps, respectively. The controller time step
Ts was 2ms. The maximum delay values were determined
through a combination of calculation and empirical testing
Fig. 21. Statechart showing the signal interactions between a sheetcontroller and a module controller.
50
65 66 67 68 69 70 710
0.5
1
1.5
Time (s)
Contr
ol S
ignals
(m
/s)
Fig. 22. Control signals in the printer prototype for a single sheet. Differentnips are represented in a rotating set of four colors; as they are synchronized,the plots are overlaid. Time values are from local controller time stamps.The print engine has constant-speed internal nips that are not part of thecontrol group; the empty space on the plot corresponds to where the sheetis entirely inside the print engine.
65.1 65.12 65.14 65.16 65.18 65.2 65.22 65.24 65.26 65.28 65.31.2
1.22
1.24
1.26
1.28
1.3
1.32
1.34
1.36
1.38
1.4
Time (s)
Co
ntr
ol S
ign
als
(m
/s)
Fig. 23. Closeup of the control signals. x-es are recorded control values.As the red and blue nips join the control group, their actions are exactlysynchronized.
of delay during peak network load. There were 5 CAN
buses used for control, and several additional buses used for
debugging and for some of the clock synchronization.
A sample trace of the controls generated when sending a
single sheet through the system is shown in Fig. 22, with a
closeup in Fig. 23. The path of the sheet is the one shown in
Fig. 2. Control signals for the 18 different nips on the path
are plotted, but the controllers are successfully synchronized
so the traces cannot be distinguished. The control values are
the commanded velocities for the nips. The control trace
follows a sheet as it comes out of the feeder at a nominal
highway speed of 1.25m/s, slows down to enter the print
engine, speeds up again after exiting the print engine, and
then continues out of the machine at the highway speed. The
bumps in the control signals correspond to the apply times
for the data obtained from sheets crossing sensor locations.
The reference tracking performance in the system is shown
in Fig. 24, with a closeup in Fig. 25. The tracking is quite
good, with errors within a few millimeters even in challeng-
ing points for tracking (during the speed-ups and slow-downs
on the on- and off-ramps), where the PTOS inflection-free
tracking comes into play. The successful synchronization of
the system can also be seen in the indistinguishable reference
tracks for the different module controllers. The tracking
performance of the system with respect to the high-level
planner timepoints is shown for a set of twenty sheets in
Fig. 26.
64 65 66 67 68 69 70 710
0.5
1
1.5
2
2.5
3
3.5
Time (s)
Dis
tan
ce
(m
)
Sensor data
Controller reference
Fig. 24. Reference tracking in the module controllers for a single sheet.Different nips are represented in a rotating set of four colors; as they aresynchronized, the reference plots are overlaid. Time values are from localcontroller time stamps.
65 65.1 65.2 65.3 65.4 65.5 65.6 65.7 65.80.2
0.3
0.4
0.5
0.6
0.7
0.8
Time (s)D
ista
nce
(m
)
Sensor data
Controller reference
Fig. 25. Closeup of the reference tracking, showing the reference reset.
B. Conclusion
In this tutorial paper, we have discussed challenges, solu-
tions, and lessons learned from the design of a hierarchical
distributed networked control approach for a highly reconfig-
urable printing system. We have described: the hardware plat-
form, along with the physical parameters and performance
requirements; the system dymanics along with a hierachical
control approach, including planning, coordination and time-
optimal based feedback/feedforward control; synchronization
and handoff mechanisms for distributed deployment; realtime
embedded software architecture; and real data collected from
the system that was built at PARC.
Perhaps some of the most important general lessons that
we have learned from this experience are quite widely
applicable. The first is due to Einstien: “Everything should
be made as simple as possible - but not simpler”. This is
particularly true when it comes to system integration, where
any unnecessary design complexity generally incurs far more
effort and delays in system integration than expected. Using
a more complex design should be explored only when the
simpler alternatives have been proven to deliver insufficient
performance. The second lesson is: “Start with feasibility;
strive for optimality”. Again, we have found it extremely
helpful to get a working prototype with full functionality up
and running as soon as possible, even if the performance is
suboptimal. The point here is not to encourage “hacking”,
or discourage rigorous analytical thinking. Rather, the point
is that, when designing a new system for the first time,
51
48 49 50 51 52 53 54 55 56 57 58 590
0.5
1
1.5
2
2.5
3
Time (s)
Dis
tan
ce
(m
)
Sensor data
Planner timepoints
Fig. 26. Tracking performance with respect to planner time points, fortwenty sheets. Line style indicates which print engine a sheet used: blue/red= top/bottom, solid/dashed = left/right.
until the system is up and running, the relative importance
of different design problems is sometimes very difficult to
assess. The real system can help to put these problems in
proper perspective. New simulation and verification tools are
making things easier all the time, but in the end, it seems
that some iteration is always necessary to optimize the final
product.
In closing, we hope that this material, along with that
presented in the supporting three sessions, will demonstrate
that modern printing systems are some of the most sophisti-
cated devices in use today, which require a truly remarkable
range of engineering, and in particular, control techniques,
to enable them to deliver the level of performance that we
enjoy today.
VIII. ACKNOWLEDGMENTS
We would like to thank all of our colleagues at PARC and
at Xerox for their collaboration on this project.
REFERENCES
[1] K.J. Astrom and B. Wittenmark. Computer Controlled Systems.Prentice Hall, 1997.
[2] M. Athans and P. Falb. Optimal Control. McGraw Hill, 1966.[3] A. Bemporad, F. Borrelli, and M. Morari. Model predictive control
based on linear programming - the explicit solution. IEEE Trans. Aut.
Contr., 2002.[4] J. Bobrow, S. Dubowsky, and J. Gibson. Time-optimal control of
robotic manipulators along specified paths. Int. J. Robotic Research,1985.
[5] F. Borrelli. Constrained Optimal Control of Linear and Hybrid
Systems. Springer, 2004.[6] A.E. Bryson and Y.-C. Ho. Applied Optimal Control. Academic Press,
1975.[7] C.-L. Chen and G. Chiu. Incorporating human visual model and spatial
sampling in banding artifact reduction. In Proceedings of the American
Control Conference, Boston, MA, June 2004.[8] H. Choset, K. Lynch, S. Hutchinson, G. Kantor, W. Burgard,
L. Kavraki, and S. Thrun. Principles of Robot Motion. MIT Press,2005.
[9] J. Craig. Introduction to Robotics: Mechanics and Control. PrenticeHall, 2004.
[10] L. S. Crawford, H. Hindi, R. Zhou, and Dan Larner. Synchronizedcontrol in a large-scale networked distributed printing system. In Pro-
ceedings of the International Conference on Robotics and Automation,2009.
[11] G. Franklin, D. Powell, and M. Workman. Digital Control of Dynamic
Systems. Addison Wesley, 1998.
[12] M. P. J. Fromherz, L. S. Crawford, and H. A. Hindi. Coordinatedcontrol for highly reconfigurable systems. In Hybrid Systems: Com-
putation and Control (HSCC), Zurich, Switzerland, 2005. Springer-Verlag.
[13] M. Gergeleit and H. Streich. Implementing a distributed high-resolution real-time clock using the CAN-bus. In 1st International
CAN-Conference 94,, Mainz, Germany, September 1994.[14] E. Hamby and E. Gross. A control-oriented survey of xerographic
systems: basic concepts to new frontiers. In Proceedings of the
American Control Conference, Boston, MA, June 2004.[15] D. Harel. Statecharts: a visual formalism for complex systems. Science
of Computer Programming, 8, 1987.[16] H. Hindi, L. Crawford, R. Zhou, and C. Eldershaw. Efficient waypoint
tracking hybrid controllers for double integrators using classical timeoptimal control. In Proceedings of the IEEE Conference on Decision
and Control, December 2008.[17] H. A. Hindi, L. S. Crawford, and M. P. J. Fromherz. Synchronization
of state based control processes with delayed and asynchronousmeasurements. In IEEE Conf. Decision and Control, December 2005.
[18] M. Krucinski, C. Cloet, R. Horowitz, and M. Tomizuka. A mecha-tronics approach to copier paperpath control. In Proceedings of the
first IFAC conference on mechatronics systems, Darmstadt, Germany,September 2000.
[19] F. Lewis and V. Syrmos. Optimal Control. Wiley, 1995.[20] P. Li, T. Sim, and D. Lee. Time sequential sampling and reconstruction
of tone and color reproduction functions for xerographic printing. InProceedings of the American Control Conference, Boston, MA, June2004.
[21] B. Lincoln and B. Bernhardsson. Optimal control over networks withlong random delays. In Proceedings of the International Symposium
on Mathematical Theory of Networks and Systems, 2000.[22] R. Luck and A. Ray. An observer-based compensator for distributed
delays. Automatica, 26(5):903 – 908, May 1990.[23] J. Nilsson. Real-time control systems with delays, 1998.[24] M. Samek. Practical statecharts in C/C++. CMP Books, 2002.[25] R. Sanfelice, R. Goebel, and A. Teel. Invariance principles for hybrid
systems with connections to detectability and asymptotic stability.IEEE Trans. Aut. Contr., 2007.
[26] B. Selic, G. Gullekson, and P. Ward. Real-Time Object Oriented
Modeling. Wiley, 1994.[27] A. S. Tanenbaum and M. van Steen. Distributed Systems: Principles
and Paradigms. Prentice Hall, New Jersey, 2002.[28] W.M. Wonham. Linear Multivariable Control: A geometric Approach.
Springer-Verlag, 1985.
52
top related