1

ASCENT: An Algorithmic Technique forDesigning Hardware and Software in Tandem

Jules White, Brian Doughtery, and Douglas C. SchmidtVanderbilt University, EECS Department, Nashville, TN, USA

Email:{jules, briand, schmidt}@dre.vanderbilt.edu

✦

Abstract —Search-based software engineering is an emergingparadigm that uses automated search algorithms to help designersiteratively find solutions to complicated design problems. For example,when designing a climate monitoring satellite, designers may wantto use the minimal amount of computing hardware to reduce weightand cost, while supporting the image processing algorithms runningonboard. A key problem in these situations is that the hardwareand software design are locked in a tightly-coupled cost-constrainedproducer/consumer relationship that makes it hard to find a goodhardware/software design configuration. Search-based softwareengineering can be used to apply algorithmic techniques to automatethe search for hardware/software designs that maximize the imageprocessing accuracy while respecting cost constraints.

This paper provides the following contributions to research on search-based software engineering: (1) we show how a cost-constrainedproducer/consumer problem can be modeled as a set of two multidi-mensional multiple-choice knapsack problems (MMKPs), (2) we presenta polynomial-time search-based software engineering technique, calledthe Allocation-baSed Configuration Exploration Technique (ASCENT),for finding near optimal hardware/software co-design solutions, and (3)we present empirical results showing that ASCENT’s solutions averageover 95% of the optimal solution’s value.

1 INTRODUCTION

Current trends and challenges. Increasing levels of program-ming abstraction, middleware, and other software advance-ments have expanded the scale and complexity of softwaresystems that we can develop. At the same time, the ballooningscale and complexity have created a problem where systemsare becoming so large that their design and developmentcan no longer be optimized manually. Current large-scalesystems can contain an exponential number of potential designconfigurations and vast numbers of constraints ranging fromsecurity to performance requirements. Systems of this scaleand complexity—coupled with the increasing importance ofnon-functional characteristics [8] (such as end-to-end responsetime)—are making software design processes increasinglyexpensive [22].

Search-based software engineering [17, 16] is an emergingdiscipline that aims to decrease the cost of optimizing systemdesign by using algorithmic search techniques, such as geneticalgorithms or simulated annealing, to automate the designsearch. In this paradigm, rather than performing the searchmanually, designers iteratively produce a design by using a

search technique to find designs that optimize a specific systemquality while adhering to design constraints. Each time anew design is produced, designers can use the knowledgethey have gleaned from the new design solution to craftmore precise constraints to guide the next design search.Search-based software engineering has been applied to thedesign of a number of software engineering aspects, rangingfrom generating test data [27] to project management andstaffing [5, 3] to software security [9].

Open Problem. A common theme in domains wheresearch-based software engineering is applied is that the designsolution space is so large and tightly constrained that the timerequired to find an optimal solution grows at an exponentialrate with the problem size. These vast and constrained solu-tions spaces make it hard for designers to derive good solutionsmanually. One domain with solution spaces that exhibit thesechallenging characteristics is hardware/software co-design.

Hardware/software co-design is a process whereby a sys-tem’s hardware and software are designed at the same-time in order to produce optimizations that would not bepossible in either hardware or software alone. Traditionally,hardware/software co-design has focused on determining howto partition application functionality between hardware andsoftware. For example, developers can take a time-criticalimage processing step in an application and determine whetherto implement it in hardware or software. Typically, thereare limited resources available to implement functionalityin hardware and thus determining which pieces of func-tionality to implement in hardware versus software becomesextremely challenging. A number of search-based engineeringtechniques, ranging from particle swarm optimization [1, 29]to genetic algorithms [34, 28, 13] have been used to helpautomate this process.

This paper examines another type of hardware/software co-design problem that is common in the domain of distributedreal-time and embedded (DRE) systems. The problem wefocus on is the need to choose a set of hardware and soft-ware configuration options that maximizes a specific systemcapability subject to constraints on cost and the productionand consumption of resources, such as RAM, by the hardwareand software, respectively. This problem assumes that thepartitioning of functionality between hardware and software

2

is fixed but that there are different configuration options inthe hardware and software that developers must set.

A configuration option is a setting that can be changed inthe hardware or software design, such as the image resolutionused in an image processing algorithm. Selecting a higherresolution will yield better image processing results but willin turn require more CPU time. Similarly, the hardware hasmultiple configuration options, such as the clock speed of theprocessor used. The goal of engineers is to find a series ofsettings for these configuration options that maximize a utilityfunction that describes the quality of the system.

For example, when designing a satellite for monitoringearth’s magnetosphere [11], the goal may be to maximizethe accuracy of the sensor data processing algorithms onthe satellite without exceeding the development budget andhardware resources. Ideally, to maximize the capabilitiesof thesystem for a given cost, system software and hardware shouldbe configured in tandem to produce a design with a precise fitbetween hardware capabilities and software resource demands.The more precise the fit, the less budget is expended on excesshardware resource capacity.

A key problem in these design scenarios is that theycreate a complex cost-constrained producer/consumer probleminvolving the software and hardware design. The hardwaredesign determines the resources, such as processing powerand memory, that are available to the software. Likewise, thehardware consumes a portion of the project budget and thusreduces resources remaining for the software (assuming a fixedbudget). The software also consumes a portion of the budgetand the resources produced by the hardware configuration. Theperceived value of system comes from the attributes of thesoftware design,e.g., image processing accuracy in the satelliteexample. The intricate dependencies between the hardware andsoftware’s production and consumption of resources, cost,andvalue makes the design solution space so large and complexthat finding an optimal and valid design configuration is hard.

Solution approach → Automated Solution Space Explo-ration. This paper presents a heuristic search-based softwareengineering technique, called theAllocation-baSed Configu-ration Exploration Technique(ASCENT), for solving cost-constrained hardware/software producer/consumer co-designproblems. ASCENT models these co-design problems as twoseparate knapsack problems [21]. Since knapsack problems areNP-Hard [10], ASCENT uses heuristics to reduce the solutionspace size and iteratively search for near optimal designs byadjusting the budget allocations to software and hardware.Inaddition to outputting the best design found, ASCENT alsogenerates a data set representing the trends it discovered inthe solution space.

A key attribute of the ASCENT technique is that, in theprocess of solving, it generates a large number of optimaldesign configurations that present a wide view of the trendsand patterns in a system’s design solution space. This papershows how this wide view of trends in the solution spacecan be used to iteratively search for near optimal co-designsolutions. Moreover, our empirical results show that ASCENTproduces co-design configurations that average over 95%optimal for problems with more than 7 points of variability

in each of the hardware and software design spaces.Paper organization. The remainder of this paper is or-

ganized as follows: Section 2 presents a motivating exampleof a satellite hardware/software co-design problem; Section 3discusses the challenges of solving software/-hardware co-design problems in the context of this motivating exam-ple; Section 4 describes the ASCENT heuristic algorithm;Section 5 analyzes empirical results from experiments weperformed with ASCENT; Section 6 compares ASCENT withrelated work; and Section 7 presents concluding remarks andlessons learned from our work with ASCENT.

2 MOTIVATING EXAMPLE

This section presents a satellite design example to motivate theneed to expand search-based software engineering techniquesto encompass cost-constrained hardware/software producer/-consumer co-design problems. Designing satellites, such asthe satellite for NASA’s Magnetospheric Multiscale (MMS)mission [11], requires carefully balancing hardware/softwaredesign subject to tight budgets. Figure 1 shows a satellite witha number of possible variations in software and hardwaredesign. For example, the software design has a point of

Fig. 1. Software/Hardware Design Variability in a Satellite

variability where a designer can select the resolution of theimages that are processed. Processing higher resolution imagesimproves the accuracy but requires more RAM and CPUcycles.

Another point of variability in the software design is theimage processing algorithms that can be used to identifycharacteristics of the images captured by the satellite’s cam-eras. The algorithms each provide a distinct level of accuracy,while also consuming different quantities of RAM and CPUcycles. The underlying hardware has a number of points ofvariability that can be used to increase or decrease the RAMand CPU power to support the resource demands of different

3

image processing configurations. Each configuration option,such as the chosen algorithm or RAM value, has a costassociated with it that subtracts from the overall budget. Akey design question for the satellite is:what set of hardwareand software choices will fit a given budget and maximizethe image processing accuracy. For example, will an imageprocessing algorithm that consumes more memory but requiresless CPU resources or an algorithm that requires buying a moreexpensive processor and less RAM better fit the budget?

Many similar design problems involving the allocationof resources subject to a series of design constraints havebeen modeled asMultidimensional Multiple-Choice KnapsackProblems(MMKPs) [20, 23, 2]. A standard knapsack prob-lem [21] is defined by a set of items with varying sizes andvalues. The goal is to find the set of items that fits into afixed sized knapsack and that simultaneously maximizes thevalue of the items in the knapsack. An MMKP problem is avariation on a standard knapsack problem where the items aredivided into sets and at most one item from each set may beplaced into the knapsack.

Figure 2 shows an example MMKP problem where twosets contain items of different sizes and values. At most

Fig. 2. An Example MMKP Problem

one of the items A,B, and C can be put into the knapsack.Likewise, only one of the items D, E, and F can be putinto the knapsack. The goal is to find the combination oftwo items, where one item is chosen from each set, that fitsinto the knapsack and maximizes the overall value. A numberof resource related problems have been modeled as MMKPproblems where the sets are the points of variability in thedesign, the items are the options for each point of variability,and the knapsack/item sizes are the resources consumed bydifferent design options [23, 7, 2].

The software and hardware design problems are hard tosolve individually. Each design problem consists of a numberof design variability points that can be implemented by ex-actly one design option, such as a specific image processingalgorithm. Each design option has an associated resource con-sumption, such as cost, and value associated with it. Moreover,the design options cannot be arbitrarily chosen because thereis a limited amount of each resource available to consume.

It is apparent that the description of the software designproblem directly parallels the definition of an MMKP problem.An MMKP set can be created for each point of variability(e.g., Image Resolution and Algorithm). Each set can thenbe populated with the options for its corresponding point of

variability (e.g., High, Medium, Low for Image Resolution).The items each have a size (cost) associated with them andthere is a limited size knapsack (budget) that the items canfit into. Clearly, just selecting the optimal set of softwarefeatures subject to a maximum budget is an instance of theNP-Hard [10] MMKP problem.

For the overall satellite design problem, we must contendwith not one but two individual knapsack problems. Oneproblem models the software design and the second problemmodels the hardware design. The first of the two MMKPproblems for the satellite design is its software MMKPproblem. The hardware design options are modeled in aseparate MMKP problem with each set containing the potentialhardware options. An example mapping of the software andhardware design problems to MMKP problems is shown inFigure 1.

We call this combined two problem MMKP model aMMKPco-design problem. With this MMKP co-design model of thesatellite, a design is reached by choosing one item from eachset (e.g., an Image Resolution, Algorithm, RAM value, andCPU) for each problem. The correctness of the design canbe validated by ensuring that exactly one item is chosenfrom each set and that the items fit into their respectivesoftware and hardware knapsacks. This definition, however,is still not sufficient to model the cost-constrained hardware/-software producer/consumer co-design problem since we havenot accounted for the constraint on the total size of the twoknapsacks or the production and consumption of resources byhardware and software.

A correct solution must also uphold the constraint that theitems chosen for the solution to the software MMKP problemdo not consume more resources, such as RAM, than areproduced by the items selected for the solution to the hardwareMMKP problem. Moreover, the cost of the entire selection ofitems must be less than the total development budget. We knowthat solving the individual MMKP problems for the optimalhardware and software design is NP-Hard but we must alsodetermine how hard solving the combined co-design problemis.

In this simple satellite example, there are 192 possiblesatellite configurations to consider, allowing for exhaustivesearch to be used. For real industrial scale examples, however,there are a significantly larger number of possibilities whichmakes it infeasible to use an exhaustive search technique. Forexample, a system with design choices that can be modeledusing 64 MMKP sets, each with 2 items, will have264

possible configurations. For systems of this scale, manualsolving methods are clearly not feasible, which motivates theneed for a search-based software engineering technique.

2.1 MMKP Co-design ComplexityBelow, we show that MMKP co-design problems are NP-Hardand in need of a search-based software engineering technique.We are not aware of any approximation techniques for solvingMMKP co-design problems in polynomial time. This lack ofapproximation algorithms—coupled with the poor scalabilityof exact solving techniques—hinders DRE system designers’sabilities to optimize software and hardware in tandem.

4

To show that MMKP co-design problems are NP-Hard, wemust build a formal definition of them. We can define anMMKP co-design problem,CoP , as an 8-tuple:

CoP =< Pr, Co, S1, S2, S, R, Uc(x, k), Up(x, k) >

where:• Pr is the producer MMKP problem (e.g., the hardware

choices).• Co is the consumer MMKP problem (e.g., the software

choices).• S1 is the size of the producer,Pr, knapsack.• S2 is the size of the consumer,Co, knapsack.• R is the set of resource types (e.g., RAM, CPU, etc.)

that can be produced and consumed byPr and Co,respectively.

• S is the total allowed combined size of the two knapsacksfor Pr andCo (e.g., total budget).

• Uc(x, k) is a function which calculates the amount ofthe resourcek ⊂ R consumed by an itemx ⊂ Co (e.g.,RAM consumed).

• Up(x, k) is a function which calculates the amount of thethe resourcek ⊂ R produced by an itemx ⊂ Pr (e.g.,RAM provided).

Let a solution to the MMKP co-design problem be definedas a 2-tuple,< p, c >, wherep ⊂ Pr is the set of itemschosen from the producer MMKP problem andc ⊂ Co isthe set of items chosen from the consumer MMKP problem.A visualization of a solution tuple is shown in Figure 3. We

Fig. 3. Structure of an MMKP Co-design Problem

define the value of the solution as the sum of the values ofthe elements in the consumer solution:

V =

j∑

0

valueof(cj)

wherej is the total number of items inc, cj is the jth itemin c, andvalueof() is a function that returns the value of anitem in the consumer soution.

We require thatp and c are valid solutions toPr andCo,respectively. Forp and c to be valid, exactly one item fromeach set inPr and Co must have been chosen. Moreover,the items must fit into the knapsacks forPr and Co. This

constraint corresponds to Rule (2) in Figure 3 that eachsolution must fit into the budget for its respective knapsack.

The MMKP co-design problem adds two additional con-straints on the solutionsp andc. First, we require that the itemsin c do not consume more of any resource than is producedby the items inp:

(∀k ⊂ R),

j∑

0

Uc(cj , k) ≤l∑

0

Up(pl, k)

wherej is the total number of items inc, cj is thejth item inc, l is the total number of items inp, andpj is the jth itemin p. Visually, this means that the consumer solution can fitinto the producer solution’s resources as shown in Rule (1) inFigure 3.

The second constraint onc andp is an interesting twist ontraditional MMKP problems. For a MMKP co-design problem,we do not know the exact sizes,S1, S2, of each knapsack. Partof the problem is determining the sizes as well as the items foreach knapsack. Since we are bound by a total overall budget,we must ensure that the sizes of the knapsacks do not exceedthis budget:

S1 + S2 ≤ S

This constraint on the overall budget corresponds to Rule (3)in Figure 3.

To demonstrate that solving for an exact answer to theMMKP problem is NP-Hard, we will show that we can reduceany instance of the NP-completeknapsack decision problemto an instance of the MMKP co-design problem. The knapsackdecision problem asks if there is a combination of itemswith value at leastV that can fit into the knapsack withoutexceeding a cost constraint.

A knapsack problem can easily be converted to a MMKPproblem as described by Akbar et al. [2]. For each item, a set iscreated containing the item and the∅ item. The∅ item has novalue and does not take up any space. Using this approach, aknapsack decision problem,Kdp, can be converted to a MMKPdecision problem,Mdp, where we ask if there is a selectionof items from the sets that has value at leastV .

To reduce the decision problem to an MMKP co-designproblem, we can use the MMKP decision problem as theconsumer knapsack (Co = Mdp), set the producer knapsack toan MMKP problem with a single item with zero weight andvalue (∅), and let our set of produced and consumed resources,R, be empty,R = ∅. Next, we can let the total knapsacksize budget be the size of the decision problem’s knapsack,S = sizeof(Mdp).

The co-design solution, which is the maximization of theconsumer knapsack solution value, will also be the optimalanswer for the decision problem,Mdp. We have thus setup theco-design problem so that it is solving for a maximal answerto Mdp without any additional producer/consumer constraintsor knapsack size considerations. Since any instance of theNP-complete knapsack decision problem can be reduced toan MMKP co-design problem, the MMKP co-design problemmust be NP-Hard.

5

3 CHALLENGES OF MMKP CO-DESIGNPROBLEMS

This section describes the two main challenges to buildingan approximation algorithm to solve MMKP co-design prob-lems. We discuss the challenges that make it infeasible todirectly apply existing MMKP algorithms to MMKP co-designproblems. The first challenge is that determining how to setthe budget allocations of the software and hardware is notstraightforward since it involves figuring out the precise sizeof the software and hardware knapsacks where the hardwareknapsack produces sufficient resources to support the optimalsoftware knapsack solution (which itself is unknown). Thesecond challenge is that the tight-coupling between producerand consumer MMKP problems makes them hard to solveindividually, thus motivating the need for a heuristic to de-couple them.

3.1 Challenge 1: Undefined Producer/ConsumerKnapsack Sizes

One challenge of the MMKP co-design problem is that theindividual knapsack size budget for each of the MMKPproblems is not predetermined,i.e., we do not know how muchof the budget should be allocated to software versus hardware,as shown in Figure 4. The only constraint is that the sum of the

Fig. 4. Undefined Knapsack Sizes

budgets must be less than or equal to an overall total budget.Every pair of budget values for hardware and software resultsin two new unique MMKP problems. Even minor transfersof capital from one problem budget to the other can thereforecompletely alter the solution of the problem, resulting in anewmaximum value. Existing MMKP techniques assume that theexact desired size of the knapsack is known.

There is currently no information to aid designers in deter-mining the allocation of the budgets. As a result, many design-ers may choose the allocation arbitrarily without realizing theprofound impact it may have. For example, a budget allocationof 75% software and 25% software may result in a solutionthat, while valid, provides far less value and costs considerablymore than a solution with a budget allocation of 74% and 26%percent.

There is, however, useful information in the solution spacethat can be determined by solving instances of the problem

with unique sequential divisions of the total budget. Typically,designers would choose a budget with little information onthe ramifications of a potential budget choice. By samplingthe solution space at a number of distinct budget allocations,the algorithm can show designers the best solution that it canproduce at each budget allocation. The information produced,may not show the actual best budget allocations to choose,but should help designers to make better budget allocationchoices than blindly choosing a budget allocation with noinformation at all. A key challenge is figuring out how tosample the solution space and present the results to designers.In Section 4.4 we discuss ASCENT’s solution to this problemand in Section 5 we present empirical data showing howASCENT allows designers to sample design spaces for anumber of MMKP co-design problems.

3.2 Challenge 2: Tight-coupling Between theProducer/Consumer

Another key issue to contend with is how to rank the solutionsto the producer MMKP problem. Per the definition of anMMKP co-design problem from Section 2.1, the producersolution does not directly impart any value to the overallsolution. The producer’s benefit to a solution is its abilityto make a good consumer solution viable. MMKP solversmust have a way of ranking solutions and items. The problem,however, is that the value of a producer solution or item cannotbe calculated in isolation.

A consumer solution must already exist to calculate thevalue of a particular producer solution. For example, whetheror not 1,024 kilobytes of memory are beneficial to the overallsolution can only be ascertained by seeing if 1,024 kilobytes ofmemory are needed by the consumer solution. If the consumersolution does not need this much memory, then the memoryproduced by the item is not helpful. If the consumer solutionisRAM starved, the item is desperately needed. A visualizationof the problem is shown in Figure 5.

Fig. 5. Producer/Consumer MMKP Tight-coupling

The inability to rank producer solutions in isolation ofconsumer solutions is problematic because it creates a chickenand the egg problem. A valid consumer solution cannot bechosen if we do not know what resources are availablefor it to consume. At the same time, we cannot rank thevalue of producer solutions without a consumer solution asa context. This tight-coupling between the producer/consumeris a challenging problem. We discuss the heuristic ASCENTuses to solve this problem in Section 4.3.

6

4 THE ASCENT A LGORITHM

This section presents our polynomial-time approximation algo-rithm, called theAllocation-baSed Configuration ExploratioNTechnique(ASCENT), for solving MMKP co-design prob-lems. The pseudo-code for the ASCENT algorithm is shownin Figure 6 and explained throughout this section.

4.1 ASCENT Algorithm Overview

A MMKP co-design problem,CoP , as defined as an 8-tuple:

CoP =< Pr, Co, S1, S2, S, R, Uc(x, k), Up(x, k) >

The ASCENT algorithm solves for a series of potentialsolutions toCoP using an iterative heuristic algorithm. Theinput to the algorithm is the problem definitionCoP and astep size increment,D, which is discussed in Section 4.2. TheASCENT algorithm then proceeds as shown in Figure 6

4.2 Producer/Consumer Knapsack Sizing

The first issue to contend with when solving an MMKPco-design problem is Challenge 2 from Section 3.1, whichinvolves determining how to allocate sizes to the individualknapsacks. ASCENT addresses this problem by dividing theoverall knapsack size budget into increments of sizeD. Thesize increment is a parameter provided by the user. ASCENTthen iteratively increases the consumer’s budget allocation(knapsack size) from 0% of the total budget to 100% of thetotal budget in steps of sizeD. The incremental expansion ofthe producer’s budget can be seen in thefor loop in step 1of Figure 6 and the setting of values for ofS1, S2 in step 2.

For example, if there is a total size budget of 100 andincrements of size 10, ASCENT firsts assign 0 to the consumerand 100 to the producer, 10 and 90, 80 and 20, and so forthuntil 100% of the budget is assigned to the consumer. Theallocation process is shown in Figure 7. ASCENT includesboth the 0%,100% and 100%,0% budget allocations to handlecases where the optimal configuration includes producer orconsumer items with zero cost.

In some systems, such as enterprise application servers, asystem’s CPU load may become saturated. In these instances,understanding how the system degrades in an over utilizedscenario is important. Since we are focused on hard real-timeand embedded systems, we do not handle scenarios wherethe CPU is overloaded. Instead, we assume that the resourcesof the system are fixed and that 100% utilization cannot beexceeded.

4.3 Ranking Producer Solutions

At each allocation iteration, ASCENT has a fixed set of sizesfor the two knapsacks. In each iteration, ASCENT must solvethe coupling problem presented in Section 3.2, which is: howdo we rank producer solutions without a consumer solution.After the coupling is loosened, ASCENT can solve for a highlyvalued solution that fits the given knapsack size restrictions.

To break the tight-coupling between producer and consumerordering, ASCENT employs a special heuristic. Once the

Inputs:

CoP = < Pr, Co, S1, S2, S, R, Uc(x, k), Up(x, k) >D = stepsize

Algorithm:1) For int i = 0 to ⌊S/D⌋, set S1 = i ∗ D and S2 =

S − S1

2) For each set of values forS1 andS2:

2.1) Solve for a solution,tc, to Co, givenS2

2.2) Calculate a resource consumption heuristicV r(k) for the resource inr ∈ R:

V r(r) =

∑|tc|j=0 Uc(tcj , k)

∑|R|j=0

∑|tc|k=0 Uc(tcj , k)

2.3) Solve for a solution,p, to Pr that maximizes thesum of the values of the items selected for theknapsack,

∑|p|k=0 V alue(pk), where the value of

the kth item is calculated as:

V alue(pk) =

|R|∑

j=0

V r(rj) ∗ Up(pk, rj)

2.4) For each resourcesrj ∈ R, calculate the amountof that resource,P (r), produced by the items inp:

P (r) = Up(p0, rj)+Up(p1, rj) . . . Up(p|p|−1, rj)

2.5) Create a new multidimensional knapsack prob-lem,Cmo, from Co, such that the maximum sizeof each dimension of the new knapsack is definedby the vector:

Sm2 = (S2, r0, r1, . . . r|R|−1)

2.6) Solve for a solution,c, to Cmo and add a solutiontuple < p, c > to the list of candidate solutions,lc, for CoP

3) Sort the potential solutions,lc, of CoP and outputboth the highest valued solution and the list of otherpotential solutions

Fig. 6. The ASCENT Algorithm

Fig. 7. Iteratively Allocating Budget to the ConsumerKnapsack

7

knapsack size allocations are fixed, ASCENT solves for amaximal consumer solution that only considers the currentsize constraint of its knapsack and not produced/consumedresources. This process is shown in step 2.1 of Figure 6.

The process of solving the consumer knapsackCo, in Step2.1, uses an arbitrary MMKP approximation algorithm to finda solution that only considers the consumer’s budget. Thisapproach is similar to asking “what would the best possiblesolution look like if there were unlimited produced/consumedresources.” Once ASCENT has this idealized consumer solu-tion, it calculates a heuristic for assigning a value to producersolutions.

ASCENT leverages a commonly used type of metric fromprior work on MMKPs to assign values for ranking potentialsolutions [2, 26, 30]. The heuristic that ASCENT uses to as-sign value to producer items is:how valuable are the resourcesof a producer item to the idealized consumer solution. Thisheuristic is calculated as a set of values for theVr variables inStep 2.2 of Figure 6. We calculate the value of a resource as theamount of the resource consumed by the idealized consumersolution divided by the sum of the total resources consumedby the solution. In Step 2.3 of Figure 6, the resource ratios(Vr values) are known and each item in the producer MMKPproblem is assigned a value by multiplying each of its providedresource values by the corresponding ratio and summing theresults.

4.4 Solving the Individual MMKP Problems

Once sizes have been set for each knapsack and the valuationheuristic has been applied to the producer MMKP problem,existing MMKP solving approaches can be applied. First, theproducer MMKP problem, with its new item values, is solvedfor an optimal solution, as shown in Step 2.3 of Figure 6.In Step 2.5, a new consumer MMKP problem is created withconstraints reflecting the maximum available amount of eachresource produced by the solution from the producer MMKPproblem. The consumer MMKP problem is then solved for ansolution in Step 2.6. The producer and consumer solutions arethen combined into the 2-tuple,< p, c > and saved.

In each iteration, ASCENT assigns sizes to the producerand consumer knapsacks and the solving process is repeated.A collection of the 2-tuple solutions is compiled duringthe process. The output of ASCENT, returned in Step 3 ofFigure 6, is both the 2-tuple with the greatest value and thecollection of 2-tuples.

The reason that the 2-tuples are saved and returned as partof the output is that they provide valuable information on thetrends in the solution space of the co-design problem. Each 2-tuple contains a high-valued solution to the co-design problemat a particular ratio of knapsack sizes. This data can be usedtograph and visualize how the overall solution value changes as afunction of the ratio of knapsack sizes. As shown in Section 5,this information can be used to ascertain a number of usefulsolution space characteristics, such as determining how muchit costs to increase the value of a specific system property toa given level or finding the design with the highest value perunit of cost.

4.5 Algorithmic Complexity

The overall algorithmic complexity of ASCENT can be brokendown as follows:

1) there areT iterations of ASCENT2) in each iteration there are 3 invocations to an MMKP

approximation algorithm3) in each iteration, values of at mostn producer items

must be updated.

This breakdown yields an algorithmic complexity of O(T (n+MMKP )), where MMKP is the algorithmic complexity ofthe chosen MMKP algorithm. With M-HEU (one of the mostaccurate MMKP approximation algorithms [2]) the algorith-mic complexity is O(mn2(l − 1)2), wherem is the numberof resource types,n is the number of sets, andl is maximumitems per set. Our experiments in Section 5 usedT = 100 andfound that it provided excellent results. With our experimentalsetup that used M-HEU, the overall algorithmic complexitywas therefore O(100(mn2(l − 1)2 + n)).

5 ANALYSIS OF EMPIRICAL RESULTS

This section presents empirical data we obtained from exper-iments using ASCENT to solve MMKP co-design problems.The empirical results demonstrate that ASCENT producessolutions that are often near the maximum value that canbe achieved while not exceeding resource constraints. Theresults also show that ASCENT can not only provide nearoptimal designs for the co-design problems, such as thesatellite example, but also scale to the large problem sizesofproduction DRE systems. Moreover, we show that the data setsgenerated by ASCENT—which contain high valued solutionsat each budget allocation—can be used to perform a numberof important search-based software engineering studies ontheco-design solution space.

These results are applicable to systems that have hard real-time timing constraints and resource consumption characteris-tics. In particular, resources, such as CPU utilization, musthave fixed limits. Moreover, the calculations are based onstatic worst-case bounds on resource consumption that mustbe known at design time. The results do not apply to systemswhere design decisions need to be based on dynamicallychanging resource consumption profiles.

Our experiments were designed to compare ASCENTagainst other competing configuration approaches based onCLP, Genetic algorithms, and Particle Swarm Optimization(PSO) algorithms. Our experiments were designed to testthe key properties of MMKP co-configuration problems thataffect configuration optimality and scalability. In particular, theexperiments test scalability and optimality in terms of thetotalnumber of configuration options and settings, which determinehow large the configuration solution space is. The larger theconfiguration space is, the more difficult it is to produce a goodconfiguration in a reasonable amount of time. The results showthat ASCENT produces solutions with superior optimality. Atthe same time, ASCENT runs roughly 10 times faster thaneither the Genetic or PSO algorithms.

8

Each experiment used a total of 100 budget iterations(T = 100). We also used the M-HEU MMKP approxima-tion algorithm as our MMKP solver. All experiments wereconducted on an Apple MacBook Pro with a 2.4 GHz IntelCore 2 Duo processor, 2 gigabyes of RAM, running OS Xversion 10.4.11, and a 1.5 Java Virtual Machine (JVM) run inclient mode. The JVM was launched with a maximum heapsize of 64mb (-Xmx=64m).

We chose the M-HEU algorithm since it is straight-forwardto implement and provided good results in our initial ex-periments. Many other excellent MMKP heuristic algorithmsare available that may produce better results at the expenseof increased solving time and implementation complexity.ASCENT does not require the use of any specific MMKPalgorithm, such as M-HEU, and thus designers can choosealternate MMKP heuristic algorithms if they prefer.

5.1 MMKP Co-design Problem Generation

A key capability needed for the experiments was the ability torandomly generate MMKP co-design problems for test data.For each problem, we also needed to calculate how good AS-CENT’s solution was as a percentage of the optimal solution:valueof(ASCENTSolution)valueof(OptimalSolution) . For small problems with less than

7 sets per MMKP problem, we were able to use a constraintlogic programming (CLP) [32] technique built on top ofthe Java Choco constraint solver (choco-solver.net) toderive the optimal solution.

For larger scale problems the CLP technique was simplynot feasible,e.g., solutions might take years to find. Forlarger problems, we developed a technique that randomlygenerated MMKP co-design problems with a few carefullycrafted constraints so we knew the exact optimal answer.Others [2] have used this general approach, though with adifferent problem generation technique.

Ideally, we would prefer to generate completely randomproblems to test ASCENT. We are confident in the validityof this technique, however, for two reasons: (1) the trends weobserved from smaller problems with truly random data wereidentical to those we saw in the data obtained from solving thegenerated problems and (2) the generated problems randomlyplaced the optimal items and randomly assigned their valueand size so that the problems did not have a structure clearlyamenable to the heuristics used by our MMKP approximationalgorithm.

Our problem generation technique worked by creating twoMMKP problems for which we knew the exact optimal answer.First, we will discuss how we generated the individual MMKPproblems. LetS be the set of MMKP sets for the problem,~Rbe aK-dimensional vector describing the size of the knapsack,Iij be the jth item of the ith set, size(Iij , k) be the kth

component ofIij ’s size vector ~Szij , andsize(S, k) be thekth

component of the knapsack size vector, the problem generationtechnique for each MMKP problem worked as follows:

1) Randomly populate each set,s ⊂ S, with a number ofitems

2) Generate a random size,~R, for the knapsack

3) Randomly choose one item,Iopti ⊂ OptItems fromeach set to be the optimal item.Iopti is the optimalitem in theith set.

4) Set the sizes of the items inOptItems, so that whenadded together they exactly consume all of the space inthe knapsack:

(∀k ⊂ R), (

i∑

0

size(Iopti, k)) = size(S, k)

5) Randomly generate a value,V opti, for the optimal item,Iopti, in each set

6) Randomly generate a value delta variable,Vd <min(V opti), where min(V opti) is the optimal itemwith the smallest value

7) Randomly set the size and values of the remaining non-optimal items in the sets so that either:

• The item has a greater value than the optimal itemin its set. In this case, each component of theitem’s size vector, is greater than the correspond-ing component in the optimal item’s size vector:(∀k ⊂ R), size(Iopti, k) < size(Iij, k)

• The item has a smaller value than the optimal item’svalue minusVd, valueof(Iij) < V opti − Vd. Thisconstraint will be important in the next step. In thiscase, each component of the item’s size vector israndomly generated.

At this point, we have a very random MMKP problem. Whatwe have to do is further constrain the problem so that wecan guarantee the items inOptItems are truly the optimalselection of items. LetMaxVi be the item with the highestvalue in the ith set. We further constrain the problem asfollows:

For each itemMaxVi, we reset the values of the items (ifneeded) to ensure that the sum of the differences between themax valued items in each set and the optimal item are lessthanVd:

i∑

0

(MaxVi − V opti) < Vd

A visualization of this constraint is shown in Figure 8.

Fig. 8. A Visualization of Vd

This new valuation of the items guarantees that the items inOptItems are the optimal items. We can prove this propertyby showing that if it does not hold, there is a contradiction.Assume that there is some set of items,Ibetter, that fit intothe knapsack and have a higher value. LetV bi be the value

9

of the better item to choose than the optimal item in theithset. The sum of the values of the better items from each setmust have a higher value than the optimal items.

The itemsIbi ⊂ Ibetter must fit into the knapsack. Wedesigned the problem so that the optimal items exactly fitinto the knapsack and that any item with a higher value thanan optimal item is also bigger. This design implies that atleast one of the items inIbetter is smaller and thus alsohas a smaller value,V small, than the optimal item in its set(or Ibetter wouldn’t fit). If there areQ sets in the MMKPproblem, this implies that at mostQ−1 items inIbetter havea larger value than the optimal item in their set, and thus:

V optQ +

Q−1∑

0

V opti < V small +

Q−1∑

0

V bi

We explicitly revalued the items so that:

i∑

0

(MaxVi − V opti) < Vd

By subtracting the∑Q−1

0 V opti from both sides, we get:

V optQ < V small +

Q−1∑

0

(V bi − V opti)

the inequality will still hold if we substituteVd in for∑Q−10 (V bi − V opti), becauseVd is larger:

V optQ < V small + Vd

V optQ − Vd < V small

which is a contradicton of the rule that we enforced for smalleritems:valueof(Iij) < V opti − Vd

This problem generation technique creates MMKP problemswith some important properties. First, the optimal item in eachset will have a random number of larger and smaller valueditems (or none) in its set. This property guarantees that agreedy strategy will not necessarily do well on the problems.

Moreover, the optimal item may not have the best ratio ofvalue/size. For example, an item valued slightly smaller thanthe optimal item may consume significantly less space becauseits size was randomly generated. Many MMKP approximationalgorithms use the value/size heuristic to choose items. Sincethere is no guarantee on how good the value/size of theoptimal item is, MMKP approximation algorithms will notautomatically do well on these problems.

To create an MMKP co-design problem where we know theoptimal answer, we generate a single MMKP problem with aknown optimal answer and split it into two MMKP problemsto create the producer and consumer MMKP problems. Tosplit the problem, two new MMKP problems are created. OneMMKP problem receivesE of the sets from the originalproblem and the other problem receives the remaining sets.The total knapsack size for each problem is set to exactly thesize required by the optimal items from its sets to fit. The sumof the two knapsack sizes will equal the original knapsack size.Since the overall knapsack size budget does not change, theoriginal optimal items remain the overall optimal solution.

Next, we generate a set of produced/consumed resource val-ues for the two MMKP problems. For the consumer problem,we randomly assign each item an amount of each producedresourcek ⊂ R that the item consumes. LetTotalC(k) be thetotal amount of the resourcek needed by the optimal consumersolution andV opt(p) be the optimal value for the producerMMKP problem. We take the consumer problem and calculatea resource production ratio,Rp(k), where

Rp(k) =TotalC(k)

V opt(p)

For each item,Iij , in the producer problem, we assign ita production value for the resourcek of: Produced(k) =Rp(k) ∗ valueof(Iij).

The optimal items have the highest feasible total value basedon the given budget and the sum of their values times theresource production ratios exactly equals the needed valueofeach resourcek:

TotalC(k) =TotalC(k)

V opt(p)∗

i∑

0

V opti

Any other set of items must have a smaller total value andconsequently not provide sufficient resources for the optimalset of consumer items. To complete the co-design problem,we set the total knapsack size budget to the sum of the sizesof the two individual knapsacks.

5.2 Comparison of ASCENT, a Genetic Algorithm,and PSO

Experiment 1: Comparing ASCENT’s Optimality Versusa Genetic Algorithm, and PSO. For our first experiment,we created semi-random MMKP co-design problems thatwe knew the optimal answer to using the technique fromSection 5.1. We generated MMKP co-design problems thatranged in size from 2 to 30 sets per MMKP. Each set contained15 items. These experiments yielded solution space sizesof between154 and 1560 (2 problems with 30 sets of 15items). For each problem size, we generated and solved 30problem instances using ASCENT, a genetic algorithm, and aPSO algorithm. We graphed and compared the optimality andsolving time of the three algorithms.

ASCENT’s function for determining the value of a solu-tion is domain-specific and typically provided by the systemdesigner. The approach is the same as PSO and geneticalgorithms where the value of a solution is computed usinga function supplied by the user that is specific to the problem.In the case of software configuration for DRE systems, thevalue function is typically a calculation of the expected cost,power consumption, or real-time schedulability based on thesoftware/hardware configuration. In domains, such as the cloudcomputing domain, resource utilization prices, such as thecosts of utilizing computing resources on Amazon’s EC2computing cloud (http://aws.amazon.com/ec2/#pricing) can beused to produce the valuation function.

Genetic/PSO Design: The Genetic and PSO algorithmsboth used a common representation of the problem and penaltyfunction. The problem was represented as an n-dimensional

10

vector, where the positions in the vector corresponded to theitem that was selected from each set. For example, a problemwith 3 sets per MMKP problem would produce a vector with6 components. The first 3 components would represent theitems selected from the consumer MMKP problem’s sets. Thesecond 3 components would represent the items selected fromthe producer MMKP problem’s sets.

Each position in the vector was limited to values corre-sponding to the valid 0-based indices of items in the sets. Forexample, a set with 5 items would allow values of 0-4. A valueof 2 would correspond to selecting the3rd item in the set.

The penalty function scored solutions based on 1) if thesolution’s overall value and 2) whether or not the solutionwas correct. If a solution was not valid, the score of thesolution was set to 0 - (resource overconsumption). Thatis, solutions that did not properly adhere to the budget orthe production and consumption of resources would producenegative values. Although repair functions can sometimesprovide better results than a penalizing function, repairingan arbitrary invalid MMKP co-design solution is extremelycomplex and a research endeavor in its own right [12].

For the genetic and PSO algorithms, we used population(total particle) sizes of 20, 200, and 2000 members. Weconducted various experiments to tune the parameters of thealgorithms. For the PSO algorithm, we used a local learningrate of 2, a global learning rate of 2, and an inertial value of0.5. For the Genetic algorithm, we mated the top 25% of thepopulation using a strict cutoff. We also allowed up to 50% ofthe population to survive and cross over from one generationto the next. Finally, we used a mutation probability of 0.05%.Each algorithm was run for a total of 20 generations/iterations.

Experiment Results with Semi-Random Data: The resultsfor the first experiment are shown in Figure 9.

Fig. 9. Solution Optimality vs Number of Sets Comparedfor ASCENT, a Genetic Algorithm, and a PSO Algorithm

As can be seen from the results, for up to 5 sets perMMKP problem, the Genetic algorithm with 2000 populationmembers provided the best results. The Genetic Algorithmwith 2000 population members, required 9,024ms to solve aproblem with 5 sets. ASCENT, in contrast, required 73ms.When the problems were scaled up to 30 sets per MMKPproblem, ASCENT provided far superior optimality and runtime. ASCENT produced solutions that averaged roughly

99.2% optimal versus the Genetic algorithms 54.9% optimal-ity. Furthermore, ASCENT solved the problems in an averageof 317ms versus the Genetic algorithms average runtime of64,212ms.

Experiment Results with Random Data: We also com-pared the algorithms on a series of problems that werecompletely randomly generated. For these problems, we didnot know the true optimal value. We generated 100 problemswith 50 sets per MMKP problem and 15 items per set. Thisyielded a solution space size of15100. In order to ensure thatwe generated tractable problem instances, we set extremelyloose resource constraints on the problems to create a highprobability that a solution existed.

Figure 10 shows a graph of the solution scores of thealgorithms on these 100 random problems.

Fig. 10. Solution Score for 100 Randomly GeneratedProblems

As can be seen from Figure 10, ASCENT produced superiorsolution scores across all 100 problem instances. The Geneticalgorithm, which was the second best algorithm, producedsolutions that were at most 90.9% of the value of ASCENT’ssolution and at least 65.8% of the value. The PSO producedsolutions that were at most 43.5% of the value of ASCENT’ssolutions and at least 27.5% of the value. The average solvingtime for the Genetic 2000 was 101,453ms. The average solvingtime for the PSO 2000 was 55,203ms. The average solvingtime for ASCENT was 672ms.

5.3 ASCENT Scalability and Optimality

Experiment 2: Comparing ASCENT scalability to an exactCLP technique. When designing a satellite it is critical thatdesigners can gauge the accuracy of their design techniques.Moreover, designers of a complicated satellite system needtoknow how different design techniques scale and which tech-nique to use for a given problem size. This set of experimentsevalutes these questions for ASCENT and a constraint logicprogramming (CLP) co-design technique.

Although CLP solvers can find optimal solutions to MMKPco-design problems they have exponential time complexity.For large-scale co-design problems (such as designing a com-plicated climate monitoring satellite) CLP solvers thus quicklybecome incapable of finding a solution in a reasonable time

11

frame. We setup an experiment to compare the scalabilityof ASCENT to an CLP technique. We randomly generateda series of problems ranging in size from 1 to 7 sets perhardware and software MMKP problem. Each set had 10items. We tracked and compared the solving time for ASCENTand the CLP technique as the number of sets grew. Figure 11presents the results from the experiment. As shown by the

Fig. 11. Solving Time for ASCENT vs. CLP

results, ASCENT scales significantly better than an CLP-basedapproach.

Experiment 3: Testing ASCENT’s solution optimality.Clearly, scalability alone is not the only characteristic of agood approximation algorithm. A good approximation algo-rithm must also provide very optimal results for large problemsizes. We created an experiment to test the accuracy ofASCENT’s solutions. We compared the value of ASCENT’sanswer to the optimal answer,

valueof(ASCENTSolution)

valueof(OptimalSolution)

for 50 different MMKP co-design problem sizes with 3 itemsper set. For each size co-design problem, we solved 50different problem instances and averaged the results.

It is often suggested, due to the Central Limit Theorem [19],to use a sample size of 30 or larger to produce an approx-imately normal data distribution [15]. We chose a samplesize of 50 to remain well above this recommended minimumsample size. The largest problems, with 50 sets per MMKPproblem, would be the equivalent of a satellite with 50 pointsof software variability and an additional 50 points of hardwarevariability.

For problems with less than 7 sets per MMKP problem, wecompared against the optimal answer produced with an CLPsolver. We chose a low number of items per set to decreasethe time required by the CLP solver and make the experimentfeasible. For problems with more than 7 sets, which couldnot be solved in a timely manner with the CLP technique, weused our co-design problem generation technique presentedin Section 5.1. The problem generation technique allowedus to create random MMKP co-design problems that weknew the exact optimal answer for and could compare againstASCENT’s answer.

Figure 12 shows the results of the experiment to testASCENT’s solution value verusus the optimal value over 50MMKP co-design problem sizes.

With 5 sets, ASCENT produces answers that average 90%optimal. With 7 sets, the answers average∼95% optimal.

Fig. 12. Solution Optimality vs Number of Sets

Beyond 20 sets, the average optimality is∼98% and continuesto improve. These results are similar to MMKP approximationalgorithms, such as M-HEU, that also improve with increasingnumbers of sets [2]. We also found that increasing the numberof items per set also increased the optimality, which parallelsthe results for our solver M-HEU [2].

Experiment 4: Measuring ASCENT’s solution spacesnapshot accuracy. As part of the solving process, ASCENTnot only returns the optimal valued solution for a co-designproblem but it also produces a data set to graph the optimalanswer at each budget allocation. For the satellite example,the graph would show designers the design with the highestimage processing accuracy for each ratio of budget allocationto software and hardware. We created an experiment to testhow optimal each data point in this graph was.

For this experiment, we generated 100 co-design problemswith less than 7 sets per MMKP problem and comparedASCENT’s answer at each budget allocation to the optimalanswer derived using an CLP technique (more sets improveASCENT’s accuracy). For problems with 7 sets divided into98 different budget allocations, ASCENT finds the same,optimal solution as the CLP solver more than 85% of thetime. Figure 13 shows an example that compares the solutionspace graph produced by ASCENT to a solution space graphproduced with an CLP technique. The X-axis shows the

Fig. 13. Solution Value vs. Budget Allocation

percentage of the budget allocated to the software (consumer)MMKP problem. The Y-axis shows the total value of theMMKP co-design problem solution. The ASCENT solutionspace graph closely matches the actual solution space graphproduced with the CLP technique.

12

5.4 Solution Space Snapshot Resolution

Experiment 5: Demonstrating the importance of solutionspace snapshot resolution. A complicated challenge of apply-ing search-based software engineering to hardware/softwareco-design problems is that design decisions are rarely asstraightforward as identifying the design configuration thatmaximizes a specific property. For example, if one satelliteconfiguration provides 98% of the accuracy of the mostoptimal configuration for 50% less cost, designers are likely tochoose it. If designers have extensive experience in hardwaredevelopment, they may favor a solution that is marginally moreexpensive but allocates more of the development to hardware,which they know well. Search-based software engineeringtechniques should therefore allow designers to iteratively teasethese desired designs out of the solution space.

ASCENT has a number of capabilities beyond simplyfinding the optimal solution for a problem to help designersfind desirable solutions. First, as we describe below, ASCENTcan be adjusted to produce different resolution images of thesolution space by adjusting the granularity of the budget allo-cation steps (e.g., make a larger number of smaller allocationchanges). ASCENT’s other solution space analysis capabilitiesare presented in Section 5.5.

The granularity of the step size greatly impacts the res-olution or detail that can be seen in the solution space. Toobtain the most accurate and informative solution space image,a small step size should be used. Figure 14(a) shows a solutionspace graph generated through ASCENT using 10 allocationsteps. The X-axis is the percentage of budget allocated tosoftware, the Y-axis is the total value of the solution. Itappears that any allocation of 30% or more of the budget tosoftware will produce a satellite with optimal image processingaccuracy.

(a) Low Resolution Solution Space Snapshot

(b) High Resolution Solution Space Snapshot

Fig. 14. A Solution Space Graph at Varying Resolutions

The importance of a small step size is demonstrated inFigure 14(b), which was produced with 100 allocation steps.Figure 14(a) suggests that any allocation of greater than30% for software would result in an optimal satellite design.Figure 14(b) shows that there are many pitfalls in the 70%to 99% range that must be avoided. At these precise budgetallocation points, there is not a combination of hardware andsoftware that will produce a good solution.

5.5 Solution Space Analysis with ASCENTAlthough ASCENT’s ability to provide variable resolutionsolution space images is important, its greatest value stemsfrom the variety of questions that can be answered from itsoutput data. In the following results, we present representativesolution space analyses that can be performed with ASCENT’soutput data.

Design analysis 1: Finding designs that produce budgetsurpluses. Designers may wish to know how the resourceslack values, such as how much RAM is unused, with differentsatellite designs. Another related question is how much of thebudget will be left-over for designs that provides a specifiedminimal level of image processing accuracy. We can use thesame ASCENT output data to graph the budget surplus at arange of allocation values.

Figure 15 shows the budget surplus from choosing variousdesigns. The graph has been filtered to adhere to a requirementthat the solution provide a value of at least 1600. Any datapoint with a value of less than 1600 has had its surplus set to0. Looking at the graph, we can see that the cheapest designthat provides a value of at least 1,600 is found with a budgetallocation of 80% software and 20% hardware. This designhas a value of 1,600 and produces budget savings of 37%.Once the design that produces the biggest surplus is found,developers use algorithms other than ASCENT to determinehow to best use the excess.

Fig. 15. Budget Surplus vs. Budget Allocation

Design analysis 2: Evaluating design upgrade/-downgrade cost. In some situations, designers may have agiven solution and want to know how much it will cost orsave to upgrade or downgrade the solution to a different imageprocessing accuracy. For example, designers may be asked toprovide a range of satellite options for their superiors that showwhat level of image processing accuracy they can provide ata number of price points. Figure 16 depicts another view ofthe ASCENT data that shows how cost varies in relation tothe minimum required solution value.

13

This graph shows that 5 cost units can finance a designwith a value up to 900, but a design of a value of 1,000units will cost at least 124 cost units. This information graphdemonstrates the increased financial burden of requiring aslightly higher valued design. Alternatively, if the necessaryvalue of the system is near the left edge of one of theseplateaus, designers can make an informed decision on whetherthe increased value justifies the significantly increased cost.

Fig. 16. Cost of Increasing Solution Value

6 RELATED WORK

Search-based software engineering has a large number offacets ranging from the design of general approximation algo-rithms to the construction of search-based software engineer-ing methods for specific problems. This section compares andcontrasts ASCENT to search-based software engineering tech-niques related to (1) approximation algorithms for solvingsim-ilar problems to the MMKP co-design problem, (2) methodsfor using search-based techniques to solve hardware/softwarepartitioning problems, (3) methods for using approximationtechniques for solving hardware/software scheduling prob-lems, and (4) search-based software engineering techniquesfor determining project staffing.

Hardware/software co-design. A number of co-designtechniques [6, 25, 31, 1, 29, 34, 28, 13]—that can be viewedas search-based software engineering techniques—examinetheproblem of partitioning system functionality into hardware andsoftware. These approaches use a number of search techniquessuch as heuristic search techniques [34, 31], PSO [28, 13],Genetic Algorithms [1, 29], and Clustering Algorithms [6].Inthe partitioning problem, a system’s required operations aregrouped into tasks or functions, which are then implementedineither hardware or software. The goal is to correctly partitionthe tasks into hardware and software to meet a predefinedperformance goal. Some tasks may operate with higher perfor-mance if the functionality is placed on the hardware rather thanon software. The performance of the system is thus determinedby the location and placement of tasks in hardware versussoftware.

The MMKP co-design problem, which ASCENT focuseson, is complementary to this research. In particular, theserelated approaches do not deal with maximizing a measureof system value subject to producer/consumer resources andcost. Similarly, ASCENT does not examine the impact of theplacement of tasks on the hardware and software. Each ap-proach fills an important, although distinct, role in the search-

based software engineering landscape for hardware/softwareco-design.

Another related problem in hardware/software co-design isthe scheduling of hardware/software tasks subject to resourceconstraints. This type of co-design problem tries to determinethe optimal ordering of a series of tasks implemented in bothhardware and software. Scheduling with resource constraintsis a challenging problem that has led to the developmentof large number of co-design search and design explorationtechniques [18, 24, 14]. This co-design technique is attackinga different facet of software/-hardware co-design that doesnot deal with how to select a software and hardware designthat maximizes system value subject to producer/consumer andcost constraints. ASCENT, however, focuses directly on thismaximization of system value subject to these constraints.

MMKP approximation. Many problems similar to thehardware/software co-design problem presented in this paperhave been solved with MMKP techniques. In multimediasystems, determining the quality settings that maximize thevalue of a series of data streams has been modeled as anMMKP problem [23]. Other usages of MMKP include meta-scheduling for grid applications [33]. A number of excellentheuristic approximation algorithms, such as M-HEU [2] andC-HEU [2], with near optimal results have been devised.

These existing MMKP algorithms and techniques, however,cannot be directly applied to the MMKP-codesign problemdescribed in this paper. First, as described in Section 3.1,theexisting techniques assume that there are predefined individualknapsack sizes, which is not the case in the MMKP co-designscenario. Second, as described in Section 3.2, producer MMKPitems cannot be valued separately from a consumer MMKPproblem, causing a coupling problem. Existing MMKP ap-proaches are not designed to handle this type of couplingproblem. In contrast, ASCENT addresses these issues andprovides high-quality solutions to MMKP co-design problems.

Project management and staff allocation. Accurate plan-ning of large projects are essential to estimate project cost,determine the formation of employee project teams, and toassign these teams to tasks in a manner that gives the largestprobability for successful completion. The placement of eachindividual employee can change the profile of the entire projectplan, resulting in an exponential number of possible configu-rations [4]. Moreover, parameters of a project are dynamic andmay change several times before project completion, requiringthat multiple staffing solutions be calculated. This research isrelated to MMKP co-design problems in that it deals with twotightly-coupled activites–the ordering and staffing of projectparts subject to resource constraints. Although the work isrelated, it cannot be used to solve MMKP co-design problems.In contrast, ASCENT is specifically designed for solvingMMKP co-design problems.

7 CONCLUDING REMARKS

Designing hardware and software in tandem to maximize asystem capability can be an NP-hard activity. Search-basedsoftware engineering is a promising approach that can be usedto leverage algorithmic techniques during system co-design.This paper presented a polynomial-time search-based software

14

engineering technique, calledAllocation-baSed ConfigurationExploration Technique(ASCENT), for finding near optimalhardware/software co-design solutions.

We showed how ASCENT’s heuristic-based solutions tohardware/software co-design problems average over 95% opti-mal when there are more than seven points of variability in thehardware and software design. Moreover, ASCENT’s output(which is a data set showing the optimal design configurationsat each ratio of budget allocation to hardware and software)can be used to search for and answer important softwareengineering questions, such as how much more it will costfor increasing the value of system capability.

An implementation of ASCENT is available from theASCENT Design Studio project (http://code.google.com/p/ascent-design- studio/).REFERENCES[1] M. Abdelhalim and S.-D. Habib. Modeling Communication

Cost and Hardware Alternatives in PSO Based HW/SWPartitioning. InProceedings of the International Conferenceon Microelectronics, pages 111–114, Dec. 2007.

[2] M. Akbar, E. Manning, G. Shoja, and S. Khan. HeuristicSolutions for the Multiple-Choice Multi-dimension KnapsackProblem. InProceedings of the International Conference onComputational Science-Part II, pages 659–668.Springer-Verlag London, UK, 2001.

[3] E. Alba and J. Francisco Chicano. Software ProjectManagement with GAs.Information Sciences,177(11):2380–2401, 2007.

[4] G. Antoniol, M. Di Penta, and M. Harman. A RobustSearch-based Approach to Project Management in thePresence of Abandonment, Rework, Error and Uncertainty. InProceedings of the International Symposium on SoftwareMetrics, pages 172–183, Sept. 2004.

[5] A. Barreto, M. Barros, and C. Werner. Staffing a SoftwareProject: A Constraint Satisfaction and Optimization-basedApproach.Computers and Operations Research,35(10):3073–3089, 2008.

[6] E. Barros, C. Universitaria-Recife-PE, W. Rosenstiel,andX. Xiong. A Method for Partitioning UNITY Language inHardware and Software.In Proceedings of EURO-VHDL,Grenoble, France, Sept. 1994.

[7] P. Chiu, Y. Chen, and K. Lee. A Request SchedulingAlgorithm to Support Flexible Resource Reservations inAdvance.Proceedings of the Canadian Conference onElectrical and Computer Engineering, Niagara Falls, Ontario,Canada, May 2004.

[8] L. Chung. Non-Functional Requirements in SoftwareEngineering. Springer, 2000.

[9] J. Clark and J. Jacob. Protocols are Programs Too: theMeta-heuristic Search for Security Protocols.Information andSoftware Technology, 43(14):891–904, 2001.

[10] T. H. Cormen, C. E. Leiserson, and R. L. Rivest.Introductionto Algorithms. MIT, Cambridge, Massachussetts, 1990.

[11] S. Curtis. The Magnetospheric Multiscale Mission...ResolvingFundamental Processes in Space Plasmas.NASA STI/ReconTechnical Report N, Dec. 1999.

[12] K. Deb. An Efficient Constraint Handling Method for GeneticAlgorithms. Computer methods in applied mechanics andengineering, 186(2-4):311–338, 2000.

[13] R. Dick and N. Jha. MOGAC: a Multiobjective GeneticAlgorithm for Hardware-Software Cosynthesis of DistributedEmbedded Systems.IEEE Transactions on Computer-AidedDesign of Integrated Circuits and Systems, 17(10):920–935,1998.

[14] C. Gebotys and M. Elmasry.Optimal VLSI ArchitecturalSynthesis: Area, Performance and Testability. KluwerAcademic Publishers Norwell, MA, USA, 1992.

[15] J. Gosling.Introductory Statistics. Pascal Press, Glebe,Australia, 1995.

[16] M. Harman. The Current State and Future of Search BasedSoftware Engineering.International Conference on SoftwareEngineering, Minneapolis, MN, pages 342–357, May 2007.

[17] M. Harman and B. Jones. Search-based software engineering.Information and Software Technology, 43(14):833–839, 2001.

[18] P.-A. Hsiung, P.-H. Lu, and C.-W. Liu. Energy efficientco-scheduling in dynamically reconfigurable systems. InProceedings of the International Conference onHardware/software Codesign and System Synthesis, Salzburg,Austria, pages 87–92, October 2007.

[19] P. Huber.Robust Statistics. Wiley Series in Probability andMathematical Statistics, New York, New York, 1981.

[20] T. Ibaraki, T. Hasegawa, K. Teranaka, and J. Iwase. TheMultiple Choice Knapsack Problem.J. Oper. Res. Soc. Japan,21:59–94, 1978.

[21] O. Ibarra and C. Kim. Fast Approximation Algorithms fortheKnapsack and Sum of Subset Problems.Journal of the ACM(JACM), 22(4):463–468, 1975.

[22] S. E. Institute. Ultra-Large-Scale Systems: SoftwareChallengeof the Future. Technical report, Carnegie Mellon University,Pittsburgh, PA, USA, Jun 2006.

[23] M. Islam, M. Akbar, H. Hossain, and E. Manning. AdmissionControl of Multimedia Sessions to a Set of MultimediaServers Connected by an Enterprise Network. In2005 IEEEPacific Rim Conference on Communications, Computers andsignal Processing, 2005. PACRIM, pages 157–160, 2005.

[24] M. Kim, S. Banerjee, N. Dutt, and N. Venkatasubramanian.Design Space Exploration of Real-time Multi-media MPSoCswith Heterogeneous Scheduling Policies. InProceedings ofthe International Conference on Hardware/software Codesignand System Synthesis, Seoul, Korea, pages 16–21, 2006.

[25] E. Lagnese and D. Thomas. Architectural Partitioning forSystem Level Design.International Conference on DesignAutomation, pages 62–67, June 1989.

[26] S. Martello and P. Toth. Algorithms for Knapsack Problems.Surveys in Combinatorial Optimization, 31:213–258, 1987.

[27] P. McMinn. Search-based software test data generation: asurvey. Software Testing, Verification & Reliability,14(2):105–156, 2004.

[28] D. Saha, R. Mitra, and A. Basu. Hardware softwarepartitioning using genetic algorithm. InIn Proceedings of theInternational Conference on VLSI Desig, Hyderabad, India,pages 155–160, Jan 1997.

[29] Q. Tong, X. Zou, Q. Zhang, F. Gao, and H. Tong. TheHardware/Software Partitioning in Embedded System byImproved Particle Swarm Optimization Algorithm. pages43–46, Oct. 2008.

[30] Y. Toyoda. A Simplified Algorithm for ObtainingApproximate Solutions to Zero-one Programming Problems.Management Science, pages 1417–1427, 1975.

[31] F. Vahid, D. Gajski, and J. Gong. A Binary-constraint SearchAlgorithm for Minimizing Hardware DuringHardware/Software Partitioning.In Proceedings of theConference on European Design Automation, Grenoble,France, pages 214–219, Sept. 1994.

[32] P. Van Hentenryck, H. Simonis, and M. Dincbas. Constraintsatisfaction using constraint logic programming.Constraint-Based Reasoning, 1994.

[33] D. Vanderster, N. Dimopoulos, and R. Sobie. Metaschedulingmultiple resource types using the MMKP. InProceedings ofthe 7th IEEE/ACM International Conference on GridComputing, Barcelona, Spain, pages 231–237, Sept. 2006.

[34] T. Wiangtong, P. Cheung, and W. Luk. Comparing threeheuristic search methods for functional partitioning inhardware–software codesign.Design Automation forEmbedded Systems, 6(4):425–449, 2002.