vrem: an advanced virtual environment for micro assembly an advanced virtua… · the virtual...

10
ORIGINAL ARTICLE VREM: An advanced virtual environment for micro assembly J. Cecil & James Jones Received: 22 May 2013 /Accepted: 7 January 2014 /Published online: 5 February 2014 # Springer-Verlag London 2014 Abstract Micro assembly involves the assembly of micron- sized devices. Given the complexity of this domain, the role of virtual environments becomes important as they provide a basis to propose and compare assembly alternatives virtually prior to physical assembly. This paper proposes an integrated approach which includes the use of virtual reality-based as- sembly environments that interface with physical micro as- sembly environments. Such an approach can be an intrinsic part of a collaborative manufacturing framework that seeks to support the rapid assembly of micro devices. In this paper, the design of VREM (Virtual Reality based Environment for Micro Assembly) is discussed which is based on this integrat- ed approach involving use of virtual and physical resources. Keywords Micro assembly . Virtual reality environments . Cyber frameworks 1 Introduction Micro devices assembly (MDA) involves the assembly of micron-sized parts or objects. Certain micro part designs cannot be manufactured using micro-electrical mechanical systems (MEMS) technologies. Fabrication techniques to pro- duce MEMS devices typically involves a monolithic integrat- ed circuit method that requires no or little assembly; MEMS techniques involve selectively etching away parts of a silicon wafer or adding new structural layers. Examples of products manufactured utilizing MEMS techniques are accelerometers and inkjet printer heads. However, micron-sized designs that possess complex geometry and varying material properties cannot be manufactured using MEMS techniques. Such micro devices have to be assembled using MDA techniques [25, 26]. Recently, there has been more interest in the field of MDA due to its manufacturing potential in producing biomedical sensors, chemlab-on-a-chip, surveillance devices, and other miniature designs. 2 Literature review Cecil and Vasquez [2] presented a comprehensive review of micro assembly techniques. The predominant emphasis in micro assembly systems research has focused on the creation of automated and semi automated approaches to micro assem- bly [311]. In the domain of micro devices assembly, there has been less of a focus in general on cyber and IT-oriented approaches; however, these cyber-based approaches become increasingly important as global collaborations become more commonplace with rapid advances in cyber networks and technologies. Such cyber-based approaches are part of evolv- ing next generation collaborative methods and frameworks which hold the potential to facilitate agile strategies where partners and resources are geographically distributed. A lim- ited number of papers have focused on the creation of virtual reality environments to facilitate the rapid assembly of micro devices. A brief discussion of these papers follows. Ralis et al. [12] presented a technique of visual servoing to get both a global and local estimate of the workspace. The depth is obtained by focusing the workspace. In [ 13], Feddema et al. presented insight into fine motion planning for the micro-domain. The authors emphasized that free-space motion planning and geometric assembly constraints in macro-world planners would directly apply to the micro world. Precise motion and fine motion planning would differ J. Cecil (*) Center for Information Centric Engineering, School of Industrial Engineering, Oklahoma State University, Stillwater, OK, USA e-mail: [email protected] J. Jones Sandia National Laboratories, Albuquerque, NM, USA Int J Adv Manuf Technol (2014) 72:4756 DOI 10.1007/s00170-014-5618-9

Upload: others

Post on 05-Jul-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: VREM: An advanced virtual environment for micro assembly An advanced virtua… · the virtual environment is the design of the target part to be assembled along with any specified

ORIGINAL ARTICLE

VREM: An advanced virtual environment for micro assembly

J. Cecil & James Jones

Received: 22 May 2013 /Accepted: 7 January 2014 /Published online: 5 February 2014# Springer-Verlag London 2014

Abstract Micro assembly involves the assembly of micron-sized devices. Given the complexity of this domain, the role ofvirtual environments becomes important as they provide abasis to propose and compare assembly alternatives virtuallyprior to physical assembly. This paper proposes an integratedapproach which includes the use of virtual reality-based as-sembly environments that interface with physical micro as-sembly environments. Such an approach can be an intrinsicpart of a collaborative manufacturing framework that seeks tosupport the rapid assembly of micro devices. In this paper, thedesign of VREM (Virtual Reality based Environment forMicro Assembly) is discussed which is based on this integrat-ed approach involving use of virtual and physical resources.

Keywords Micro assembly . Virtual reality environments .

Cyber frameworks

1 Introduction

Micro devices assembly (MDA) involves the assembly ofmicron-sized parts or objects. Certain micro part designscannot be manufactured using micro-electrical mechanicalsystems (MEMS) technologies. Fabrication techniques to pro-duce MEMS devices typically involves a monolithic integrat-ed circuit method that requires no or little assembly; MEMStechniques involve selectively etching away parts of a siliconwafer or adding new structural layers. Examples of productsmanufactured utilizing MEMS techniques are accelerometers

and inkjet printer heads. However, micron-sized designs thatpossess complex geometry and varying material propertiescannot be manufactured using MEMS techniques. Such microdevices have to be assembled usingMDA techniques [2–5, 26].

Recently, there has been more interest in the field of MDAdue to its manufacturing potential in producing biomedicalsensors, chemlab-on-a-chip, surveillance devices, and otherminiature designs.

2 Literature review

Cecil and Vasquez [2] presented a comprehensive review ofmicro assembly techniques. The predominant emphasis inmicro assembly systems research has focused on the creationof automated and semi automated approaches to micro assem-bly [3–11]. In the domain of micro devices assembly, there hasbeen less of a focus in general on cyber and IT-orientedapproaches; however, these cyber-based approaches becomeincreasingly important as global collaborations become morecommonplace with rapid advances in cyber networks andtechnologies. Such cyber-based approaches are part of evolv-ing next generation collaborative methods and frameworkswhich hold the potential to facilitate agile strategies wherepartners and resources are geographically distributed. A lim-ited number of papers have focused on the creation of virtualreality environments to facilitate the rapid assembly of microdevices. A brief discussion of these papers follows.

Ralis et al. [12] presented a technique of visual servoing toget both a global and local estimate of the workspace. Thedepth is obtained by focusing the workspace. In [13],Feddema et al. presented insight into fine motion planningfor the micro-domain. The authors emphasized that free-spacemotion planning and geometric assembly constraints inmacro-world planners would directly apply to the microworld. Precise motion and fine motion planning would differ

J. Cecil (*)Center for Information Centric Engineering, School of IndustrialEngineering, Oklahoma State University, Stillwater, OK, USAe-mail: [email protected]

J. JonesSandia National Laboratories, Albuquerque, NM, USA

Int J Adv Manuf Technol (2014) 72:47–56DOI 10.1007/s00170-014-5618-9

Page 2: VREM: An advanced virtual environment for micro assembly An advanced virtua… · the virtual environment is the design of the target part to be assembled along with any specified

in micro world. Path planning is traditionally studied in robot-ics to find optimal collision-free paths for manipulators [1].Path planning approaches can be proposed and studied in thecontext of virtual environments to find collision-free paths forvirtual avatars in games, entertainment, and simulations.

Hollis [6, 25] discussed the concept of a miniature factoryto accomplish MDA using an innovative agile assembly ar-chitecture. The factory considered consists of multiple-courierrobots, overhead manipulators, and platen tiles. Overheadmanipulators include end effectors capable of force and torquemeasurements. Examples of end effectors were screw drivers,lasers, orbital head formers, welders, and gluers. The processof creating a microphone assembly was used as a processcontext to highlight the agile assembly architecture.

Monferrer and Bonyuet [14] outlined a system to controlrobots in difficult or dangerous tasks using a virtual reality(VR)-based framework. A set of guidelines is proposed to definean ideal user interface that would use VR to help an operatorcontrol an underwater robot. The authors also highlighted therole of human users in the design of VR-based collaborativeenvironments. Alex et al. [15] outlined an approach involving theintegration of a VR modeling language (VRML)-based virtualmicroworld with visual servoing-micromanipulation strategies.The authors underscored the complexities involved inteleoperated micromanipulation and assembly. An operator wasable to remotely accomplish micro assembly although someInternet delays were experienced. In [13, 16], Cassier et al.described a combination of visual servoing and virtual realityto assemble micro devices. Visual servoing techniques are usedfor efficient and reliable force feedback during micro assemblytasks. Automated movements within the work cell were accom-plished through offline programming and are based on CAD-CAM data of the micro devices being processed.

In [17], a visual-servo system for peg-in-hole micro assem-bly was proposed. Dynamic position-based servo providesaccuracy, efficiency, and robustness. Regional scanning withedge fitting tracks the position of the micro object, andshadow-aided positioning algorithm is used to complete thefinal operation of placing micro pins in the hole. Luo and Xiao[18] discussed the assembly of optical fibers using hapticdevices. The overall objective was to simulate an opticalfiber-assembly process using VR with force feedback to fa-cilitate the creation of an automated fiber-assembly process.

Probst et al. [19] outlined the use of a VR-based environmentto control the assembly tasks using a microassembly systemcalled the IRIS microassembly system V2. The physical work-station consists of a 6-DOF microassembly system includingseveral microgrippers, a base unit, a top unit, cameras, and anillumination dome. In [20, 21], the creation of general frame-work for a micromanipulation robot based on virtual environ-ments is discussed. AVR-based framework is outlined for themicromanipulation robot configuration. Successful experimentsare discussed related to peg-in-hole micro assembly tasks.

In [22], a micro vision system is described to control theposition of a micro robot using a virtual reality interface. Theimportant principle in this approach is to accomplish microrobotic tasks with a human in the loop who has access tomeasurement and visualization resources.

Some of our past work involving virtual environments formicro assembly has been presented in [23, 24, 28, 38, 39].These approaches were developed as part of an InformationCentric Engineering (ICE) framework that can be used torespond to changing customer requirements. The role ofinformation models, virtual environments and physical re-sources working collaboratively and linked by advancedcyberinfrastructure was emphasized in these approaches. Asemantic web based test bed for the assembly of micro devicesis outlined in [38]; this test bed demonstrated an advancedapproach where a distributed set of software and physicalresources are discovered and subsequently used in the plan-ning, simulation and physical assembly of target micro de-vices based on changing customer requirements. Virtual en-vironment was created using.

Virtual prototyping-based approaches have been investigat-ed in other manufacturing and process domains including sat-ellite assembly [29] and emerging areas such as biomedical/bioengineering and nano manufacturing [30]. The primarybenefits of adopting virtual prototyping approaches lies in beingable to propose process design alternatives in assembly andmanufacturing contexts virtually and comparing the impact ofdesign changes on downstream activities.

One of the major benefits of using virtual environments formicro assembly is that due to the extremely small size andnature of the parts, fixtures, and the general assembly area, it iseasier and beneficial to propose and compare assembly alter-natives virtually prior to physical assembly. This allows usersand process engineers to gain a better idea of the assemblyconstraints and alternatives which can then be compared,studied and validated either individually or as a group [19].

3 Design of the virtual environment for micro assembly

When customer requirements change, there is a need to createand use simulation environments where candidate assemblyplans can be proposed, compared, evaluated, and studied virtu-ally before any physical assembly activities are initiated. Suchan approach will also enable cross-functional analysis by engi-neering team members. An important part of such an approachis the use of a virtual reality-based assembly environment wheresuch process design options can be evaluated prior to physicalassembly. In micro assembly, given the extremely small scale(in microns), a virtual environment also allows the user tosimulate and visualize various process alternatives effectively;subsequently, these process details and plans can be executedby computer interfaces linked with physical equipment

48 Int J Adv Manuf Technol (2014) 72:47–56

Page 3: VREM: An advanced virtual environment for micro assembly An advanced virtua… · the virtual environment is the design of the target part to be assembled along with any specified

including automated and semi automated work cells. The vir-tual environment allows users to “scale up” and study processalternatives at the micro scale (such as the angle to access andmanipulate a micro device or complete a part manipulation)which enables better decision making [24]. It also enablesengineers with various backgrounds to work together and iden-tify problems in a virtual collaborative manner.

An integrated virtual reality-based environment for microassembly (VREM) activities has been developed for suchmicro assembly contexts (Fig. 1). In this paper, the focus ison the design of a non immersive virtual environment. It islower in cost than the semi immersive virtual environmentwhich requires 3D eye wear and motion trackers. VREM’ssimulation environment was created on a PC windows plat-form using Unity3D and Javascript.

The physical environment considered The virtual environmenthas been developed for various work cells to demonstrate thefeasibility of the outlined approach. One of the physical envi-ronments considered is an advanced micro assembly work cellwhose primary function is to assemble micron-sized devices inthe scale of 40 to 1,000 μm. This work cell has several majorelements including an assembly table, a gripping mechanism,and various micro positioners. The basic movement duringassembly is achieved by several micro positions which arecomposed of a translation stage, one ACmotor, and two controlsignal lines. The assembly worktable, by itself, consists of twomicro translation stages along x- and y-axes and one rotationalstage, which give the worktable three degrees of freedom intotal. The whole system is mounted on a vibration isolator tableto damp the effects of undesired vibrations. The layout of thephysical cell can be seen in Fig. 2.

The gripping mechanism has a vertical degree of freedom(which can move the gripper up or down depending on thetarget assembly); it utilizes various mechanical tweezers toperform the task of picking and placing micro objects. Anassortment of tweezers for this work cell with different shapesis available from a variety ofmaterials. The assembly table hasa rotational degree of freedom, which enables the gripper toavoid obstacles or approach a certain part from more intricateangles. The work cell has a two-camera-based vision systemwhich can provide precise feedback on the position of thetweezers, and the target micro parts during assembly.

Figure 3 shows a view of the virtual environment whichserves as the simulator for this work cell where assembly andpath planning details can be compared. The virtual environ-ment has several key modules which are discussed in thefollowing sub sections. These include an Assembly Analysismodule, User Interface module, Collision Detection module,and a Command Generation module.

A collaborative cyber-physical integration module functionsas the overall coordinator to ensure accomplishment of thevarious tasks including supervising obtaining user input andexchanging information/data between the virtual (cyber) andphysical environments. As indicated in Fig. 1, the main input tothe virtual environment is the design of the target part to beassembled along with any specified precedence constraints(during assembly). The outcomes from the virtual environmentto the physical environment include the validated assemblyplan, a validated path plan, and the assembly instructions forthe physical part to be assembled by the available work cellresources. Feedback from the virtual environment to the cus-tomer can include assembly problems (related to being unableto assemble the target part due to obstacles or design

Fig. 1 Using virtualenvironments for physical microassembly

Int J Adv Manuf Technol (2014) 72:47–56 49

Page 4: VREM: An advanced virtual environment for micro assembly An advanced virtua… · the virtual environment is the design of the target part to be assembled along with any specified

constraints) which, in turn, can be used to modify a givendesign prior to physical assembly and a comparison of candi-date plans (should a user provide more than one assemblyalternatives). The outcomes from the physical environmentinclude a report on the successful completion of a given assem-bly, problems encountered, as well as live video which can beused to monitor the progress of the physical assembly activities.

Assembly analysis (AA) module This module allows a user topropose an assembly plan based on assembly and obstacleconstraints. Using a menu-driven interface, a user can study atarget assembly virtually. A view of the main screen is shownin Fig. 3. There are two options in general: an assembly plancan be generated by a user manually and input to the system orit can be generated automatically using a genetic algorithm(GA)-based approach. The user is allowed to select the targetparts to be assembled as well as make changes to the processlayout if necessary (including the type and positions of the partholding fixtures, the type of tweezers to be used, etc.).

Depending on the complexity of the task, the user can alsosplit the screen into multiple views (apart from the defaultisometric view). This will enable being able to see the sideview and top view during a candidate virtual assembly (Fig. 4)which may be important in certain process contexts (forexample, ensuring a pin clears an obstacle which can be bettervisualized in a side view). As the assembly area is very small(about 2 in. in diameter), being able to propose assembly planswhich include sequence in which a set of parts need to beassembled and ensuring the gripper does not collide with otherparts (or sensors in a given part layout) becomes an importantstep in the overall process. The various objects of interest(such as the gripper, micro objects, etc.) and their behaviorin the virtual environment are managed as a scene graph.Hierarchy attributes (involving parent–child relationships)are maintained to reflect dynamic object interactions.

User interface module This module enables a user to interactwith the virtual environment using a keyboard and a menu-

Fig. 2 View of the advancedmicro assembly environment

Fig. 3 A close-up view of thevirtual reality-based assemblyenvironment

50 Int J Adv Manuf Technol (2014) 72:47–56

Page 5: VREM: An advanced virtual environment for micro assembly An advanced virtua… · the virtual environment is the design of the target part to be assembled along with any specified

driven interface. The main interface screen first provides anoverview of the capabilities of VREM as well as how tointeract with it. A set of tutorials also guides new users. Theassembly user interface provides multiple options for a user tosimulate and study a target assembly plan. As indicated earli-er, users can choose to use the manual option (to enter anassembly sequence) or can use the automated assembly se-quence generator. Under the manual input category, users caneither use the basic option (where they can jog the variousmicro positioners (see Fig. 4) as well as grasp/release parts) oruse the advanced assembly input procedures.

There are two assembly input options:(a) A user can input (or manually enter) a set of coordinates

including the location of part holders (fixtures, etc.) as well asthe final destinations. This can be directly entered on the screen.

(b) Users with more experience can input a candidateassembly plan as a data file. In a team-based concurrentengineering context, multiple engineers can study a target partto be assembled and propose candidate alternatives. The partinteraction module facilitates bringing in a desired set of CADmodels relevant to a given assembly.

Automated assembly sequence generator module The assem-bly generator module plays a key role in VREM.GAs are usedto optimize the assembly sequences involving assembly ofmicro devices. Two types of genetic operators are used togenerate new child links: they are crossover and inversionoperators. Inputs to this module are the location of part feedersand destination locations of the micro devices.

The main steps for the GA-based approach are the following:First, generate n child sequences from n random parent se-quences (where n is the number of parts to be assembled).Genetic operators such as crossover (60 %) and mutation(40 %) are used in generating the new candidate or childsequences. The objective function is the traveling distance ofthe gripper for a candidate assembly sequence (this includesdistance from home to the part feeder, picking up target part, and

then completing placement or assembly task; this is repeated forall parts in an assembly sequence). For each iteration, (based onthe lowest traveling distance of the robot gripper to accomplishthe micro assembly activities), the best n child sequences areselected. As before, genetic operators are applied to this set ofchild sequences and the most feasible child sequences are iden-tified based on traveling distance. This process is repeated untilthere is no significant decrease in the traveling distances of anew (or child sequence) compared to the parent sequence. Thisfinal sequence can be generated automatically and is then sim-ulated in the virtual reality environment.

Collision detection module The function of this module is todetect potential collisions when a candidate assembly plan isproposed. There are numerous collision detection approachesproposed in the literature [1]. Our approach uses a basicenclosing box approach where collisions are detected betweenrelevant enclosing boxes of active or passive entities in thevirtual environment during assembly planning activities; al-ternate paths are generated based on such detection of colli-sions for a given assembly plan.

An assembly plan specifies a sequence of target parts, theirstart and final destinations, as well as detailed 3D path coor-dinates (for the various micro positioners to move, the gripperpick/insert/release tasks, etc.). This plan is input to the visual-ization engine where the assembly tasks are simulated. After afinal assembly plan is identified, the sequence is input to thecommand generation module.

Command generation module This module is responsible forgenerating the physical micro robotic commandswhich can bedownloaded to the physical assembly work cell to complete atarget assembly. There are three categories of commands:

1. The linear move instructions (from one point to another):this is used for the micro positioner’s movement to helptransfer a part from pickup positions to destinations.

Fig. 4 Multiple views in VREM

Int J Adv Manuf Technol (2014) 72:47–56 51

Page 6: VREM: An advanced virtual environment for micro assembly An advanced virtua… · the virtual environment is the design of the target part to be assembled along with any specified

2. The rotational move instructions (for rotating the workarea surface as needed). This enables avoiding obstaclesand improving access to a pickup or destination pointwhile reducing assembly completion time.

3. The gripper open and close instructions (for picking upand placing target parts in destinations).

For the physical assembly, based on the selected assemblyplan, a detailed set of instructions are generated. An exampleof an assembly program used by the advanced work cell isgiven in Fig. 5. In this program, a part moves to a micropin’s location, grips that pin, moves to another destina-tion, and then places (inserts) that pin in target destina-tion. It then picks up the same pin and returns it tooriginal position. The detailed commands appear on theleft side with a brief description of each command onthe right side of each command.

Fig. 6 is sequence diagram showing a summary of the maininformation / data exchanges between the various collaborat-ing components in VREM.

4 Experiments, test cases, and discussion

Using VREM and the physical work cell, a number of manip-ulation and assembly tasks were completed. Several compos-ite part designs were created to demonstrate the feasibility ofVREM for physical micro assembly tasks.

The first set of examples involved assembly involving pickand place as well as insertion tasks involving assortment ofparts such as gold pin and gears (see Fig. 7). The compositepart shown in the figure is a black substrate which has anumber of holes where target parts can be placed or inserted.

Fig. 5 Physical micro assemblycommands used during assembly

Fig. 6 A sequence diagramillustrating the overall flow ofactivities using VREM

52 Int J Adv Manuf Technol (2014) 72:47–56

Page 7: VREM: An advanced virtual environment for micro assembly An advanced virtua… · the virtual environment is the design of the target part to be assembled along with any specified

In this example, the manipulation involves a gear and a goldpin. The gear is inserted into the target hole location (thediameter of the gear shaft is 400 μm); the gold pin is placedin the destination with the gripper holding the ring-shapedhead (the pin cylinder is 200 μm and the ring diameter seen inFig. 7 is 500 μm). As this involved only few parts, theassembly plan was input manually. More important than thesequence, a user can experiment with the angle of picking upthe pin and placing it in the target location using VREM. Thegear insertion was more time consuming as the gear shaft hadonly a 700-μm region (in height) which could be used by thegripper for the part handling and insertion activity.

The second set of experiments involved manipulation ofseveral micron pins and two sets of gears in a composite partinvolving precision assembly. As can be seen in Fig. 8, theassembly task involved the insertion of a micro pin and a gear(gear 1) into target locations which were in between microfibers. Gear 1 (made of plastic with a shaft diameter of300 μm) was inserted into a target hold between the whiteplastic fibers. The steel micro pins were 70 μm in diameter.One of them was inserted between the plastic fibers while theother two were placed in target holes in a wax substrate(shown in Fig. 8).

The third set of experiments involved both micro and mesoscale part manipulation on a gold wire micro mesh (Figs. 9and 10). This experiment involved manipulation ofmeso/micro part designs where some of the target parts maybe in the meso scale while others are in the micro scale range.While there is no universally accepted definition of meso

assembly, in this paper, we use the description from [26] inwhich meso scale is described as including part sizes greaterthan 1 mm, with accuracies greater than 25 μm. The targetmanipulation involves assembling meso/micro devices(“stacked”) on a micro cylindrical pin; a view of the complet-ed virtual assembly in VREM is shown in Fig. 9. The physicalassembly sequence is shown in Fig. 10 (images i–v). A micropin (70-μm diameter) is first precisely inserted into a targetlocation on a micro scale mesh (which is made of gold wireand is 180×180-μm square). Subsequently, an aluminumbracket-shaped sieve with multiple holes (which was used asa composite sieve for various assembly scenarios in our lab) isplaced over this micron pin; the locating hole feature in thissieve has a diameter of 400 μm (see image i in Fig. 10); thebracket is positioned so that the micro pin goes through thislocating hole. The bracket is 800 mm in length and 300 μm inthickness. Then, a gear is picked up by the gripper and placedover one of the bracket’s locating holes ensuring that the70-μm diameter pin passes through the gear’s hollow shaftwhich has an internal hole diameter of 100 μm (see image iiand image iii, Fig. 10). Finally, a hexagonal plate is placedover the gear shaft (the width of hexagon is nearly 1 mm withinternal locating hole of 150 μm) as shown in image iv and vof Fig. 10. Part v in Fig. 10 shows the top view of thiscompleted assembly.

These examples demonstrate the feasibility of using such avirtual environment for micro assembly activities. The virtualenvironment provides a 3D scenario where candidate assem-bly plans can be generated and evaluated virtually. The virtualenvironment can detect problems with infeasible assemblyplans including collision with obstacles and reachability ofthe grippers to access a specific destination as well as allowinga user to compare candidate assembly alternatives. It can alsoestimate the assembly time as well as allow a user to introducea new gripper configuration, changing the work cell layout,and introducing new fixtures, among other aspects of theassembly process design context.

The virtual environment-based approach outlined in thispaper has limitations in its ability to predict certain problemsduring gripping due to the presence of the adhesive forces

Fig. 7 Virtual and physicalassembly views of a target partinvolving a bracket, gear and pin

Fig. 8 Assembly involving micro pins and gears

Int J Adv Manuf Technol (2014) 72:47–56 53

Page 8: VREM: An advanced virtual environment for micro assembly An advanced virtua… · the virtual environment is the design of the target part to be assembled along with any specified

coming into play. For example, a micro object can stick to thegripping surface or gripper after it is released (due to theadhesive forces coming into play between the gripper andthe micro part or object). Research investigations are continu-ing to address this ability to compute the adhesive forcescoming into play during micro manipulation and then simu-lating the impact of these forces which can negatively affectthe successful completion of target micro assembly tasks.regarding an accurate manner to calculate the adhesive forces(van der Waals, electrostatic and surface tension) coming intoplay between a candidate gripping surface and a target part tobe manipulated [3]. We are building on our past work dealingwith this key research area [3, 27]; one of the long term goalsof the Information Centric Engineering (ICE) framework is todevelop a simulation capability where such gripping relatedproblems (among others) are predicted and minimized in the

virtual environment prior to physical assembly; such a simu-lation capability will enable users to analyze the feasibility ofusing a gripper or gripping approach (with a view towardspreventing parts to stick to the gripper, apart from being ableto compare candidate gripping approaches and other assemblyactivities).

Currently, the physical environment discussed earlier (Fig. 2)provides feedback to the customer through the cyber physicalintegration module which includes status of the assembly taskscompleted (including video camera feedback to the users of thework in progress and completed assembly activities) and prob-lems encountered during assembly. Research is also continuingon investigating distributed collaboration through the Internetwhere customers at different locations can use a “cloud”-basedframework to analyze assembly alternatives, conduct simula-tions, as well as monitor assembly activities [37]; such ap-proaches will lay the foundation for the next generation of cyberintensive collaborative frameworks which can be termed as‘cyber physical frameworks’; such cyber physical frameworksand approaches will greatly benefit from innovative networkingtechnologies which are being investigated by Future Internetinitiatives such as Global Environment for Network Innovation[31, 33, 34] and US Ignite [32, 35, 37].

The virtual environment created is also being used to teachmicro assembly concepts and principles to undergraduate andgraduate engineering students at Oklahoma State University[35, 36]. Virtual Learning Environments (VLEs) are VirtualReality based environments used to teach engineering andother STEM topics at both the university and K-12 levels.

Fig. 10 Meso and micro part manipulation and assembly involving micro mesh, pin, plate, and gear

Fig. 9 Close-up view of the completed virtual assembly in VREM

54 Int J Adv Manuf Technol (2014) 72:47–56

Page 9: VREM: An advanced virtual environment for micro assembly An advanced virtua… · the virtual environment is the design of the target part to be assembled along with any specified

VLEs have been created to teach micro assembly concepts toengineering students at both the undergraduate and graduatelevels at Oklahoma State University [35, 36]. They are alsobeing used as part of a successful outreach program targetingK-12 school students to encourage them towards engineeringand science careers [35, 36].

5 Conclusion

Micro assembly is an evolving field of research with signifi-cant economic potential. The approach discussed in this paperalong with the VREM environment can be used to proposeand study assembly alternatives prior to physical assembly.Such a virtual environment can be used to interface withphysical micro assembly devices, grippers, and other equip-ment. The automated assembly sequence generator uses ge-netic algorithms to produce assembly sequences. Subsequent-ly, after an assembly plan is identified, the correspondingphysical commands for a target assembly can be producedby a command generator module. These commands can bedownloaded to physical micro assembly resources where tar-get assembly tasks can be completed. A variety of parts havebeen assembled using VREM to demonstrate its feasibility.

With the advent of cyber technologies, the use of virtualenvironments assumes importance as it enables various assem-bly and process scenarios to be studied virtually prior to physicalassembly. When coupled with web based and other cyber tech-nologies, such cyber intensive approaches hold the potential tousher in a new era of manufacturing collaborations using dis-tributed resources and tools. Research activities are continuinginvolving extending the capabilities of VREM to enable agreater degree of cyber collaborations involving distributedteams of engineers.With the development of the next generationof Internets [31, 32], the approach outlined in this paper holdsthe potential to evolve into advanced cyber physical approachesand frameworks which are more agile in responding to globallychanging customer requirements. holds the potential to becomepart of cyber physical frameworks which support cyber collab-orations for global design—manufacturing contexts.

Acknowledgments This research work was supported through grantsfrom the National Science Foundation (NSF Grants 0965153, 0951421,1032359, and 1256431), Sandia National Laboratories and Los AlamosNational Laboratory. Other grants were provided by Oklahoma StateUniversity (OSU) including an Interdisciplinary Grants program fromthe Office of Provost at OSU.

References

1. Lozano-Perez T, Wesley M (1979) An algorithm for planningcollision-free paths among polyhedral obstacles. Commun ACM22:560–570

2. Cecil J, Vasquez D (2005) A review of gripping and manipulationtechniques for micro assembly applications. Int J Prod Res 43(4):819–828

3. Vasquez D, Cecil J (2005) Investigation of van der Waals Forces inthe assembly of micro devices, Proceedings of the 2005 IndustrialEngineering Research Conference, Atlanta, GA, May 14–18

4. Böhringer KF, Ronald S, Fearing RS, Ken Y, Goldberg KY (1999)Microassembly. In: Shimon Nof (ed) The Handbook of industrialrobotics, vol 2E. John Wiley & Sons pp. 1045–1066

5. Greminger M, Yang G, Nelson B (2002) Sensing nanonewton levelforces by visually tracking structural deformations. Proceedings ofthe 2002 IEEE International Conference on Robotics andAutomation (ICRA) 2:1943–1948.

6. Rizzi AA, Gowdy J, Hollis RL (2001) Distributed coordination inmodular precision assembly systems. Int J Robot Res 20(10):819–838

7. Cecil J, Information Centric Engineering (ICE) Frameworks foradvanced manufacturing enterprises, proceedings of industry appli-cations and standard initiatives for cooperative information systemsfor interoperable infrastructure, OnTheMove (OTM) Conferences,Sept 10, 2013, Graz, Austria, pp. 47–56.

8. Thompson JA, RS Fearing RS (2001) Automating microassemblywith ortho-tweezers and force sensing, IROS 2001, Maui, HI, Oct.29-Nov. 3. pp 1327–1334

9. Palaria A, Enikov ET (2006) Experimental analysis of the stability ofelectrostatic bits for assisted nano-assembly. J Electrost 64:1–9

10. Piybongkarn D, Sun Y, Rajamani R, Sezen AS, Nelson BJ (2005)Travel range extension for aMEMS electrostatic microactuator. IEEETrans Control Syst Technol 13(1):138–145

11. Gorman JJ, Dagalakis N G(2003) Force control of linear motor stagesfor micro assembly, Proceedings of IMECE’03, Washington DC, pp.615–623

12. Popa D, Kang B, Sin J, Zou J (2002) Reconfigurable micro-assemblysystem for photonics applications. Proceedings of the IEEEInternational conference on Robotics and Automation, Washington,DC, pp 1495–1502

13. Cassier C, Ferreira A, and Hirai S (2002) Combination of visionservoing techniques and VR based simulation for semi-autonomousmicro assembly workstation, Proceedings of the 2002 I.E.International Conference on Robotics and Automation. Washington,DC: pp. 1501–1506

14. Monferrer A., Bonyuet D, Cooperative robot teleoperation throughvirtual reality interfaces, first international symposium on collabora-tive information visualization environments, July 2002, London, UK,pp. 243–248

15. Alex J, Vikramaditya B, & Nelson, B (1998) A virtual realityteleoperator interface for assembly of hybrid MEMS prototypes,Proceedings of DETC’98 ASME Design Engineering TechnicalConference, September 13–16, Atlanta, GA

16. Ferreira A, Hamdi M (2004) Microassembly planning using physi-cally based models in virtual environment. Proceedings of the 2004International Conference on Intelligent Robots and Systems. IEEE 4:3369–3374

17. Chang RJ, Lin CY, Lin PS (2011) Visual-based automation of peg-in-hole microassembly process. ASME J Manuf Sci Eng 133:1–12

18. Luo Q, & Xiao J (2006) Haptic simulation for micro/nano-scaleoptical fiber assembly, Proceedings of the 2006 I.E. InternationalConference on Intelligent Robots and Systems, pp. 1353–1358

19. Probst M, Hürzeler C, Borer R, Nelson BJ (2009) A microassemblysystem for the flexible assembly of hybrid robotic MEMS devices.International Journal of Optomechatronics 3(2):69–90

20. Sun L, Tan F, Rong W, Zhu J (2005) A collision detection approachin virtual environment of micromanipulation robot. High TechnolLett 11:371–376

21. Tan FS, Sun LN, Rong BW, Zhu J, Xu L (2004) Modeling ofmicromanipulation robot in virtual environment. Acta metallurgicasinica (English Letters) 17:194–198

Int J Adv Manuf Technol (2014) 72:47–56 55

Page 10: VREM: An advanced virtual environment for micro assembly An advanced virtua… · the virtual environment is the design of the target part to be assembled along with any specified

22. SulzmannA, Breguet JM, Jacot J (1995)Microvision system (MVS):a 3D computer graphic-based microrobot telemanipulation and posi-tion feedback by vision, 1995. Proc SPIE Microrobotics MechanicalSyst 2593:38–49

23. Cecil J, Powell D, Gobinath N (2007) Micro devices assembly usingvirtual environments. J Intell Manuf 18:361–369

24. Cecil J, Gobinath N (2005) Development of a virtual and physicalcell to assemble micro devices, Special Issue of the Journal ofRobotics and CIM, pp. 431–441

25. Hollis R, Gowdy J., Miniature factories for precision assembly,Proceedings of International Workshop on Micro-Factories,Tsukuba, Japan. December 1998. 99, pp. 1–6

26. Popa D, StephanouH (2004)Micro and meso scale robotic assembly.SMEJ Manuf Process 6(1):52–71

27. Zhang, L., Cecil, J., Vasquez, D., Jones, J., Garner, B., Modeling ofvan der Waals forces during the assembly of micro devices, the2006 I.E. Conference on Automation Science and Engineering(CASE) Conference, October 8–10, Shanghai, China

28. Cecil J, Huber J (2010) Virtual prototyping in engineering, chapter 1,virtual engineering. Momentum Press, New Jersey, pp 1–15

29. Muthaiyan, A., Cecil, J., Avirtual environment for satellite assembly,computer aided design and application, Vol.5, Nos.1-4, 2008

30. Cecil J, Trivedi A, Jones J (2006) An information based manufacturingframework for nanomanipulation, World Scientific and EngineeringAcademy and Society. Transs Info Sci Appl 3(2):462–470

31. Global environments for network innovation, GENI, www.geni.net

32. US Ignite, What is US Ignite, http://us-ignite.org/33. Cecil J, Ramanathan P, et al (2013) Collaborative virtual environ-

ments for orthopedic surgery, Proceedings of the 9th annual IEEEInternational Conference on Automation Science and Engineering(IEEE CASE 2013), August 17 to 21, Madison, WI

34. Cecil J, Gunda R, Calyam P, Seetharam S (2013) A Next generationcollaborative framework for advanced manufacturing, Proceedingsof the 9th annual IEEE International Conference on AutomationScience and Engineering (IEEE CASE 2013), August 17 to 21,Madison, WI

35. Cecil J, Ramanathan P, Mwarumba M (2013) Virtual learning envi-ronments in engineering and STEM education, IEEE Frontiers inEducation Conference, Oklahoma City, Oct 23–26

36. Center for Information Centric Engineering (CICE), http://www.okstate.edu/cinbm/outreach.htm and http://www.okstate.edu/cinbm/education.htm.

37. Cecil, J., Cloud computing for collaborative advanced manufacturinghttps://mozillaignite.org/apps/415/ Accessed Jan 2013

38. NarayanasamyG, Cecil J, Son TC (2006) A collaborative frameworkto realize virtual enterprises using 3APL. Declarative AgentLanguages and Technologies IV, Lecture Notes in ArtificialIntelligence. Springer Berlin Heidelberg pp. 191-206.

39. Cecil J et al (2004). VIRAM: AVirtual Reality Environment for theassembly of micro devices. Proceedings of the ASME 2004Computers and Information in Engineering Conference, Sept. 28–Oct. 2, 2004 Salt Lake City, Utah, USA, 4:671–682

56 Int J Adv Manuf Technol (2014) 72:47–56