Eager Computation and Lazy Propagation of Modifications for Reducing Synchronization Overhead in File Replication System

Manu Vardhan, Shailendra Verma, Priyank Bhatnagar, Dharmender Singh Kushwaha Department of Computer Science and Engineering

MNNIT Allahabad Allahabad, INDIA

{rcs1002, cs1020, cs084062, dsk}@mnnit.ac.in

Abstract— Files are typically replicated in a distributed system to improve the data access, response time, data availability and increase fault tolerance capability. Many researchers discusses about the applications that works on read only data, however many applications need to alter the data. If all replicas of a file are not synchronized this drives the system into inconsistent state. To keep system consistent, the changes made at one replica of a file should be reflected in other replicas in minimum time, this time is called synchronization overhead. Proposed approach reduces this overhead by using lazy update propagation and ordered storage of updates. Unlike conventional approaches that propagate the updates to a fixed master replica of a file, the proposed approach transfer the role of master replica to the last modified replica. This mechanism reduces the overhead of propagating the update messages to master replica immediately. The master replica computes the modification messages eagerly but propagates lazily. Experimental results shows how various factors viz., file size, size of modifications and number of replicas to be updated affects the time to propagate the changes to other replicas.

Keywords- Distributed Systems; Modification Propagation; Synchronization Overhead; Replication

I. INTRODUCTION Multiple copies of file engender the issue of confliction

between file copies; it is called inconsistency. To keep the replicated copies consistent, it is mandatory that all the copies of a file strictly have the same content at a time; it means changes made at a file must be reflected on other copies in zero time (immediately). It is practically impossible because of network delays. Basically proposed approach is based on two operators (�) and ( ). Operator (�) compares two files and writes the differences made in a Diff file (difference file) named D(f, seq_no, timestamp), where f is the name of file on which (� ) operator is applied, seq_no is for differencing multiple Diff files, and the timestamp shows when Diff file is created. The ( ) operator applies the modifications stored in Diff file on stale copy of file to update it.

If a file (f) have two or more replicas(r1, r2, r3…), and a replica (r1) get changed by a client than all read or write operations on other replicas (r2, r3,….) will put on hold until changes made in (r1) are reflected in other replicas. The proposed approach minimizes this reflection time. Instead of transferring the complete file (f) from (r1) to (r2, r3,…), proposed approach extracts modification on (r1) using (�) operator, and joins them on stale copy of (f1) available on

(r2, r3,...) using ( ) operator. As in most of the real world applications, the amount (size) of modifications done on a file is much lesser than the file size. So the total time of (�), propagation of modification file, and ( ) with stale replica is much lesser than complete file transfer. Proposed approach stores the Diff (D) files on master replica, so it needs not to be computed every time when Diff (D) file is needed.

Conventional approaches of modification propagation maintains a master replica, and any changes made in any other replica have to be propagated immediately to the master replica, this overhead is reduced in proposed approach by notifying all replicas that the last modified copy of (f) that is (r1) is the new master replica or new owner for file (f).

Rest of the paper is organized as follows. Section 2 discusses the related work that is followed by proposed approach in section 3. Section 4 presents the results and simulation, followed by conclusion in section 5.

II. RELATED WORK Primary-copy (master-slave) approach for updating the

replicas says that only one copy could be updated (the master), secondary copies are updated by the changes propagated from the master. There is only one replica which always has all the updates. Consequently the load of the primary copy is large. Domenici [1] discusses several replication and data consistency solutions, including Eager (Synchronous) replication and Lazy (Asynchronous) replication, Single-Master and Multi-Master Model, and pull-based and push-based. Author presented various replication and consistency maintenance algorithms to deal with huge scientific data. Jaechun [4] proposed two kinds of data replication techniques, called owner-initiated replication and client-initiated replication. Proposed replication techniques do not need to use file system-level locking functions, so they can easily be ported to any of file systems. Guy [5] proposed a replica modification approach, a replica is designated either a master or a secondary replica. Only master replica is allowed to be modified whereas secondary replica is treated as read-only, i.e. modification permission on secondary replica is denied. A secondary replica is updated in accordance with the master replica if master replica is modified. Sun [6] proposes two coherence protocols viz., lazy-copy and aggressive-copy. Replicas are only updated as needed, if someone accesses it in the lazy-copy based protocol. It can save network bandwidth resources without transferring up-to-date replicas every time

2012 Third International Conference on Computer and Communication Technology

978-0-7695-4872-2/12 $26.00 © 2012 IEEE

DOI 10.1109/ICCCT.2012.73

331

when some modifications are made. However, lazy-copy protocol has to pay the penalties for access delay when inter-site updating is required. For the aggressive-copy protocol, replicas are always updated immediately when the original file is modified. In other words, full consistency for replica is guaranteed in aggressive-copy, whereas partial consistency is applied to lazy-copy. Compared with lazy-copy, access delay time can be reduced by aggressive-copy based mechanism without suffering from long update time during each replica access. Nevertheless, full consistency with frequent replicas updates could exhaust considerable amount of network bandwidth resources. Furthermore, some updates may be invalid and inefficient because it is probable that it will never be used. Huang et al. [11] proposed the differentiated replication to improve accessing performance and replicas availability. They make effort on performance, availability and consistency. But maintenance consistency algorithm does not take storage capacity into account. Some replicas are not to be accessed for a long time by grid users, which will waste the free space of storage device. Dirk et al. [14] proposed a high-level replica consistency service, called Grid Consistency Service (GCS). The GCS allows updating file synchronization and consistency maintenance. The literature proposed several different consistency levels ranging from entirely synchronized data to loosely synchronized data. Gird users can choose different consistency services dynamically adjusting replicas consistency degree. Jiafu Hu et al. [19] proposed an asynchronous model, despite the system failure or network traffic congestion, this model avoids the replicas inconsistency in grid environment. The consistency problem mentioned in the literature could be classified into two kinds, one was the metadata replica consistency and the other one the data content consistency. Chang [21] proposes architecture called Adaptable Replica Consistency Service (ARCS) with the capability of dealing with the replica consistency problem. It has the better performance and load balance for file replication in Data Grids.

III. PROPOSED APPROACH Fig 1 shows a group of File Replication Servers (FRSs)

along with the Requesting Nodes (Ns). FRS maintains the information about the shared files stored on them. The figure represents the logical connections between FRSs and Ns. All FRSs are logically connected to each other. FRSs will communicate/exchange information with each other as and when required. Initially a node sends request to any FRS known to the requesting node (Ns) i.e. IP address of that FRS. When load on a FRS increases, it replicates the files on other FRSs, so that the response time to fulfill the request get reduced. Now, if a requesting node associated with a FRS modifies a file, these modifications need to be propagated to the FRS having its replica. This propagation can be done in two ways, either Complete File Transfer (CFT) or only the modifications. This paper focuses on the approach that minimizes the time to update the replicas.

A file (f1)’s replica1 (r1) denoted by (r1,f1) located at FRS1 is changed, and changes committed at time (t1) and a client reads another copy of that file on replica2, (r2,f1) located at (FRS2) than this operation is not permitted until

(r2,f1) is not replaced with most updated copy that is (r1,f1). It includes the time for file transfer from (r1) to (r2) through network. Until file copy (r1, f1) does not replace (r2, f1) on FRS2, all operations on (r2) on file (f1) will be put on hold. This holding period is called the stuck time/synchronization overhead denoted by (S t).

St = Pt + Tt Where, St= synchronization overhead Pt = propagation time= distance (r2-r1) / speed of signal Tt = transfer time= File Size / link speed Our goal is to minimize St, which is the sum of two

positive values Pt and Tt. Pt depends on the distance between replicas. In some contexts i.e. intranet, the replicas may be situated within premises, but in different Domain Name Server (DNS). This distance might be of thousand miles that depend on the type of application.

RN2

FRS1

RN1 RN8

FRS4

RN9

RN12FRS5

RN6

RN5FRS3

RN7

RN10

RN11

RN4

FRS2

RN3

Figure 1. FRS architecture.

Tt depends on the size of file to transfer. Instead of replacing stale replica of file f1 (r2, 1f ) with updated replica (r1) of file (f1) i.e. (r1,f1) located on FRS1 will extract only the changes made using the operator � ((r1,f1), (r1, 1f )). Here (r1, 1f ) is the stale replica of (f1) on FRS1. These changes will be stored in modification file denoted by D(f1, sequence_no, timestamp) & propagated to stale replica (r2) of file (f1) on FRS2. After applying these changes on stale copy (r2, 1f ) it will be updated to (r2,f1)

using join operation �((r2, 1f ), D(f1, sequence_no, timestamp)). Time of � operator is extraction time denoted

332

by t�, time to join the Diff file with stale file is t and time of propagation for D(f1,seq_no, timestamp) is tpD, so total time to reflect changes made on (r1,f1) in (r2, 1f ) is Trf.

Trf = t�+ tpD + t� Trf= total time to reflection of changes in (r2, 1f ) t�= time for extraction of modification content, and store

them in modification file (D) tpD= time for propagation of modification file D from

(r1) to (r2) t�= time for joining stale file ( 1f ) with modification

file (D) Tcft= time in Complete File Transfer (CFT) In case with one replica, if Trf < Tcft, proposed approach

is beneficial as compared to complete file transfer. If there are more than one replica of file, than for updating each replica (rn) where n >1, will take only tpD + t�, as the previously existing modification file (D) can be propagated to all the replicas.

In previous approaches every file has a primary replica and other replicas are considered as the secondary replica, this primary replica is called the master replica [13]. When a replica of file is updated on secondary replica, than primary copy have to be updated immediately. This approach need to wait for file writing completion on secondary copy and then transfer this updated file to the master copy. In proposed approach as soon as a replica get request for file writing it notifies other replicas that he is the new master replica. As all replicas knows that new master copy is the replica on which last write operation done. So there is no need to update any other replica immediately. Every FRS maintains a data structure given below in Table 1. The entries in data structure keep tracks of information, when a file was last time modified (tlw) and by which FRS as Owner FRS. Detailed working of read and write operations for a file replica on a FRS is given below:

TABLE I. DATA STRUCTURE MAINTAINED ON FRS

FileID OwnerID Last write time stamp

Modification File

f1………...................fn

FRS1…………..………………..FRSm

f1(tlw)……………....……………fn(tlw)

D(f1, 1, tf) .......................D(fn, n, tf)

Read Operations: When the node (Ni) associated with

FRS1 makes a request to FRS1 for read operation on file (fi), it first checks weather f is stale copy or not, by comparing the f’s timestamp (tf ) with f’s last write time (tlw) stored in data structure, if (tf < tlw ) it means FRS1 does not have the latest updated copy of file f. So first it sends a Req_modification_list message to owner FRS of (fi) with tf. In response of the Req_modification_list message, owner FRS of (fi), response back by sending all modification files D[fi, seque_no, timestamp], created after (tf). FRS1 will now apply join operation (�) on stale replica of (fi) and all received modification files D[fi, seq_no,timestamp] in increasing order of seq_no. After applying all these join operations, file fi (stale copy of fi) turns into an updated one.

Now the read operation can be performed on the updated version of the requested file (fi).

Write Operation: A node (Ni) associated with a FRS1 generates a request for write operation on file (fi). If FRS1 is owner of (fi), it waits for completion of write operations on that file if any, and then acquire write lock on (fi), and provides write permission. If FRS1 is not owner of that file it multicast a message for granting write permission on (fi) called RW(fi) message with f‘s timestamp (tf) to all FRSs. The receiving FRSs will make a entry in their data structure that shows, the new file owner of (fi) is FRS1. If the receiving FRS is not the owner FRS of (fi) it sends an ACK for write permission. If it is owner FRS of (fi), then it acknowledges with all modification files created after (tf) and a write permission ACK. FRS1 will then updates stale copy of (fi) using join operations. FRS1 acquire write lock on (fi) and gives write permission to node Ni.

IV. SIMULATION AND RESULTS The proposed model is simulated on Linux platform and

LAN of 10.0 Mbps. Proposed approach is compared with the conventional Complete File Transfer approach (CFT). Proposed approach outperforms the CFT in terms of time and size of data transfer to replace a stale replica of file with the modified one.

Figure 2. Time required to update replica with increasing file size

Fig. 2 shows that as the file size increases, time to update the replica with complete file increases linearly, whereas with the proposed approach, it is almost constant. As the ratio (file size : modification size) i.e. 1MB:100KB, 2MB:100KB, 3MB:100KB and 4MB:100KB increases, the percentage reduction in time to update the file also increases.

V. CONCLUSION This paper presents a mechanism to reduce the

synchronization time between multiple replicas of a file, using modification propagation approach and dynamically selecting the master replica. The master replica computes the file modifications eagerly but propagates them lazily and stores an ordered list of modifications. The simulation results shows that proposed approach gives a far better performance in terms of time and the benefits increases if the ratio (file size : modification size) increases, with varying file size.

REFERENCES [1] Andrea Domenici, Flavia Donno, Gianni Pucciani, Heinz

Stockinger, Kurt Stockinger, “Replica Consistency in a Data

333

Grid,” Nuclear Instruments and methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Volume 534, Issues 1-2, pp. 24-28, 21 November 2004.

[2] Jiang Lan, Xiaotao Liu, Prashant Shenoy and Krithi Ramamritham, “Consistency Maintenance In Peer-to-Peer File Sharing Networks”, Proceedings of the Third IEEE Workshop on Internet Applications, WIAPP pp. 90- 94, 23-24 June 2003.

[3] Donno, G. Pucciani, H. Stockinger "Relaxed data consistency with CONStanza", A. Domenici, F. Sixth IEEE International Symposium on Cluster Computing and the Grid, vol.1, 16-19 May 2006, pp.-429.

[4] J. No, et al. “Data Replication Techniques for Data-Intensive Applications”, International Conference on Computational Science (4) 2006: 1063-1070.

[5] L. Guy, P. Kunszt, E. Laure, H. Stockinger and K. Stockinger “Replica Management in Data Grids”, Technical report, GGF5 Working Draft, Edinburgh, Scotland, July 2002.

[6] Yuzhong Sun and Zhiwei Xu, “Grid Replication Coherence Protocol”, The 18th International Parallel and Distributed Processing Symposium (IPDPS'04) - Workshop, Santa Fe, USA, April 2004, pp.232-239.

[7] Jin Xiong; Jianyu Li, Rongfeng Tang, Yiming Hu,"Improving data availability for a cluster file system through replication," Parallel and Distributed Processing, 2008. IPDPS 2008. IEEE International Symposium on , 14-18 April 2008, pp.1-8.

[8] Watanabe, T., Kanzaki, A., Hara, T., Nishio, S., “A Selective Update Propagation Based on Degree of Data Update in Peer-to-Peer Networks," International Conference on Network-Based Information Systems, 19-21 Aug. 2009, pp.52-59.

[9] Ahmed, A., P. D. D. Dominic, and H. Ibrahim. 2008. “A binary hybrid replication strategy for improving availability and maintaining consistency of data in large scale mobile environments,” Paper read at Information Technology, ITSim 2008. International Symposium on, 26-28 Aug. 2008, vol.3, pp.1-9.

[10] K.V.K.Chaitanya, Smt. S.Vasundra “EARM: An Efficient and Adaptive File Replication with Consistency Maintenance in P2P Systems”, In International Journal of Electrical and Electronics Engineering (IJEEE), Interscience Open Access Journals, Vol. 1, Issue 1, Oct. 2011.

[11] Changqin Huang, Fuyin Xu, and Xiaoyong Hu, “Massive Data Oriented Replication Algorithms for Consistency Maintenance in Data Grids”, ICCS 2006, Part I, LNCS 3991, 2006, pp. 838-841.

[12] Haiying Shen, "IRM: Integrated File Replication and Consistency Maintenance in P2P System", Proceedings of 17th International Conference on Computer Communications and Networks, 3-7 Aug. 2008, pp.1-6.

[13] Yuanyuan Zhou , Liviu Iftode , Kai Li, “Performance Evaluation of Two HomeBased Lazy Release Consistency Protocols for Shared Virtual Memory Systems”, Proceedings of the Operating Systems Design and Implementation Symposium, 1996, pp. 75-88.

[14] Dirk Düllmann, Wolfgang Hoschek, Javier Jaen Martinez, Ben Segal, “Models for Replica Synchronisation and Consistency in a Data Grid”, Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing (HPDC-10’01), Oct. 2001, pp. 67.

[15] Nazanin Saadat, Amir Masoud Rahmani, PDDRA: A new pre-fetching based dynamic data replication algorithm in data grids, Future Generation Computer Systems, Volume 28, Issue 4, April 2012, pp. 666-681.

[16] R. Ladin, B. Liskov, L. Shrira, "Lazy replication: exploiting the semantics of distributed services ", Proceedings of Workshop on the Management of Replicated Data, 8-9 Nov 1990, pp.31-34.

[17] Chao-Tung Yang, Chun-Pin Fu, Chien-Jung Huang, Ching-Hsien Hsu,"FRCS: A File Replication and Consistency Service in Data Grids", International Conference on Multimedia and Ubiquitous Engineering, MUE 2008., 24-26 April 2008, pp.444-447.

[18] A. Mamat, M. Radi, M. Mat Deris , H. Ibrahim, "Performance of update propagation techniques for data grid," International Conference on Computer and Communication Engineering, ICCCE 2008, pp.332-335, 13-15 May 2008

[19] Jiafu Hu, Nong Xiao, Yingjie Zhao, and Wei Fu, “An Asynchronous Replica Consistency Model in Data Grid”, Parallel and distributed processing and applications (ISPA 2005 Workshops), 2005, pp. 475-484.

[20] Huang, Fuyin Xu and Xiaoyong Hu, “Massive Data Oriented Replication Algorithms for Consistency Maintenance in Data Grids”, Changqin Computational Science – ICCS, vol. 3991, 2006, pp. 838-841.

[21] Ruay-Shiung Chang and Jih-Sheng Chang, “Adaptable Replica Consistency Service for Data Grids”, Third International Conference on Information Technology: New Generations (ITNG’06), 2006, pp. 646-651.

334