![Page 1: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/1.jpg)
Distributed QoS Evaluation for Real-World Web Services
Zibin Zheng, Yilei Zhang, and Michael R. [email protected]
July 07, 2010
Department of Computer Science & EngineeringThe Chinese University of Hong Kong
Hong Kong, China
ICWS 2010, Miami, Florida, USAJuly 05 - 10, 2010
![Page 2: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/2.jpg)
2
Outlines
• Introduction
• Distributed Evaluation of Web Services
• How to Use the Datasets
• Conclusion and Future Work
![Page 3: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/3.jpg)
1. Introduction
![Page 4: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/4.jpg)
44
IntroductionWeb applications are becoming more and more important!
![Page 5: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/5.jpg)
55
Introduction• The age of Web 2.0
– Web pages and Web services
• Web services are Web APIs that can be accessed over a network and executed on remote systems.– Open standards– Interoperability
![Page 6: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/6.jpg)
6
Quality-of-Service
• Quality-of-Service (QoS): Non-functional performance. – User-independent QoS properties.
• price, popularity.• No need for evaluation
– User-dependent QoS properties. • failure probability, response time, throughput.• Different users receive different performance.
![Page 7: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/7.jpg)
7
QoS-Driven Approaches
• Web service selection
• Web service composition
• Fault tolerant Web services
• Web service ranking
• Web service recommendation
Limited real-world Web service QoS datasets for experimental studies
![Page 8: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/8.jpg)
8
Real-world Web Service Evaluation
• Obtain 21,358 publicly available Web services from the Internet.
• WS invocation and evaluation:– 235,262,555 lines of Java codes.
• Two large-scale distributed evaluations are conducted and first hand experiences are provided.– Dataset 1: 150 users * 100 Web services– Dataset 2: 339 users * 5825 Web services
![Page 9: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/9.jpg)
2. Distributed Evaluation of Web services
![Page 10: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/10.jpg)
10
Location Information
• 21,358 Web services from 89 countries. • The top 3 countries provide 55.5% of the obtained Web s
ervices.– United States: 8867 Web services, – United Kingdom: 1657 Web services, – Germany: 1246 Web services
![Page 11: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/11.jpg)
11
Obtaining Web Service Address
• Web service portals or directories – xmethods.net, – webservicex.net,
– webservicelist.com,
• Web service searching engines– seekda.com, – esynaps.com,
• Obtain 21,358 addresses of WSDL files
![Page 12: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/12.jpg)
12
WSDL File Infomation
![Page 13: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/13.jpg)
13
Java Code Generation
• Axis 2 to generate Java codes for 13,108 Web services. • Totally 235,262,555 lines of Java codes are produced.
![Page 14: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/14.jpg)
14
Dataset 1: Failure Probability
![Page 15: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/15.jpg)
15
Overall Performance
• Mean of failure probability
• Standard deviation of failure probability
![Page 16: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/16.jpg)
16
Overall Performance
• Average failure probabilities of all of the 100 Web
services and all the 150 service users are larger th
an 0.
• The standard deviation first becomes larger with th
e increase of mean and begins to decrease after a
certain threshold.
• The standard deviations are large, indicating that p
erformance of the same Web service observed by
different service users can vary widely.
![Page 17: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/17.jpg)
17
Failure Types(1) Web service invocations can fail easily.
(2) Providing reliable Web services is not enough for building reliable service-oriented system.
(3) The Web service invocation failures are unavoidable in the unpredictable Internet environment; therefore, service fault tolerance approaches are becoming important.
(4) Service fault tolerance mechanisms should be developed at the client-side.
![Page 18: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/18.jpg)
18
Dataset 2: Response-time & Throughput
• Each service user makes one invocation on each Web services.
![Page 19: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/19.jpg)
19
Overall Response-Time• Web services with poor average response time performance tend to have la
rge performance variance to different users.
• Large response time of a Web service can be caused by the long data transferring time or the long request processing time at the server-side.
• Influenced by the client-side network conditions, different service users observe quite different average response time performance on the same Web services.
![Page 20: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/20.jpg)
20
Overall Throughput
• Influenced by the poor server-side network conditions, there is a small part of Web services providing very poor average throughput performance (<1 kbps).
• Service users with large average throughput values are more likely to observe large throughput performance variance on different Web services.
![Page 21: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/21.jpg)
3. How to Use the Datasets?
![Page 22: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/22.jpg)
22
Web Service Recommendation
Zibin Zheng, Hao Ma, Michael R. Lyu, Irwin King, “WSRec: A Collaborative Filtering based Web Service Recommender System”, ICWS2009.
![Page 23: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/23.jpg)
23
Fault Tolerant Web Services
Zibin Zheng and Michael Lyu, “A QoS-Aware Fault Tolerant Middleware for Dependable Service Composition ”, DSN2009.
Global constraint: Success-rate of the whole service plan > 99%.
Stateless Web services
Stateful Web services
![Page 24: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/24.jpg)
24
More Research on the Datasets
• Web service selection.
• Web service search.
• Web service ranking.
• Other QoS-driven approaches of Web services.
![Page 25: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/25.jpg)
4. Conclusion and Future Work
![Page 26: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/26.jpg)
26
Conclusion and Future Work
ConclusionDistributed evaluation of Web servicesDataset 1: 150 * 100, failure probabilityDataset 2: 339 * 5825, response-time and throughput
Future work Investigating more QoS properties Incentive mechanisms for collecting user dataEvaluating more Web services from more locations
![Page 27: Distributed QoS Evaluation for Real- World Web Services Zibin Zheng, Yilei Zhang, and Michael R. Lyu lyu @cse.cuhk.edu.hk July 07, 2010 Department of Computer](https://reader036.vdocument.in/reader036/viewer/2022062314/56649e8a5503460f94b8f046/html5/thumbnails/27.jpg)
Thank you!
Web service QoS datasets: http://www.wsdream.net