25 novembre 2017 | 9 AM - 5 PM
Cap-Haitien, Haiti.
Open Source Software and Open DataThe way to move ahead
Dr. Waqar [email protected]
Al-Khawarizmi Institute of Computer Science UET, Lahore
Al-Khawarizmi Institute of Computer Science UET, Lahore
“Share this code with your fellow users. Learn from it. Improve upon it. And when you're done, please give something back to the community”
- Richard Stallman Kidd, father of free software, GNU Initiator
• The open-source movement was started to spread the idea of open-source software. Programmers who support the open-sourcemovement philosophy contribute to the open-source communityvoluntarily.
• The goal is to promote the production of "high quality programs" aswell as "working cooperatively with other similarly minded people" toimprove technology.
Open Source Initiative
Al-Khawarizmi Institute of Computer Science UET, Lahore
History of Open Source Software (OSS)• 1960's Bundled software with IBM • 1976 The original EMACS was a set of Editor MACroS for the TECO editor written by Richard Stallman • 1980’s Stallman founded the GNU project to write a complete open source operating system. • 1984 X originated at MIT. The X Org foundation leads. • 1987 Perl, the dynamic programming language was created and released by Larry Wall • 1980's – 1990's early development continues • 1990's late Acceptance of open source products • 1991-1992 final version Linux kernel; creation GNU/Linux
• 1995- Release of MySql• 1997 GNOME project leaders Miguel de lcaza and Federico Mena• 1998 Liberation of Netscape; Mozilla web browser officially launched• 1998 term “Open Source Software” replaces “Open Software”.• 2003 development of open source CMS such as WordPress
• 2008 Google released Chrome, an open source browser• 2013 Open source libraries for Deep Learning
• 2016 Microsoft joins Linux foundation
Al-Khawarizmi Institute of Computer Science UET, Lahore
Open Source Adoption Trends
Al-Khawarizmi Institute of Computer Science UET, Lahore
25 novembre 2017 | 9 AM - 5 PM
Cap-Haitien, Haiti.
Market Trends
Al-Khawarizmi Institute of Computer Science UET, Lahore
Developing a Commercially Viable Product with OSS
Identify Business Need
Research & Customization
Commercialization
“Without open source, Facebook, Google, Amazon and nearly every other modern technology company would not exist” (Andreessen-
Horowitz – leading OSS firm, 2016)
Al-Khawarizmi Institute of Computer Science UET, Lahore
Global Financial Benefits of OSS
• Savings of over $60 billion per year
• The Android OS, which runs on over 80 % of all smartphones in the world, is based on the open source operating system Linux.
• As of 2016, open source software served 75 % of the top 10,000 websites on the Internet.
• Open source has been embraced by for-profit businesses
– 78 % of companies use open source software substantially, only 3 % don’t use open source software
Al-Khawarizmi Institute of Computer Science UET, Lahore
• IBM – Apache SystemML
– Robotic calculations and inference agent– Correlate documents– Detect change points in IoT sensor data– …
• Google– Chrome– TensorFlow– Google Maps– …
• Facebook• Infer (Identify bugs in code)• Redex (Byte code optimizer for android)
Open Source Empower Corporations ?
Al-Khawarizmi Institute of Computer Science UET, Lahore
Microsoft Adapters for OS
https://opensource.microsoft.com/
• Open Source Growth has compelled Microsoft to develop adapters and repositories in the form of applications to connect Microsoft applications with Open Source Systems
• Microsoft would not survive if it did not take this step.
Al-Khawarizmi Institute of Computer Science UET, Lahore
Benefits of Open Source
● FREE, Secure, Scalable
● Customizable (Software and OS)
● Simple and lightweight software
● More learning opportunity for students
● Opportunities for researchers, entrepreneurs and industry
Al-Khawarizmi Institute of Computer Science UET, LahoreAl-Khawarizmi Institute of Computer
Science UET, Lahore
A Brief Comparison
Open-Source
● Free
● Free source available
● Better performance
● Secure
● Difficult installation & configuration
● Simple & Rigid
● Configure from scratch
● Google for support
● Poor documentation or missing
Propriety
Licensed
Source not available
Relatively slow
Prone to threats
Relatively easy installation & configuration
Appealing & Attractive
Pre-configuration
Support by vendor
Properly documented
Al-Khawarizmi Institute of Computer Science UET, Lahore
Al-Khawarizmi Institute of Computer Science UET, Lahore
KICS Contributions Towards OSS
• Research Labs in house developments in OSS
• Releases in Open Source
• Short Trainings
• ICOSST
• Open Data Contributions
– CVML, CLE, AERIL
Al-Khawarizmi Institute of Computer Science UET, Lahore
KICS Open Source Contributions
High Performance and Computer Lab
• Characterization, Evaluation, and Development of High Performance Network Services on Multi-Core Architectures.
• Multicore Processor Architecture and Communication (MPAC) library release to open source
• Urdu Search Engine (USE) development on open tools• Runtime Parallelization of Sequential Binary Code through JVM• Development of Type-II Hypervisor for MIPS64 Based Systems• MIPS64 - System Mode Emulation in QEMU• Analysis of Virtual Machine Scheduling
Al-Khawarizmi Institute of Computer Science UET, Lahore
KICS Open Source ContributionsCenter for Language Engineering (CLE)
Localization • OSS Localization
– Windows Installer for Urdu NVu (Open Source Web Authoring System)
– Urdu Localization Terminology (For localization of OSS)
• Keyboards– CLE Urdu Mobile Keyboard
– CLE Urdu Keyboard for Android Based Devices
– CRULP Urdu Phonetic Keyboards
– CRULP BTK Phonetic Keyboards
Language Processing– Word Level Processing (word sense tagging, segmentation, spell checker ..)
– Phrase Level Processing (English to Urdu machine translator, Urdu Text to Speech system, Screen Reader etc..)
Al-Khawarizmi Institute of Computer Science UET, Lahore
Open Source System : In-House Development
University Management System (UMS)• Using an open source framework Odoo, UMS Research Team
has developed a secure, robust, reliable, and professional University Management System, UMS
• This System covers OBE (Outcome Based Education) embraced by PEC (Pakistan Engineering Council) in 2014
• A first in the country in-house developed OBE based UMS on open tools
KICS ERP System• Odoo is also serving as KICS ERP system to manage our
resources like, accounting, purchases, stores, HR and salary.
Al-Khawarizmi Institute of Computer Science UET, Lahore
Open Source System : In-House Development
Open Source Content Management System• UET & KICS Websites (+70 sub-domains using OSS)• UET & KICS Profile Management System• Conferences and Workshops websites (+30 Websites using OSS)
All Above portals are based on Open Source Systems using following tools;– OpenCMS– CodeIgniter– Wordpress– Moodle – Joomla– Laravel– Drupal – Yii etc
Al-Khawarizmi Institute of Computer Science UET, Lahore
Open Source System : In-House Development
Library Management System using open sources Integrated Library System (ILS)
• UET Library Management System is developed in JavaEE using open-source tools KOHA and is deployed in complete open-source environment to provide a zero cost solution to public sector libraries
• Koha is a fully featured, scalable library management system. Development is sponsored by libraries of varying types and sizes, volunteers, and support companies worldwide.– An automated, secure and cost effective secure library management
system based on RFID (Radio Frequency Identification) technology.
• UET LMS integrates data for the books and journals in Urdu as well
Al-Khawarizmi Institute of Computer Science UET, Lahore
Open Source System : In-House Development
Other Applications developed using OSS– IPAS: Invoice processing and Automation System for Power Industry
– Wi-CAM: Auto Configuration Server for Wi-Tribe internet service provider
– uHands: Emergency Management System
– mKisan is An Interactive Advisory System for Farmers
– OMR-Based Solutions
– RFID-Based Vehicle Access Control and Parking Solution
– 4G TRx developed on Open Source GNU Radio Plat form
– 4G TRx also developed for Virginia Tech Open Radio Platform
Al-Khawarizmi Institute of Computer Science UET, Lahore
ICOSSTInternational Conference on Open Source Systems & Technologies for 11 years
Wi-CAM: Auto Configuration ServeruHands: Emergency Management SystemmKisan is An Interactive Advisory System
for Farmers
Al-Khawarizmi Institute of Computer Science UET, Lahore
Trainees trained on OSS
367
315
417
310
586603
NUMBER OF TRAINEES TARINED ON OPEN SOURCE TOOLS & TECHNOLOGIES
2012 2013 2014 2015 2016 2017
Al-Khawarizmi Institute of Computer Science UET, Lahore
1. Business Idea Competition
2. Xtreme Programming
3. Web Design Competition
4. Workshop – Bootstrap
5. Circuit Mania
21
BRAINIAC : IEEE Student Activities
Al-Khawarizmi Institute of Computer Science UET, Lahore
Case Study
● KICS employees triggered a change from Proprietary to Open Source Systems in 2015.
● Problem Statement: Shift from
Windows(OS) —› Ubuntu
Licensed Softwares —› Open Source Applications
Al-Khawarizmi Institute of Computer Science UET, Lahore
Office Applications
● Libre Office– Writer, Impress, Calc, Math, Draw, Base, CAD
● Tex Editor– TexMaker, lyx Document Processor
● PDF Reader– Acrobat Reader, Document Viewer(builtin)
● Project – Project Libre, Open Proj, Gantt Project
Al-Khawarizmi Institute of Computer Science UET, LahoreAl-Khawarizmi Institute of Computer
Science UET, Lahore
Windows Software Alternative Software
MS Office 2010 Libre Office
MS Project Project Libre, Open Project
MS Visio Libre Draw
MS Outlook Thunder Bird
Dreamweaver Aptana
Photoshop GimpShop
Corel Draw Inkscape, Xara
AutoCAD FreeCAD, Libre CAD
Macromedia Flash F4L (development phase)
Internet Explorer N/A
Safari N/A
Notepad++ Emac, gedit, geany
WinSCP FileZila
Active Directory LDAP & Samba server
Crystal Report iReport
MS SQL MySQL, MariaDB
Tortoise SVN Kde SVN
Drivers Most of drivers are Built-in
KICS Attendance Management System N/A
Al-Khawarizmi Institute of Computer Science UET, Lahore
Open Data
“Numerous scientists have pointed out the irony that right at the historical moment when we have the technologies to permit worldwide availability and
distributed process of scientific data, broadening collaboration and accelerating the pace and depth of discovery ... we are busy locking up that data and preventing the use of correspondingly advanced technologies on
knowledge.”
- John Wilbanks, VP Science, Creative Common
Al-Khawarizmi Institute of Computer Science UET, Lahore
Open Data• Open Data is:
– Freely available on the internet;
– Without financial, legal, or technical barriers
– Include both structured and unstructured formats
Al-Khawarizmi Institute of Computer Science UET, Lahore
Open Data in Science
• Accelerates the process of research and discovery by reuse and validate previous results.
– Human Genome Project (HGM)• All human genomic sequence information should be freely available and in the
public domain in order to encourage research and development and to maximize its benefit to society
– Pubmed• Abstracts of Biomedical scientific documents used for research in text mining
– Youtube DatasetData set comprising of youtube video segments focusing on research for face recognition, human activities identification, visual scene interpretation.
– CodaLab Platform• Microsoft initiative. Run reproducible machine learning experiments for data driven
research. Include all video, textual and audio data.
– Telemetry Data, Weather Data, Demographic Data, Geographic Data, Agriculture Data, Economic Data, Space exploration Data, etc..
Al-Khawarizmi Institute of Computer Science UET, Lahore
Open Data in Governance
• Facilitates government transparency, accountability and public participation
– Punjab School Monitoring• Real-time reporting with pictorial evidence, geo-tagging of sites
visited, and automatic SMS alerts.
– Punjab Child Vaccination• Free, Open and real time data about vaccination of Punjab.
– Judicial data
– Police Station crime and criminals related data
– Land Record data
– School Solarization Project with on-line monitoring and data availability
Al-Khawarizmi Institute of Computer Science UET, Lahore
Our Contribution to Scientific Open Datasets
• Computer Vision and Machine Learning (CVML)
• Center for Language Engineering (CLE)
• Bioinformatics Research Lab
• Alternate Energy Research and Innovation Lab
Al-Khawarizmi Institute of Computer Science UET, Lahore
Computer Vision and Machine Learning Lab (CVML)
• Data specific to the subcontinental conditions, i.e. Skin tone of people, clothing etc.
• Datasets for Face, Emotion and Gender classification.
• To find interaction between multiple/groups of human, Multi human activity dataset.
• Objects dataset which consists of objects from surveillance videos.
Al-Khawarizmi Institute of Computer Science UET, Lahore
Facial Datasets | CVML KICS
Face recognition dataset (varying camera angles, with & without glasses, beard etc.):
– Subjects : 4000 males, 600 females
• Clothes dataset:
– Total subjects: 200 males, 40 females
– Types of clothes: shalwar qameez, pants and dress shirt, wasket
• Role of Safe City Project?
Al-Khawarizmi Institute of Computer Science UET, Lahore
Center for Language Engineering (CLE)
Linguistic Resources
• Text Corpora (15+ Corpus )
• Image Corpora (45+ Corpus)
• Speech Corpora (10+ Corpus)
• Lexical Resources
– Urdu Wordnet
– Urdu Books N-Grams
Al-Khawarizmi Institute of Computer Science UET, Lahore
Bioinformatics Research Lab
• Brain Data(EEG): For 30 subjects
Set of 30 emotions + actions such
– Move left, right, straight, backward.
– Eye Blink, Eye movement
– Hand movements
– Head based movements etc.
• Local Biological Databank
– Local data additions in ChemSpider and PubChem
Al-Khawarizmi Institute of Computer Science UET, Lahore
Alternate Energy Research Lab
• Weather Station Data
– Parameters:
• Humidity, Air Temperature, Pressure, Wind Speed, Sun light (direct/indirect)
– Potential Benefits:
• Forecast energy generation
• Scheduled maintenance
Al-Khawarizmi Institute of Computer Science UET, Lahore
Way to move ahead!
• Adaption of OSS by Government and Private Sector.
• Release of local data for community specific problems solving.
• Students involvement in international Open Source initiatives
• Industry academia linkage for data sharing