newlug presentation- opensolaris

of 39 /39
04/15/2010 1 OpenSolaris Overview Ivan Richwalski [email protected]

Author: newlug

Post on 19-May-2015




0 download

Embed Size (px)


April 2010 NEWLUG (North East Wisconsin Linux Users Group) meeting. Subject: OpenSolaris


  • 1. 04/15/2010 1 OpenSolaris Overview Ivan Richwalski [email protected]

2. 04/15/20102 A brief history SunOS 1 - 4 Based on original BSD Sun 680x0 workstations Later added Sun SPARC AT&T & Sun collaboration in 1987 Merging the best parts to create SVR4 New operating system named Solaris 2 ( SunOS 4 retroacively named Solaris 1 ) Solaris brand to identify whole system OS, Graphics system, Desktop software 3. 04/15/20103A brief history 1989 AT&T forms Unix System Labs 1991 First Solaris 2 release 1993 AT&T sells USL to Novell 1994 Sun buys out license from Novell No more license/royalties, Sun owns the code Early 2004 OpenSolaris planning begins Late 2004 OpenSolaris pilot project starts 2005 Open to the public Dtrace source, website & lists, more code mid-year 2007 Ian Murdock joins Sun, project Indiana starts 4. 04/15/20104Solaris/OpenSolaris Today Supports SPARC, x86 CPUs Both 32 and 64 bit kernels SPARC systems 64 bit kernel only Mixed 32 & 64 bit programs on 64 bit kernel Highly scaleable 32 CPUs on 32 bit x86, 256 on 64 bit, 512 on SPARC 5. 04/15/20105 Licensing CDDL - Common Development & Distribution License Kernel, drivers, and most of userland programs OpenSolaris Binary License Not open source, but is freely redistributable Lots of other code with their own licences GPL, BSD, MIT, Apache, Perl Artistic, etc. 6. 04/15/2010 6 About the CDDL Comon Development & Distribution License Based on Mozilla Public License v1.1 File Based Allows combination with code that has other compatible licenses Requires modifications shared back Explicit patent license, patent peace clause 7. 04/15/20107CDDL Misconceptions Gives Sun ownership of your code You retain ownership/copyright of your code Seperate contributor agreement for including your source code in Sun's code base Designed to be GPL incompatible Compatible with lots of other licences File based, does not require everything to be under CDDL 8. 04/15/2010 8Building an operating system Many seperate consolidations Operating System/Networking ( aka ON ) Kernel, drivers, networking, libraries, services, & commands X11 Desktop Gnome, Firefox, Thunderbird SFW ( Sun Freeware, various open source software ) Globalization ( includes internationalization & localization ) Not all consolidations open sourced SPARC Graphics Drivers OpenWindows, CDE ( obsolete, removed ) 9. 04/15/2010 9SMF: Service Management Facility Framework for defining persistent system services Each service identified by FMRI( Fault Management Resource Identifier ) A service can have multiple instances 10. 04/15/2010 10SMF: Service Management Facility Configuration stored in repository Persistant, transaction based, safe rollback XML manifests used to import/export service Tracks dependencies, manages parallel startup 11. 04/15/201011SMF: Service Management Facility Restarters responsible for managing services New process attribute: contract Restarter alerted if contract becomes empty Decides what action to take 12. 04/15/2010 12SMF: Service Management Facility Long running & transient ( run once ) services Service manifest defines how service runs User/Group Credentials Process Privileges Working Directory Environment variables Project ID / Resource Pool Delegated administration of services Users assigned profiles for managing services [ demo 1 ] 13. 04/15/201013 Fault Management Daemon Manages diagnosis engines, response agents, logging Recieves telemetry from other parts of system Hardware error reports ( CPU, memory, IO bus bit errors ) Driver faults ( NIC reporting corrupted data ) Disk errors ( bad blocks, disk not responding ) Diagnoses problem, takes action when needed Offline/retire faulty CPUs & memory pages Reroute IO ( Multipath disks, aggregated NICs ) Activate disk hot spares Restart affected services via SMF Send alerts via syslog, SNMP 14. 04/15/2010 14 ZFS Pooled storage Does away with dealing with partitions/slices Eliminates extra volume manager layer Simplifies administration, improves performance Volume FilesystemFS Storage Pool 15. 04/15/2010 15 ZFS Pool built on one or more vdevs Stripe, Mirror, Raidz1, Raidz2, Raidz3 Able to add additional vdevs to a pool NFSv4/NTFS style ACLs Instant read-only snapshots Can be cloned to create new writeable filesystem Delegated administration Allow users & groups ability to manage own filesystems Fine grained control of allowed tasks Compression, deduplication In place upgrades for new capabilities 16. 04/15/2010 16ZFS Guaranteed data protection Checksums used end to end, catches & repairs corruption Transactional writes, always consistent state on disk Copy on write, never alters live data, no write hole Checksum Trees, checksum stored in parent block pointer Avoids corruption hitting both data & checksum together Validate entire pool by walking whole tree Background scrubbing & resilvering Maximum performance Writes spread across all devices in pool Most disk writes sequential due to copy on write/transactions Smart prefetch, detects length & striding reads 17. 04/15/2010 17 ZFS Self healing ( as long as there's redundancy ) Redundant copies of all filesystem metadata Copies of user data optional, setable on the flyAppAppAppMirror Mirror Mirror 18. 04/15/201018 ZFS & SSDs Seperate Intent Logs ( aka slogs ) Intent log used for synchronous writes Normally uses space from disks in pool Move to SSD for fast, short lived writes ( NFS, iSCSI ) Level 2 ARC ( aka L2ARC ) ARC == Adaptive Replacement Cache in memory L2ARC best for read optimized SSDs Faster than disk, slower & cheaper than RAM 19. 04/15/201019 ZFS vs btrfs 2001 start, 2005 release 2007 development start CDDL License GPL License Checksum: fletcher2, Checksum: crc32, off fletcher4, sha256, off Always copy-on-write Selective copy-on-write Device removal, defrag No fsck needed Online/offline fsck Rollback from snapshot 20. 04/15/201020DTrace Dynamic Tracing of kernel & userland code Simplified C-like D language Added functions and variables for tracing Thread-local variables Completely safe to run on production systems Zero probe effect when dtrace not running 21. 04/15/2010 21 DTrace Provider: syscall, pid, proc, io, fbt ( functional block trace ) Module: Process executable, library, driver Function Name: entry, return, connection-start, connection-done Applications can also have their own providers 22. 04/15/2010 22 IPS: Image Packaging System Replaces old SVR4 packaging system Network based software repository & delivery GUI and CLI package tools Multi-variant packages 23. 04/15/201023IPS: Image Packaging System 24. 04/15/201024 IPS: Image Packaging System Uses ZFS to create & manage boot environments 25. 04/15/201025Desktop Gnome 2.28 Xorg 1.7 Compiz ( on live CD ) Firefox 3.5.8 Thunderbird 3.0.1 Device Driver Utility 26. 04/15/2010 26Desktop Time Slider Automated ZFS Snapshots Addon to Nautilus File Manager Enable/disable per filesystem [ demo ] Future improvements Backup to external media Better retention configuration 27. 04/15/2010 27Crossbow: Network Virtualization Great for server & network consolidation Add virtual NICs, switches, & vlans Assign cpu resources, priorities, bandwidth limits Local IP/network & remote IP/network Local & remote network port Transport type ( tcp, udp, sctp, icmp, icmpv6 ) Limit max bandwidth, set relative priority Extended accounting for flows for tracking usage 28. 04/15/201028Zones OS level virtualization, chroot on steroids One kernel, isolated userland instances Seperate naming services inside each zone ( DNS, NIS, LDAP ) Processes isolated within zones Broad resource controls Processor pools, process, thread, memory, swap limits Limited privileges & device access Must be granted by configuration Shared ip stack networking or exclusive access to NIC 29. 04/15/201029 Branded Zones Support for non-native environments Interpretation layer to translate from brand to native System calls, signals, process & thread handling Different brand types native: on Solaris 10 / Solaris Express releases ipkg: for OpenSolaris releases lx: Linux 2.4 kernel & 32 bit apps lx26: Experimental Linux 2.6 support solaris8 & solaris9: on SPARC ( commercial support ) s10brand: to support transition of Solaris 10 to future versions [ zones demo ] 30. 04/15/2010 30xVM OpenSolaris as dom0 on top of Xen Xen 3.4 hypervisor & tools Live migration support Full & Paravirtualized domU on top of hypervisor Intel VT or AMD-V instruction set needed for Full HVM OpenSolaris as full PV guest Use Crossbow for managing networking Can use ZFS volumes for storage 31. 04/15/201031 COMSTAR Common Multiprotocol SCSI Target Seperates storage backend from access methods LUN masking & mapping Multipathing over different transportsCOMSTAR 32. 04/15/201032 COMSTAR Logical Unit Providers Disk, Tape, Virtual Tape Library logical units Actual storage via ZFS volume, raw disk, file, tape Supports thin provisioning COMSTAR 33. 04/15/2010 33 COMSTAR Port Providers Exports LUNs to other hosts Looks like normal block device to the host iSCSI, FibreChannel, FCoE, SAS, iSER ( iSCSI over RDMA ) Server 1 Server 2 COMSTARServer 3 34. 04/15/201034 CIFS Server Native, in kernel CIFS server Can join Windows domain LDAP/Windows AD user maps Map Windows Security ID to Solaris user/group NT ACL stored natively in ZFS ZFS snapshots work with Windows Shadow Copy Virus scan engine support Integrated into ZFS, uses ICAP Protects NFS & local access too 35. 04/15/2010 35 Trusted Extensions Replaces previous seperate Trusted Solaris OS System enforced mandatory access control policies Explicit authorization needed to cross security labels Device & network access also labeled [ demo ] 36. 04/15/2010 36Drawbacks Uses more memory No KDE ( Other distros include KDE ) ZFS block pointer rewrite a ways off yet For online pool resizing, re-compress/dedup/defrag ZFS encryption still being developed IPS going through heavy development Power Management, Suspend/Resume Limited Wi-Fi chipsets supported Proprietary media codecs Fluendo commercial gstreamer plugins 37. 04/15/201037Other Distributions OpenSolaris: Official Sun/Oracle sponsored distribution BeleniX: First non-Sun Live CD & GUI distro KDE, GNOME, & XFCE packages Nexenta: OpenSolaris kernel, Ubuntu userland & packaging Easy NAS/SAN Seutp Commercially supported NexentaStor storage platform 38. 04/15/201038 Other Distributions EON: Embedded Operating System/Networking Small NAS focused distribution MilaX: Started as mini/rescue CD experiment Full CD/USB based distro, enough for older hardware Jaris: Japanese based distribution 39. 04/15/2010 39Other Distributions Schillix: First non-sun distribution Lead by Joerg Schilling MarTux First SPARC distribution Korona: KDE4 based test drive Live DVD Experimental, currenly on KDE 4.3.98