gemstone/64 3.0: a look under the hood - norm green
TRANSCRIPT
![Page 1: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/1.jpg)
GemStone/64 3.0 - A Look Under the Hood
Norman R. Green
Director, R & D, VMware Inc.
Smalltalk Solutions 2011
![Page 2: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/2.jpg)
2
Agenda
New Features
Smalltalk Programming Changes
Performance Improvements
Beyond 3.0
![Page 3: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/3.jpg)
3
New Features
Native Code Virtual Machine•Converts byte codes to native instructions• JIT design – conversion occurs at first method invocation.• 25% to 100% speed improvement
![Page 4: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/4.jpg)
4
New Features
Foreign Function Interface (FFI)• Load shared libraries.•Call C functions without writing C code.•All coding can be done in Smalltalk•Martin will present FFI in detail on Wednesday @ 9:15 am
![Page 5: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/5.jpg)
5
New Features
Mid-Level Shared Caches•Caches that sit between the stone cache and a remote
SPC•Benefits
• Better caching.• 3 caches to look for pages rather than 2
• Reduced Network I/O on Stone Server• 500 remote caches can red-line a 10 GB/s network!
![Page 6: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/6.jpg)
6
Classic Remote SPC “Star” Configuration
StoneSPC
DB
SPC
SPC
SPC
SPC
SPC
SPC
![Page 7: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/7.jpg)
7
Mid-Level Cache Configuration
StoneSPC
DB
SPC
SPC
SPC
SPC
SPC
SPC
SPCSPC
Mid-Level Caches
Leaf Caches
![Page 8: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/8.jpg)
8
New Features
Mid-Level Shared Caches•Creating a Mid-Level Cache1.Start the netldi on the remote (leaf cache) host, mid-level
host, and stone server2.From the leaf-cache host, do a normal remote login3.Run this code to create the mid-cache:
System midLevelCacheConnect: midLevelHostName cacheSizeKB: aSize maxSessions: nSess
![Page 9: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/9.jpg)
9
New Features
Mid-Level Shared Caches•Connecting to an Existing Mid-Level Cache1.From the leaf-cache host, do a normal remote login2.Run this code to connect to the mid-cache:
System midLevelCacheConnect: hostName
![Page 10: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/10.jpg)
10
New Features
External Password Validation•Passwords may now be stored and managed outside of
GemStone.• 2 New Options
• LDAP = Lightweight (ha!) Directory Access Protocol• UNIX password
![Page 11: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/11.jpg)
11
New Features
External Password Validation•User ID Aliases are Supported
• GemStone user ID may or may not match external user ID.• Example:
• GemStone UID: ‘normg’• LDAP UID: ‘norm.green’
![Page 12: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/12.jpg)
12
New Features
External Password Validation• LDAP – Uses OpenLDAP and OpenSSL libraries (shipped
with GemStone).•UNIX – Uses getpwnam() / getspnam() UNIX calls.
![Page 13: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/13.jpg)
13
New Features
External Password Validation•Enabling LDAP authentication for a UserProfile:
| u |
u := AllUsers userWithId: ‘normg’.
u enableLDAPAuthenticationWithAlias: ‘norm.green’
baseDN: ‘uid=%s,ou=LdapTesting,dc=gemstone,dc=com’
filterDN: nil .
System commitTransaction .
![Page 14: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/14.jpg)
14
New Features
Multi-threaded Operations•The following operations are now multi-threaded:
• markForCollection• objectAudit• findDisconnectedObjects• countInstances• listInstances• listReferences• GsObjectInventory• findObjectsLargerThan• Epoch Garbage Collection• Write Set Union Sweep (part of GC finalization)
•Native (OS) threads are used.•Each thread is a unique GemStone session•Each thread takes a shared page cache slot.
![Page 15: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/15.jpg)
15
New Features
Multi-threaded Operations•Aggressiveness Is Controlled By 2 Parameters:
• Number of threads• Percent CPU Active
• Means total CPU load on the server across all cores.• Threads will start sleeping if the threshold is exceeded.
•Both Parameters Can Be Adjusted On The Fly• SystemRepository setMultiThreadedScanThreads: numThreadsmaxCpu: aPercent
![Page 16: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/16.jpg)
16
New Features
Multi-threaded Operations•The default methods are NOT aggressive• If you want speed, use the fast* methods
• markForCollection – uses 2 threads• fastMarkForCollection – uses up to 16 threads
![Page 17: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/17.jpg)
17
New Features
ProfMonitor Improvements•New implementation supports sample intervals down to 1
microsecond (1000 ns).•New Class Protocol
• ProfMonitor monitorBlock: aBlock downTo: hits intervalNs: ns
• ProfMonitor monitorBlock: aBlock intervalNs: ns
![Page 18: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/18.jpg)
18
New Features
Built-In Monticello Support•Monticello Support is now available “out of the box”•Protocols:
• GSPackageLibrary loadMczFile: aFile fromRepositoryPath: aPath
• GSPackageLibrary loadLastVersionOf: packageName fromRepositoryPath: aPath
![Page 19: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/19.jpg)
19
Agenda
New Features
Smalltalk Programming Changes
Performance Improvements
Beyond 3.0
![Page 20: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/20.jpg)
20
Smalltalk Changes
Greater ANSI Compliance•Correct Method results
• Arg vs receiver
•Add Missing Methods•Add Missing Classes
Many Smalltalk hacks that were added because “GemStone doesn’t do it that way” can be removed in 3.0
![Page 21: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/21.jpg)
21
Smalltalk Changes
Array Literal Syntax•Run-time Constructor
• 2.x: #[ 1, 2, 3 ]• 3.0: { 1 . 2 . 3 }•Compile-time Constructor (unchanged)
• #( 1, 2, 3)
In 3.0, the old 2.x Array syntax can be enabled by running this code after login:• System gemConfigurationAt:#GemConvertArrayBuilder put: true
ByteArray Literal Syntax (new)•#[ 0 1 254 255 ]
![Page 22: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/22.jpg)
22
Smalltalk Changes
Exception Handling• In 3.0, ANSI exceptions are fully supported.•ANSI exceptions are class-based.•Old GemStone Exception class is deprecated, but still
works in some cases.•GemStone error numbers are deprecated.
![Page 23: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/23.jpg)
23
Smalltalk Changes
AlmostOutOfMemeory Exception ExampleautoCommitForBlock: aBlock withThreshold: aPercentage
| errNumAlmostOutOfMemory |
errNumAlmostOutOfMemory := ErrorSymbols at: #rtErrSignalAlmostOutOfMemory .
"Enable AlmostOutOfMemory signal"
System signalAlmostOutOfMemoryThreshold: aPercentage .
[
aBlock onException: AlmostOutOfMemory do:[ :ex |
ex number == errNumAlmostOutOfMemory ifTrue:[ | systm |
systm := System .
systm commitTransaction
ifFalse:[ systm abortTransaction. ^ false ].
systm _vmMarkSweep ; enableAlmostOutOfMemoryError.
ex resume .
] ifFalse:[
ex outer
]
]
] ensure: [
"disable AlmostOutOfMemory signal"
System signalAlmostOutOfMemoryThreshold: -1.
].
^ true
%
![Page 24: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/24.jpg)
24
Smalltalk Changes
New Special Selectors•The following selectors are inlined by the compiler and
may not be overloaded:• ifNil:
• ifNotNil:
• ifNotNil:ifNil:
• ifNil:ifNotNil:
• _isSmallInteger
• _isInteger
• _isNumber
• _isFloat
• _isSymbol
• _isExceptionClass
• _isExecBlock
• _isArray
![Page 25: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/25.jpg)
25
Smalltalk Changes
Segment is now GsObjectSecurityPolicy•We’ve wanted to change that for a loooooong time.
![Page 26: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/26.jpg)
26
Smalltalk Changes
Dynamic Instances Variables•Add instance variables on the fly!•No instance migration required.•Dynamic Inst. Vars. are “per object”•Methods:
• Object dynamicInstVarAt: aSymbol• Object dynamicInstVarAt: aSymbol put: anObject• Object removeDynamicInstVar: aSymbol
![Page 27: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/27.jpg)
27
Smalltalk Changes
LargeInteger Changes•New class: LargeInteger
• Replaces both LargePositiveInteger and LargeNegativeInteger• Existing Instances:
• LargePositiveInteger -> ObsoleteLargePositiveInteger • LargeNegativeInteger -> ObsoleteLargeNegativeInteger
• Instances are automatically converetd when loaded into object memory.
• To convert manually, send:• anObsoleteInt convert
![Page 28: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/28.jpg)
28
Smalltalk Changes
ScaledDecimal Changes•New format:
• mantissa (SmallInteger or LargeInteger)• scale (SmallInteger) – power of 10
•New Literal Notation (ANSI compliant)• 1.23s2
• Mantissa: 123• Scale: 2
![Page 29: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/29.jpg)
29
Smalltalk Changes
ScaledDecimal Changes• GS/64 v2.x ScaledDecimal is now FixedPoint in 3.0• Instances are automatically converted• New Literal Syntax:
• 1.23p2• Numerator: 123• Denominator: 100• Scale: 2
![Page 30: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/30.jpg)
30
Smalltalk Changes
New Class/Metaclass Hierarchy
• Object
• Behavior
• Metaclass
• Class
• Object
• Behavior
• ObsoleteMetaclass
• Module
• Metaclass3
• Class
2.x 3.0
• After upgrade, existing Metaclasses become ObsoleteMetaclass
• Newly created classes will have a class of Metaclass3
• Applications which are sensitive to the superclass of Class or Metaclass will need to be adjusted.
![Page 31: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/31.jpg)
31
Agenda
New Features
Smalltalk Programming Changes
Performance Improvements
Beyond 3.0
![Page 32: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/32.jpg)
32
Performance Improvements
Aborting A Transaction •Round trips to stone reduced from 2 to 1
![Page 33: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/33.jpg)
33
Performance Improvements
Old SMC Design
Gem Procedure
• Get SMC queue lock
• Add self to SMC queue
• Release SMC queue lock
• Wait on semaphore for stone’s response.
Stone Procedure
1. Get SMC queue lock.
2. Remove all requests from SMC queue.
3. Release SMC queue lock
4. Process Requests
Stone
Gem
Gem
Gem
SMC Queue
Spin Lock
![Page 34: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/34.jpg)
34
Performance Improvements
New SMC Design – No Lock or Queue
Gem Procedure
• Set my bit (atomic OR)
• Wait on semaphore for stone’s response.
Stone Procedure
1. For each 64-bit session word:
• Read and clear all bits (atomic AND).
2. Process Requests
Stone
GemGem
SMC Words
![Page 35: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/35.jpg)
35
Performance Improvements
Session Priority• Elimination of SMC queue means sequence of sessions
requesting service from stone is not preserved.• Session Priority added to compensate.• Session Priority Values:
0 – lowest (Reclaim and Admin GC gems)1 – low2 – medium (default)3 – high4 – highest
• Session holding (or about to receive) the commit token always has highest priority.
![Page 36: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/36.jpg)
36
Performance Improvements
Session Priority Protocol• System setSessionPriority: anInt forSessionId: aSessionId
• System priorityForSessionId: aSessionId
![Page 37: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/37.jpg)
37
Performance Improvements
Shared Cache Hash Table Read Locks• 2.x Design
• All hash table row accesses were serialized by spin lock.• Result: Lookups are done in series.
• 3.0 Design• Each hash table row now has a reference count and a write
lock.• Result: Lookups are done in parallel.
![Page 38: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/38.jpg)
38
Performance Improvements
Serialized Hash Table Lookups In 2.x
Gem1
Spin Locks
9 2 6
Collision Chain
SPC Hash Table
14Gem2
Gem3
• 3 Gems want to lookup page 6.
• Each gem must hold the lock to perform the lookup.
If the lookup time is n, then:• Gem1: n• Gem2: 2n• Gem3: 3n
![Page 39: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/39.jpg)
39
Performance Improvements
Parallelized Hash Table Lookups In 3.0
Gem1
Spin Locks
9 2 6
Collision Chain
SPC Hash Table
14Gem2
Gem3
• 3 Gems want to lookup page 6.• All 3 lookups can be done
simultaneously.
If the lookup time is n, then:• Gem1: n• Gem2: n• Gem3: n
![Page 40: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/40.jpg)
40
Performance Improvements
New Tranlog AIO System• POSIX AIO calls (aio_write()) removed.
• Too many OS bugs
• Replaced With Our Own Code• Based on Native OS Threads• New Config Parameter:
• STN_NUM_AIO_WRITE_THREADS - Number of native threads started to perform tranlog writes.
![Page 41: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/41.jpg)
41
Agenda
New Features
Smalltalk Programming Changes
Performance Improvements
Beyond 3.0
![Page 42: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/42.jpg)
42
Beyond 3.0
“True” Hot-Standby Databases• What Oracle calls “database clustering”
• 1 or more stand-by databases
• Kept in-sync by applying tranlogs.
• Failover to a standby on fault
• Read-only access to standby DBs.
• Minimal DBA setup required.
![Page 43: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/43.jpg)
43
Hot Standby Design
StoneSPC
DB TL
Agent
StoneSPC
DB TL
Agent
StoneSPC
DB TL
Agent
Primary Database
Hot-Standby Databases
![Page 44: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/44.jpg)
44
Beyond 3.0
Additional Multi-threading• Full Backup
• Restore from Backup
• markGcCandidates
Database Monitoring Support
• SNMP or JMX
Automatic Load Balancing
• When a new gem is started, dynamically decide which server to run it on.
![Page 45: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/45.jpg)
45
Commit Performance Improvement
GemStone/64 2.4.4.4
GemStone/64 3.0
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Commit Throughput
Commits per Second
• Solaris 10 SPARC
• Tranlogs on solid state disks
• Extents on raw partitions.
• 16 gems
![Page 46: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/46.jpg)
46
Index Performance Improvement
• Solaris 10 SPARC
• Tranlogs on solid state disks
• Extents on raw partitions.
• 2M element collection
Index Performance
0 100 200 300 400 500 600 700 800
Build Indexes
Audit indexes
Remove indexes
Op
erat
ion
Seconds
3.0
2.4
• 5 Equality Indexes
![Page 47: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/47.jpg)
47
For More Information…..
Beta drops available via anonymous ftp:• ftp://ftp.gemstone.com/pub/GemStone64
![Page 48: GemStone/64 3.0: A Look Under the Hood - Norm Green](https://reader035.vdocument.in/reader035/viewer/2022062511/5515b0704979591b1e8b4e69/html5/thumbnails/48.jpg)
48
VMware Inc.1260 N.W. Waterhouse Ave.Suite 200Beaverton, Oregon, 97006
www.vmware.com
Norman R. GreenDirector, R&[email protected]
(503) 533-3410 direct(503) 804-2041 mobile
Questions ?