optimizing application architecture (.net/java topics)
Post on 30-Jun-2015
335 Views
Preview:
TRANSCRIPT
OPTIMIZING YOUR APPLICATION
ARCHITECTURERavi Okade
Slides from Philly.NET code camp June 2014
All architecture is design but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change.
Source: wikiquote.org
Grady Booch
Intro
We will talk about ideas to optimize your application architecture.
We will talk about some theory and some practice! Hope you will like it
Source: wikiquote.org
Donald Knuth
If you find that you're spending almost all your time on theory, start turning some attention to practical things; it will improve your theories. If you find that you're spending almost all your time on practice, start turning some attention to theoretical things; it will improve your practice.
Agenda: (Re)Visiting the programming 101 Data Structures and Algorithms Serialization Synchronization Other stuff (Rx, functional and so
on)
Data Structures and Algorithms
Do you know ..
What sorting algorithm is used by Array.Sort ?
Describe internals of List<T>. Compare and contrast with LinkedList<T>
Array.Sort
Source: http://msdn.microsoft.com/en-us/library/6tf1f0bc.aspx
Quicksort honored as one of top 10 algorithms of 20th centuryin science and engineering.
Click icon to add picture
Source: Wikipedia.org
Quicksort invented by Tony Hoare (1960)
Why does it matter ?
Robert Sedgwick, Princeton University http://www.cs.princeton.edu/~rs/
Lists
List<T> in .NET is highly optimized You typically never have to use anything
else Initial size makes a difference
Consider an array!
Arrays are written using native code Much faster than all other data
structures
Interesting Lists - Skiplist
A Skip list is a data structure that allows fast search within an ordered sequence of elements. Fast search is made possible by maintaining a linked hierarchy of subsequences, each skipping over fewer elements
From Wikipedia - http://en.wikipedia.org/wiki/Skip_list
Skiplist - searching example
From Scott Mitchell’s article in MSDN “An Extensive Examination of Data Structures Using C# 2.0”
.NET Specialized Collections
ListDictionary Implements IDictionary using a singly linked list. Recommended for collections that typically include fewer than 10 items.
HybridDictionary Implements IDictionary by using a ListDictionary while the collection is small, and then switching to a Hashtable when the collection gets large.
NameValueCollection Represents a collection of associated String keys and String values that can be accessed either with the key or with the index.
StringDictionary Implements a hash table with the key and the value strongly typed to be strings rather than objects.Many more – see: System.Collections.Specialized Namespace
No Guarantees! Question everything
https://www.youtube.com/watch?v=YQs6IC-vgmo#t=6
Serialization
Serialization is a solved problem! BinaryFormatter NetDataContractSerializer BinaryNetDataContractSerializer DataContractJsonSerializer XmlSerializer BinaryXmlSerializer DataContractSerializer BinaryDataContractSerializer
Or is it ?
Serialization matters..
Data exchange across clear boundaries works very well with xml/json.
But Lot of data exchange is still internal. Examples:Data Services for Thick clientsStoring data in a cacheDistributed computing, cloud storage
Popular protocols
Protocol Buffer (Protobuf)Created 2001; Opensourced 2008Designed and used extensively by Google
Apache ThriftCreated in 2007Used extensively in Facebook
Apache Avro Created by LinkedIn and open sourced in
2011
See a good overview of different protocols hereSee performance comparison of various formats here
So many ways to serialize!Serialization time for various protocols
https://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking
ProtoBuf.NET Size comparison
prot
obuf
-net
Binar
yFor
mat
ter
SoapF
orm
atte
r
XmlS
eria
lizer
DataC
ontra
ctSer
ializ
er
DataC
ontra
ctJs
onSer
ializ
er
5
251
928
170 212
32
ProtoBuf.NET Time comparison
prot
obuf
-net
Binary
Form
atte
r
SoapF
orm
atte
r
XmlS
erial
izer
DataC
ontra
ctSer
ialize
r
DataC
ontra
ctJs
onSer
ialize
r
ProtoBuf.NET Performance
Serializer Size Serialization Time
Deserialization Time
Protobuf-net 5 842 3,535
BinaryFormatter 251 11,618 15,178
SoapFormatter 928 47,179 79,218
XmlSerializer 170 15,373 22,216
DataContractSerializer 212 3,778 11,455
DataContractJsonSerializer 32 4,129 18,728
https://code.google.com/p/protobuf-net/wiki/Performance
ProtoBuf.NET Attributes
From ProtoBuf.Net Quickstart (Download from SVN here code). This example project is in QuickStart folder
ProtoBuf.NET Serialization code
Nice samples here and here
ProtoBuf schema
package QuickStart;
message Contact { optional string Name = 1; optional string ContactDetails = 2;}message Customer { optional string CustomerId = 1; optional string Name = 2; optional double MaximumOrderAmount = 3; repeated Contact Contacts = 4;}
ProtoBuf.NET Serialization without attributes
Protobuf Demo
Serialize/Deserialize example in .NET Protobuf Schema Using data serialized using .NET in Java
Avro Comparison Dynamic typing: Avro does not require that code be
generated. Data is always accompanied by a schema that permits full processing of that data without code generation, static datatypes, etc.
Untagged data: Since the schema is present when data is read, considerably less type information need be encoded with data, resulting in smaller serialization size.
Schema evolution: Avro requires schemas when data is written or read. Most interesting is that you can use different schemas for serialization and deserialization, and Avro will handle the missing/extra/modified fields.
Avro Demo
Using Avro Schema and GenericRecord (Both in .NET and Java)
Comparing the output sizes of various Codecs (Java sample)
Serialize in Java and consume in .NET and vice-versa.
Simplicity does not precede complexity, but follows it.
Alan J. Perlis (1922 – 1990)
First recipient of the Turing Award.
If you have a procedure with 10 parameters, you probably missed some.
Alan J. Perlis
SynchronizationAvoiding and Eliminating
Avoid using locks. Simply say "no" to locks. Locks slow programs, reduce their scalability, and are the source of bugs in parallel programs.
James Reinders (Intel)From Rules for Parallel Programming for Multicore
Reduced lock and lock-less collections ConcurrentDictionary Immutable Collections in .NET 4.5
ConcurrentDictionary
Available in .NET 4.0 Needs adjustment to your code
Immutable Collections in .NET 4.5 System.Collections.Immutable namespace
ImmutableDictionary<TKey, TValue>ImmutableSortedDictionary<TKey, TValue>ImmutableHashSet<T>ImmutableList<T>ImmutableQueue<T>ImmutableSortedSet<T>ImmutableStack<T>
Needs adjustment. See this Also immutable collections are slower.
Benchmark! Also see this nice series of blogposts by Eric Lippert
Immutable<T> Demo
Follow conventional wisdom
Preaching .. DRY - Don’t repeat yourself Abstraction Principle KISS (Keep it simple, stupid!) Avoid Creating a YAGNI (You aren’t going to need
it) Do the simplest thing that could possibly work Don’t make me think Open/Closed Principle Write Code for the Maintainer Principle of least astonishment Single Responsibility Principle Minimize Coupling Maximize CohesionMostly Quoted from: http://www.artima.com/weblogs/viewpost.jsp?thread=331531
More preaching Hide Implementation Details Law of Demeter Avoid Premature Optimization Code Reuse is Good Separation of Concerns Embrace Change Write Unit tests, know your mock Have good code coverage Use Dependency Injection Lint your code, cover your coverage (80%+) Use frequent builds/test runs
Mostly Quoted from: http://www.artima.com/weblogs/viewpost.jsp?thread=331531
"How to test?" is a question that cannot be answered in general. "When to test?" however, does have a general answer: as early and as often as possible.
Bjarne Stroustrup
01010100 01101000 01101001 01101110 01101011 00100000 01101111 01110101 01110100 01110011 01101001 01100100 01100101 00100000 01110100 01101000 01100101 00100000 01100010 01101111 01111000
Think outside the box
Consider Functional Consider Rx Read up on new developments in
the Big Data/High speed computing fronts:AKKA, Apache Spark, DataTorrentDisruptor pattern, Mechanical Sympathy,
Work Stealing pattern
Object-oriented programming is an exceptionally bad idea which could only have originated in California.
Edsger Dijkstra
http://en.wikiquote.org/wiki/Talk:Edsger_W._Dijkstra
Consider Functional
Helps you think differently Immutable data-structures No side affects Functions don’t care about state Partial evaluations (higher order
functions)
QuickSort in functional languages
Haskell (1990)
qsort [] = [] qsort (x:xs) = qsort (filter (< x) xs) ++ [x] ++ qsort (filter (>= x) xs)
F# (2005)
let rec quicksort l = match l with | [] -> [] | h::t -> quicksort (List.filter (fun x -> x < h) t)
@ h :: quicksort (List.filter (fun x -> x >= h) t)
http://en.wikibooks.org/wiki/Algorithm_Implementation/Sorting/Quicksort
C# is functional too! Quicksort using Linq
public IEnumerable<T> Quicksort(List<T> v, IComparer<T> comparer) { if (v.Count < 2)
return v; T pivot = v[v.Count / 2]; return Quicksort(v.Where(x => comparer.Compare(x, pivot) < 0),
comparer) .Concat(new T[] { pivot }) .Concat(Quicksort(v.Where(x => comparer.Compare(x, pivot) > 0),
comparer)); }
http://en.wikibooks.org/wiki/Algorithm_Implementation/Sorting/Quicksort
Consider Rx
Great for pub-sub scenarios with impedance mismatch:AsyncEvent streams
Well suited for front-end (WPF, Winforms) It makes your code simple and safe If you like Linq, you will love Rx! Rx is getting popular and is now available
for Java and JavaScript
Learning Rx
MSDN has very good examples Excellent book by Lee Campbell (Free!)
Rx in 15 minutes
http://channel9.msdn.com/Blogs/Charles/Erik-Meijer-Rx-in-15-Minutes
Rx Demo:WPF Autocomplete TextboxSource – code sample from Oliver Lohmann MSFT
Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it.
Alan J. Perlis
</End>
top related