network and distributed systems
DESCRIPTION
TRANSCRIPT
![Page 1: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/1.jpg)
Lecture 3 – Networks and Distributed Systems
CSE 490h – Introduction to Distributed Computing, Spring 2007
Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution 2.5 License.
![Page 2: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/2.jpg)
Outline Networking Remote Procedure Calls (RPC) Transaction Processing Systems
![Page 3: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/3.jpg)
Fundamentals of Networking
![Page 4: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/4.jpg)
Sockets: The Internet = tubes?
A socket is the basic network interface Provides a two-way “pipe” abstraction
between two applications Client creates a socket, and connects to
the server, who receives a socket representing the other side
![Page 5: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/5.jpg)
Ports
Within an IP address, a port is a sub-address identifying a listening program
Allows multiple clients to connect to a server at once
![Page 6: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/6.jpg)
Example: Web Server (1/3)
1) Server creates a socket attached to port 80
80
The server creates a listener socket attached to a specific port. 80 is the agreed-upon port number for web traffic.
![Page 7: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/7.jpg)
Example: Web Server (2/3)
The client-side socket is still connected to a port, but the OS chooses a random unused port number
When the client requests a URL (e.g., “www.google.com”), its OS uses a system called DNS to find its IP address.
2) Client creates a socket and connects to host
80Connect: 66.102.7.99 : 80
(anon)
![Page 8: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/8.jpg)
Example: Web Server (3/3)
Server chooses a randomly-numbered port to handle this particular client
Listener is ready for more incoming connections, while we process the current connection in parallel
3) Server accepts connection, gets new socket for client
80
(anon) (anon)
4) Data flows across connected socket as a “stream”, just like a file
![Page 9: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/9.jpg)
What makes this work? Underneath the socket layer are several more
protocols Most important are TCP and IP (which are used
hand-in-hand so often, they’re often spoken of as one protocol: TCP/IP)
Your dataTCP header
IP header
Even more low-level protocols handle how data is sent over Ethernet wires, or how bits are sent through the air using 802.11 wireless…
![Page 10: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/10.jpg)
IP: The Internet Protocol
Defines the addressing scheme for computers
Encapsulates internal data in a “packet”
Does not provide reliability Just includes enough information for the
data to tell routers where to send it
![Page 11: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/11.jpg)
TCP: Transmission Control Protocol Built on top of IP Introduces concept of “connection” Provides reliability and ordering
Your dataTCP header
IP header
![Page 12: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/12.jpg)
Why is This Necessary?
Not actually tube-like “underneath the hood” Unlike phone system (circuit switched), the
packet switched Internet uses many routes at once
you www.google.com
![Page 13: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/13.jpg)
Networking Issues
If a party to a socket disconnects, how much data did they receive?
… Did they crash? Or did a machine in the middle?
Can someone in the middle intercept/modify our data?
Traffic congestion makes switch/router topology important for efficient throughput
![Page 14: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/14.jpg)
Remote Procedure Calls (RPC)
![Page 15: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/15.jpg)
How RPC Doesn’t Work Regular client-server protocols involve
sending data back and forth according to a shared state
Client: Server:
HTTP/1.0 index.html GET
200 OK
Length: 2400
(file data)
HTTP/1.0 hello.gif GET
200 OK
Length: 81494
…
![Page 16: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/16.jpg)
Remote Procedure Call RPC servers will call arbitrary functions in
dll, exe, with arguments passed over the network, and return values back over network
Client: Server:
foo.dll,bar(4, 10, “hello”)
“returned_string”
foo.dll,baz(42)
err: no such function
…
![Page 17: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/17.jpg)
Possible Interfaces
RPC can be used with two basic interfaces: synchronous and asynchronous
Synchronous RPC is a “remote function call” – client blocks and waits for return val
Asynchronous RPC is a “remote thread spawn”
![Page 18: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/18.jpg)
Synchronous RPC
s = RPC(server_name, “foo.dll”, get_hello, arg, arg, arg…)
RPC dispatcher
String get_hello(a, b, c){ … return “some hello str!”;}
foo.dll:
print(s);...
client server
time
![Page 19: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/19.jpg)
Asynchronous RPC
h = Spawn(server_name, “foo.dll”, long_runner, x, y…)
RPC dispatcher
String long_runner(x, y){ … return new GiantObject();}
foo.dll:
GiantObject myObj = Sync(h);
client server
time
(More code
...
keeps running…)
![Page 20: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/20.jpg)
Asynchronous RPC 2: Callbacks
h = Spawn(server_name, “foo.dll”, callback, long_runner, x, y…)
RPC dispatcher
String long_runner(x, y){ … return new Result();}
foo.dll:
void callback(o) { Uses Result}
client server
time
(More code
...
runs…)
Thread spawns:
![Page 21: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/21.jpg)
Wrapper Functions
Writing rpc_call(foo.dll, bar, arg0, arg1..) is poor formConfusing codeBreaks abstraction
Wrapper function makes code cleanerbar(arg0, arg1); //just write this; calls “stub”
![Page 22: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/22.jpg)
More Design Considerations
Who can call RPC functions? Anybody? How do you handle multiple versions of a
function? Need to marshal objects How do you handle error conditions? Numerous protocols: DCOM, CORBA,
JRMI…
![Page 23: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/23.jpg)
Transaction Processing Systems
(We’re using the blue cover sheets on the TPS reports now…)
![Page 24: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/24.jpg)
TPS: Definition
A system that handles transactions coming from several sources concurrently
Transactions are “events that generate and modify data stored in an information system for later retrieval”*
* http://en.wikipedia.org/wiki/Transaction_Processing_System
![Page 25: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/25.jpg)
Key Features of TPS: ACID
“ACID” is the acronym for the features a TPS must support:
Atomicity – A set of changes must all succeed or all fail Consistency – Changes to data must leave the data in a
valid state when the full change set is applied Isolation – The effects of a transaction must not be
visible until the entire transaction is complete Durability – After a transaction has been committed
successfully, the state change must be permanent.
![Page 26: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/26.jpg)
Atomicity & Durability
What happens if we write half of a transaction to disk and the power goes out?
![Page 27: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/27.jpg)
Logging: The Undo Buffer
1. Database writes to log the current values of all cells it is going to overwrite
2. Database overwrites cells with new values
3. Database marks log entry as committed
If db crashes during (2), we use the log to roll back the tables to prior state
![Page 28: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/28.jpg)
Consistency: Data Types
Data entered in databases have rigorous data types associated with them, and explicit ranges
Does not protect against all errors (entering a date in the past is still a valid date, etc), but eliminates tedious programmer concerns
![Page 29: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/29.jpg)
Consistency: Foreign Keys
Purchase_idPurchaser_nameItem_purchased FOREIGN
Item_idItem_nameCost
Database designers declare that fields are indices into the keys of another table
Database ensures that target key exists before allowing value in source field
![Page 30: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/30.jpg)
Isolation
Using mutual-exclusion locks, we can prevent other processes from reading data we are in the process of writing
When a database is prepared to commit a set of changes, it locks any records it is going to update before making the changes
![Page 31: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/31.jpg)
Faulty Locking
Lock (A)
Unlock (A)
Lock (B)
Unlock (B)
Unlock (A)
Lock (A)
Write to table A
Read from A
Write to table B
time
Locking alone does not ensure isolation!
Changes to table A are visible before changes to table B – this is not an isolated transaction
![Page 32: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/32.jpg)
Two-Phase Locking
After a transaction has released any locks, it may not acquire any new locks
Effect: The lock set owned by a transaction has a “growing” phase and a “shrinking” phase
Lock (A)
Unlock (A)
Lock (B)
Unlock (B)
Unlock (A)
Lock (A)
Write to table A
Read from A
Write to table B
time
![Page 33: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/33.jpg)
Relationship to Distributed Comp
At the heart of a TPS is usually a large database server
Several distributed clients may connect to this server at points in time
Database may be spread across multiple servers, but must still maintain ACID
![Page 34: Network and distributed systems](https://reader033.vdocument.in/reader033/viewer/2022061219/54b897b44a79590e6f8b456c/html5/thumbnails/34.jpg)
Conclusions
We’ve seen 3 layers that make up a distributed system
Designing a large distributed system involves engineering tradeoffs at each of these levels
Appreciating subtle concerns at each level requires diving past the abstractions, but abstractions are still useful in general