COMP 410 Spring 2015 Project

May 6, 2015

Contents

Project Overview.........................................................................................................................................5

The Purpose of this Document................................................................................................................5

The Customers.....................................................................................................................................5

The Developers....................................................................................................................................5

The Scope................................................................................................................................................5

The Problem............................................................................................................................................6

The Requirements...................................................................................................................................6

The Users.............................................................................................................................................7

The Use Case.......................................................................................................................................7

The Plan...................................................................................................................................................7

Timeline...............................................................................................................................................7

The Framework Documentation..................................................................................................................9

Motivation...............................................................................................................................................9

Framework Concepts.............................................................................................................................10

A single-machine application.............................................................................................................10

Running an application......................................................................................................................11

System Block Diagram.......................................................................................................................11

Networking........................................................................................................................................13

Getting Started......................................................................................................................................14

Base-Level Components - Processors................................................................................................14

Writing a Simple Processor................................................................................................................15

How to Run the Processor We've Written?.......................................................................................16

Connecting Components Together - The Subgraph Component.......................................................16

Adding More Processors to a Subgraph.............................................................................................17

Registering a Subgraph in a Subgraph...............................................................................................18

Adding Components Dynamically - The Pool Component.................................................................19

Standard Networking Components.......................................................................................................21

Motivation.........................................................................................................................................21

Network Components........................................................................................................................22

Networking Pools..............................................................................................................................24

How to Use the Network Components..............................................................................................25

Building a Distributed Network Graph with NetFlow............................................................................30

Creating the Initial Application..........................................................................................................30

Designing the Application Graph.......................................................................................................31

Designing the Distributed Graph.......................................................................................................32

Updating an Application....................................................................................................................33

Common Graph Operations...............................................................................................................34

Proving Scalability..................................................................................................................................35

THE SCALABILITY TESTING HARNESS.................................................................................................36

RESULTS.............................................................................................................................................36

The Example Application Documentation.................................................................................................36

Application Description.........................................................................................................................36

Network Topology.............................................................................................................................37

Check-in Application..............................................................................................................................37

GUI.....................................................................................................................................................38

Use Cases...........................................................................................................................................38

Hospital Application..............................................................................................................................40

GUI.....................................................................................................................................................40

Use Cases...........................................................................................................................................41

Disaster Simulator.................................................................................................................................41

GUI.....................................................................................................................................................42

Warehouse Application.........................................................................................................................43

Use Cases...........................................................................................................................................44

Helicopter Application...........................................................................................................................44

GUI.....................................................................................................................................................45

Emergency Beacon Application.............................................................................................................45

GUI.....................................................................................................................................................46

Use Cases...........................................................................................................................................47

Central Manager Application.................................................................................................................48

Use Cases...........................................................................................................................................50

How to set up the application...............................................................................................................50

Message Types:.....................................................................................................................................51

1. RestockRequest.............................................................................................................................51

2. StateUpdate...................................................................................................................................51

3. DeliveryOrder................................................................................................................................51

3. RequestConfirm.............................................................................................................................51

4. Registration...................................................................................................................................51

5. GetHelpRequests...........................................................................................................................51

6. HelpRequest..................................................................................................................................51

7. HelpConfirm (acknowledges the help request)..............................................................................51

8. HospitalChoice (Location of hospital)............................................................................................51

Prototype Use Case Diagrams...............................................................................................................51

Prototype 1........................................................................................................................................52

Prototype 2........................................................................................................................................53

...........................................................................................................................................................53

The System Diagnostics Tool Documentation............................................................................................55

How it works:.........................................................................................................................................55

Fields:................................................................................................................................................55

Project OverviewThe Purpose of this DocumentThe purpose of this document is to serve as a reference for both the customers and the developers throughout the project and after the project has completed. It is meant to facilitate the development process and help the developers communicate their understanding of the problem and plan for their solution to the customers. It will cover the following sections:

An overview of the problem that the customers presented to the developers

The requirements of a solution to that problem

The developers design of a solution to the problem

The developer’s plan for creating that solution

The CustomersThe customers are representatives from two companies. Three representatives from Schlumberger, “the world’s largest supplier of technology, integrated project management and information solutions to customers working in the oil and gas industry worldwide;” Danny Dykes, Andrea Pound, Matthew Worthington, and Chuk Ovuworie; all out of the Schlumberger Houston Technology Center. One representative from Invisible Connect, LLC, “a well-funded start-up company located in Eau Claire, WI;” Jim McDougall.

The DevelopersThe developers are a class of 20 Rice University Computer Science graduate and undergraduate students. We are assisted by two professors: Dr. Stephen Wong and Dr. Dung Nguyen; two teaching assistants: Anastasia Novinskaya and Ryan Lear; and an undergraduate Human-Computer Interaction Consultant: Julia Hossu. We have organized ourselves into this structure.

The ScopeThis will define the scope of the class. Addressing the following aspects:

Timeframe - The project officially began on January 12, 2015. We first met with the customers on January 30th (Schlumberger) and February 2nd (Invisible Connect). In these meetings, the customers presented the problem that they wanted us to solve. We began the project with a Design Competition phase for the first two weeks.

Expectations - The project is expected to be complete by April 24th, the end of the Rice Spring semester, with a final presentation of the project to follow. The developers are expected to deliver a product to the customers at the end of the semester.

Setting - The project is being completed by students at Rice University as part of a class regarding Software Engineering practices. From the course website: "COMP 410 is a unique experience in learning fundamental principles of software engineering. As a pure "discovery-process" learning environment, students are placed into a realistic software development scenario and are given the freedom to explore and develop their own interpretations of the guiding ideas and skills behind successful software development."

The ProblemThis section will present the problem that we were tasked to solve as we understand it.

The customers presented the problem as two different scenarios. Schlumberger asked us to build an Emergency Resource Management System. This system is an analog for a similar system they would like to build for their oil well system. The story is as follows: data flows into the system from many different devices and sensors. That data then needs to be processed, transmitted, or acted upon. The data is telling a story of the status of the system. The system will need to be large and complex, and should be reconfigurable and extensible. The system must protect the integrity of the data and keep its transfer secure. The system should be robust and handle disconnections within it in an intelligent way.

Invisible Connect asked us to build a network of connected devices that allow those devices to do more than they could previously do because they are now connected and have information from external sources. This network would need to manage data systems for scenarios such as food and water supply quality, security services, livestock or population health and growth, weather patterns, etc. It was stressed that the network connections would be largely unreliable and often intermittent. The devices in this network would be from all sorts of manufacturers and vary in their computational ability and support.

We synthesized these presentations into one problem that encompassed both customer's concerns. The system we would need to build would be a distributed network of devices from a variety of manufacturers with varying levels of computational ability, which supports massive amounts of information flow into and throughout the system. This information must be aggregated and synthesized into different formats. The network could not be guaranteed reliable and the system must handle disconnections intelligently so as to not lose data. The system needs to act in real-time, being able to respond to changes of the system status quickly. The system must protect the integrity of the data and keep its transfer secure. The network topology would need to be easily reconfigured during the life of the system. The data in the system will be of varying types.

The RequirementsBased on the customer's presentations we defined the problem areas and priority of each of these areas. The following is a list of the problem areas in priority order (highest priority first):

The system will need to be:1. distributed - the system can be spread across multiple devices2. reconfigurable - the system can support a network topology that is dynamic3. extensible to various platforms and networking protocols - platforms that are not readily compatible

4. extensible to various applications - applications that are not readily compatible5. extensible to various data types6. secure - information is protected7. efficient/capable of processing (receiving, transmitting, and reacting to) data in real-time8. robust/capable of handling disconnections and intermittent connections9. capable of supporting integration with 3rd party devices10. reliable - the system can confirm successful transmissions and handle unsuccessful transmissions11. capable of supporting an arbitrarily large and complex network of ground and cloud based devices12. capable of supporting arbitrarily large volumes of data, specifically high bandwidth devices

Assumptions: Those that are installing a node in the network will know who this node wants to communicate with.

The UsersThe users are divided into two types: System Administrators and Application Users System Administrators - responsible for setting up and maintaining the network. Application Users - use the system for its intended business purpose.

The Use CaseSee the use case diagram page on the SharePoint.

The PlanThe plan for developing the solution to the problem is to develop the product in stages. We set out to have 3 prototypes and a final product. Every two weeks, the developers met with the customers to demonstrate the current prototype. Prototype 0's goal was to build the basic framework and prove that it could pass message and integrate with 3rd party devices. We developed a burglary alarm system, details can be found here. Prototype 1's goal was to tie the story back to the idea the customer's presented and show the ability for information to flow in multiple directions throughout the network. We developed the Hospital-Warehouse-Central Manager & emergency beacon rescue system. At this point the framework was given the ability to support dynamic network topologies of known types. Prototype 2's goal was to test the scalability of the framework. We had laid down the infrastructure in P0 and P1 and were ready to test the framework.

TimelineThe timeline page is here. See the feature list development timeline as well for development progress.

The approach for developing the features was based on the priority of the features and the dependencies between them, further we factored in the time constraints and the complexity of each feature when planning each phase. We began by determining the fundamental features of the system, sending data between nodes. We then also determined that working with physical hardware presents technical troubles and wanted to address that feature early on in development. In the next phase, we gave the framework the ability to support dynamic connections of known types. We also laid the infrastructure for testing the scalability of the framework. In this phase we also developed began basic system diagnostics and tackled the data storage feature. In both of these phases, we developed user interfaces for the application layer of the system. In the third phase, we tested the scalability of the framework and developed the robust connections feature. We also further expanded the system

diagnostics development. In the final phase of development, the team determined what work was remaining to achieve the desired functionality and documentation. We discovered that we would not have the capacity to complete the scaling test that we had originally planned due to technical issues surrounding the scaling of the application that we wrote. Details can be found here. Instead, we decided to create a simple application to test the ability of the framework to support a high volume of data and many connections. We also spent this time planning the final demo, presentation, and refining documentation and code quality.

The Framework DocumentationMotivationSuppose we’re building a distributed network of devices: sensors that capture data from their environment, aggregators that synthesize sensor data into information that’s useful to the business, and actuators that make decisions based on this information. That is, we’re building a computational graph: the nodes are components that input/output data, and the edges are the connections between these nodes. This graph may be deployed across many machines.

When building such a computational graph, we have a few goals in mind: Graph implementation should use the same concepts as graph design. When we plan a

computational graph, we talk about the inputs and outputs of each component, the internal behavior of each component, and the connections between them. The framework should allow us to design in terms of these concepts, too, rather than rethink our design for some other architecture.

Business logic should be decoupled from connection logic. A camera just knows how to take photos. It’s not interested in knowing which node handles its photos, if any; some other part of the system should connect the camera’s output to some node’s input. This encapsulation allows us to more easily test the camera, and reuse it in other parts of our application without modification.

When an edge goes across machines, this should be an implementation detail of the edge. Suppose we have a camera on the moon sending photos to an archive on Earth. While this gap is incredibly important when deploying the application, it doesn’t affect our abstract computational graph: the gap doesn’t affect the camera’s business logic of taking photos, or the archive’s business logic of archiving photos, or the message-passing semantics of the edge between them. Instead, we’ll expect the framework to provide some tools to help us keep the edge robust in the face of communication failure, and we’ll leave our business logic untouched.

This computational graph abstraction should be usable at any level of the system. Suppose our graph contains a camera that uses its sensor, shutter, and flash to take a photo. We could implement it as a single monolithic black box that encapsulates the interactions with all three hardware devices. We could also implement each hardware device as a top-level component of our system. But the ideal abstraction is to have these three hardware components act as a subgraph of our larger application: high-level, we have a Camera component that takes photos, but we can also dive into the subgraph and understand the interactions between the individual hardware components. This makes the low-level hardware components reusable elsewhere in the graph, while still allowing us to think about the camera as its own semantic entity.

The graph topology should be static wherever possible. This allows us to visualize the graph even when it’s not running, and be confident that we have a full understanding of exactly how the components interact. None of the madness caused by unexpected changes in the graph topology at runtime, thanks.

But the graph topology needs to dynamically support new devices at runtime. Suppose we’re crowdsourcing traffic data from users who install an app on their phone. Under a strictly static graph topology, to add a new edge between a new user and the central server, we’d have to reconfigure and restart the central server. Since users install and uninstall apps all the time, that’s not acceptable. Even though static topology is great, we need to include some limited dynamism somewhere. So, we’ll

make the following compromise: we don’t need the ability to introduce brand new types of devices at runtime, but we do need the ability to introduce new devices of a known type.

Framework ConceptsThe framework is fundamentally a C# implementation of flow-based programming, extended to support new devices by including dynamic pools of same-type components.

We’ll cover the basics of flow-based programming when discussing the framework, but here’s Wikipedia’s summary:

Flow-based programming (FBP) is a programming paradigm that defines applications as networks of "black box" processes, which exchange data across predefined connections by message passing, where the connections are specified externally to the processes. These black box processes can be reconnected endlessly to form different applications without having to be changed internally. A network is a set of devices that can communicate with each other.

A single-machine applicationAn application is a running instance of an application graph, whose nodes are components and whose edges are the connections between the inputs and outputs of the components.

A component includes: Input channels: streams of incoming messages. Each has a name and a C# type. Output channels: streams of outgoing messages. Each has a name and a C# type. Behavior: a component defines how to handle an incoming message, and can emit outgoing messages

at any time.

The framework supports three different types of components: processors, subgraphs, and pools.

ProcessorsA processor is a C# object that implements a component by running C# code. It registers a handler for each input and output channel, and the rest of its behavior is up to the C# class.

A connection with a hardware sensor, for example, would be a good fit for a processor: it encapsulates all interactions with the proprietary sensor API, wrapping them in a component that the rest of the system can use. Similarly, a GUI usually belongs inside its own processor, as does any kind of direct computation, like taking a running average.

SubgraphsA subgraph is a computational graph nested inside another computational graph.

Subgraphs are configured in the same way as an application graph, but they can also expose some of their children's channels to the rest of the application.

Consider the Camera we described earlier. The Camera itself has its own semantics to the rest of the application, but is itself a computational graph that connects the hardware sensor, the hardware shutter, the hardware flash, and a component that coordinates their behaviors.

A subgraph can contain any combination of processors, pools, and even subgraphs—but, like an application graph, these components and connections cannot be changed at runtime. (That's the "static" part. When dynamism is necessary, see pools.)

PoolsA pool is a dynamic collection of components of same-type components. (It’s the one kind of dynamic subgraph we currently provide.)

Pools are configured by specifying the type of component that it contains: the set of input and output channels that each member of the pool must implement.

The pool then serves as a multiplexer for messages addressed to and from the members of the pool. That’s why each member of the pool must be of the same type: the pool is configured to forward specific input/output channels, but, if a component doesn’t implement those channels, then the multiplexing won’t make sense, so it can’t join the pool.

Other components can add new components to and remove existing ones from the pool at run-time through the pool's "AddProcessor", "AddSubgraph", and "Remove" channels. Furthermore, other components can listen to pool state updates concerning the pull by forwarding notifications that are emitted out of the "Added" and "Removed" channels.

Running an applicationAn application runs on a device, or hardware on which we can run an instance of NetFlow, such as a microcontroller or a cloud server.

The user boots the application, and provides to the framework: The types of components that will appear in the graph:o Processors: directly implemented as C# classes (example)o Subgraphs: configured using the framework’s C# configuration protocol (example)o Pools: configured using the framework’s C# configuration protocol (example )

The application grapho Nodes: The set of running components that will appear in the graph. Each includes a name and a type.o Edges: The set of connections between input and output channels. Each includes the sending

component’s name, the sending channel’s name, the receiving component’s name, and the receiving channel’s name.

The framework then combines these into a running application. It creates a running instance of each component in the graph. It establishes the connections between the running component instances. It allows the user to start/stop the application at any time.

System Block Diagram

NetworkingBefore we start adding networking concepts to our framework, let’s try to think about networking in terms of flow-based programming. Can we model networking using the tools we already have?

Consider a distributed burglar alarm: a Photo Collector takes photos of a secure area, a Photo Monitor watches those photos for suspicious behavior, and an Administrator is notified whenever the Photo Monitor flags a photo as suspicious. These three processes must live on three different machines, connected by standard sockets; what application graph will we deploy to each?

When building a flow-based program, we first think about the flow of data through our system: how is photo data transformed as it flows from its source to its destination? If you’ve written a networking application before, you’ll realize that network communication requires some additional data transformations, including serialization and interaction with the socket API.

And, as it happens, this diagram is pretty much all we need! Sockets and serializers are not special; they’re just components like any other that handle input and emit output.

For this reason, networking is not native to the framework. Instead, we provide a set of standard components that implement the socket-based edges of your network.

You’ll note that, in the diagram above, we went more low-level than we usually would like; rather than thinking about serialization and sockets, we’d rather have a component that represents a remote service and encapsulates all these low-level details. As it happens, we provide exactly that! Keep reading the User Manual for more information.

Networking is also where pools become relevant. A Photo Monitor might listen for cameras to connect, and then put a corresponding Remote Photo Collector Service component into its pool of photo collectors; see RemoteListenerServicePool.

The Photo Monitor might also have a GUI to manage its connections with Administrators; when the GUI requests a connection, it sends that request to the RemoteRequesterServicePool, which then adds a Remote Administrator Service to its pool of administrators.

Getting StartedIn the next sections we will show you how to build up a simple application using the NetFlow framework.

First, we will show you how to write an instance of a Processor, which is the simplest of components in the framework. It is a component that exposes input/output channels and defines start/stop methods.

Next we will show you how to run an application using the simple processor through writing an instance of a Subgraph, which is a component that encapsulates a subgraph of an application graph, and exposes only some of its children's input/output channels to the rest of the application.

Finally, we will show you how to write an instance of a Pool, which is the third and final type of component. It is a component that contains multiple components of the same type and multiplexes messages to their input channels and from their output channels. It allows for a dynamic network topology.

Base-Level Components - ProcessorsProcessors are the lowest-level components in a system. They register C# Actions as input channels, and request C# Actions as output channels, and then hook these channels together however they see fit. To build one, inherit from the IProcessor interface.

A component has four states: uninitialized, initialized, stopped, and started. We can Init, Start, and Stop a component to move between these states.

  A component will only receive messages on its input channels when Started, and a component must only send messages on its output channels when Started.

IProcessors must both implement Init and must implement Start and Stop. This is where the user running the application can start and stop any external services that your processor uses, like a GUI or a database connection.

Writing a Simple Processor

Consider a simple processor that performs capitalization of characters. It accepts a string in one of its input channels, capitalizes the string, and passes the capitalized string out through one of its output channels.

  The StringCapitalizer component accepts a String into its "UncapitalizedInput" input channel, and outputs a String through its "CapitalizedOutput" output channel.

This is what it would look like in C# code, using the framework IProcessor interface:

class StringCapitalizer : IProcessor{ private Action<String> SendString; public void Init(ProcessorConfig config) { config.RegisterInput<String>("UncapitalizedInput", this.CapitalizeString); this.SendString = config.RegisterOutput<String>("CapitalizedOutput"); } public void CapitalizeString(String toCapitalize) { this.SendString(toCapitalize.ToUpper());

} public void Start(){} public void Stop(){}}

Although the processor above has both input and output channels, some processors only register output channels; they emit messages on those output channels when somthing in their business logic provokes them to emit a message. These are useful when, say, encapsulating a third-party sensor: when the component starts, use the proprietary sensor API to listen for events; when an event occurs, emit it on an output channel, perhaps after reformatting it in some way to be more useful to other components.

Some processors only register input channels; they receive messages, then handle them in some way that's not visible to the rest of the system. These are useful when, say, encapsulating a GUI: when the component starts, launch the GUI; when a message arrives on an input channel, display it to the user in some way.

How to Run the Processor We've Written?

Now that we've written some business logic (Capitalization of a String) and housed it in a Processor, how do we run our code? This requires registering the processor in a top-level subgraph, which will be covered in the next section.

Connecting Components Together - The Subgraph ComponentA subgraph is a component that encapsulates other components (processors or other subgraphs). It configures the connections between its child components, and can even connect its own inputs and outputs to its children. This enables you to build component trees. So, even if the system you're building system has 12 components total, you could abstract some of the components into separate subgraphs so that the system is a lot less complex at the highest layer. To build a subgraph, implement ISubgraph.

An important thing to note is that a subgraph's configuration cannot change at run-time. In other words, if you wanted to wire in another component while the subgraph is running, you cannot do this. We do provide a component, the Pool that provides this functionality. We'll go over Pools in the next section.

As was mentioned in the previous section, in order to run a simple processor, it must be registered in a subgraph. This is true because an application is run by specifying a top-level subgraph that contains the entire application graph.

We could register an instance of the Capitalizer processor from before like so:

class CapitalizerSystem : ISubgraph{ public void Init(SubgraphConfig config) { config.RegisterProcessor("Capitalizer", new Capitalizer()); }}

Adding More Processors to a SubgraphThis subgraph containing only a Capitalizer is not very interesting or useful. To demonstrate how to add more components, and how to wire them together we will add two more components to the system: a component that generates strings and one that writes strings to the console.

The StringGenerator is a simple processor that does not have any input channels, and sends "Hello World" out of its output channel "StringOutput" every three seconds:

class StringGenerator : IProcessor{ private Action<String> SendString; public void Init(ProcessorConfig config) { this.SendString = config.RegisterOutput<String>("StringOutput"); } public void Start() { while(true) { this.SendString("Hello World"); Thread.Sleep(3000); } } public void Stop() { }}

The StringPrinter is another simple processor that does not have any output channels, but accepts input and writes it to the console:

class StringPrinter : IProcessor{ public void Init(ProcessorConfig config) { config.RegisterInput<String>("StringInput", this.PrintString); } private void PrintString(String s) { Console.WriteLine(s); } public void Start() { } public void Stop() { }}

To wire them together in the subgraph, we can use the ForwardOutput method:

class CapitalizerSystem : ISubgraph{ public void Init(SubgraphConfig config) { config.RegisterProcessor("Generator", new StringGenerator()) .ForwardOutput("StringOutput", to: "Capitalizer.UncapitalizedInput"); config.RegisterProcessor("Capitalizer", new StringCapitalizer()) .ForwardOutput("CapitalizedOutput", to: "StringInput"); config.RegisterProcessor("Printer", new StringPrinter());

}}

We've registered three components, and are: Forwarding output from the StringGenerator via its "StringOutput" output channel to Capitalizer's

"UncapitalizedInput" channel

Forwarding output from the Capitalizer via its "CapitalizedOutput" output channel to the StringPrinter's "StringInput" input channel.

Something important to note: In our design, one output channel can only have one input channels as a destination.

Now that we have an interesting subgraph to run, we can run it using the ApplicationRunner class:

class CapitalizerProgram{ public static void Main(String[] args) { ApplicationRunner runner = new ApplicationRunner(new CapitalizerSystem()); runner.Run(); }}

Registering a Subgraph in a SubgraphSubgraphs can also be registered in subgraphs. Furthermore, subgraphs can have their input channels forwarded to input channels within, and they can have the output channels from components within forwarded to the output channel of the entire subgraph.

To demonstrate this, we can wrap the StringCapitalizer in a subgraph:

class WrappedStringCapitalizer : ISubgraph{ public void Init(SubgraphConfig config) { config.RegisterProcessor("CapitalizerProcessor", new StringCapitalizer()); config.AsInput<String>("CapitalizerProcessor", "UncapitalizedInput",

"UncapitalizerInput"); config.AsOutput<String>("CapitalizerProcessor", "CapitalizedOutput",

"CapitalizerOutput"); }}

AsInput and AsOutput forward input/output from the subgraph's channels to the input/output channels of an internal component.

In other words:

The first parameter is the registered name of an internal component ("CapitalizerProcessor"), The second parameter is the name of the channel that belongs to the internal component. The third parameter is the name of the channel we would like to expose on the subgraph.

To replace StringCapitalizer with WrapperStringCapitalizer, we can keep the channel names the same, but use RegisterSubgraph instead:

class CapitalizerSystem : ISubgraph{ public void Init(SubgraphConfig config) { config.RegisterProcessor("Generator", new StringGenerator()) .ForwardOutput("StringOutput", to: "Capitalizer.UncapitalizedInput"); config.RegisterSubgraph("Capitalizer", new WrappedStringCapitalizer()) .ForwardOutput("CapitalizedOutput", to: "StringInput"); config.RegisterProcessor("Printer", new StringPrinter()); }}

Adding Components Dynamically - The Pool Component

A pool is a composition of the same dispatcher or the same processors, i.e. each pool only one type of child, the child can be a dispatcher or a processor. Inside the pool, the children do not interact with each other. Although there are these limitations, it allows the pool to dynamically add and remove children. It accomplishes this via its input channels that take in the component to add or remove. This feature is especially important for networking components, because it allows a system to dynamically add connections; the feature enables a dynamic networking topology.

In terms of the input and output channels of a pool, they reflect the input and output channels of its children, except that the pool's input types are addressed to/from specific children. What this means is that input can be addressed to a specific dispatcher of processor in the pool, and output is labeled as coming from a specific dispatcher or processor in the pool.

The basic structure of Pool class is shown as follows:

 

public sealed class Pool<T> T: object of type T in the poolpublic function:public Pool<T> AcceptAddressedInput<I>(String inputChannelName)//register an input channel of its child as pool's inputpublic Pool<T> SupportAddressedOutput<O>(String outputChannelName) //register an output channel of its child as pool's outputpublic sealed class Pool<T>

The main purpose of a pool is that a child can be selected to execute the message and it supports dynamically adding and removing a child.

Name Data Type

"AddProcessor" Addressed<T>

"AddDispatcher" Addressed<T>(only one of these two is available at run time)

"Remove" int

x: register by the user.

inputName Addressed<I> (the int type here represents the id of the child in the pool)

A pool have 2+y user accessible output channels:

Name Data Type

"Removed" int

"Added" int

y: register by user

OutputName Addressed<O> 

Five of the channels are fixed. Others are defined by the user according to its child.

Example of registering a pool of processors:

appConfig.RegisterPool<StringNode>("Pool") .AcceptAddressedInput<string>("StringIn") .AcceptAddressedInput<int>("IntIn") .SupportAddressedOutput<int>("IntOut") .SupportAddressedOutput<string>("StringOut") .EndConfig() .ForwardOutput("StringOut", to: "Display.Display") .ForwardOutput("IntOut", to:"Display.DisplayInt") .ForwardOutput("Removed", to: "Display.ShowRemoved") .ForwardOutput("Added", to: "Display.ShowAdded");Example of registering a pool of subgraphs:appConfig.RegisterPool<StringDispatcher>("Pool") .AcceptAddressedInput<string>("StringIn") .SupportAddressedOutput<string>("StringOut") .EndConfig() .ForwardOutput("StringOut", to: "Display.Display") .ForwardOutput("Removed", to: "Display.ShowRemoved") .ForwardOutput("Added", to: "Display.ShowAdded");

Standard Networking ComponentsMotivationNote that, as we've described our framework so far, all components seem to run in a single process on a single machine. But our key goal is to build a distributed system; how can we do that when an ISubgraph only routes messages between components on a single machine?

First, as per our core mantra, let's think about what networking is in terms of data flow through our system. How is data transformed during its full journey from the Photo Collector to the Photo Monitor?

Since it'll almost always be appropriate to use socket-based communication in the networks we build, let's start there. (There are a lot of steps in this diagram! We don't always think about them, but they're always present). Note that each circle corresponds to a component, and each box represents a physical, machine boundary.

An interesting observation: The serializers and sockets have exactly the same interface as any other component in the system. A serializer takes an object from the part of the system that generated it, transforms it into a byte array, and passes those bytes to the socket. A socket, like the Photo Collector or the Administrator, is just an encapsulation of interacting with an external service.

So, we don't need to introduce new concepts to our framework to implement networking; networking can just be implemented as standard components that we provide! This decoupling keeps our invariant framework slim and therefore more maintainable and easier to reason about.

Network ComponentsTo that end, we've built some super-low-level components for you to use when networking:

DataContractJsonSerializer under the hood, so they'll be able to work with any T that implements C#'s Data Contracts.

RemoteServiceSocketListener listens for a single incoming connection, RemoteServiceSocketRequest attempts to establish a single outgoing connection. You can then send/receive bytes in either direction

RemoteRequesterService wraps a RemoteServiceSocketRequest together along with serializers and a multiplexer.

RemoteRequesterServicePool is a pool of RemoteRequesterService's.

RemoteRawConnectionService is used to to wraps a connection along with a multiplexer, demultiplexer, serializers, and deserializers.

The Raw Connection Service Pool (RCSPool) is used to enable a dynamic topology. It has a listener socket that accepts connections, and spawns RemoteRawConnectionServices in a pool.

We will provide more on how to use them below:

RemoteRequesterServiceA RemoteRequesterService is a subgraph containing several components: a RemoteServiceSocketRequest that can request connections from listeners, serializers, a multiplexer, deserializers, and a demultiplexer.

When you register a serializer for a RemoteRequesterService (Or the a pool, which will be explained later) with a given name, we framework will automatically add "In" and "Out" to the channel name. So when you are going to use the input channels and output channels of the serializer, remember to add "In" and "Out" to your defined name.

Naming of the components: For the components (subgraphs, processors, and pools) that are encapsulated inside the components that are included in the standard components we provide, we will add '[]' to include the name of components(For example "[Requester]"). So when naming a component, try to avoid the naming convention and thus the outer component's name will not have conflit with inner invisible ones.To register a requester, register each of its input & output channels using RegisterSerializer.

To register an input and output channel, specify a type T of the channel, the channel name, the serializer function, the deserializer function, and the resend policy {Policy.Discard, Policy.Ram, Policy.Disk}. The resend policy determines the caching policy if the connection is lost. There are three policies, Ram, Disk, and Discard. Ram will cache lost messages in ram, Disk will cache lost messages on disk, and Discard will discard all lost messages.

RemoteRequesterService rn1 = new RemoteRequesterService(); //This will automatically add "In" and "Out" to the channel name

rn1.RegisterSerializer<T1>("channelName1", serialize1, deserialize1, Policy.Discard) .RegisterSerializer<T2>("channelName2", serialize2, deserialize2, Policy.Ram) .RegisterSerializer<T3>("channelName3", serialize3, deserialize3, Policy.Disk) .RegisterDefaultRequester(ip, port); // Or you can use your own requester RegisterRequester(IProcessor requester)

Networking PoolsWhat if an application needs a dynamic network topology? For instance take an example where there is a Central Manager that manages hospitals that are subgraphs running on separate physical devices far away, and the Central Manager needs to communicate over the network between each hospital. Using RemoteService components, one could set up the network so that there is a one-to-one link between a network component that the Central Manager has, and that Hospital has. The problem with this approach is that it assumes that the Central Manager knows all of the hospitals that will connect at compile-time. What if the hospital wanted to be able to connect to new hospitals on the fly? Using the provided networking pools, the central manager could accomplish such a feat. To visualize how a pool works, take a look at this figure:

The Central Manager has a pool that listens for incoming requests from hospitals. When a request is accepted, a bi-directional connection component is created and added to the pool.

To initialize a Remote Requester Service Pool:

// RRS stands for RemoteRequesterServiceRRSPool rp1 = new RRSPool();

// .RegisterSerializer takes ("ChannelName", SerializerMethod, DeserializerMethod) //This will automatically add "In" and "Out" to the channel namerp1.RegisterSerializer("StringChannel", String2Bytes, Bytes2String, Policy.Discard) .RegisterSerializer("IntChannel", Int2Bytes, Bytes2Int, Policy.Discard) .RegisterDefaultNetworkService();

// the source node has the ability to send strings, ints, tell the pool to add a RRS, and remove a RRS.

SourceNode source = new SourceNode();TestSubgraph t1 = new TestSubgraph((appConfig) =>{ appConfig.RegisterComponent("RRSP", rp1); appConfig.RegisterComponent("Source", source) // RRSP has two special channels, AddRemoteRequesterService and

RemoveRemoteRequesterService // They take in the type RemoteServiceConnectionInfo, and

create/remove RRSs to/from the pool. .ForwardOutput("Add", to: "RRSP.AddRemoteRequesterService") .ForwardOutput("Remove", to: "RRSP.RemoveRemoteRequesterService") .ForwardOutput("SendString", to: "RRSP.StringChannel") .ForwardOutput("SendInt", to: "RRSP.IntChannel");});

To initial an RCSPool:

// RCSPool use the local host ip address and only need to pass in one port number to start listeningRCSPool rp2 = new RCSPool(); rp2.RegisterSerializer("StringChannel", String2Bytes, Bytes2String,

Policy.Discard) .RegisterSerializer("IntChannel", Int2Bytes, Bytes2Int,

Policy.Discard) .RegisterListeningPort(18038);

To register an RCSPool component to a subgraph:

//For example if we have a subgraph call tdTestSubgraph td = new TestSubgraph((appConfig) =>{ appConfig.RegisterComponent("RCSP", rp2)

.ForwardOutput("StringChannelOut", to: "Display.ReceiveString") .ForwardOutput("IntChannelOut", to: "Display.ReceiveInt") .ForwardOutput("Added", to: "Display.Added") .ForwardOutput("Removed", to: "Display.Removed"); appConfig.RegisterComponent("Display", display);});

The RCSPool now has two channels that emit status messages: "Added" and "Removed". When a new component is added, the GUID of the added component is emitted from this channel. Similarly, when component is removed from the pool, the GUID of the removed component is emitted via the "Removed" channel.

How to Use the Network ComponentsThis page aims to serve as a short tutorial on how to configure an instance of IDispatcher to have RemoteServiceSocketRequest or RemoteServiceSocketListener components that can send and receive messages over a network.

In the Init portion of the IDispatcher you are using, you must configure the dispatcher. Registering components is part of the dispatcher configuration, and creating and registering an instance of a RemoteServiceSocketRequest or RemoteServiceSocketListener component is how to enable network communications for a node. Note that although only the requester component can initiate connections to listener components, once a connection is established communication is bidirectional.

Here are the constructors for the two components:

RemoteServiceSocketRequest(String host, int port, string inputChannelName, string outputChannellName)

RemoteServiceSocketListener(IPAddress address, int port, string inputChannelName, string outputChannelName)

Parameters: host, address, and port should be self-explanatory.inputChannelName - The name of the input channel for the network component. What gets put in here gets sent over the network.outputChannelName - The output channel for the network component. What is received over the network goes out through this channel.

An example instantiation for running both on the same machine on port 18000 would be.RemoteServiceSocketRequest requesterComponent =  new RemoteServiceSocketRequest(

NetworkUtil.GetHostIPv4Address().ToString(), 18000, "InputR1", "OutputR1");

RemoteServiceSocketListener listenerComponent =  new RemoteServiceSocketListener(NetworkUtil.GetHostIPv4Address(),

18000, "InputL2", "OutputL2");

Now let's say you had some component that generates a string "ping", and wants to send it over the network, and a different component that can reply to "ping" with "pong" (let's name them PingComponent and PongComponent), and they want to send these messages over the network. For this example, lets say that the PingComponent is the requester component and the PongComponent is the listener component.

This is how you would initialize the components for this example, where PingGenerator and PongGenerator are concrete component classes:

RemoteServiceSocketRequest requesterComponent = new RemoteServiceSocketRequest(

NetworkUtil.GetHostIPv4Address().ToString(), 18000, "RequesterInput", "RequesterOutput");

RemoteServiceSocketListener listenerComponent = new RemoteServiceSocketListener(NetworkUtil.GetHostIPv4Address(), 18000,

"ListenerInput", "ListenerOutput");PingGenerator pingGenerator = new PingGenerator();PongGenerator pongGenerator = new PongGenerator();

Remember if the input/output channels for the network components get confusing to think back, refer to my explanation above in parameters section.

Next, the dispatcher you are using needs to be configured to use these components. In this example I will be using FrameworkTests.TestDispatcher, which allows you to define the configuration in the constructor. Let's begin with the TestDispatcher for ping:

TestDispatcher pingDispatcher = new TestDispatcher((appConfig) =>{ appConfig.RegisterComponent("PingGenerator", pingGenerator ) .ForwardOutput("ToNetworkComponent", to: "Requester.RequesterInput") .ForwardOutput("Start", to: "Requester.Start") .ForwardOutput("Stop", to: "Requester.Stop"); ;

appConfig.RegisterComponent("Requester", requesterComponent) .ForwardOutput("RequesterOutput", to: "PingGenerator.ProcessMsg");});

Here, we are registering the two components for the ping application:

1. pingGenerator:1. We are making all its input for channel "ToNetworkComponent" go to the input channel

for the Requester (which will get sent out over the network).2. To demonstrate the built-in "Start" and "Stop" channels we will

be allowing pingGenerator to control the connection status of requestComponent. When a network component receives a void input to "Stop", it is to close the connection. When it receives a void input to "Start", it is supposed to either re-connect if it is a requester. Listener is always listening after initialization and the listening does not stop after a "Stop"; "Stop" for a listener simply closes the connection.

2. requesterComponent1. We are forwarding output from requestComponent's Output (what requestComponent

recieves over the network from the pong application) to go to PingGenerator's ProccessMsg channel.

Now it's time to make a dispatcher for pong.

TestDispatcher pongDispatcher = new TestDispatcher((appConfig) =>{ appConfig.RegisterComponent("PongGenerator", pongGenerator) .ForwardOutput("ToNetworkComponent", to: "Listener.ListenerInput");

appConfig.RegisterComponent("Listener", listenerComponent) .ForwardOutput("ListenerOutput", to: "PongGenerator.ProcessMsg");});

This is very similar to the dispatcher for ping, except we are not giving pongGenerator the ability to give the listener components; this system we are building is actually a system where only the requester has the ability to disconnect/reconnect, while the listener is always listening.

Now that we have our dispatchers configured, we can now throw them in runners, spawn some threads, and see some action! To get them up and running, you could do the following:

var pingRunner = new Runner(pingDispatcher);var pongRunner = new Runner(pongDispatcher);

Thread pingThread = new Thread(() => pingRunner.Start());Thread pongThread = new Thread(() => pongRunner.Start());pingThread.Start();pongThread.Start();

Now how do we check that the components actually receive the ping/pong? For this example, we will check to see if the "ping" and replying "pong" are received by having a private variable owned by each ping/pong component, that is initially null, but is changed to the result of what it receives. We could check it in the following way, where getRecievedMsg returns what message it received over the network (via the ProcessMsg channel):

Thread.Sleep(100);Assert.IsTrue(pongGenerator.getReceievedMsg() == null);Assert.IsTrue(pingGenerator.getReceievedMsg() == null);pingGenerator.Ping();Thread.Sleep(100);Assert.IsTrue(pongGenerator.getReceievedMsg() == "ping");Assert.IsTrue(pingGenerator.getReceievedMsg() == "pong");

When I set up this example, the Assert.IsTrue calls all proved to pass, thus "ping" and "pong" were actually sent over the network!

Next, let's say we want to initially have the connection alive, but eventually kill it and bring it back up. We could do the following:

Thread.Sleep(100);Assert.IsTrue(pongGenerator.getReceievedMsg() == null);Assert.IsTrue(pingGenerator.getReceievedMsg() == null);

pingGenerator.StopConnection();Thread.Sleep(100);pingGenerator.EnableConnection();

pingGenerator.Ping();Thread.Sleep(100);Assert.IsTrue(pongGenerator.getReceievedMsg() == "ping");Assert.IsTrue(pingGenerator.getReceievedMsg() == "pong");

The PingGenerator and PongGenerator classes are listed for reference below:

PingGenerator:class PingGenerator : IProcessor{ private string receivedMsg;

private Action<byte[]> ToNetworkComponent; private Action SendStartCmd; private Action SendStopCmd;

private void ProcessMsg(byte[] msg) {

String data = Encoding.UTF8.GetString(msg); receivedMsg = data; }

public void Init(ProcessorConfig config) { this.ToNetworkComponent = config.RegisterOutput<byte[]>("ToNetworkComponent"); config.RegisterInput<byte[]>("ProcessMsg", this.ProcessMsg); this.SendStartCmd = config.RegisterVoidOutput("Start"); this.SendStopCmd = config.RegisterVoidOutput("Stop"); }

public void Start() { // no-op for example; required by IProcessor }

public void Stop() { // no-op for example; required by IProcessor }

public void Ping() { byte[] ping = Encoding.UTF8.GetBytes("ping"); ToNetworkComponent(ping); }

public void EnableConnection() { SendStartCmd(); }

public void StopConnection() { SendStopCmd(); } public string getReceievedMsg() { return this.receivedMsg; }}

PongGenerator:class PongGenerator : IProcessor{ private string receivedMsg;

private Action<byte[]> ToNetworkComponent;

private void ProcessMsg(byte[] msg) { String data = Encoding.UTF8.GetString(msg); receivedMsg = data; // send a pong back this.Pong(); }

public void Init(ProcessorConfig config) { this.ToNetworkComponent = config.RegisterOutput<byte[]>("ToNetworkComponent"); config.RegisterInput<byte[]>("ProcessMsg", this.ProcessMsg); }

public void Start() { // no-op for example; required by IProcessor }

public void Stop() { // no-op for example; required by IProcessor }

private void Pong() { byte[] pong = Encoding.UTF8.GetBytes("pong"); ToNetworkComponent(pong); }

public string getReceievedMsg() { return this.receivedMsg; }}

Building a Distributed Network Graph with NetFlowCreating the Initial ApplicationNow that we’ve talked a bit about how to use NetFlow to design an application graph and run it on a single device, we want to get to the good stuff: how do we design a distributed application graph that’s deployed across many devices?

In this document, a device is hardware on which we can run an instance of NetFlow, like a microcontroller or a cloud server. A network is a set of devices that can communicate with each other. A component is software that receives and emits data in some specified way. An application graph describes the semantics of some application’s data flow, by drawing edges between the

components of your system. When we divide an application graph into subgraphs, each of which will be deployed to a different type of device, we call the result a distributed graph.

Creating an application therefore follows these steps: Design the application graph. This step should come first, because the semantics of your data is

invariant with respect to the network through which it flows. Design the distributed graph. Divide the application graph into subgraphs that we’ll deploy to different

devices, according to your business constraints. Write NetFlow code for each device’s subgraph. Deploy each subgraph to each device.

Designing the Application GraphThe first step to designing our distributed system is the same as designing any data flow application: draw the graph describing the flow of information. Let’s not think about device boundaries yet; just where data starts, where the resulting information used, and how it’s transformed along the way.

For example, consider a burglar alarm system. We attach a motion sensor to a door and face a camera toward that same door, take photos of people entering, and notify the alarm system when these photos are suspicious. (This is the very first NetFlow application we built! Good times.)

1. The motion sensor constantly emits data in a proprietary format along a proprietary connection API.

2. The motion sensor adapter encapsulates a proprietary connection with the third-party motion sensor hardware, and analyzes that data to decide when to emit a door-is-open event

3. The photo requester transforms that event into a request that the camera take a photo.

4. The camera adapter uses the proprietary camera API to take a photo.

5. The photo labeler attaches metadata to the photo, like a timestamp.

6. The photo monitor filters photos by suspiciousness (in our case, whether the photo was taken after hours).

7. The database adapter uses the database connection API to save incoming photos for future examination.

8. The alarm relay forwards the suspicious photo to the administrator GUI.

9. The administrator GUI adapter encapsulates a GUI that shows the suspicious photo to the building’s security administrator.

Designing the Distributed GraphNow that we have our application graph, we need to consider our hardware constraints. What hardware do we have access to? What components should live on the same machine in order to avoid excessive network traffic? In short: How should we distribute the application graph across devices?

Dividing the GraphFor our hypothetical burglar alarm, we’ll divide the graph as follows: Each door is equipped with a door microcontroller that coordinates the photo-capturing process,

including interactions with the door sensor and camera hardware.

An administrator console is the computer in the administrator’s office, and runs the administrator GUI.

Between them, a central manager analyzes incoming photos and decides when to forward suspicious photos to administrator consoles.

Burglar alarm diagram, dividing the application graph into these three subgraphs.

Considering Pools of DevicesIn our burglar alarm system, we’d like to be able to add a new door or administrator without creating a brand new application. That is, the central manager should maintain a pool of connections with door microcontrollers and a pool of connections with administrator consoles, such that a new device can connect and join the pool at runtime.

We’ll talk more about how this process works in Updating the Application. For now, we just want to denote whether a connection between devices is one-to-one, one-to-many, or many-to-many.

Handling Edges across DevicesNote that we have edges that cross device boundaries, but it’s not entirely clear what that means. This communication must involve some kind of networking, but networking isn’t fundamental to flow-based programming. This interaction must therefore use some kind of third-party communication like sockets

—which makes it a perfect fit for the adapter pattern! Let’s express these edges more clearly by implementing each endpoint as an adapter over a network connection.

Burglar alarm diagram, with each connection’s endpoints updated to be adapters over network connections. Draw it the same as we draw other third-party communications, like with Kinect.

Build and DeployWe now have a self-contained application subgraph for each device in the system, so we can code up each subgraph as a NetFlow application, and deploy it to the appropriate device.

To learn how to use the provided networking components, check out the NetFlow user manual . It includes how to use the point-to-point networking components, as well as the networking pool components.

Once we boot up each device, we should have a fully functional distributed application. Hooray, success!

Updating an ApplicationAs we know, though, most applications change over time. What happens when we want to deploy a new version of the application? How will that affect the state of the network? If we’re only deploying new code to some devices in the network, how will that affect the others?

In order to think about these questions, we’ll want to more explicitly talk about the fundamental device operations that a system administrator can perform, how these operations affect network state, and how higher-level graph operations can be expressed in terms of these device operations. In this way, we’ll be able to understand how high-level changes to the deployment graph (adding a new type of device) lead to low-level network effects (updating and restarting the new device’s neighbors).

Device OperationsWe’ll define two fundamental device operations: adding a device that’s compatible with its neighbors, and removing a device. (When two devices are connected, we call them neighbors.)

For two devices to be compatible means that they will use the same protocol once a connection exists between them—and that protocol includes the types of messages that each device will send and receive.

You might be wondering, how can I add a device if it’s not compatible with its neighbors? Well, in this document, we consider it to be a bug when two connected devices are incompatible: behavior is undefined, and can lead to crashes, or, worse, incorrect output. Therefore, if you’d like to add a new type of device, you’ll first need to update the neighbors’ logic in order to make them compatible with the new device; this is one of our graph operations that we’ll discuss later.

Adding a Device That’s Compatible With Its NeighborsTo add a device to the network, push the relevant application subgraph code to it, and start it up. Easy!

The addition of this new device will not affect most of the graph, but may require its neighbors to restart, depending on how they establish the connection between them.

Using NetFlow’s standard socket-based components, one device must listen on a particular port for new connections, and the other must know the hostname and port and request a connection.

If the new device is the requester, then the job is simple: on startup, the new device requests a connection with the neighbor, and the neighbor accepts it without any need to restart. This is the more common scenario: when you build your network, you’ll choose the more transient devices to be requesters and the more permanent devices to be listeners.

If the new device is the listener, then the need to restart the neighbor depends on how it’s handling the previously missing connection. If you’re using NetFlow’s robust connection tools, then connections can be given a policy to try and reconnect. By default, this is exponential backoff, so requesters will automatically try and reestablish connection with the listener. We recommend always using robust connections, in order to remove the headache of having to restart the requesters when a listener restarts; the potential to cascade restarts across your network is just a bit too scary.

Removing a DeviceThis operation, thankfully, is dead simple: the connection with its neighbors drop, and that’s okay, because there’s nothing to connect to. The neighbors don’t need to restart, nor does any other part of the graph.

Common Graph OperationsNext, we’ll talk about some graph operations: changes that a developer might make to the graph and expect a system administrator to deploy.

Updating a Device’s Application SubgraphIf we discover that we’d like to change something about how a device behaves, it’s easy to model this use case with our fundamental operations:

1. Remove the old version of the device from the network. (That is, stop the device.)2. Add the new version of the device to the network. (That is, push new code to the device and start it.)

So, updating a device’s application subgraph requires us to restart the device itself, and no other devices in the network.

Note that Step 2 requires that the new version of the device remains compatible with its neighbors. If you’re looking to change how a pair of devices interact, see below.

Updating the Communication Protocol along a ConnectionWhen we’re changing the way in which a device interacts with other devices, we’ll need to be more careful. We don’t ever want to have incompatible devices trying to connect to each other; that’s just asking for bugs.

To update the communication protocol between Device A and Device B: Remove the old version of Device A from the network.

Remove the old version of Device B from the network.

Add the new version of Device A to the network.

Add the new version of Device B to the network.

So, updating the communication protocol between a pair of devices requires us to restart each device concurrently, and no other devices in the network.

Adding a Device of a New Type to the NetworkWe’ve already covered the rather common use case of adding a device of a known type to the graph: If you’re adding another door to the burglar alarm system, the deployment graph doesn’t change at all; just add the new door device to the network, and it’ll request to the central manager that’s always listening for new doors. Heck, the new door doesn’t even need to use the same hardware as the existing doors; so long as it’s compatible with the central manager, we can add it without touching any other part of the network.

But what happens when we have a new type of device with different input/output semantics than anything else we’ve seen before? Clearly, something else in the network will have to change: something will have to communicate with this new device, and, since no code currently exists for communicating with this new type of device, its neighbors will need to receive new code.

For each neighbor N of the new device:

1. Update N’s application subgraph by adding components to communicate with the new device. (That is, remove the old version of N, then add the new version of N.)

2. Add the new device to the network.

So, adding a new type of device requires us to restart its neighbors, and no other devices in the network.

Composing These Graph OperationsStill, when you change an application in the real world, you might be deploying many changes at once: adding a new type of device, updating all the logic downstream to handle its new information, etc. So long as you think of this larger change as a sequence of adding/removing devices, sometimes concurrently if the protocol along a connection is changing, then everything should work out cleanly: your network will never be in an incompatible state, because the fundamental operations disallow it, and you can roll out some parts of the change separately in order to avoid full-system downtime.

Proving ScalabilityWe will show the scalability of the Framework by measuring the operating costs of the Framework, and scaling those costs to appropriate limitations.

We hold that our Framework could be considered 'scalable' if the following criteria are met:

1. Each additional network connection to the Framework incurs a constant, additive overhead in performance.

2. Given that additional network connections will inevitably create additional overhead in the Framework, having overhead that is directly proportional to the number of connections is required for scalability - otherwise, if each additional network connection requires additional overhead costs, a large amount of connections might become exponentially dependent.

3. We require that each component in the Framework operates in O(N) time (with respect to the number of connections).

THE SCALABILITY TESTING HARNESSScaling tests are comprised of scaling connections between a two node system, in which both nodes share multiple connections. We compare the difference between:

1. A two-node system with N bare-socket connections between themselves, and 2. A two-node system with N Framework connections between themselves.

Comparing these two nodes will provide two different sources of the same features, to ameliorate comparative measurement bias.

We will compare bandwidth and memory, and our results will be in the form of a ratio between the results for the two node system with bare socket connections and that of the Framework connections.

RESULTSPreliminary results have unearthed small bugs in the Framework. Additional tests are blocked until the Framework Team conducts a thorough code review.

Most recent framework patch happened the day before the final presentation and we did not have the time or resources to adapt the tests to the updated framework within a day's timeframe

The Example Application DocumentationApplication DescriptionOur system simulate a disaster relief scenario where resources need to be distributed between different locations and buildings. All the decisions about how to distribute the resources are done by our central manager application. Our application and framework were built using C# for Windows.

Network Topology

Check-in ApplicationState: Stateless

Connection: Listener

External Libraries: Relayr C# SDK, Newtonsoft.Json, Kinect 2.0 SDK, XZing Barcode.

Message Types: Send PatietInfo.

Operations: Automatically insert a patient information into the system after read a QR code which contains the patient information.

The patient check-in application assumes that a patient arrived into a hospital and received some quantity of needed resources to attend the patient. After that a QR Code with the patient category and the associated resources was generated. Then, when the patient walks into the door, a accelerometer sensor receives the motion information and triggers the kinect camera which is in front of the door. The patient should wait until the kinect scan the QR code and insert the patient data into the system. The GUI will show the patient category and the time of the check-in on the screen.

GUI

Use Cases

Hospital ApplicationState: Stateful

Connection: Requester for CM, Check-in and Disaster Simulator

Message Types:

Receives - PatientInfo,

Send

Operations: Automatically send requests each 5 seconds if necessary restock some item.

GUI

Use Cases

Disaster SimulatorState: Stateless

Connection: Pool for Hospitals

Message Types:

Receives - Registration

Sends - PatientInfo

Operations: Manually operated using GUI

Disaster Generator allows users to configure and add new disasters through the GUI, and it simulates the disasters by generating patients every day for connected hospitals. 1 day in the simulation is 5 seconds in real life.

The Disaster Generator will begin simulation once it starts running. It listens for incoming connections from hospitals. A hospital could connect and register at any point during the simulation, and for any active disaster whose range includes the hospital, the hospital will receive a number of patients every day until the disaster expires.

When a hospital disconnects, its registration in the Disaster Generator will not be removed, which means that new patients could still be generated for it. These patient infos will be cached by the framework and sent to the hospital when it reconnects.

GUI

The left part of the disaster generator GUI allows users to configure and generate new disasters. There are five configuration options:

Disaster Type: a drop down list with three choices, namely Blizzard, Hurricane, and Flood.

Duration: duration of the disaster; should be a positive integer.

Latitude, and Longitude: coordinates of the disaster's center; should be non-negative integers.

Radius: radius of the disaster area; should be a positive integer.

After setting the configs, click "Generate" to generate a disaster. This disaster will start on the next day, i.e. if a user sets the duration to 5 and then clicks "Generate" on Day 1, the disaster will be active from Day 2 to Day 6.

The right part of the GUI has a grid of active disasters and their configurations. On the bottom, the GUI shows the current date and the number of registered hospitals. Currently the simulation starts at Day 1 and increases the date every 5 seconds. When a hospital connects and registers itself for the first time, the registration counter will increase by one. If a hospital is disconnected, it will not be removed and the counter will not decrease, because we expect it to reconnect back again after some time. We will continue generate patients for it, and framework's robust connection feature will ensure that the patient messages are cached and delivered when the connection restores.

Warehouse ApplicationState: Stateful

Connection: Requester for CM

Message Types:

Receives - DeliveryOrder

Sends - Registration, StateUpdate, RequestConfirm

Operations: Automatically confirm requests and restock itself.

A warehouse stores a large number of resources in its inventory. It receives DeliveryOrder from the Central Manager, which tells the warehouse how many resources are requested by which hospital. Then the warehouse delivers the order to the requesting hospital offline and confirms the delivery to the Central Manager through a state update message. A state update also includes a copy of the most updated inventory so that the Central Manager has an accurate. A warehouse automatically restocks itself to full inventory periodically. All the data of warehouse is saved in a local database.

Use Cases

Helicopter ApplicationState: Stateless

Connection: Pool for Beacons and requester to CM

Message Types:

Receives - HelpRequest, HospitalChoice, HelpConfirm

Send - Registration, GetHelpRequest, HelpRequest

Operations: Manually operated using GUI

The Helicopter simulates a mobile device that connects the disconnected part of the system (the Emergency Beacon) to the rest of the system (the Central Manager).

The Helicopter broadcasts its location to nearby Beacons, and if the Beacon has patients that need help, the Beacon responds to the Helicopter.

The Helicopter then connects to the Central Manager, which tells the Helicopter the location of a Hospital that currently has the resources to handle the patients. The Central Manager then sends the

total number of resources needed to the Hospital, which updates its inventory. The Helicopter then takes the patients directly to that Hospital, bypassing the Hospital's Check-in process.

GUI

On the left side, there’s a button called Search for Help, which broadcast a getHelpRequest message to all beacons in range. The current location shows the current helicopter position, the latitude and longitude fields are used to set a new position of the helicopter and the move button update the helicopter position according to the latitude and longitude fields.

On the right side, there’s a log which shows all the actions of the helicopter.

Emergency Beacon ApplicationState: Stateful

Connection: Requester

Message Type:

Sends - HelpRequest

Receives - GetHelpRequest

Operations: Manually operated using GUI

The Emergency Beacon application works together with the Helicopter application to simulate a disconnected device in our framework. The Emergency Beacon is a device that is isolated from the rest of the system. It can only connect to the network by the Helicopter and also just can contact the

Helicopter, when the Helicopter is close to it. The Emergency Beacon answers a Helicopter broadcast looking for some emergency in a one way communication.

The Emergency Beacon simulates a diagnostic of an Emergency First Response team who find injured people and assess their condition. The Emergency Beacon stores what total resources are needed by the injured people. Then, when the Beacon makes contact with a Helicopter, it sends the sum of the resources required by the injured patients.

GUI

On the left side, there is a list of the patients and status of the patients which were received by the Emergency Beacon. On the right side, there is a list of resources and the sum of the resources which are needed to attend all the patients.

This is window to add new patients with name and resources that the patient needs.

Use Cases

Central Manager ApplicationState: Stateful

Connection: Pool for Helicopters, Warehouse and HospitalsThe Central Manager maintains a historical and current record of each components inventory. In particular, it is aware of each Hospital's and Warehouse's inventories. When a Hospital makes a restock request, the Central Manager will find the closest set of Warehouses that can serve the request and asks

the Warehouses to dispatch deliveries to the requesting hospital. In the event that there are no Warehouses that can serve the request, the request is stored in the backlog until new inventory is introduced into the Warehouses and the request can be satisfied. All the backlog requests are saved on the database, so there’s no limit of backlog requests. Finally, Central Managers maintain connections to Helicopters, which are mobile components. Once a Helicopter picks up Patients from an Emergency Beacon, the Central Manager reroutes the Helicopter to the Hospital that can best serve the Patients' needs, and routes Warehouse deliveries for extra resources if needed. All the data of the central manager is saved in a cloud database on Azure.

Use Cases

How to set up the application

1. Configure the P2Config.cs inside P2Library file with all the IP Address and port number of the components which are going to run.

2. Start the components which have connection listeners or pools. (Central Manager, Disaster Simulator or Check-in)

3. Start the components which have connection requesters (Hospital, Warehouse, Helicopter)4. Start the disconnected application Emergency Beacon (if necessary)

Message Types:1. RestockRequest

a) bID - UUID (buiding ID)b) Request - Dictionary<Resource, Quantity>

2. StateUpdatea) bID - UUID (building ID)b) Inventory - Dictionary<Resource, Quantity>c) Delivery - dID (marked when hospital receives delivery, or when warehouse confirms delivery,

null otherwise)3. DeliveryOrder

a) dID - UUID (delivery ID, assigned by CM)b) Destination - Tuple<Lat, Long>c) Items - Dictionary<Resource, Quantity>

3. RequestConfirma) PendingDeliveries - Dictionary<dID, Dictionary<Resource, Quantity>> (In the dictionary is also ID:

dID)4. Registration

a) bID - UUIDb) Location - Tuple<Lat, Long>c) Name - Stringd) Type - enum(H, W)

5. GetHelpRequestsa) Location of helicopter

6. HelpRequesta) Location of patientsb) Dictionary: resource -> # neededc) Request ID

7. HelpConfirm (acknowledges the help request)a) Request ID

8. HospitalChoice (Location of hospital)a) Request ID

Prototype Use Case Diagrams

Prototype 1

Prototype 2

The System Diagnostics Tool DocumentationWe analyze the network so you don't have to. The system diagnostics tool has a database of logs from the running application. There is a logger in the application that sends the logs to the database. The system diagnostics tool parses the logs and displays a visualization of the network based on the logs.

Current build can be found at:

http://comp410sysdiag.azurewebsites.net/

How it works:The System Diagnostics tool website supports a basic we API. POST requests can be sent to api/LogModelEntries with enough data to make up a log, and they are directly added to the database.

Fields:Source - doesn't matter what's put in here, it's filled in server side with the IP address of the sender of the POST request

Timestamp - a timestamp that conforms to ISO 8601 standard

Id - used to differentiate between logs from the same source taken logged to closer then timestamps have precision

Severity - The severity of the log

Category - the type of the log

Log - The actual contents of the log

This is all stored in a basic relational database

The filter works simply by displaying the logs ordered by timestamp (default). The ordering can be changed by clicking on the column names. Filtering is done by SQL queries when the page refreshed, only populating the table with the filtered results

The network display tool works by taking all the logs, sorting them by timestamp, and from them building out the network step-by-step.