coordinetz: coordinated dataflow protection for ultra-high ... · stony brook university...
TRANSCRIPT
![Page 1: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/1.jpg)
Stony Brook University
CoordiNetZ: Coordinated Dataflow Protection for Ultra-High
Bandwidth Science Networks
Vasudevan Nagendra
Joint work withVinod Yegneswaran, Phillip Porras, Samir R. Das
SRI International
ACSAC 2019
![Page 2: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/2.jpg)
Agenda of the Talk
• Background on ESNet & SDMZ
• SDMZ Requirements / Limitations
• CoordiNetZ Architecture• Functional Components
• Evaluations
![Page 3: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/3.jpg)
Agenda of the Talk
• Background on ESNet & SDMZ
• SDMZ Requirements / Limitations
• CoordiNetZ Architecture• Functional Components
• Evaluations
![Page 4: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/4.jpg)
4
Background: Energy Sciences Network (ESNet)
SwitchDTNs
SDMZ Site1
SwitchDTNs
SDMZ Site2
Project (P1) : SLAC : LBNL : DENVER : ALBUQ :
KANSAS: HOUSTON
Project (P2): SLAC : LBNL : SACREMENTO : DENVER
: KANSAS : HOUSTON : ANL : JLAB
ESNet: Largest Science Networks in the World
![Page 5: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/5.jpg)
5
Background: Energy Sciences Network (ESNet)
SwitchDTNs
SDMZ Site1
SwitchDTNs
SDMZ Site2
Project (P1) : SLAC : LBNL : DENVER : ALBUQ :
KANSAS: HOUSTON
Project (P2): SLAC : LBNL : SACREMENTO : DENVER
: KANSAS : HOUSTON : ANL : JLAB
ESNet: Largest Science Networks in the World
140 Campus networks peered to ESNet
50 Petabytes per month
40+ DoE sites connected with dedicated 100Gbps WAN links (within & across countries)
![Page 6: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/6.jpg)
Virtual Circuits
Border Router
Border Router Switch
DoE/Campus Site2 SDMZ
Switch
DTNs
DoE/Campus Site1 SDMZ
Edge Firewall
DTNs
Edge FirewallLAN Hosts
Switch
DoE/Campus Site2 LAN
LAN HostsSwitch
DoE/Campus Site1 LAN
IDSIDS
SDMZ Core / InternetX *10Gbps X *10Gbps
X*10/100 Gbps
Reference: https://www.es.net/assets/pubs_presos/sc13sciDMZ-final.pdf
Elephant Flow≥ 10Gpbs
Background: Science DMZ Network Architecture
Isolated from Stateful Firewalls & DPI devices for performance
![Page 7: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/7.jpg)
Virtual Circuits
Border Router
Border Router Switch
DoE/Campus Site2 SDMZ
Switch
DTNs
DoE/Campus Site1 SDMZ
Edge Firewall
DTNs
Edge FirewallLAN Hosts
Switch
DoE/Campus Site2 LAN
LAN HostsSwitch
DoE/Campus Site1 LAN
IDSIDS
SDMZ Core / InternetX *10Gbps X *10Gbps
X*10/100 Gbps
Reference: https://www.es.net/assets/pubs_presos/sc13sciDMZ-final.pdf
Network with 0.0046% packet drops in TCP-based elephant flows with RTT greater than 20msec could result in 10X drop in
throughput.
Elephant Flow≥ 10Gpbs
Science DMZ: Network isolated from stateful firewall/DPI devices
Background: Science DMZ Network Architecture
Isolated from Stateful Firewalls & DPI devices for performance
![Page 8: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/8.jpg)
Background: Collaboration among projects across sites
Sites{project}:
Project1: Site1, Site2
Project2: Site1, Site3 & Site4
Hosts{project}:
Project1: H1, H2, H3 H4, H5, H6
Project2: H3, H4, H5, H6, H7, H8
ESNet (Science DMZ) Core Network
Project 2
Project 1X*10/100 Gbps
Site4
Site3
Site2
H6
H7H1
H2
H3
H4
H5
H8
Site1
Lacks Isolation across projects & Infrastructure
![Page 9: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/9.jpg)
Background: Collaboration among projects across sites
Sites{project}:
Project1: Site1, Site2
Project2: Site1, Site3 & Site4
Hosts{project}:
Project1: H1, H2, H3 H4, H5, H6
Project2: H3, H4, H5, H6, H7, H8
ESNet (Science DMZ) Core Network
Project 2
Project 1X*10/100 Gbps
Site4
Site3
Site2
H6
H7H1
H2
H3
H4
H5
H8
Site1
Hosts shared across projects P1 & P2:{H3, H4, H5, H6}
Lacks Isolation across projects & Infrastructure
![Page 10: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/10.jpg)
Agenda of the Talk
• Background on ESNet & SDMZ
• SDMZ Requirements / Limitations
• CoordiNetZ Architecture• Functional Components
• Evaluations
![Page 11: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/11.jpg)
SDMZ Requirements (1): Intuitive & Unified Policy Specification
Dynamic Data flow-based policies
Data-flow policies
Policy (P1)
S1 {P1(E1)}
S2 {P1(E1)}
D1: Sensitive Data of Exp1
Temporal and spatial policies
Policy (P2)
D2: Sensitive & Export Controlled
Time >9PM <7AM
S1 {P2}
S2 {P2}X
Dynamic Policy
Policy (P3)
D3: Sensitive Data & Network under
Brute-force
Notify Admin &Quarantine
S1 {P2} X
![Page 12: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/12.jpg)
SDMZ Requirements (2): Data flow tracking & enforcement
Tracking data flows and enforcing rules is challenging
Science DMZ Core Network
Site 1
Site3
Site 2
Site 4Project 2
H1
H2
H3Private Data
“D1”Data “D12a”
Sensitive Data “D2”
D1 * D2 = “D12a”
Policies:D1: Site2 -> Site4D1: !-> {Site1}
D2: !-> {Site2, Site1}
D12a: -> {Site1, Site2} ?
Data “D12a”
![Page 13: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/13.jpg)
SDMZ Requirements (3): Isolation in Shared infrastructures (multiple projects & sites)
Isolation in Abstraction & Fine-grained Policy Enforcement
Site 1: Research / Production DTN Nodes
Site 2: Research / Production DTN Nodes
Policy Framework
SDMZ Core Network
Site1 specific policies:Project 1 & Project 2
Project1 Specific Policies
Rules pertaining to projects (1 & 2)for Host DTN 1
Project2 Specific Policies
Switch1:P1: Host1 -> InternetP2: Host1 !-> Internet
![Page 14: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/14.jpg)
SDMZ Requirements (4): Context-aware security enforcement
Context-awareness and aggregation for enforcement
Dynamic data transfer ports ( FTP PORT Command): 9001:9025
SDMZ Core
Globus FTP Control
GridFTP Source DTNsGridFTP Target DTNs
Clustered IDS
Challenge: • Proactively specifying rules for
enforcement on unknown data ports.
![Page 15: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/15.jpg)
SDMZ Requirements (4): Context-aware security enforcement
Context-awareness and aggregation for enforcement
Dynamic data transfer ports ( FTP PORT Command): 9001:9025
SDMZ Core
Globus FTP Control
GridFTP Source DTNsGridFTP Target DTNs
Clustered IDS
Challenge: Lack of context to detect distributed attacks• e.g., DDoS, data exfiltration, network scans
and so on
![Page 16: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/16.jpg)
1. No stateful inspecting devices along data path• Offline DPI & coarse-grained security• E.g., IDS, shunting techniques, Router/switch ACL
2. Lack of fine-grained data security • Varying levels of sensitivity, security, privacy and
compliance.• Light-weight data tracking• Fine-grained enforcement
3. Policy specification for non-admin SDMZ users• Multi-administrative domain
• multiple projects spans across multiple sites• E.g., Researchers, Scientists & Professors• Isolation In Abstractions
Limitations: Science DMZ
16
![Page 17: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/17.jpg)
Agenda of the Talk
• Background on ESNet & SDMZ
• SDMZ Requirements / Limitations
• CoordiNetZ Architecture• Functional Components
• Evaluations
![Page 18: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/18.jpg)
Our Contributions: Performance, Programmability & Security Challenges
18
Science DMZ Networks
CoordiNetZ:• Intent-based Graph policy
framework• Graph-based specification• Fine-grained data-flow policies• Graph-Composition Techniques
• Context-aware Security • Host & network context• Stateless microservices
• Inter-site & Intra-site context-aware tagging
![Page 19: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/19.jpg)
19
CoordiNetz: High level System Architecture
Cross-Site Coordinator for Context-aware Tagging and Security Enforcement
Project1 ProjectN
CoordiNetz Dashboard
Border Router
Border Router
SDMZ Core (tier0/1) Sites
DTNsSDMZ Site2
DTNs
IDSIDS
CNZ Controller
SDN Controller
CNZ Controller
SDN Controller
SDMZ Site1
SDN Controller
Coordinated, Context-aware & fine-grained dataflow-based policy specification
![Page 20: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/20.jpg)
20
CoordiNetz: High level System Architecture
Cross-Site Coordinator for Context-aware Tagging and Security Enforcement
Project1 ProjectN
CoordiNetz Dashboard
Border Router
Border Router
SDMZ Core (tier0/1) Sites
DTNsSDMZ Site2
DTNs
IDSIDS
CNZ Controller
SDN Controller
CNZ Controller
SDN Controller
SDMZ Site1
SDN Controller
Coordinated, Context-aware & fine-grained dataflow-based policy specification
Context-awareness (tagging)• Host• network
Graph-based Policy Specification• Tree-based Abstractions• Data-specific policies• Dataflow tracking
Inter-site & Intra-site Tagging• Enforcement beyond sites• 20 bits of IPV6 • Intra-site tag assignment• Inter-site tag space allocation
Light weight stateless Microservices• Spoofing Protection, Data Exfiltration,
Collaborative Protection • On-demand security services with
lightweight microservices
Conflict detection & resolution• Dataflow-based graph
composition algorithms
![Page 21: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/21.jpg)
CoordiNetZ: Key components
1. Host DTN• SciMon• SciFlow
2. CNZ Controller
3. CNZ Coordinator
4. Stateless microservices
![Page 22: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/22.jpg)
SciMon: Science DMZ Monitor for Host DTN
• Track data flows to generates data-flow records • Sent to CNZController (-> CNZ Coordinator) to build
data-flow graphs
# [SciMon]: username, hostname, processID, appname, execpath, execArguments, execCredential, openFileList, integrity, pProcessID, pAppname, pExecPath, sensorID, sensorVer
SciMon Flow Record:
• Contextual information required for host-level enforcement• WHO (user/applications/process/experiment), WHAT
(file/network I/O), HOW (remote login), WHEN (timestamp), WHERE (country, city, IP).
• Monitor and enforce host/process-specific data policies.• Tag-based policies (IPv6 Flow label)
![Page 23: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/23.jpg)
SciFlow: Science Flow tracking for Host DTN
• Provides additional flow-specific context compared to NetFlow• e.g., DNS transaction summaries, unfinished SYN
handshakes, unsolicited ACKs, ACK timeouts, IP address reputation, geography information (domain, country, city, latitude and longitude)
# [SciFlow]: srcIP, srcPort, dstIP, dstPort, start, end, duration, protocol, state, srcZeropaks, srcDatapaks, srcAvgpak, srcBytecnt, srcPakcnt, dstZeropaks, dstDatapaks, vlan, dstAvgpak, dstBytecnt, dstPakcnt, updateTime, updateSrcBytecnt, pdateSrcPakcnt, srcPrefix, dstPrefix, updateDstBytecnt, updateDstPakcnt, icmpPakcnt, srcDomain, dstDomain, srcCountry, srcCity, dstCountry, dstCity, srcLatitude, userID, srcLongitude, dstLatitude, dstLongitude, IPScore
SciFlow flow record:
![Page 24: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/24.jpg)
CNZ Controller (1): Context & Reconciliation
Context from host & network to tag the packet
Host DTN
• Collects host/process-layer context
• Consolidates flow records and forwards to Coordinator
• Project-specific, site-specific, and host-specific rules for policy enforcement
• Triggers SDN Controller to insert flow rules for filtering malicious traffic
SDNController
Cross-Site CNZ Coordinator(Policy & Data Management Module)
CNZController
FlowModrequest
Host DTN
Data-flow records
Flow meta-data
Consolidates flow records
Project-specific policies
Reconcile to Site / host-specific
rules (tag-based)
![Page 25: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/25.jpg)
CNZ Controller (2): Intra-site tag assignment
Tag Assignment Algorithm for optimizing tag utilization
• Traditional Approach: Bit per network attribute• Projects, experiments, hosts, users, and so on
• CNZ Controller locally assigning tags to each policy
• Assign contiguous tags to policies having: • Same action attributes• Grouped together using bit masking.
• # tags required is approximately equals the number of conflict-free policies.
25
![Page 26: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/26.jpg)
CNZ Coordinator (1): Tree-based Infrastructure Abstractions
Abstractions for Isolation & Intuitive Policy Specification26
Original data(D1)
Da1 D2
Dc1 Dd
1 Dx2
Dy2
Experiment Outcomes
OutEXP2
OutEXP1 OutEXP2
OutEXP2 OutEXP3
OutEXP1
(a) Abstraction tree for experimental data outcome (AM = data{*}.experiment
{Exp1}).
(b) Abstraction tree for dynamic host security (AM = security-state{*}.site {Site1}:hosts{*}).
(c) Network vs Host-specific abstractions of Site 1. buildings{*}. site{Site1}:networks{*}:hosts{*}
![Page 27: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/27.jpg)
CNZ Coordinator (2): Dataflow tracking Dashboard (within host & across geographies)
Lightweight dataflow tracking dashboard 27
Original data(D1)
Da1 D2
Dc1 Dd
1 Dx2
Dy2
Experiment Outcomes
OutEXP2
OutEXP1 OutEXP2
OutEXP2 OutEXP3
OutEXP1
(a) Experimental data transformation
SDMZSite1
SDMZSite3
D1DT1
DT1
DT2
DT2
D1
DT1
SDMZSite2
D2
(b) Dataflow tracking across sites
![Page 28: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/28.jpg)
CNZ Coordinator (3): Graph-based ACL policy specification
ACL-based policies 28
![Page 29: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/29.jpg)
CNZ Coordinator (4): Dataflow Policies
Fine-grained Dataflow policies
Simplified Data Policy Syntax:
site{Site1} -> data{D1} -> site{Site2}
site{Site1} -> data{D1} -> site{Site3}
site{*} -> data{D1/*} !-> site{Site3}
/* Default drop enforced on rest
of traffic automatically */
SDMZSite1
SDMZSite3
D1DT1
D1
DT2
DT1
D1DT1
SDMZSite2
D1
DT1
29
![Page 30: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/30.jpg)
CNZ Coordinator (4): Composition for Conflict detection
Conflicting Policies: {P2,P3, P7}
Policy Composition
Engine
Graph-based Composition (ACLs & Dataflow Policies):1. Normalize policies2. Graph-based composition3. Precedence for resolution
Source Entities
Devices, Applications,
Networks
Initial State
State = Monitoring
State = Compromise
Time > 10PM < 7AM
Time > 7AM < 10PM
Time > 10PM < 7AM
Time > 7AM < 10PM
Devices, Applications,
Networks
Target Entities
Target State
Turn Heater OFF
Motion = NO
.....
.....
.....
.....
.....
.....
FW DPI
States / conditions / triggers
NFC / Actions for ACL
.....
Actions for Trigger-Action Rules
Open DOOR
Open WindowState =
Quarantine
30
![Page 31: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/31.jpg)
CNZ Coordinator (5): Context-aware Inter-site Tagging (cTags)
Tag Assignment Algorithm for optimizing tag space reuse
Tag-space Assignnment:
• Efficiently assign non-overlapping tag space across sites
• Edge Coloring to assign non-overlapping tags for Optimizing Tag space reuse
*Px – Project ID, Cx – Color assigned to a project
31
Reuse the same color among other projects across sites:• C1:“Tag space should never overlaps with the tag space
assigned to its immediate adjacent sites with which the current site has project association”
• C2: Color and Tag size depends on the number of policiesenforced by the project.
![Page 32: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/32.jpg)
CNZ Coordinator (5): Context-aware Inter-site Tagging (cTags)
Tag Assignment Algorithm for optimizing tag space reuse
*Px – Project ID, Cx – Color assigned to a project
32
Reuse the same color among other projects across sites:• C1:“Tag space should never overlaps with the tag space
assigned to its immediate adjacent sites with which the current site has project association”
• C2: Color and Tag size depends on the number of policiesenforced by the project.
![Page 33: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/33.jpg)
CNZ Coordinator (5): Context-aware Inter-site Tagging (cTags)
Tag Assignment Algorithm for optimizing tag space reuse
• Allocates necessary tag space to each project • Additional slack tag space for future policies.
• Tag space allocation done globally at the CNZ Coordinator
Goal: Maximize efficient reuse of tag space cross-site projects, while avoiding overlaps.
33
![Page 34: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/34.jpg)
Stateless microservices for data flow protection
Flexibility: Stateless & functional decomposition
• Tag-based filtering • vs traditional ACL-based filters
• Tag-based rate limiting
• Tag-based connection tracking
• Preventing DTN hosts from tag spoofing • In order to bypass SDN-enforced flow controls
• Preventing malicious exfiltration of sensitive data
34
![Page 35: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/35.jpg)
Overview: High-level Architecture
Context from host & network to tag the packet
SDN Switch
SDNController
Cross-Site CNZ Coordinator(Policy & Data Management Module)
GridFTP
SciFlow + (GeoIP DB)
Host DTN
cTags: Situational-aware Flow
tagging
Tag-based flow rules to dynamically steer
/ shunt TrafficProcess
Policy Table
Flow Records
Border Router
SDMZ WAN Core
Dynamic Data Flow Policies &
Tag assignment
CNZController
SciMon
35
Inline context-aware protection
![Page 36: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/36.jpg)
1. Context-awareness • Host-process level context to tag flows
• SciMon, SciFlow modules
2. Graph-based Policy framework• Dataflow level policies• Dataflow tracking
3. Graph Composition algorithms• Dataflow-based graph composition algorithms
4. Context-aware tagging• 20 bits of IPV6 tagging• Intra-site tag assignment & Inter-site tag allocation algorithms• Optimize the tag assignment and allocation
• Edge coloring
5. Light weight security stateless microservices• Spoofing Protection, Data Exfiltration, Collaborative Protection,
On-demand security services with lightweight microservices
Recap: CoordiNetZ Capabilities
36
![Page 37: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/37.jpg)
Agenda of the Talk
• Background on ESNet & SDMZ
• SDMZ Requirements / Limitations
• CoordiNetZ Architecture• Functional Components
• Evaluations
![Page 38: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/38.jpg)
Evaluation: Policies & Dataset for benchmarking
38
• PS-1: From 2 different SDMZ network infrastructures • With ~150 & ~400 SDMZ policies • ~5325 and ~7987 enforceable rules
respectively
• PS-2: Large synthetic policy set of 20k policies • Derived from PS-1• Emulating 40 different SDMZ networks.
• DS-1: High Energy Physics - Theory collaboration network dataset • Employs ≈9.8k nodes, with ≈25k edges.
![Page 39: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/39.jpg)
Evaluation: Composition
39
Composition efficiency:
• Composing 20K policies took ~49 sec• ~30 abstraction
trees and ~15% conflicts.
![Page 40: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/40.jpg)
Evaluation: Composition
40
Composition efficiency:
• Reduced composition latency by upto 2.25X • compared to
composition with out caching
![Page 41: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/41.jpg)
Evaluation: Tagging Optimization
Tagging efficiency:
• ≈4 – 5× fewer bits than bit segmentation and ≈ 3 – 4× fewer than Alpaca and FlowTags.
• For Syn Cam Net1: 2.2 – 3× fewer than Alpaca and FlowTags.
41
![Page 42: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/42.jpg)
Evaluation: Rule Optimization
Rule space efficiency:
• ~40% – 47% rule-space improvement compared to Alpaca, FlowTags and BS.
• For Syn Cam Net1: ~55% rule-space improvement
42
![Page 43: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/43.jpg)
Evaluation: Customized Microservices
Flow processing Performance (Tag-based filtering): • ~8 – 12% throughput improvement with tag-based filtering
• ~92% (128-bytes) & ~99% (9000-bytes) packets.• Vs traditional stateless IPv6 ACL-based filtering
• ~6.6% drop in throughput for Spoofing Protection • Vs line rate.
• ~10% Improvement for Tag-based connection tracking • Vs flow-based connection tracking. 43
![Page 44: CoordiNetZ: Coordinated Dataflow Protection for Ultra-High ... · Stony Brook University CoordiNetZ: Coordinated Dataflow Protection for Ultra-High Bandwidth Science Networks Vasudevan](https://reader034.vdocument.in/reader034/viewer/2022042022/5e7a1ea4f5047435841561ef/html5/thumbnails/44.jpg)
• Provides situational-awareness, policy specification and enforcement across SDMZ sites.
44
Summary: CoordiNetZ
CoordiNetZ: Dataflow policy specification and enforcement architecture for SDMZ
Cross-Site Coordinator for Context-aware Tagging and Security Enforcement
Project1 ProjectN
CoordiNetz Dashboard / Graphical UI
Border Router
Border Router
SDMZ Core (tier0/1) Sites
DTNs SDMZ Site2DTNs
IDSIDS
CNZ Controller
SDN Controller
CNZ Controller
SDN Controller
SDMZ Site1
SDN Controller