![Page 1: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/1.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Networking –Current Status
FNAL Computer Security
Peer Review
Phil DeMar
March 22, 2005
![Page 2: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/2.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Outline
• FNAL Network Overview• Perimeter Controls & Tools• Internal Network Controls & Tools• Network Critical System*
* Termed ‘Major Application’ in the new CSPP under development
![Page 3: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/3.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
FNAL Network Overview
• A centrally-managed campus-wide network– Restricted central services (FNAL Policy on Computing…):
• Routing & bridging– Separately admin’ed AD network grandfathered in policy
• Address, name, & time services• Exemptions rarely granted
• Architecture based on work group model:– Affinity groups w/ their own dedicated LANs
• Based on experiment, organization, geography• Mostly physical LANs; a few vLANs w/ trunking• Detachable from campus network, if necessary
![Page 4: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/4.jpg)
![Page 5: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/5.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Core Network Facilities & Essential Network Services
• Core network facilities:– FCC collapsed backbone– WH core router – Border router
• Essential network services– Name service– Address allocation services
• Static addresses• DHCP service
– Time service– VPN service
ADLAN
Site 38
Off-Site[Internet]
FCC Offices
FCCComputingResources
WH OfficeLANs
FCCCollapsedBackbone
Switch/Router
WHCollapsedBackbone
Switch/Router
SiteBorderRouter
622Mb/s
TD/IC
Village
CDF
D0
SDSS
MiniBoone
CMS
FTArea
MINOS
![Page 6: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/6.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Internal Network
• A single, general network access zone:– No customized access restrictions for individual
work groups
• Critical System* LANs:– Networks supporting collection of related systems
who’s compromise could seriously impact the laboratory’s science programmatic operations
• Designated by the CSExec– Individual plans, typically with customized network
access & protections
* Termed ‘Major Applications’ in the new CSPP under development
![Page 7: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/7.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Critical Systems (aka Major Applications)
Critical System Network Access Protection Operational Management
Accelerator controls network
Firewall w/ VPN AD
Business systems network
Firewall w/ border router ACLs
BSS
CDF Online network Router ACLs CD Networking
D0 Online network Router ACLs CD Networking
Network Firewall w/ VPN CD Networking
Authentication systems Host-based protections CD Security Team
MetaSys building controls
Isolated vLAN w/ Firewall & VPN
CD Networking
![Page 8: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/8.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Off-site Network Access (I)
• Current site perimeter access policy:– Open inbound access with a few protections:
– Netbios (TCP ports 135, 137 – 139, 445)
– SunRPC* (TCP/UDP port 111)
– Web Servers (TCP ports 80, 443)
» Exemption process available– SMTP (TCP port 25) except for facility mail servers– DNS (TCP port 53) except for facility DNS servers– SNMP* (UDP port 161)
– Open outbound access with minimal restrictions:– IRC (TCP default ports 6667-6669)
* also blocked outbound
![Page 9: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/9.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Off-site Network Access (II)
• An alternate very high bandwidth offsite path now in place:– Via dark fiber connection
to StarLight– Intended use – high
impact data movement– Redundant path for
production offsite link
StarLight
ESnet
FNALBorderRouter
ESnetRouter
CERN
SD1648 SMCommunication Subsystem Shelf
SD1648 SMCommunication Subsystem Shelf
FNALDWDM
gear
FNALDWDM
gear
Onsite
Off-site
FNALDark Fiber
to StarLight FNAL
FNAL6500
@StarLight
FNALStarLight
Router
622
Mb
/s
FNAL
Network
Abilene
GeneralInternet
Production Network (10GE)
StarLight 10GE Path
Production Network (1GE)
(NBC Bldg)
UltraScience
Net
UltraLight UKLight
CAnet4
• Default-deny inbound access w/ ACL exceptions- Redundant path traffic goes thru border router
![Page 10: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/10.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Border router flow data
• Logs all off-site network connections– Useful for investigating computer security incidents
• Generates daily & hourly Top 20 reports on:– Top talkers, top listeners, top conversations– Breakouts by number of flows, bytes, or packets– Unusual traffic patterns
• Large numbers of offsite hosts contacted• Large amounts of data transferred• Unusual consumption of network resources
• Now collecting flow data on internal routers
![Page 11: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/11.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
AutoBlocker
• Based on quasi-realtime flow record analysis• Blocks “greedy” users (perceived as scanners…)
– Outbound or inbound scanners– Address-based scans or port-based scans– Automated unblocked after behavior stops
• Proven useful in blocking infected local systems– Alerts for out-of-ordinary flow patterns– Occasionally blocks “greedy”, but legit apps
• Mostly nuisance apps, such as P2P, games…• New version should minimize those disruptions
![Page 12: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/12.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Telecommuting Access
• VPN service available– Encrypted tunnel capability to the Laboratory– Assigns virtual local Fermilab address– Allows site access to protocols blocked at Border – Must use Cisco VPN client & FNAL-provided profile
• Standard configuration forced onto users• Split-tunneling restricts tunnel data flows to
FNAL-related traffic
• Dial-up: – Uses Radius authentication – Limited to on-site access only
![Page 13: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/13.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Node Registration
• System registration is required to be granted a usable address on the facility network
– Permanent registration in MISCOMP database for either static or automatic DHCP address:
• Key information required: MACs, sysadmin – Temporary DHCP service available for transient
users not registered in MISCOMP:• Provides DHCP lease good for rest of the day• Re-registration necessary every day
– 5 day limit per 30 day period
![Page 14: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/14.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Node Registration Monitoring
• Currently checking for unregistered static IP systems via simple ping utility– Doesn’t work so well with software firewalls…– Not useful at all for DHCP subnets
• Have developed a prototype to check ARP table information for proper registration:– Verifies IP/MAC tuples observed on network
correlates to registered MISCOMP information– 2-3 months away from being production use tool
![Page 15: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/15.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Node Tracking
• Router ARP & switch FDB tables gathered every 20 minutes
• Node Locator utility manipulates ARP & switch FDB data to:– Identify location of IP or MAC address on the network– Provide switch port information for the system– Provide traffic utilization for switch port
![Page 16: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/16.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Infrastructure Monitoring & Response
• Network management stations monitor status of network devices & servers:– Device and server reachability & uptime monitored– Service response (DNS, DHCP, & NTP) also monitored
• Off-hours support:– Automated device/service paging during off-hours
• Two people on call at all times– Escalation procedures to Section, Dept., then Division Heads
– User problem reporting via HelpDesk off-hours service
![Page 17: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/17.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Wireless Support
• WLANs cover major work areas of the site• Not treated differently than wired access
– Broadcast SSID– Authentication not required– Encryption not required– Node registration required
• But tightening down on vulnerabilities:– Migrating to wireless subnets (70% complete)– Rogue detection based on Cisco Wireless LAN
Solution Engine (WLSE) & war drives– Site border scans checking for offsite bleed-thru
![Page 18: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/18.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
The Network Critical System*• Network Critical System*:
– “Those parts or components of the network necessary to sustain the operation of the general facility network as a functioning entity”
– “Those parts or components of the network that are an integral part of an activity or operation whose compromise could seriously impact the Laboratory’s science programmatic operations”
• CSPP Network Critical System* Plan:– Protects network critical system components themselves– Current plan is version 2; revised 4/7/2003
• Next revision due in line with new CSPP * also known as Major Application
![Page 19: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/19.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Components
• Facility core network devices:– FCC & WH core routers
– Border router
• Servers for essential network services:– DNS, DHCP, NTP
• Run-II experiment network “core” routers– Off-line network core router
– On-line network router
![Page 20: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/20.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Network Management LAN
• Isolated LAN to controlled access to:
– Network Critical System* core & border routers
• Also other major network devices in the FCC & WH– Enterprise DNS/DHCP server & NTP time sources
• Misc other servers (ie., Radius server…)
• Used for:– Remote console access & configuration management– O/S upgrades– snmp/statistical data collection
* also known as Major Application
![Page 21: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/21.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Network Mgmt LAN Figure
Cisco PIX
WH FCC
WH FCC
EnterpriseDNS/DHCP
server
GPStime
servers
X
NetworkMgmt
system
NetworkManagement
LAN
General Facility
LAN
GPStime
servers
VPNConc.
PIX Firewall
BorderRtr
<Off-Site>
RadiusServer
DNSServer
DHCPServer
DNSServer
DHCPServer
![Page 22: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/22.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Network Mgmt LAN (cont)
• Physically separate from campus LAN– Dedicated fiber; dedicated switches
• Firewall protected w/ default deny inbound– Exceptions for necessary server traffic & monitoring:
• DNS/DHCP traffic• NTP traffic w/ stratum-2 NTP servers (ie., routers)
• Remote terminal access via VPN• Network management system dual-homed to
general LAN & network management LAN
![Page 23: 2005 FNAL Computer Security Peer Review and Self Assessment Networking – Current Status FNAL Computer Security Peer Review Phil DeMar March 22, 2005](https://reader033.vdocument.in/reader033/viewer/2022050714/56649d255503460f949fc57f/html5/thumbnails/23.jpg)
2005 FNAL Computer Security Peer Review andSelf Assessment
Questions…
?