The Fermilab Network, Computer Security,

and you….

Phil DeMar / Donna Lamore

Computer Security Awareness Day

March 8, 2005

Fermilab Network Overview

~10,000 systems Organized on model of work group LANs

Organizational: AD, CD, PPD, TD, BSS, DIR, ESH, FESS, LSS

Experiment: CDF, D0, CMS, MINOS, mBoone, SDSS

Geographical: Fixed Target, Site 38, Village

Work groups supported on switches that connect to the core network

Core Network Facilities & Essential Network Services

Core network facilities: FCC core router WH core router Border router

Essential network services: Name service Dynamic address

allocation service Time service

ADLAN

Site 38

Off-Site[Internet]

FCC Offices

FCCComputingResources

WH OfficeLANs

FCCCollapsedBackbone

Switch/Router

WHCollapsedBackbone

Switch/Router

SiteBorderRouter

622Mb/s

TD/IC

Village

CDF

D0

SDSS

MiniBoone

CMS

FTArea

MINOS

Off-site Network Access

Off-site traffic traverses border router: Delineation point between onsite & offsite Our 1st line of defense against the Internet

Flow data collected on border router: Logs all off-site network connections

Source/destination IP addresses & ports Flow timestamp & duration, bytes/packets sent & received

Useful for detecting infected systems & investigating computer security incidents

We are also collecting flow data on internal routers

Off-site Network Access (II)

Current site perimeter access policy: Open inbound access with a few protections Open outbound access with minimal restrictions

Changes to default inbound openness under discussion: Likely a multi-level security zone architecture

Green zone = default inbound allow Yellow zone = default inbound deny

Openness for open science collaboration is recognized as a requirement

Off-site Network Access (III)

An alternate very high bandwidth offsite path now in place:

Via dark fiber connection to StarLight

Intended use – high impact scientific data movement

StarLight

ESnet

FNALBorderRouter

ESnetRouter

CERN

SD1648 SM

Communication Subsystem Shelf

SD1648 SM

Communication Subsystem Shelf

FNALDWDMgear

FNALDWDMgear

Onsite

Off-site

FNALDark Fiber

to StarLight FNAL

FNAL6500

@StarLight

FNALStarLight

Router

622

Mb

/s

FNALNetwork

Abilene

GeneralInternet

ProductionNetwork (10GE)

StarLight10GE Path

ProductionNetwork (1GE)

(NBC Bldg)

UltraScience

Net

UltraLightUKLight

CAnet4

Restrictions on Network Facilities & Services at Fermilab

The network is a restricted central service Per the Fermilab Policy on Computing

http://computing.fnal.gov/cd/policy/cpolicy.pdf

Prohibited activities include: Routing & bridging (switching…) on systems

attached to the campus network Using IP addresses not assigned to you Offering DNS, DHCP, or NTP services

Routing/Bridging Restrictions:

Applies to systems directly or indirectly attached to the facility network

Backend networks with dual-homed (gateway) systems are allowed, but No forwarding of traffic through the gateway system No use of Network Address translation (NAT) Use Fermilab-assigned (RFC1918) address blocks

Private hardwire networks with no direct or indirect connection to the facility network is OK Sorry, no private wireless networks…

Accessing the Fermilab Network

System registration is required to be granted a usable address on the facility network

Two types of network addresses are allocated: DHCP – dynamic, but temporary IP address

Useful for mobile systems Convenient for proper network configuration on a system

Static – fixed, but constant IP address Immobile; address is bound to a specific subnet Necessary for systems offering services

Static IP address registration

Static IP address : Requested via MISCOMP

https://fncdug1.fnal.gov/misnet/

MAC address(es) required to receive an IP address Additional necessary information:

Sysadmin Location Hardware information

Plan to require static IP renewal once a year

DHCP address registration

Two Types of DHCP address registration Permanently registered DHCP (Normal)

Register via MISCOMP (https://fncdug1.fnal.gov/misnet/) MAC address(es) must be registered Same sysadmin, location, & hardware info as for static IP Yearly renewal will become necessary soon

Temporary – Cinderella Registration Initial browser access forces Web Registration page

− Registration info: name, e-mail addr., contact info IP address good till midnight; then you must re-register Maximum 5 Cinderella leases per 30 days

Wired Connection to Site LAN

DHCP supported on most subnets: Plug in & registered systems are on the network

Static IP address requires proper configuration for the local subnet Contact local support person for assistance

Helpdesk – 2345 to report problems

Accessing the Wireless Network

DHCP support only

Wireless LAN support covers most of the site 802.11B – 11 Mbs Beginning to deploy 802.11G – 54 Mbs

Authentication: Currently no authentication for wireless access SSID is broadcast Likely to change in the future

Wireless Network No-No’s

You can’t install your own Access Points (AP): See Fermilab Policy on Computing – a restricted

central service Or enable any AP capability on your notebook Developing automated rogue AP detection tool

Bridging must be turned OFF on user devices A known problem with Windows XP Switches set to shutdown ports on systems with

bridging enabled

Remote Access – Dial-up

Dial-up: Now uses Radius authentication V.34 – typically 28.8kbps

No plans for further upgrades If the obsolete, out of warranty modem pool dies, no

replacement…

Limited to on-site access only Last resort ?

Dial-up ISDN phased out completely

Remote Access – VPN

VPN Provides encrypted tunnel through internet Assigns virtual local Fermilab address

Allows access to Fermilab machines restricted from offsite Allows access to protocols blocked at Border

Must use Cisco VPN client & FNAL-provided profile Yearly renewal necessary:

Involves updating FNAL-provided VPN profile

Request account at: https://www-dcn.fnal.gov/vpn/vpn_reg.cgi Need ID number, Associated Workgroup

Appropriate Use

From the Fermilab Policy on Computing:

“ Fermilab encourages effective use of computing technologies in all aspects of its activities. Fermilab maintains an open scientific environment where the free exchange of ideas is encouraged and protected. We permit a wide range of computer activities including incidental use for private purposes. We encourage use of the Web and other Internet communication channels. With this comes the responsibility for every Fermilab employee and user to exercise common sense and good judgment. “

Appropriate Use (cont.)

Network Appropriate Use primary concerns: Potential public embarrassment to the Laboratory Consuming Significant Resources (excessive use)

Examples of traffic where common sense and good judgment should come into play : Acting as a server for P2P distributed file systems

Kazaa, eDonkey, Gnutella, NAPster, Skype, etc…

Game Sites Auctions

Traffic monitoring thru the border router

Flow data generates daily & hourly Top 20 reports on: Top talkers, top listeners, top conversations Breakouts by number of flows, bytes, or packets

Primarily checking for: Unusual consumption of network resources Unusual traffic patterns

Large numbers of offsite hosts contacted Large amounts of data transferred

Border Router Network Blocks

Border Router static blocks: Exceptions to inbound default-allow

Netbios IRC Web Servers require exception

Autoblocker: Based on quasi-realtime flow record analysis Blocks “greedy” users (perceived as scanners…)

Automated unblocked after behavior stops

Occasionally blocks “greedy”, but real applications New version should minimize those disruptions

Internal Network Blocks

DHCP service: When requested by Computer Security Team (CST)

Typically to isolate a vulnerable or infected system Unblocked only upon approval from CST

For network Infractions – excessive use, restricted central service

Unblocked when corrected

Static IP address internal block: Normally at the request of CST

Unblocked only after approval from CST

MAC address black-hole Implemented on local switch At request of FCIRT – during an incident

Unblocked at request of FCIRT

Network Infractions – illegal IP address use, excessive use, restricted central service

Unblocked when corrected

Switch port block Occasionally used for expedient network disconnect

Too easy to get around Can affect other users/systems on same switch port

Internal Network Blocks (cont)

Helpful Links

Network info available on Data Comm web site http://www-dcn.fnal.gov/

Network Stats: http://fndcg0.fnal.gov/~netadmin/onsite/stats.html Node Locator: to find point-of-attachment & associated

switch traffic graphs NDT Tester: useful in testing for connectivity/duplex

problems

Trouble Reporting – x2345 – helpdesk

Questions…

??