GTRI_B-1

GaETC 2006: Open Source Email Server Solutions

and Email Filtering Techniques for the K-12 Environment

11/16/06

Jason Kau

Georgia Tech Research Institute (GTRI)Georgia Tech Information Security Center (GTISC)jason.kau@gtri.gatech.edu404-407-8806

GTRI_B-2

About Me• specialize in applied network security, design, implementation, benchmarking

• current assignments: - [CISAnet] primary network and security engineer for the Criminal Information Sharing Alliance Network (CISAnet), a private network connecting ten state police agencies, Department of Homeland Security (DHS), the Drug Enforcement Agency’s El Paso Intelligence Center (EPIC) and the Regional Sharing Information Systems Network (RISSnet)- [JSIC] Assisting the Joint Systems Integrated Command (JSIC) of the US Military Joint Forces Command (JFCOM) on their wireless networking Security and Technical Implementation Guide (STIG) - [Jasper] consulting to Jasper County School District in Monticello, GA- [F3] member of GTRI Foundations for the Future

GTRI_B-3

F3: Foundations for the Futurehttp://www.f3program.org

“a collaboration of Georgia Tech researchers working with government and industry support to ensure universal K-12 technology access and effective use in Georgia…"

"focus of F3 is to help accelerate the application of telecommunications technology for interconnecting K12 schools for collaborative learning, remote access to educational facilities, and Internet-based resources…"

"the mission is to leverage existing investments and expertise to promote powerful, effective, and feasible alternatives that improve educational practice in Georgia through the innovation applications in technology…”

GTRI_B-4

F3: Foundations for the Future

F3 has been active in 80 county and city school systems in the areas of:

• technical assistance

• professional development workshops

• proposal writing assistance

• technology demonstrations

• conference sponsorship and planning

GTRI_B-5

So what is the purpose of this session?

• Illustrate why you can't fit these two topics (servers and filtering) into a single 1 hour session (hint: I'm probably not going to get to everything)

• Part 1: Make school systems, especially K12, aware of some of the closed source, semi-open source and open source email/messaging/groupware/collaboration alternatives

• Part 2: Discuss email threats and the techniques to mitigate their impact on a school system and its users

• Part 3: Jasper County School District case study

GTRI_B-6

Part 1:

Email/Messaging/Groupware/Collaboration Server Software

Suites

GTRI_B-7

Background: Email/Messaging/Groupware/Collaboration Servers

• We expect more from our email server these days: email, calendaring, journal/notes, forums/bulletin boards, wiki's, integrated messaging (IM, VoIP), document management, project management, knowledge base, time tracking, etc. (future attack vector?)

• So we now call them Messaging/Groupware/Collaboration server software suites

• The big four: Microsoft Exchange, Lotus Notes, Novell Groupwise, Sun Java Communications Suite

• Strong market of smaller (smaller companies, smaller installed bases) closed source, semi-open source and open source alternatives to the big four

GTRI_B-8

What do I mean by closed source vs. semi-open source vs. open source?

• “Open-source software is computer software whose source code is available under a copyright license that permits users to study, change, and improve the software, and to [freely] redistribute it in modified or unmodified form.” – Wikipedia

• Open Source Initiative (OSI)'s Open Source Definition (OSD):http://www.opensource.org/docs/definition.php

• semi-open source: 1) a portion of the suite is open source or being open sourced or 2) there is an open source edition/version of the suite (usually missing functionality, containing restricted functionality or poor documentation)

• warning: closed source suites often conflated with open source when they run on Linux servers

GTRI_B-9

What do I mean by proprietary/closed vs. semi-open source vs. open source?

• open source does not equal free (as in beer); free (as in beer) software can be closed source; can charge for open source but freely distributable part of the license makes it unlikely you'll receive revenue by charging a fee; open source is free (as in liberty)—the freedom to inspect it, modify it, extend it, etc.

• all of these suites including the proprietary/closed and non-open source editions probably utilize open source under the hood (some more extensively than others)

• most rely on proprietary/closed source/not free (as in beer) MAPI Connector for Microsoft Outlook support; not counting that against them as to their degree of open-source-ness

• open source MAPI connectors not mature

GTRI_B-10

So what are these closed source, semi-open source, and open source alternatives?

• Zimbra• Gordano Messaging Suite• Opengroupware• Scalix • Open-Xchange• @Mail• CommuniGate Pro• Bynari Insight• eGroupware• Horde• Samsung Contact (dead)• PostPath• Kerio MailServer• Kolab

GTRI_B-11

What are the advantages of these messaging/groupware/collaboration suites

over the big four?• Innovation

first anti-spam efforts were implemented in open source first suites to implement integrated anti-spam first to fully leverage AJAX/Web 2.0 (Zimbra and Scalix) first to offer linked attachment handling (Horde)

• Cost costs less per user (debatable in the educational world) arguably can run on less costly hardware

• Lesser dependency/vendor tie-in Microsoft Exchange requires Active Directory/Internet Explorer Better support for 3rd party browsers (Firefox, Safari)

• Greater ability customize and extend• Better understanding by looking under the hood• Ability to write your own patches

GTRI_B-12

Differentiators of these suites

• applications they include (e.g. some do not have unified messaging) and applications they depend on

• Outlook support/compability: native MAPI support (PostPath), MAPI connector (Zimbra, Gordano, Scalix, Kerio, etc.), fullness of MAPI connector (e.g., Opengroupware's does not do email– expect you to use IMAP)

• degree to which they are open source (already discussed) and the open source license

• differ on the support offerings: direct commercial support (Zimbra, Gordano, Scalix, Kerio, etc.), certified partners (Horde), effectively only community support (eGroupware)

• country of origin (support issues, language issues)

GTRI_B-13

Example: Zimbra Collaboration Suite (ZCS) 4.0

• open source edition missing functionality: attachment search, online backup/restore, clustering/high availability, storage management, rebranding, domain-level administrators, Outlook support, mobile device support

• Full Outlook support via MAPI connector in standard and professional additions

• Heavy use of underlying open source (Apache, Tomcat, MySQL, amavisd-new, ClamAV, postfix, Swatch)

GTRI_B-14

Making the decision

• decide what groupware/collaboration applications you need, functionality you need within those apps and what you can live without

• determine how important Outlook support is and compare how that functionality is delivered (MAPI connector required, fullness of MAPI connector [does it do email, how complete is the shared support], how MAPI connector is distributed [automatic, manual])

• consider the expertise/experience of your staff and obtainable staff and thus the level of support you need

• read the vendor product literature; search for trade magazine reviews

• consider the suite’s conversion tools from your current platform

GTRI_B-15

Making the decision

• for suites that are largely community driven efforts (Horde, eGroupware, Kolab), determine how active the community is at development efforts and fixing bugs

• try to determine financial status (hard with private companies)

• based on the above, pare down to three or four and perform "bake-off" (in-house evaluation)

GTRI_B-16

Part 2:

Mitigation of Email Threats (Filtering)

GTRI_B-17

The Email Threats

• Spam reputation harm liability data storage bandwidth

• Phishing defrauding your users or school

• Virus/Malware/Worm/Trojan liability data loss productivity loss

GTRI_B-18

The Business Model

- Wikipedia

GTRI_B-19

Anti-spam filtering techniques

• heuristic spam filtering *• statistical spam filtering *• challenge-response• collaborative checksum or signature filtering *• sender policy framework *• internal inoculation *• external inoculation *• URL blacklists *• IP address blacklists *• tarpitting• greylisting *• honeypots *• disposable email addresses *

• litigation• reputation analysis• sender checks *• address munging *• OCR *• recipient verification *

Key:

* = ones that are practical and/or effective (in my opinion)

Red = effective by itself

GTRI_B-20

Anti-spam filtering techniques

• with the exception of statistical spam filtering (trained to your environment), generally speaking none of these techniques should be used exclusively to judge an email as ham vs. spam

• instead these techniques should be used to assign a score; the higher the score, the more spammy; the lower the score the more hammy

• the administrator or user makes the determination as to what score level determines spam and what score level determines ham based and what do with them; you could take this further with more than two levels, e.g. ham, low-scoring spam, high-scoring spam

GTRI_B-21

Heuristic Spam Filtering

• scoring emails “using rules to identify specific characteristics of spam and non-spam”, e.g. performing a textual analysis of the message looking for specific words, phrases, URLs; looking for malformed or forged headers; etc.

• cons: 1) rule sets were designed for everyone to use and thus are water downed to reduce false positives, 2) spammers have access to these rules to improve their emails, and 3) scores for matching rules require constant tweaking/maintenance

• pros: provides out of the box filtering

• examples: SpamAssassin, Barracuda Spam Firewall

GTRI_B-22

Statistical Spam Filtering

• kind of document/language classification system were mathematical algorithms are used to develop a statistical or probabilistic model of spam vs. ham

• cons: requires user training (but most filters provide a way to pre-train user accounts using spam and ham corpus, share training among users [groups] or from a global user)

• pros: can provide the best filtering, up to 99.99% accurate, without any supplementary spam checking

• examples: DSPAM, CRM411, Death2Spam, bogofilter

GTRI_B-23

Challenge-Response

• send a challenge to first time senders asking them to do something to verify they're a human and not a bot

• cons: 1) people just won't email you, 2) slows down email, 3) generates a lot of email (each spam generates a challenge), 4) subject to forgeries • pros: 1) very accurate and 2) requires no training for users

NOT RECOMMENDED

GTRI_B-24

collaborative checksum or signature filtering

• create a checksum of key parts of spam email and submit it to a collaborative network that can be checked by others • cons: 1) subject to false positives if it is sole indicator, 2) subject to injection by malicious party

• pros: 1) benefit from others misery, 2) supplements heuristic filters by increasing scores

• examples: Distributed Checksum Clearinghouse (DCC), Vipul's Razor, Pyzor

GTRI_B-25

Sender policy framework

• use DNS records to define the the valid servers allowed to send email out for @domain.com • cons: nothing keeps spammers from setting up their own domains and SPF DNS records, though it may help track them down

• pros: reduces spam forgeries

GTRI_B-26

Greylisting

• each incoming email is met with a "temporarily unavailable" or "try again later" error

• spammers generally don't try again; legitimate email servers almost always try again

• cons: 1) some broken legitimate email servers out there give up immediately meaning you may have to create whitelists and 2) delays legitimate email

• pros: can significantly reduce the amount of spam received

• Test at Jasper County School District: 11/13/06 – no greylisting – 6,759 spam emails received 11/14/06 – greylisting – 1,320 spam emails received

GTRI_B-27

Anti-virus filtering

• Are A/V vendors interchangable? Nope. Response times to outbreaks vary greatly. 2005 results from AV-Test.org:

Average Response Time       Product NameBetween 0 and 2 hours……………KasperskyBetween 2 and 4 hours……………BitDefender, Dr. Web, F-Secure,

Norman, SophosBetween 4 and 6 hours……………AntiVir, Command, Ikarus, Trend MicroBetween 6 and 8 hours……………F-Prot, PandaBetween 8 and 10 hours…………..AVG, Avast, eTrust-INO, McAfee,

VirusBuster Between 18 and 20 hours…………eTrust-VET

• Kaspersky is hero in Russia and virus writers send them their viruses as a courtesy; DoD won't use Kaspersky

GTRI_B-28

Anti-virus filtering

• in a non-scientific ~100,000 email test at Jasper County School District in late 2004 to early 2005 ClamAV was able to identify more phishing fraud than Sophos (none) or McAfee (some)

• Linux a good choice as a filter platform because ClamAV, BitDefender and Panda anti-virus are free; but ClamAV does a poor job at identifying malware trojans (according to 9/26/06 PC-Welt article) and Panda has slow response times (according to 2005 AV-Test results) so you'll need to supplement them with another A/V vendor

• consider diversity of anti-virus: two different A/V vendors at the email gateway and one A/V vendor on file servers/end-systems as it mitigates 0-day, vendor update variance; but it may increase your chances of a false positive

GTRI_B-29

Anti-virus filtering

• keep in mind that the slow response time of vendors like McAfee and Trend-Micro may be due to their higher percentage of large corporate customers where false positive could be a big disaster; they are less likely to release update before they've done extensive QA

• new propagation patterns such as "serial variant attacks" making signature-based systems ineffective

GTRI_B-30

Protecting Against Phishing

• user education (e.g. don't click on links in emails)

• only use a filtering product that can detect, disable, and/or warn about a possible phishing attacks (especially numeric) within an email

GTRI_B-31

Other filtering

• JavaScript/VBScript

• Web bugs

• Dangerous HTML

• Accounts for children: All HTML? All images?

GTRI_B-32

Implementing filtering

• four topology choices: 1) use built-in anti-spam/anti-virus in your groupware software, 2) use anti-spam/anti-virus software plug-in for your groupware software, 3) dedicated anti-spam/anti-virus filtering appliance (e.g. Barracuda Spam Firewall, CipherTrust IronMail, etc.) or server (e.g. MailScanner+SpamAssassin+dspam, Sophos PureMessage, etc.), 4) managed service (e.g. Postini, Cloudmark, etc.)

built-in: can be cheaper/easier but probably doesn't provide the best filtering; allows spam to impact interactive performance of your mail server(s)

plug-in: may not be able to use overlapping products to achieve best filtering and allows spam to impact interactive performance of your mail server(s)

GTRI_B-33

Implementing Filtering

• dedicated appliance/server: more flexible (many filtering options), more secure (keeps your groupware server out of the DMZ), performance of your mail server(s) not impacted by incoming threats

• managed service: excellent choice if you don't have in-house expertise; keeps the threats out of our network entirely; reduces bandwidth

• implement per-user statistical filtering because "one man's ham is another man's spam"

• implement per-user quarantines that purge after X amount of days to reduce data storage requirements

GTRI_B-34

Implementing Filtering

• dedicated appliance/server: more flexible (many filtering options), more secure (keeps your groupware server out of the DMZ), performance of your mail server(s) not impacted by incoming threats

• managed service: excellent choice if you don't have in-house expertise; keeps the threats out of our network entirely; reduces bandwidth

• implement per-user statistical filtering because "one (wo)man's ham is another (wo)man's spam"

• implement per-user quarantines that purge after X amount of days to reduce data storage requirements

GTRI_B-35

Implementing Filtering

• implement filtering solutions that are outside the context of the filtering solution (SPF DNS records, address munging, etc.)

• open source a strong alternative to commercial solutions; for example, no commercial offering that does external inoculation

• MailScanner, MIMEDefang, and amavisd-new are the major open source filtering frameworks that interact with a open source mail transport agent (postfix, sendmail, exim), open source anti-spam filters (DSPAM, bogofilter, SpamAssassin, etc.) and open source, commercial, or freeware anti-virus to create a full anti-spam, anti-phishing, anti-virus all for $0.00 + cost of A/V.

GTRI_B-36

Making the decision

• pare down to three or four appliances/dedicated servers (based on what criteria?)

• perform "bake off" that subjects them to your actual incoming mail and mail load

• "bake-off" should look at accuracy, ease of integration, scalability, friendliness of user quarantine, ease of administration

• consider the experience/expertise of your staff: pure open source requires expert staff; commercial offerings require competent staff; managed service requires little from staff

• 95% accuracy is not good! 99.5% accuracy should be the minimum acceptable level as the difference between 95% and 99.5% is not 4.5%; 95% = 5 errors per 100 messages and 99.95% means one error per 200 messages. 99.95% is ten times better!

GTRI_B-37

Part 3:

Case Study: Jasper County School District

GTRI_B-38

• 600 users: mix of web-based, Eudora, and Thunderbird• RedHat Enterprise Linux 4.0 AS on single Dell PowerEdge 2850 (dual 3 GHz Xeon; 4 GB of RAM), 136 GB hardware RAID5 (3 x 72 GB)• Filtering Framework: MailScanner• Mail Transport Agent: Postfix• IMAP/POP3 Server: Dovecot• Web-based Groupware (Email, Shared Calendaring, Notes, Address Book, File Manager): Horde Project• Mailing Lists: Mailman• Spam Filtering: SpamAssassin (shared heuristic, shared statistical filtering, sender policy framework, some sender checks), Vipul's Razor and DCC (collaborative checksum), several DNS RBLs, Postgrey (greylisting), DSPAM (per-user statistical filtering with per-user quarantines)• Web-based Reporting/Logging: MailWatch for MailScanner• Anti-Virus Filtering: ClamAV, Sophos• Total Cost: $60/year for RHEL; $400/year for Sophos

GTRI_B-39

• some users refuse to train and high variance of what the users consider spam so DSPAM cannot be used exclusively

• SpamAssassin is in front of DSPAM; SpamAssassin is mostly heuristic but also does SPF, DNS-based RBL, collaborative checksum (Vipul's Razor, DCC), to assign a score to all email: 0-5 = non-spam, 5-10 = low-scoring spam, 10+ = high scoring spam

• MailScanner deletes all email that is "high scoring spam" while "low-scoring spam"and "non-spam" is delivered to DSPAM

• MailScanner deletes all email that is on three or more DNS-based RBLs (i.e. assume it is "high-scoring spam")

• MailScanner quarantines disallowed attachments in an administrator quarantine and sends warning to receipient and sender

GTRI_B-40

• MailScanner disarms phishing fraud by rewriting HTML so that it points out that the displayed hyperlink does not match the the actual hyperlink

• MailScanner disarms HTML contain <Form> so users cannot input data (phishing fraud) and <Script> (so HTML rendering engines of email clients do no execute the code), and <Img> tags with very small images to block web bugs

• MailScanner can also convert all HTML emails to Text which could be very effective at protecting children (but we do not currently do this because children do not have email accounts)

• as a non-programmer, I was able to write several patches to fix issues in Horde and DSPAM