E-mail archiving and DB2 CommonStore for Lotus Domino
IBM INFORMATION ON DEMAND COMES TO YOUFebruary 5, 2008Bangkok, Thailand
2
Agenda
CSLD - Overview of base functions Sub bullet
CSLD - Overview of advanced functions
CSLD - Summary and highlights
3
Extract e-mails from the messaging system– Option: together with meta data (like subject, sender, date, …)
… and store them in an electronic archive system (repository)– On hard drive, tape, or optical disks (WORM)– For many years
Access archived e-mails (retrieval)– Through the Domino client– Through the client of the electronic archive
What is e-mail archiving?
Messaging systemElectronic archive
system (repository)
Client
Archive
Retrieve
Retrieve
4
CommonStore
Electronic archive and content management solution
Designed for– SAP (CSSAP)– Lotus Notes and Domino (CSLD)– Microsoft Exchange (CSX)
IBM Content ManagerIBM Content Manager OnDemandTivoli Storage Manager
Mail system
CommonStore
Electronic Archive
5
What drives e-mail archiving today?
Growth of the messaging system
– More users– Larger mail boxes– More expensive operation
Requirement– Reduce the growth of the
messaging system– Offload old e-mails from
production messaging system– Keep old e-mails in near on-
line access
Legal requirements or industry regulations
– E-mails treated as business records (based on procedures)
– Provide e-mails on demand during audit or investigation
Example 1– User selects e-mails that should
be business records
Example 2– All e-mails of a certain user group
have to be captured and retained for a certain number of years
– Example: Securities Exchange Commission (SEC), regulations for broker and dealers
6
The Architecture
Application or User
DB
Lotus DominoLotus DominoEnvironmentEnvironment
DB2 CommonStore for Lotus Domino (CSLD)
Archiveoptions
Notes user
Job DB
CM Agent
archpro
Notes API
Archive specific APIs
Config. DB
CM iSeries
CM v8
CMOD
TSM
CM v8Agent
CM iSeriesAgent
CMODAgent
TSMAgentHTTP
dispatcher
Domino dispatcher
CSLD tasks
CSLD crawler
Notes API
archint.ini
csmimes.properties
ARCHWIN dispatcher
Local CSLD archive DB
7
Click to edit Master title style
DB2 CommonStore for Lotus Domino
Overview of base functions
8
CommonStore base functions
Archive
Retrieve
Search
Update
Delete
9
Click to edit Master title style
Base function - Archive
10
Archiving Types
Entire messages– Mail header (To, From, Subject, ...) can be kept as a
placeholder (“stub“) in Notes views or folders– Mail becomes very small or is removed completely
Attachments only– Only the attachments are archived, – Text in mail body remains unmodified
Convert format of message– Line data, Rich text– Any other format (like TIFF, PDF) through program exit
11
Archiving Type - Entire messages
Highlights– 100 % document fidelity (no format conversion)– Includes Notes specific Rich text elements
• Database links, buttons, sections, …
– Works for messages in Notes Richtext and MIME format– Works for signed and encrypted messages
Several options for mail placeholder– Text only (configurable)– Retrieval hotspot– Intelligent abstract of message
12
Example: Archived message with retrieval hotspot
Retrieval hotspot
New
13
Archiving Type - Entire messages (cont.) Intelligent abstracting (optional)
– A short summary of the mail is inserted in the mail body– Intelligent text analysis based on IBM’s Text Analysis Framework
(TAF) identifies most relevant sentences (special training for e-mails)
– Length (number of phrases) can be configured– Mail placeholder remains small (around 3 KB)
Benefit– Better search results for user since the abstract is included in the
Domino full-text index– No additional license cost for intelligent text analysis
14
Example: Archived message in iNotes with abstract
Intelligent abstract
New
15
Archiving Type - Attachment
Highlights– Archives only the attachment– Works for messages in Notes Richtext and MIME format– Works even for attachments outside of mail body – Works for signed messages– Mail body remains intact– Can save 60-75 % of entire message storage space
Attachment placeholder– In the exact location (not at end of message)
• Option 1: Text with hidden URL for Web retrieval
• Option 2: Text only (no URL)New
16
Example: Attachment archiving
Attachment placeholders
17
Archiving Method – Automatic archiving (Crawler)
Rules (policies)– Size of documents– Age of documents (absolute or relative)– Size of mail database– Notes formula (maximum flexibility)
New
18
Example: Policy with Notes users and groups
New
19
Archiving Method - User-driven archiving
Archive action in Notes folder(Notes Client)
Archive action in open document
(Web Mail)
20
Click to edit Master title style
Base function - Retrieve
21
Retrieval Types
Simple click on a button for reload on Domino server– No Client code installed on user work station
• Retrieval button added as action to mail template or
• Retrieval hotspot inserted in mail body
– Temporary restoration of message• Periodic clean-up of reloaded messages
– For reply or forward to external recipients– Works for both entire message and attachment archiving
CommonStore web viewing– Simple click on web link (URL) created during archiving– Attachments can be viewed individually– No reload on Domino server– Works for attachment archiving only
22
Example: Reload on Domino Server
Notes Client- Retrieval button- Retrieval hotspot
Web Mail- Retrieval button- Retrieval hotspot
23
1. Click on CommonStore Web link (URL)2. View archived attachment in Web Browser
Example: Web retrieval of an archived attachment
24
Click to edit Master title style
Base function - Search
25
Search and retrieve of archived messages
Search– Based on attributes
• Subject, To, From, Date, ...
• Stored in relational database of archive
• Requires mapping during archiving
– Full-text (requires Content Manager v8.2)• On a text attribute
• On the content of the archived item
Environment– Notes Client or Domino Web Access
• Entire messages are restored in original state to Domino
– Content Manager Client (or Content Manager OnDemand Client)• Access point to Enterprise Content Management (ECM)
26
Example: Search in Notes Client
1. Enter a search2. Get a hit list (results)3. Click to open or view
document
27
Example: Search in Content Manager pClient
1. Enter a search2. Get a hit list (results)3. Click to open
document
28
Click to edit Master title style
Base function – Update and Delete
29
Other features: Update and delete in archive Update
– Attributes stored in repository
Delete– Trigger removal in the archive from Notes interface– Important for keeping documents in Domino in synch with archived
items (no orphans)– Important for implementing document retention policies
30
DB2 CommonStore for Lotus Domino
Overview of advanced functions
31
Single-instance store (SIS) for CM v8.2
Archiving of entire Notes folders
Other Domino applications
CommonStore advanced functions
32
Single-instance store (CM v8.2)
Feature– Identical messages are automatically recognized and
archived just once (based on intelligent hash code algorithm)– Example: 1 e-mail is sent to 50 recipients
• 51 copies in Domino, only 1 copy archived by CommonStore
– Optional setting– Work when archiving entire messages
Benefit– Minimizes storage space in archive– Removal from Notes in archive supported
New
33
Single-instance store (CM v8.2) – cont’d
1 item linked to several mail owners– Multi-value attributes in CM v8.2 (child component)– Any attribute that is different for messages with same
content is part of the child component
New
34
Archiving of entire Notes folders Archive
– Notes folder and all subfolders and all contained documents stored with just one archiving request
– CommonStore stores the Notes folder information in the archive– Recommended: archive entire message without leaving document
stubs Restore
– Select root folder and restore all subfolder and contained documents with just one retrieval request
Benefit– One folder becomes the placeholder for many documents and
folders– Useful for user-driven archiving in Domino applications
35
Other Domino applications
All CommonStore functions available– Automatic and user driven archiving– Retrieve, search, update, delete, …– Any attribute mappings– Any Notes forms
Template changes– Based on CommonStore Script libraries– Easy customization
36
Example: Customer & Co application
37
DB2 CommonStore for Lotus Domino
Summary and highlights
38
Highlights
Central, rule-based, automatic archiving– Makes distributed local archives obsolete
Ease of use– Available in Notes, Domino Web Mail, and iNotes– User driven archiving– Flexible retrieval (without reload on Domino server)– Fast retrieval for mobile users (local archive copy)
Simple implementation and administration– No code install on Client workstation– Domino database for system administration– Tightly integrated in Notes/Domino
39
Highlights – cont’d
100 % document fidelity– Preserves Notes format (links, sections, signature, …)
Flexibility– Several archiving options (attachments only, entire
message, format conversion, ...)– Arbitrary meta data (attributes) stored in archive– Many additional features (update, delete …)– Can be used for other Domino applications
Part of an Enterprise Content Management (ECM) solution– No point solution– Choice of archive platform (Windows, Unix, iSeries, zOS)– Mature repository with proven scalability
• 10,000+ archive installs world-wide