incontrol™ processing - amazon s3 · the incontrol processing module is capable of processing...

InControl™ Processing

Web Interface User Guide

InControl™ Processing Web Interface User Guide

CONFIDENTIAL

Contents 1. Introduction .......................................................................................................................................... 3

2. Case Management ................................................................................................................................ 3

2.1. Login .............................................................................................................................................. 3

2.2. Case List ........................................................................................................................................ 4

2.3. Creating New Clients .................................................................................................................... 4

2.4. Creating New Cases ...................................................................................................................... 5

2.5. Adding Search Strings ................................................................................................................... 6

2.6. Adding Date Filters ....................................................................................................................... 7

2.7. Creating Workflows ...................................................................................................................... 9

2.8. Creating Workflow Templates ................................................................................................... 15

2.9. Adding Custodians ...................................................................................................................... 16

2.10. Creating Workflow Templates ............................................................................................... 17

2.11. Manage Items ......................................................................................................................... 17

2.12. Creating a Review Set ............................................................................................................. 18

2.13. Deactivate ............................................................................................................................... 18

3. Item Manangement............................................................................................................................ 19

3.1. Creating an Item ......................................................................................................................... 19

3.2. Applying Tags .............................................................................................................................. 20

3.3. Applying Custodians ................................................................................................................... 21

3.4. Running a Workflow ................................................................................................................... 22

4. Drive Management ............................................................................................................................. 22

4.1. Mounted Evidence Drives – Drives mounted for ingesting data. ................................................... 22

4.2. Mounted Production Drives – Drives mounted to save exports or productions. .......................... 22

5. Events.................................................................................................................................................. 23

5.1. Smart Filters................................................................................................................................ 23

5.2. Details ......................................................................................................................................... 23

6. Glossary of Exported Fields ................................................................................................................ 24


CONFIDENTIAL

1. Introduction InControl is a full end-to-end eDiscovery platform. This user guide is designed to take an administrator through the web interface of the InControl Processing module. The web interface allows an administrator to ingest native data, process the data, cull based on various search methodologies, export data, and prepare sets of documents for review.

The InControl Processing module is capable of processing tasks across multiple processors and servers. The system can import directly from forensic images and file systems, and it can extract over 60 file container types including: virtual machine hard drives, ISO CD-ROM images, and email containers. Once ingested, the user has the ability to cull data sets using Boolean expressions to include: known file filter, internal metadata, filesystem metadata, date filter, search text queries, and file type. Post processing, the system provides detailed exception, ingestion, and responsiveness reports.

2. Case Management 2.1. Login

Users can access the Processing module through any Internet browser. Passwords should have an upper-case letter, lower-case letter, a number, and a special symbol.

Once logged in, users can change/update their passwords.


CONFIDENTIAL

2.2. Case List The list of cases is accessed from the Case Management tab.

The Search box allows you to search ID, Client, or Description fields in order to filter the case list. You have the option of displaying active cases, disabled cases, or both.

2.3. Creating New Clients In order to create new clients that will be added when creating a new case, click the “Create Client” button. Type the name of the client in the box and click submit. This will add the client to the list of clients available when creating a new case.


CONFIDENTIAL

2.4. Creating New Cases New cases can be created by clicking the “Create Case” button on the Case Management tab.

x Case ID - Enter a case ID specific to the new case you are adding. This can be an alpha or numeric value.

x Case Description – Enter a case description (example: Client Document Review). x Client Name – Select client from the drop-down box.

Once the case is created, click on the details to track the various date filters, tags, or search terms.


CONFIDENTIAL

2.5. Adding Search Strings Search strings can be added from the case management tab that can be used as filters (both inclusive or exclusive) to be used when processing data. Once the strings are added, they will be able to be selected as filters within a workflow.

1. Select “Add Search Strings” from the Select Action drop-down box.

2. Enter in a prefix for the search strings. The prefix will act as a unique identifier for the set of search terms you are entering.


CONFIDENTIAL

3. Add in search string into the box. Boolean expressions are allowed as search strings. 4. Click “Add String” to add additional Search Strings. 5. Multiple strings can be entered at one time by copy-and-pasting from another list. In this

case, each term will be separated into its own line or string. 6. Click “Submit” once your list is complete.

2.6. Adding Date Filters Each entry in a view is checked against all of the date filters defined for the case. A Boolean string is stored for each date filter indicating whether the file is responsive to the date filter. Once date filters are created, they can be applied as either an inclusive or exclusive filter when importing data.

1. Name – Enter a unique name for the date filter. Multiple date filters can be applied during the import process.

2. Start Date – Select the beginning date of the filter in mm/dd/yyyy format. Clicking on the row will bring up a calendar in order to select a specific day.

3. Start Time – Select the “All day” option if there is not a specific start time. To select a specific start time, either enter a specific time in the row, enter the time in hh:mm:ss format or select from the visual clock. Start times will use Coordinated Universal Time (UTC) for the time zone. The system maintains all times in UTC in order to standardize times across time zones. When defining a date filter, the start and stop times are defined in UTC. Each time zone is an offset from UTC by a certain number of hours. For instance,


CONFIDENTIAL

Eastern time in the United States is five hours behind UTC in the winter and four hours behind UTC in the summer. If you select this option, then documents that have any modification time in the date filter range will be responsive to the date filter.

4. End Date – Select the ending date of the filter in mm/dd/yyyy format. Clicking on the row will bring up a calendar in order to select a specific day.

5. End Time – Select the “All day” option if there is not a specific start time. To select a specific start time, either enter a specific time in the row, enter the time in hh:mm:ss format or select from the visual clock.

In general, date filtering is a complex task because a document may have multiple dates stored in filesystem and internal metadata. Our date filters are complex in that they allow the user to define whether to use the following set of times as part of date filtering:


CONFIDENTIAL

1. Modify time – If you select this option then any documents that have any modification time in the date filter range will be responsive to the date filter.

2. Access time – If you select this option then documents that have any access time in the date filter range will be responsive to the date filter.

3. Create time – If you select this option then documents that have any creation time in the date filter range will be responsive to the date filter.

4. Metadata Change time – If you select this option then documents that have any filesystems metadata change time in the date filter range will be responsive to the date filter.

5. Delete time – If you select this option then documents that have any creation time in the date filter range will be responsive to the date filter.

6. Internal metadata – If you select this option then the date filter will consider internal metadata to determine date filter responsiveness. For example, if you select internal metadata and modify time, then a document having internal metadata indicating that the document was modified in the date filter range will be considered responsive to the date filter.

7. Filesystem metadata – If you select this option then the date filter will consider filesystem metadata to determine date filter responsiveness. For example, if you select filesystem metadata and modify time, then a document having filesystem metadata indicating that the document was modified in the date filter range will be considered responsive to the date filter.

Additionally, the ICPU date filter allows the user to decide whether files with no relevant times should be included as responsive to the filter.

2.7. Creating Workflows Workflows are a list of commands that you would like to run for a given processing job and/or case. Workflows are needed in order to process data, apply search and/or date filters. You can start from an empty or blank template, or choose from a saved template (discussed in section 2.8).


CONFIDENTIAL

1. Select a name for the workflow and type it in the Workflow Name box. 2. Select from the list of commands and Add them to the workflow:

a. Import – The import function would be the first command if you are starting a new case. You can give a description of what you are importing, ex. John Doe’s email. When running the work flow, the step in that command is to select the source from which you wish to import. You can select the OCR box that will kick off the option to OCR documents that do not have extractable text.

b. Select Source – Select this command if you wish to select a source of data that has already been imported for the purpose of applying a date filter, etc. It is not necessary to use the Select Source command if you are importing new documents.

c. Filter – A variety of filters can be applied either as inclusive or exclusive to import documents. Each filter is a separate Boolean string that is separated with AND or OR operators, and parenthesis are available for grouping and order of operations. The NOT operator can be applied in order to use the filter as an exclusive filter.


CONFIDENTIAL

The filters are: i. Attachment – Use this filter to limit only to documents that are

attachments to other documents. ii. Searchable PDFs – This filter will match any entry that has been detected a

searchable PDF. iii. Non-searchable PDFs – This filter will match any entry that has been

detected as a non-searchable PDF iv. Known to NIST – This filter will match any file that is known to the list of

common of file types that are system files defined by the National Institute of Science and Technology (NIST).

v. User Document – This filter will match any entry that is part of the file group Document or Productivity.

vi. All documents – This filter is not really a filter at all. It returns all entries without filtering.

vii. Any email address – This filter will match any email entry where metadata indicate that the email includes a named party.

viii. Email Between – This filter will match any email entry where metadata indicate that the email was between two parties.

ix. Responsive to Searches – This filter will match any entry that is responsive to the search string identifiers. Select the prefix or term to be included in the filter. Hold down the control or shift key to select multiple terms to be included in the filter.

x. In Document Set – This will match any entry that is part of the file that is in a specific set of documents allowing you to filter only on certain folders or sets of data.

xi. Tagged – This will match any entry that has a specific tag assigned to the entry.

xii. In Type Group – This filter will match any entry that has been detected as belonging to any of the type groups provided as arguments (ex. Container Files, Spreadsheets, Email Messages, etc).

xiii. Responsive to Date Filter – This filter will match any entry that is responsive to the date filter name that is provided.


CONFIDENTIAL

xiv. Extension Matches – This filter will match any entry that has an extension matching those provided.

d. Produce – This command contains settings for the export of documents. You can provide a description of the command in the Description box. Then select to Dedupe either globally or within a custodian if desired. You will then have the option to numerate the production documents with a unique prefix, remove file containers, include email and PDF families, and keep only email or PDF files.

i. Description – Place a description of the production in the box.

ii. Dedup – Select whether to deduplicate the documents either globally or within a custodian’s data set. This method will remove items from the production that have the same characteristics as files that have already been produced. For email files, the Message-ID field, from, and sent timestamp are used as a unique identifier. For nonemail files, the SHA1 hash is used as a unique identifier. If the global argument is used, then dedup will remove all files with a matching identifier that have already been produced in the given prefix for the case. If the custodian argument is used, then dedup will remove all files with a matching identifier that have already been produced in the given prefix for the custodian.

iii. Prefix – Place a prefix in the box to number the documents accordingly. iv. Remove file containers – This command will remove all file containers

(unwound files) from the production. Very often container files will be responsive just as the internal files are responsive. Typically, the client does not want the container but wants the responsive internal files. So for this case, all of the responsive items that have been unwound from container files will be maintained. Because Email and PDF files only unwind the attachments to these items and not the main body content, these files are not removed in this process because they should ordinarily be included for review.


CONFIDENTIAL

v. Include email and PDF families – This command will add any children objects of email or PDFs to the production set. Generally, clients will want the entire email or PDF family to be produced if the email/PDF or any of the attachments are responsive.

vi. Keep only EML or PDF – This command will remove the children of any email or PDF file whilst keeping the email or PDF file for production. When a client will be doing unassisted native review, they will ordinarily want a single file such as an EML rather than the EML file and one file for each of the attachments. This process automatically runs the includeEmailAndPDFFamilies command to ensure that all Email/PDF are included and then all of their children are trimmed from the production.

e. Process – This command combines other commands and creates additional settings and filters when processing documents. This command takes a source view and a command string that enumerates the processes that should be performed on the data set.

i. Description – Place a description of the production in the box.


CONFIDENTIAL

ii. Hash - Run the hash algorithm against all native files in a view and store the results in the database. The supported hash algorithms are MD5, SHA1, and SHA256.

iii. Export – Choose to select the text and/or native files all items responsive to the user’s production request.

iv. Extract text – This command Extracts rendered HTML text from all native files in a data set. The extracted text is imported into the extractedText ICPU store. The reason for choosing HTML rather than plaintext is that HTML allows for the encoding of metadata into the file in a way that will allow our search process to find metadata that are responsive to search terms. After text is extracted, an indexing process runs in the background that indexes the extracted text to allow for searching.

v. KFF – This command Use the SHA1 hash associated with each entry in a view and determine if these entries are in the NIST known file database (NSRL). A boolean is stored in the database for each entry indicating whether the file is known.

vi. Mime type – This command will analyze the content and extension of each entry and store several computed file types in the database. This process will also run the Grouping process.

vii. Grouping – This command will look at the content mimetype and other parameters within an entry to determine a type group that should be assigned to that entry. For example, a document with a mimetype of application/msword would be assigned a type group of Document.Word Processing.MS Word, while a file with a mimetype of application/vnd.ms-outlook would be assigned a type group of Productivity.Email. All items that are not known will be assigned a type group of Unknown. Grouping is run as part of the Mimetype process but if there are changes made to the grouping algorithm then there may be a need to run the type grouping function separately without having to redo the time-intensive process of typing.

viii. OCR – This command will OCR all documents in the selected data set. After the documents are indexed, an indexing process runs in the background that indexes the text for searching.

ix. Extract Internal Metadata – This command will extract the internal metadata from the native files for each entry in a view using Apache Tika and other methods. This internal metadata will be parsed as a dictionary and stored in the database document for each entry in the view.

x. Date filter – This command will prompt the user to apply a date filter(s) that are defined in the case prior to processing.


CONFIDENTIAL

xi. Search – This will prompt the user to perform a search using all of the search strings stored for the case. Apply tags to each item indicating which search strings were responsive.

xii. Report – This process exports a report providing detail of various exceptions and other items of note regarding the objects in a data set.

f. Send to Review – This command line has all of the same commands as the Process command. This is designed to create a review set to be sent to the InControl Review module.

3. Click “Submit” once you have all of the commands entered and this will save the Workflow and apply it to the case.

2.8. Creating Workflow Templates Workflow Templates can be created in the event that a list of commands or tasks need to set up to be repeated on a continual basis. Multiple templates can be created, providing the user with the ability to have repeatable steps for every client or specific set of tasks. Templates are created using the same commands and functions as when creating a Workflow.

When creating a Workflow from a template, you will need to edit the commands so that they conform to the specifications of the case to which they are being applied.


CONFIDENTIAL

2.9. Adding Custodians Custodians can be added to a case at any time. Once a list of custodians is established, they can be applied to specific data sets or tasks. Custodians can be entered manually on each line and click the “Add name” button to add more lines, or you can copy and paste from a list of custodians. Once you have your list, click “Submit” to save the list.


CONFIDENTIAL

2.10. Creating Workflow Templates Tags can be created so that they can be applied during various processing tasks for a specific case. These tags can be used to select or filter items as additional processing and filtering is performed. These tags are free form and can be arbitrarily chosen by the user. For instance, a user could apply a tag named ToProduce to items that should be produced irrespective of other filtering criteria. A new view could then be created that included all documents labeled with this tag.

2.11. Manage Items Manage Items action will take the user to the Item Management tab. All of the options will be covered in Section 3 of this guide.


CONFIDENTIAL

2.12. Creating a Review Set Selecting this option allows the user to select from Exports created during the processing phase of the case workflow for the purpose of creating a review set to be imported into the InControl Review module.

2.13. Deactivate Select “Deactivate” in order to remove a case from the Active case list. Cases can be re-activated at any time.


CONFIDENTIAL

3. Item Manangement Items are processing events associated with a case. Access the Item Management by clicking on the tab or by selecting the action from the case drop-down menu. Once on the Item Management tab, select the case from the drop-down menu.

3.1. Creating an Item Create items in order to run a particular workflow for a case. Click on the “Create Item” button.

1. Item ID – Enter a unique identifier for the item. The entry can be alpha-numeric. 2. Item Description – Provide a description for the item for organizational purposes. 3. Custodian – Select a custodian from the list if you would like to associate them with item.


CONFIDENTIAL

3.2. Applying Tags Tags can be used to select or filter items as additional processing and filtering is performed. These tags are free-form and can be arbitrarily chosen by the user.

For instance, a user could apply a tag named ToProduce to items that should be produced irrespective of other filtering criteria. A new data set could then be created that included all documents labeled with this tag.

1. Tag – Select the tag to be applied from the drop-down. (Tags are created from the case action menu)

2. Recursive – Check this option if you would like to include all of the folders and sub-folders of a data set.

3. Untag – Select this option if you wish to remove tags from the selected data set. 4. Path – Double-click on the source folder to navigate to the appropriate source or mounted

drive. 5. Click Submit


CONFIDENTIAL

3.3. Applying Custodians Custodians can be applied to items either during import or after the fact.

1. Custodian – Select the custodian to be applied from the drop-down. (Custodians are created from the case action menu)

2. Recursive – Check this option if you would like to include all of the folders and sub-folders of a data set.

3. Path – Double-click on the source folder to navigate to the appropriate source or mounted drive.

4. Click Submit


CONFIDENTIAL

3.4. Running a Workflow Once set up of an item is complete, the user can select to run an associated workflow. The workflows are created from the Case Management tab.

1. Select from the list of workflows that are associated with a given case.

2. Follow the steps and modify the workflow as they pertain to the specific task and the desired outcome.

4. Drive Management The Drive Management tab provides the user the ability to manage the physical location for the data to either be imported or exported. These can be either physical hard drives that are plugged into the processing server or virtual network locations.

4.1. Mounted Evidence Drives – Drives mounted for ingesting data.

4.2. Mounted Production Drives – Drives mounted to save exports or productions.


CONFIDENTIAL

5. Events The Events tab is designed to track progress of executed workflows and view reports such as search hit reports and exceptions. Any time a workflow is executed, and Event is created.

5.1. Smart Filters Smart Filters are available to easily access Events by case, item, status, type, and date.

5.2. Details Event Details provide a summary of a processing event. Hit Reports and Exception reports can be accessed from the details.


CONFIDENTIAL

6. Glossary of Exported Fields Below is a summary of the fields that are created during export. These will be mapped to fields in the review module.

DOCID The ID number assigned to the document. If the document was produced as images then this will be the first image number.

CUSTDIAN The custodian associated with the document.

AUTHOR The author of the document as defined by internal metadata.

BEGDOC The beginning document ID number for the document.

If the document was produced as images then this will be the first image number and is the same as the document ID field.

ENDDOC The ending document ID number for the document. If the document was produced as images then this will be the

last image number.

PGCOUNT The number of pages of the document. When only a

native is produced then this will always be one.


CONFIDENTIAL

PARENTID The document ID number of the parent document. If the document does not have a parent then this will be the same as the document ID field.

ATTCHLST A semicolon separated list of attachments to this document. If this document has no attachments then this will be blank.

BEGATTCH The beginning document ID number of the first document in the family.

ENDATTCH The ending document ID number of the last document in the family.

MEDIA This field describe the type of document and can contain either: eMail, eDoc, or Attachment.

ALTID The unique ID number associated with this document. This number is globally unique.

FILETYPE The mimetype of the document.

SUFFIX The file extension or suffix of the document.

APP If the internal metadata details the application that was used to open this document this field will contain that value, otherwise this field will be blank.

ATTCHCNT This field will indicate the number of attachments to this document.

SHA1 The SHA1 hash of the document. NAME The original document file name. FILESIZE The size of the document in bytes.

FSDTCRTD The date when the file was created on the filesystem.

FSDTMOD The date when the file was last modified as indicated by the filesystem metadata.

FSDTACCD The date when the file was last accessed as indicated by the filesystem metadata.

FSDTMCHG The date when the filesystem metadata was last updated for this document.

IMDTMOD The date when the document was last modified as

indicated by the internal metadata.

FSTMCRTD The time when the document was created as indicated by the filesystem metadata.

FSTMMOD The time when the document was last modified as indicated by the filesystem metadata.

FSTMACCD The time when the file was last accessed as indicated by the filesystem metadata.

FSTMMCHG The time when the filesystem metadata was last updated for this document.

IMTMMOD The time when the document was last modified as indicated by the internal metadata.


CONFIDENTIAL

IMDTCRTD The date when the document was created as indicated by the internal metadata.

IMTMCRTD The time when the document was create as indicated by the internal metadata.

FROM The sender of an email, otherwise blank.

TO Email addresses listed in the To field of an email, otherwise blank.

CC Email addresses listed in the Cc field of an email, otherwise blank.

BCC Email addresses listed in the Bcc field of an email, otherwise blank.

SUBJECT The subject of an email, otherwise blank. DATESENT The date that an email was sent, otherwise blank. TIMESENT The time that an email was sent, otherwise blank. FILEPATH The original path of the document as preserved.

PROPERTIES Any notes added to the document during the discovery processing.

SEARCHES Blank.

SUBJECT_OTHER The subject of an email, otherwise blank.

THREAD_IN_REPLY_TO The In-Reply-To field from an email header.

MESSAGEID The Message-Id of an email.

THREAD_REFERENCES The References field from an email

header.

FILEGRP The dot-separated value describing the file group or

type of document.

FILEGRP1 The first value in the file group. FILEGRP2 The second value in the file group. FILEGRP3 The third value in the file group.

KNOWN A Boolean value indicating whether the file is known to the known file filter.

NATVLINK The location of the native file.

TEXTLINK The location of the text file.

IMGLINKS A semicolon-separated value indicating the location of any image files.

DEDUPLNK A list of custodian’s that have possession of this document.

RCPADDS The recipient addresses as stored in MSG files. ATTRNG The range of attachments. This is BEGDOC-ENDDOC. MD5 The MD5 hash.


CONFIDENTIAL

RCVDATE The date that an email was received.

RCVTIME The time that an email was received.

TITLE The document title.

incontrol™ processing - amazon s3 · the incontrol processing module is capable of processing...

Documents