6.1 administration guide...10-0-0-90.securonix.com/10.0.0.90:8032 cs_event_enrichment running...

SNYPR 6.1

ADMINISTRATIONGUIDE

Securonix Proprietary Statement

Thismaterial constitutes proprietary and trade secret information of Securonix, andshall not be disclosed to any thirdparty, nor usedby the recipient except under the terms andconditions prescribedby Securonix.

The trademarks, servicemarks, and logos of Securonix andothers usedherein are the property of Securonix or theirrespective owners.

Securonix Copyright Statement

Thismaterial is also protectedby FederalCopyright Lawand is not to be copiedor reproduced in any form,using anymedium,without the prior written authorization of Securonix.

However, Securonix allows the printing of the Adobe Acrobat PDF files for the purposes of client training and reference.

Information in this document is subject to change without notice. The software described in this document is furnishedunder a license agreement or nondisclosure agreement. The softwaremay be usedor copiedonly in accordance withthe terms of those agreements.Nothing herein shouldbe construedas constituting an additionalwarranty. Securonixshall not be liable for technical or editorial errors or omissions containedherein.Nopart of this publicationmay bereproduced, stored in a retrieval system,or transmitted in any formor anymeans electronicormechanical, includingphotocopying and recording for any purpose other than the purchaser's internal use without the written permission ofSecuronix.

Copyright 2018©Securonix All rights reserved.

Contact Information

Securonix, Inc.

14665Midway Rd.Ste. 100,Addison, TX75001

www.securonix.com

855.732.6649

Revision History

Date Product Version Description

2/9/2018 6.1 First Release

Copyright©2018Securonix, Inc.Page | 2

SNYPR6.1Administration Guide

Table of Contents2

Introduction 7

Who Should Read This Guide 7

User Interface Elements 8

Configure SNYPR Hadoop Settings 14

Tenant-Config 15

Kafka 15

Solr 22

Impala/Hive 28

HBase 33

HDFS 38

Redis 45

Spark 45

Settings 47

Configuring the Application 47

Application Settings 47

General Settings 47

Data Import Settings 49

Single Sign-on 52

Quick Links 53

Startup Jobs 53

Archival Settings 54

DNS Servers 58

Data Masking 59

Hadoop 59

Housekeeping Jobs 59

Avroparquet Migration Job 61

LDAPAuthentication 62

Log Settings 65

Application Logs 65

Logging 66

Manage License 69



SAML Settings 70

SMTP Server Settings 74

UI Preferences 76

Configure SSL for Secure Data Transfer from Remote Ingester to Hadoop Cluster 77

Kafka Brokers: Creating SSL Certificates, Keystores and Truststores 77

Kafka Brokers: Enabling SSL Keystore and Truststore 85

Remote Ingester: Creating SSL Certificates, Keystores and Truststores 89

Creating an Ingester Keystore with New CA-Signed SSL Certificate 89

Remote Ingester: Configuring the Properties Files 94

Generating SSLConfig.Properties File for the Remote Ingester 94

Generating the IngesterCloud.Properties File for the Remote Ingester 95

Enable Kerberos Authentication for Cloudera Services 97

Kerberos Prerequisites 97

Host Configuration 97

Integrate SNYPRwith Kerberized Hadoop Services 99

Configure the SPARK Jobs to connect to the Kerberized Hadoop Services 99

Example of Cloudera Cluster Authentication, Authorization and Security with Kerberos 100

HDFS Extended ACLs 106

Sentry 106

Enabling Sentry for Hive 108

Enabling Sentry for Impala 109

Setting Up Global Sentry Policy 110

CDH SSL 112

LDAPAuthentication in Impala 115

Static Pools 115

References 115

Access Control 117

Setting Up Access Control 117

Creating Roles 118

Creating Users 122

Creating Groups 125

Managing Users, Groups, and Roles 128

Granular Access Control 131



Setting up Granular Access Control 132

Password Control 135

Workflows 139

Configuring Workflows 140

Connection Types 152

Managing Connection Types 152

Adding a New Connection Type 154

Example: Configure CEF Export Connection 155

Uploading or Downloading Files 156

Registering Connectors 158

Threat Library 161

Updating from the Threat Exchange 162

Exploring the Threat Library 165

Deploying Content from the Threat Library 174

Email Templates 176

Using Email Templates 176

Viewing and Editing Email Templates 176

Adding Email Templates 178

JobMonitor 182

Monitoring Jobs 182

Spark Jobs Administration 188

Administering Spark Jobs In Cloudera 188

Stopping Spark Jobs in Cloudera Manager 192

Administering Spark Jobs using Command Line 195

Viewing Yarn Application Status 202

Spark Job Properties 202

Appendix A: Access Privileges 211

Add Data 211

Administration 249

Analytics 260

Dashboard 276

Geolocation 284

InvestigationWorkbench 285



Operations Center 286

Other 288

Reports 288

Security Command Center 293

Spotter 295

Third Party Intelligence 297

Views 298

Appendix B: Administration Checklist 308



IntroductionSNYPR is a big data security analytics platform built on Hadoop that utilizes Securonix machine learning-based anomaly detection techniques and threat models to detect sophisticated cyber and insider attacks.SNYPR uses Hadoop both as its distributed security analytics engine and long-term data retention engine.Hadoop nodes can be added as needed, allowing the solution to scale horizontally to support hundreds ofthousands of events per second (EPS).

Features:

l Supports a rich variety of security data including security event logs, user identity data, access privileges,threat intelligence, asset metadata, and netflow data.

l Normalizes, indexes, and correlates security event logs, network flows, and application transactions.

l Utilizesmachine learning-based anomaly detection techniques, including behavior profiling, peer groupanalytics, pattern analysis, and event rarity to detect advanced threats.

l Provides out-of-the-box threat and risk models for detection and prioritization of insider threat, cyberthreat, and fraud.

l Risk-ranks entities involved in threats to enable an entity-centric (user or devices) approach to mitigatingthreats.

l Provides Spotter, a blazing-fast search feature with normalized search syntax that enables investigators toinvestigate today’s threats and track advanced persistent threats over long periods of time, with all dataavailable at all times.

l Provides the Investigation Workbench to detect links across disparate datasets to enable quickinvestigations and hunting for cyber threats.

Who Should Read This GuideThis guide provides detailed information about configuring and administering the SNYPR application.

The SNYPR Administration Guide is written for:

l System administrators and service providers who need information about how to monitor and administerthe platform at a systems level.

l Businessmanagers and other users in a supervisory role who need information about how to useSNYPR to grant employees and partners access to applications, check for policy violations, and managecases.

If you require additional information, the following documents are available:

l SNYPR Architecture Guide - for system administrators, system integrators, and deployment teamswhoneed to determine SNYPR deployment options in a Hadoop cluster.

l SNYPR Installation Guide – for system administrators, system integrators, and deployment teamswhoneed to install the application.


SNYPR6.1Administration GuideIntroduction

l SNYPR Integration Guide – for deployment engineers and service providers responsible for integratingdata sources and creating content, and compliance officers and IT specialists who need to configureand maintain Risk Management functionality.

l SNYPR User Guide - for information security professionals, security analysts who need to detect andmanage threats, and risk and compliance officers, and IT specialists who need to use SNYPR's reportingcapabilities to monitor and remediate compliance.

User Interface ElementsSome of the common elements found throughout the application are shown in the following image:

A. SNYPR Logo: Click from any screen to return to the Security Command Center home screen.

B. Main Menu: Click to expand navigation options.

C. Current Screen: Click to return to the home screen for the current menu item.

D. Quick Search: Enter text to search within SNYPR.

E. Connection Status: Click the to view theConnection Status for all Hadoop components running on



your environment.

The green check mark indicates the component is running; a red X indicates the component is notrunning.

Click to view details of each component.

To configure settings for Hadoop components, navigate toMenu > Administration > Settings >Hadoop and following the instructions in Configure SNYPR Hadoop Settings.

F. Notifications: View job failure notifications and download exports including Spotter reports and query



results. To delete notifications, click the red X.

To download reports, click the download icon. For information on how to export Spotter reports, seeSpotter.

G. CollapsedMenu: Access the following screens:

Geolocation

From this screen, view the geolocation of the network source of specific resources.



You can perform the following actions:

a. ToggleAnalyze Violation Data to Yes to analyze data.

b. Click refresh icon to refresh results.

c. Click erase icon to clear results

d. Select a resource from the dropdown.

e. Select a time range from the dropdown.

f. Use +/- to zoom in/out from themap.

g. Click and drag mouse around to pan and tilt map view.

h. Click icons on the right side to switchmap view:

Op Logs

From this screen, you can view messages generated while executing Spark jobs.



To view messages, complete the following:

1. Click + to start a Consumer.

2. SelectDatasource, Job,Policy, and Policy from dropdowns.

3. Specify the max number of messages. Default 1000.

4. Click Stop to stop retrieving messages.

Debug

From this screen, view error messages and associated data to debug the SNYPR application.

Click an option to see associated data.

Outbox

From this screen, view the SNYPR email queue and send or delete messages in the outbox.

H. Admin: View the user name of the current user, change current user password, and log out.



To change the current user's password, click Change Password, enter the old and new password,confirm the new password, and click Update. To log out, click Log Out.



Configure SNYPR Hadoop SettingsSNYPR by Securonix leverages Hadoop technologies including Kafka, Solr, Impala/Hive, HBase,HDFS, Redis, and Spark. After integrating Hadoop, youmust configure Hadoop settings within the SNYPRapplication.

When you log in to the SNYPR application for the first time, you will be prompted to configure your Hadoopsettings. You can access the Hadoop settings at any time from the Hadoop Settingsmenu.

For detailed information about setting up SNYPR integration with Kerberos-enabled Cloudera services, seeEnable Kerberos Authentication for Cloudera Services.

To configure the Hadoop settings in SNYPR, complete the following steps:

1. Log in to the application.

2. Navigate toMenu > Administration > Settings.

3. Click Hadoop from the left navigation panel.

Note: Click the green three bar icon to minimize and maximize the left navigation panel.


SNYPR6.1Administration GuideConfigure SNYPRHadoopSettings

4. Select the Hadoop distribution in your environment:

l Cloudera: Cloudera, Inc. provides Apache Hadoop-based software, support and services, andtraining to business customers. Cloudera's open-source Apache Hadoop distribution, CDH (Cloudera Distribution including Apache Hadoop), targets enterprise-class deployments of thattechnology.

l Hortonworks: Hortonworks is a big data software company that develops and supports ApacheHadoop for the distributed processing of large data sets across computer clusters.

5. Click the name of the component you would like to configure.

Note: The circles beside the names of the components indicate the status of completion asfollows:Gray: Not startedOrange: Incomplete or unsuccessfully configuredGreen: Successfully configured

Tenant-ConfigSNYPR supports a multi-tenant environment in whichmultiple instances of the application can run in a sharedenvironment within a cluster, either on different servers or on different ports on the same server. To select thetenant ID of the tenant for the current instance of the application, complete the following steps:

1. Select a tenant from the dropdown orCreate a New Tenant.

2. Click Save and Next.

KafkaKafka is a distributed publish-subscribe messaging system that is designed to be fast, scalable, and durable.Kafka maintains feeds of messages in topics and consumers read from topics. Since Kafka is a distributedsystem, topics are partitioned and replicated acrossmultiple nodes. In SNYPR, Kafka plays an important rolein publishing and consuming activity data and notifications.

To configure Kafka, follow these steps:



Authentication Type

1. SelectAuthentication Type from dropdown:

l NoAuth: No further action required.

l SSL: Complete the following:

l Client SSL Configuration for Console: Provide the following:

l Key Password

l Key Store Location

l Key Store Password

l Trust Store Location

l Trust Store Password

l Client SSL Configuration for Cluster: Provide the following:

l Key Password





l Broker Configuration for Console: Provide the following:

l Key Password





l Broker SSL Configuration for Cluster: Provide the following:



l Key Password





2. Enter the URLs of the brokers including port number 9092 using commas (,) to separate entries.Example: snypr-10-0-0-150:9092,snypr-10-0-0-151:9092,snypr-10-0-0-152:9092.

Note: You can find the URLs of the Kafka brokers you set up during Hadoop integration inCloudera Manager by navigating to Kafka > Instances.



Topic Details

1. Enter Zookeeper Quorum URLs using commas (,) to separate entries. Example: snypr-10-0-0-150:2181,snypr-10-0-0-151:2181,snypr-10-0-0-152:2181.

Note: Default port is 2181.

Note: You can find the URLs for Zookeeper in Cloudera Manager by navigating to Zookeeper> Instances.



2. Enter the names of the Kafka topics you created when preparing the infrastructure:

a. Preview Topic

b. Access Topic (created in Cloud mode only)

c. Enriched Topic

d. Raw Topic

e. Configuration Messages Topic

f. Indexer Counts Topic

g. Job Tracker Topic

h. LogMessage Topic

i. Violations Topic

j. Users Topic

k. Tier2 TopicTo find the names of topics you created, use the following command from a command-line interface:

a. [root@ ~]# kafka-topics -list --zookeeper :2181

Example: [root@10-0-0-90 ~]# kafka-topics -list --zookeeper10.0.0.90:2181



Kafka Message Settings

1. Complete the following information:

a. Delimiter: Specify the delimiter for raw events. Example |.

b. Publish Threshold: Specify the number of events the application publishes at one time.Default 20000.

c. Max Message Size: Specify the maxmessage size for Kafka Topics.



d. Batch Size: Specify the batch size in bytes. Default 16384.

e. Linger: Specify the linger duration inmilliseconds. Default 1.

Note: If you have fewer messages than batch size accumulated for partition, the applicationwill "linger" for the specified time waiting for more records to publish.

f. Compression Type: Select the compression type for data generated by the producer from thedropdown:

g. Failed Events Folder: Enter a folder name if you would like to move the events Kafka failed topublish to a specific location. Default none.

h. Failed Events Folder Size in Bytes: Enter the storage space for failed events in bytes.

i. Interval to check failed events: Specific an interval in milliseconds to check failed events. Default0.

j. Enrichment Compression Batch Size: Specify a value. Recommended: 10000.

k. Raw Compression Batch Size: Specify a value. Recommended: 10000.

2. Click Test to verify connection and check status.



3. Click Save and Nextwhen status is successful.

SolrSolr is a popular search platform. It can index and search activity data and return recommendations forrelated content based on the search query's taxonomy. In SNYPR, Solr is used in Spotter to create complexqueries and interactive visualization.

To configure Solr, complete the following steps:

Authentication Type

1. Specify theAuthentication Type from the dropdown.



a. NoAuth: Proceed without entering additional information.

b. Kerberos: Enter the following information:

l Host FQDN: Enter the fully qualified domain name of the host, such as test.securonix.com.

l Realm: Specify the realmwhere the Kerberos database is stored. The realm lives on onecomputer (KDC) and can have read-only slave servers (similar to a cluster). For example,test.securonix.com

l Key Tab Path: Enter a key tab path. A key tab is a file containing pairs of Kerberos principalsand encrypted keys, which are derived from the Kerberos password. For example,/home/securonix/ securonix_krb5.keytab.

l Principal: Enter a principal. A principal is an identity that Kerberos is able to authenticate.Principalsmay represent users, network hosts, or network services.

l Jaas Conf File Path: Enter the Jaas Conf file path.

l Service Name: Specify a service name, such as Solr. For example,[email protected].

l Authentication Mechanism: Specify an authenticationmechanism. For example, Kerberos.



c. LDAP: Enter the following information:

l Username: Specify the user name. For help, see help.snypr.settings.ldap.username.

l Password: Specify the LDAP password. For help, see help.snypr.settings.ldap.password.

d. Kerberos with Trust Store: Enter the following information:



l Host FQDN: Enter the fully qualified domain name of the host.

l Key Tab Path: Enter a key tab path. A key tab is a file containing pairs of Kerberos principalsand encrypted keys, which are derived from the Kerberos password.


l Realm: Specify the realmwhere the Kerberos database is stored. The realm lives on onecomputer (KDC) and can have read-only slave servers (similar to a cluster).

l Service Name: Specify a service name. Example: impala.

l Trust Store Path: Enter a trust store path.



l Trust Store Path Password: Enter the trust store password.


l SSL Value: Enter the SSL value.

l Authentication Mechanism: Specify an authenticationmechanism.

2. Enter ZKQuorum: URLs using commas (,) to separate entries.

Append /solr to the last URL after the port number.

Example: 10.0.0.62:2181,10.0.0.61:2181,10.0.0.60:2181/solr

Note: Default port is 2181.

Note: You can find the URL of Zookeeper in Cloudera Manager by navigating to Zookeeper >Instances.

Collection Details

In Solr, data is indexed into collections, which allow for faster results from search queries in Spotter. Thecollections are partitioned into individual chunks of data called shards.

For more information about how to search data collections in SNYPR, refer to the SNYPRUser Guide.

The shards and their replication factors are configured in this section.

1. Specify a uniqueName for each data collection:

l Lookup

l Watchlist

l Control Core



l IP Mapping

l TPI

l Entity Metadata

l Risk Score

l Activity

l Violation

l Daily Violations Summary

l Entity Relation

l Users

l Violation Control Core

l White List

2. Specify theNo(number) of Shards into which to split the data within each collection.

Note: A shard refers to an individual partition of data within Solr.

3. Enter a Replication Factor to specify the number of times to replicate each shard within each collection.



Solr Additional Settings

1. Enter the following information:

l Batch size: Specify the number of events indexed during a single to commit to Solr during indexing.Default 1000.

l Inter Batch Sleep: Specify the duration inmilliseconds to wait before retrying index for a failedbatch. Default 5.

l Percentage of Indexing Servers: Use the dropdown to specify the percentage of indexing serversto be used for activity indexing. Default 70.

l Enable Multi Collection Indexing: SelectYes orNo. If enabled, multiple collections will be createdby the event indexing job whenever the soft threshold is reached. DefaultYes.

l Collection Soft Threshold: Specify the size of the document each collection should have if multiplecollection is enabled. This is only a soft threshold; each collection will have documents near to theconfigured value. Default 100,000,000.

l Collection Count Threshold: Specify the collection count after which the collections getsunloaded. For example, if 100 activity collections are created and threshold is set to 50, the first 50activity collections are unloaded from Solr.

l Replication Threshold: Specify the collection count after which the replication is reduced. Forexample, if there are 100 activity collections and the replication threshold is set to 5, the older 95collection replications are reduced.

l Solr Root Directory: The root directory where the frozen Solr indexes are stored. For HDFS, it willbe /solr. For disk-based, it should be the same as solr.frozen.bucket configured in Solr.

l Solr Service Username: Specify the username of the Solr service. For HDFS, provide a usernamewho has write access to /solr directory. For disk-based, provide the gateway node solr username.

l Solr Service Gateway Node Host: Specify the SSH Hostname of the Solr Gateway node. This isonly applicable for disk-based indexes.

l Solr Service Gateway Node Password: Specify the SSH Password of the Solr Gateway node.This is only applicable for disk-based indexes.

l Solr Service Gateway Node SSH port: Specify the SSH port of the Solr Gateway node. This isonly applicable for disk-based indexes.

l Create Force Collection: Select from the drop down and click Force Create Collection. Thisallows you to create a collection forcefully from the UI whenever new cores are added to Solr.

2. Click Test to verify connection and test status.


Impala/HiveImpala is a massively scalable parallel processing (MPP) SQL query engine for data stored in a computercluster running Apache Hadoop. Impala brings scalable parallel database technology to Hadoop, enablingusers to issue low-latency SQL queries to data stored in HDFS and Apache HBase without requiring datamovement or transformation.



Apache Hive is a data warehouse software project built on top of Apache Hadoop for providing datasummarization, query, and analysis. Hive gives an SQL-like interface to query data stored in various databasesand file systems that integrate with Hadoop.

To configure Impala/Hive, follow these steps:

Authentication and Connection Details






l Service Name: Specify a service name.








l Host FQDN: Enter the fully qualified domain name of the host. This is the Impala serverhostname. For example, host1.securonix.com.


l Principal: Enter a principal. A principal is an identity that Kerberos is able to authenticate.Principalsmay represent users, network hosts, or network services. For example,[email protected]




l Service Name: Specify a service name. For example, Impala.

l Trust Store Path: Enter a trust store path. Ensure that your certificate is imported into the cacertfile. For example: usr/java/jdk/jre/lin/security/cacerts.





2. Enter theConnection URL of Impala using default port 21050.

Note: You can find the Connection URL of Impala in Cloudera Manager by navigating toImpala > Instances.

3. Enter theDatabase name. For example, securonix.

To find the Database name created during Hadoop integration, log in to the Impala shell and use thefollowing query:

[.securonix.com:21000] > show databases;

Example:

[10-0-0-90.securonix.com:21000]: # su - impala $ impala-shell > showdatabases;[10-0-0-90.securonix.com:21000]: # su - impala $ impala-shell> quit;

4. Specify the Table Prefix to use for resources. Example: securonixresource.

5. Specify the JDBC Driver. Example: com.cloudera.impala.jdbc4.driver.

6. Specify the number of Partitions per Page. Default 0.

7. Specify the Impala/Hive URL. Example: com.cloudera.impala.jdbc4.Driver.

8. Specify the Impala/Hive Username and Impala/Hive Password.



HBaseApache HBase is an open-source non-relational (NoSQL) database that runs on top of HDFS and providesreal-time read/write access to those large datasets. Hbase scales linearly to handle large datasets withbillions of rows and millions of columns, and it easily combines data sources that use a wide variety ofdifferent structures and schemas.

To configure HBase, complete the following steps:



l Host FQDN: Enter the fully qualified domain name of the host. For example, test.securonix.com.

l Realm: Specify the realmwhere the Kerberos database is stored. The realm lives on onecomputer (KDC) and can have read-only slave servers (similar to a cluster). For example,test.securonix.com.


l Principal: Enter a principal. A principal is an identity that Kerberos is able to authenticate.Principalsmay represent users, network hosts, or network services. For example,[email protected].


l Service Name: Specify a service name. For example, hbase.









2. Enter theName Space created during Hadoop integration. Example: securonix.

To find the Name Space created during Hadoop integration, log in to the HBase shell and use thefollowing command:

hbase(main):002:0: # hbase shell > list_namespacehbase(main):002:0: # hbase shell > quit

3. Use slider to selectYes orNo to Split Tables in Hbase. DefaultYes.

4. Specify the number ofRegions. Default 3.

5. Specify the Resources required to connect to HBase. Example: file:///etc/hbase/conf/hbase-site.xml


7. Click Save when status is successful.

HDFSThe Hadoop Distributed File System (HDFS) is designed to store very large data sets reliably and to streamthose data sets at high bandwidth to user applications. HDFS stores file systemmetadata and applicationdata separately. HDFS storesmetadata on a dedicated server called the NameNode. Applications data arestored on other servers called DataNodes.

To configure HDFS, complete the following steps:






l Host FQDN: Enter the fully qualified domain name of the host. For example, test.securonix.com.



l Realm: Specify the realmwhere the Kerberos database is stored. The realm lives on onecomputer (KDC) and can have read-only slave servers (similar to a cluster). For example,test.securonix.com.


l Principal: Enter a principal. A principal is an identity that Kerberos is able to authenticate.Principalsmay represent users, network hosts, or network services. For example,[email protected].


l Service Name: Specify a service name. For example, hdfs.












2. Specify theHDFS Site. HDFS site is required to connect to HDFS.Example: file:///etc/hadoop/conf/hdfs-site.xml.

3. Specify theCore Site. The Core site is required to connect to HDFS.Example: file:///etc/hadoop/conf/core-site.xml.

4. Specify theCluster HDFS Site. The Cluster HDFS Site is required to connect to HDFS Cluster.Example: file:///etc/hadoop/conf/hdfs-site.xml.

5. Specify theCluster Core Site. The Cluster Core Site is required to connect to HDFS Cluster.Example: file:///etc/hadoop/conf/core-site.xml.



HDFS Connection Details

6. Specify theUsername of HDFS.

7. Specify theWorking Directory created within the Service Account Folder for SNYPR during Hadoopintegration. Example: /user/securonix.

8. Specify the Product Directory created during Hadoop integration. Example: snypr.

9. Specify theHDFS Directory for storing Unparsed Events. Example: Invalid.

10. Specify theHDFS Directory for storing whitelists/temporary files for analyzing Proxy Events.Example: ProxyEvents.

11. Specify theHDFS Directory for storing Violations. Example: Violations.





RedisRedis is an open-source software project that implements data structure servers. It is networked and in-memory, and it stores keys with optional durability.

To configure Redis, complete the following steps:

Redis Nodes Details

1. Enter the IP Address or Hostname with port for Redis Inmemory-DB connection inNode for Redis.Example : 192.168.1.102:6379.

2. Click Add Node to add additional nodes.

Redis Connection Details

1. Specify a Password if set by the administrator.

2. Specify a Connection Pool size for Redis connections. Recommended default: 10.



SparkThemain feature of Spark is its in-memory cluster computing that increases the processing speed of anapplication. Spark is designed to cover a wide range of workloads such as batch applications, iterativealgorithms, interactive queries, and streaming. In SNYPR, Spark is used in ingestion, indexing, and analyticsalgorithms.



To configure Spark, complete the following steps:

Spark Details

1. Specify the Spark Defaults. The Spark Defaults are required to runSpark. Example: /etc/spark/conf/spark-defaults.conf.

2. Use the slider to Enable Kerberos.

l If Yes: Enter KeyTab Path for connection to YarnMaster server. Example: /Securonix/securonix_home/security/securonix.keytab.

l IfNo: Proceed without entering additional information.

3. Specify the Yarn Master IP.

Note: To find the YarnMaster IP, use Cloudera Manager to navigate to Yarn > Instances. TheYarnMaster IP corresponds to the ResourceManager (Active) IP.

4. Specify the SSH Port for the YarnMaster server. Default 22.

5. Specify the SSH UserName. Example: securonix.

6. Specify the SSH Password.

7. Specify the path of the Yarn Site xml file. Example: /etc.hadoop/conf/yarn-site.xml.

8. Click Save and Next.



SettingsSNYPR settings are configured from the Administrationmenu.

Configuring the ApplicationTo customize SNYPR settings, navigate toMenu > Administration > Settings.

Application SettingsOn the Application Settings page, configure the following options:

General SettingsFrom the General Settings option, you can configure the following information:


SNYPR6.1Administration GuideSettings

l Application Time zone: The time zone for the application server.

l Database Time zone: The time zone for the database server.

l Date Format: Select frommultiple date/time formats from the dropdown box.

l Session Timeout: Enter a timeout period for sessions in seconds.



l Web Services: Toggle to Yes to enable the application to use web services.

l Token Required for Web Services?: Toggle to YES to allow web services to use token for securitypurposes.

l IP Validation during Token authentication: Toggle to YES to allow enabled/disabled IP Validationduring token authentication.

Data Import Settings

Note: These settings are for advanced users only.

The application ismulti-threaded to perform parallel processing. Each event file is processed by spawningmultiple threads. Each thread simultaneously parses the event log file, performs correlation, and inserts theprocessed log into the database. You can configure the settings for various data import activities in thissection.



Multithreading: Use the Yes/No switch to enable or disable parallel processing in the application. If youselect Yes, youmust also configure the following settings:

For Activity Import:

l Maximum Threads: The number of threads that are spawned during the import of activities andevents. (The default is 30.)

l Maximum Lines per Thread: The number of lines provided that are processed by each thread.(The default value is 20000.) Each user file is processed by spawning multiple threads. Each threadsimultaneously parses the user file, checks for identity lifecycle changes, and inserts the processeddata into the database.

For User Import:

l Maximum Threads: The number of threads that are spawned during the import of users. (Thedefault value is 20.)

l Maximum Lines Per Thread: The number of lines provided that are processed by each thread.(The default value is 10000.)

Preview data refresh interval: Specify the number of minutes for which the preview data is cached. Duringthis period, if the Preview button is clicked again, the application retrieves preview data from cache, otherwiserefreshes from the data source. (The default value is 30.)



Do you want to set Invalid Events Threshold: Set to Yes/No to set invalid threshold and invalid eventscount threshold to skip event parsing for further events.

Save events after each file Imported (Yes/No): Enable this setting if you wish to save the events after eachfile is processed. If this is set to no, all of the filesmatching the file pattern will be processed prior to saving tothe database.

Split input event file into smaller files (Yes/No): Use this setting to split the input file to smaller chunks forprocessing. If an extremely large file is encountered (greater than 1 GB), you can split the file to increase theprocessing speed.

Lines per file: Enter the number of lines to be present in each file.

Clear correlation: (Yes/No) Makes the disabled access account (disabled during access import) anorphan and removes the past correlation.

Clear attributes: (Yes/No) Removes all access attributes from the disabled access account (disabledduring access import).

Ignore Account Name Case: (Yes/No) Imports all access accounts as all upper case. If the same accessaccount name is encountered with lower case and upper case, this setting prevents duplicate account names.

Single Sign-onThis screen enables the application for user authentication and Single Sign-on (SSO).

Hostname = Enter the URL for the host; for example, company.com.

Logout URL = Enter the logout URL, for example: http://www.google.com. Once you are logged out, youwill be redirected to this URL.



Quick LinksThis screen allows you to add items to the Menu Bar.

Quick Link Title

Menu Title: This field determines the label that appears on the Menu Bar.

Quick Link URLs

l Name: This determines the link name under the mainMenu Bar name.

l Protocol: Select either http or https.

l URL: Enter the URL that you want to link to when users click the item under the Menu Bar.

l Order: This determines the order that in which the links appear under the Menu Bar.

l +/-: Click the plus sign to add another Quick Link item. Click the minus sign to remove an existing item.

Startup JobsThis option allows you to add and control the jobs that run when the application starts. Options on this screeninclude the following:



l Name: Name of the job.

l Enable: SelectYES to initialize a job that does not exist yet. SelectNO to disable an existing job.

l Force re-schedule: SelectYES if the configuration is changed. On the next startup, the existing job willbe disabled, a new job will be created with a new configuration, and the flag will be reset to NO.

l Frequency: This determines the frequency of the job. Valid values are once, seconds, minutes, hourly,daily, weekly, monthly, or yearly.

l Time: Time at which job should be rerun, format is HH:MM:SS.What

l Interval: This specifies the interval after which the job should be rerun. This field uses the value set inFrequency; for example, if you want a job to run every minute, you could set the Frequency to minutes andthe Interval to 1, or Frequency to seconds and Interval to 60.

When you have finished making changes to the General Settings options, click Save at the bottom of thescreen.

Archival SettingsThe SNYPR application provides a tiered archival option for storing data long term. If you want to store alarge amount of data for compliance or historical analysis for a large period of time, this tiered archival optionoffers several benefits that optimize your storage, provide cost benefits and index data for quick search andretrieval when stored for relatively short periods of time.

The tiered storage option for archiving data is as follows:

l Hot - In this instance, the most recently indexed data is stored in Solr for the number of days specified.This indexed data is optimized for storage and quick search. It is primarily used by Spotter.



l Cold - In this instance, the event data is stored in HDFS parquet format for the number of days specified.The data is enriched, but not indexed, which translates to the search being slower. The data stored in theCold tier is used for reports, for example.

l Frozen - In this instance, the data is archived on an AmazonWeb Services (AWS) for long-term storage.The data format is not available for searching by Spotter.

Use the SNYPR interface to set the archival period through the Archival Settingsmenu. Go toMenu >Administration >Archival Settings.

By default, archiving is disabled. To enable, set the archival settings to Enable.

Archival settings are available in two modes: Global Mode and Datasource Mode.

Note: You can set either mode, but not both at the same time. The Global Mode allows you tospecify archival settings globally for all data sources. If you want to set archival settings at adatasource level only, go to the Datasource Mode.

Setting Hot and Cold Archival Settings at Global LevelTo set the hot and cold storage settings on theGlobal Mode tab, follow these steps:

1. From the Archival Settings window, selectGlobal Mode.

2. Set the Index Expiry Days for SOLR indexes to the number of days you want. At the end of the dayspecified, the indexed data is deleted from SOLR. This setting is set for the hot expiry of data.



3. Set theCold Expiry in Days after which the hot data from SOLR is archived to HDFS in parquet fileformat for archival. This setting is set for the cold expiry of data.

4. UnderDatasource configuration, click the > button to move a datasource to the exclusion list to theright. TheResourcesGroup list includes all the datasources to which you want to apply the globalmode settings.

5. SelectSave Policy to save your policy settings. Any changesmade to the policy take effect when the jobruns next time. It deletes the activity data from SOLR and HDFS based on the rules you configured.

Setting Hot and Cold Archival Settings at Datasource LevelTo set the hot and cold storage settings on theDatasource Mode tab, follow these steps:



1. From the Archival Settings window, selectDatasource Mode.

2. Select the datasource from the Resource Group drop-down.

3. Set the Index Expiry Days for SOLR indexes to the number of days you want. At the end of the dayspecified, the indexed data is deleted from SOLR for this datasource. This setting is set for the hot expiryof data.

4. Set theCold Expiry in Days after which the hot data from SOLR is archived to HDFS in parquet fileformat for archival. This setting is set for the cold expiry of data.

5. Click the + button to add the datasource to which the news are applied when the job runs. If you want toremove the datasource, click the - button.

6. SelectSave Policy to save your policy settings. Any changesmade to the policy take effect when the jobruns next time. It deletes the activity data from SOLR and HDFS based on the rules you configured.

Setting Frozen Archival SettingsThe fields for frozen archival settings are the same at the global or datasource level. However, the onlydifference is that the settings at the global level apply to all datasources. If you have specified the settings atthe datasource level, then the frozen archival settings only apply to the specified data sources on theDatasource Mode tab.

To set the frozen archival settings at global or datasource level, follow these steps:

1. From the Global Mode or Datasource Mode, selectCreate New Connection from the drop-down tocreate a connection name for the archival.

2. The Add New Connection window appears. Youmust provide details for the following fields to enablearchiving of data for in AWS:

l Connection Name: Enter a name for the archival connection.

l Connection Type for: The selection defaults to Archival.

l Connection Type: The selection defaults to AWS.UnderConnection Details, provide information for the following fields:



l Access Key: Enter the AWS access key. This key uniquely identifies the user who owns the AWSaccount.

l Secret Key: Enter the secret key. The secret key is used to calculate the digital signature that youinclude in the request.

l Bucket: Once you have provided the access and secret keys, you can test the connection to get theAWS bucket list. A bucket is a logical unit of storage in AWS.

l Select the list from the drop-down. This bucket list is populated if the connection to AWS issuccessful.

l Source Folder: Specify the path to the folder where the file to be uploaded is located.Default: ${SECURONIX_HOME}/import/in.

l Success Folder: Specify the folder into which you would like the file to move upon successfulupload. Default: ${SECURONIX_HOME}/import/success

l Failed Folder: Specify the folder into which you would like the file to move upon a failed upload.Default: ${SECURONIX_HOME}/import/failed.

l Incremental Field: Set this to Yes if you want incremental updates.

l Prefix: Specify the path within the bucket fromwhich logsmust be extracted. You can use this to limitthe response to folders that begin with the specified prefix. For example:aws/AWSLogs/853268358782/CloudTrail/us-east-1/2017 limits the search to logs from 2017.

3. Click Save to save your settings for the frozen, long-term storage.

4. SelectSave Policy to save your policy settings.

Note: The settings for datasource configuration apply to the frozen storage, too. If a datasource isexcluded from the list at the global level, then activity data for that datasource is permanently deletedfrom the cold storage when it expires.

DNS ServersTo access the DNS Servers to add and change the IP entries for your DNS servers:

1. Navigate to Configure > Settings.

2. On the left navigation pane, click DNS Servers.

3. Change, add, or remove IP addresses as needed.



Data MaskingFor details about the Data Masking settings, refer to Configuring Data Masking.

HadoopFor details about the Hadoop component settings, please refer to Configure SNYPR Hadoop Settings.

Housekeeping JobsSome recordsmaintained by the application do not hold much value as the data ages. These recordsmaybe deleted after a period. The application provides housekeeping jobs to delete these aging records. Decidehow long to maintain these records and configure the housekeeping jobs to remove old data. To access thehousekeeping jobs:

1. Navigate toMenu > Administration > Settings > Housekeeping Jobs.

Types of Housekeeping Jobs

Job Name DescriptionRecommendedSchedule

User ImportHistory Every time auser import is fired, the SNYPRapplicationstores the history of the number of newusers, deletedusers, updatedusers, etc. This jobclears this table basedon the input days.

For example, query fired:DELETEFROMUserimporthistoryWHERE importdate

Job Name DescriptionRecommendedSchedule

Activity User IPMapping Clears the activity user IPmapping that ismaintained forIP address attribution.

For example, query Fired:DELETEFROMActivityuseripmappingWHERE lastupdate

6. On the Schedule Housekeeping Job screen, you can enter a Job Description in the text box (optional).

7. To configure email notifications, set Enable Job Related Notifications to YES.

8. You can configure the application to send notification emails upon success, failure, or misfired. Completethe email notifications as needed.

9. From theRun Job options, select the frequency for which you want to run the housekeeping job.

10. To save the housekeeping job, click Save. To run the housekeeping job now, click Run.

Avroparquet Migration JobA nightly job consolidates events fromAvro and migrates it to compressed Parquet format in HDFS. Thisfunctionality enables you to migrate events fromAvro to Parquet format manually.

1. To run the migration job manually, navigate toMenu > Settings > Administration > AvroParquetMigration Job.

2. Use > or >> to select the resource groups from the left and move them to the right.



3. Click Run Migration to trigger the migration job. The job is triggered immediately to migrate the eventsfromAvro to Parquet.

LDAP Authentication

Prerequisites for setting up LDAP Authentication1. The LDAP account should have read permissions for the organizational unit against which the

application authenticates.

2. Identify the DN (Distinguished Name) for the account. For example: cn=svc_[DN];OU=ServiceAccounts;DC=[DN];DC=com

3. Identify the following additional parameters that are required for AD authentication:

l The IP address/hostname of the domain controller.

l The OU (organizational units) containing the different users that should be authenticated.

Understanding the ConfigurationBy default, the application authenticates against the local MySQL data store. However, this can be changed toauthenticate the users against Active Directory.

Note: The authorization for the users is performed based on locally assigned roles.

l managerDn =

l managerPassword =

l grails.plugins.springsecurity.ldap.context.server = (ex:ldap://xx.xx.xx.xx:389 orldaps://xx.xx.xx.xx:636)

l grails.plugins.springsecurity.ldap.authorities.groupSearchBase =

l grails.plugins.springsecurity.ldap.search.base =

To change the default LDAP authentication:

1. Add the following line to the ldap-config.properties file in the “/securonix/securonix_home/conf/”:

grails.plugins.springsecurity.ldap.authorities.groupSearchFilter=member={0}

2. Add the userid (same as AD login) for the application, and provide the appropriate access controls. Bydefault, the system uses the sAMAccountName for authentication. This can be changed by changing thefollowing value:

grails.plugins.springsecurity.ldap.search.filter=sAMAccountName={0}

3. Change ‘sAMAccountName’ to cn, dn, or other distinguishable value as required.

4. If local user authenticationmust be enabled, comment the following line; otherwise, authentication will beonly against AD. Uncomment it to authenticate only against AD.

grails.plugins.springsecurity.providerNames = ldapAuthProvider



5. To debug the errors faced, make the following changes to the log4j.properties files:

log4j.logger.org.springframework.security=DEBUG

Note: If there are multiple domains to be configured, create a virtual directory that has the entire list ofusers. Use the credentials of the virtual directory in the ldap-config.properties files.

Configure LDAP1. Navigate to “/securonix/securonix_home/conf/”.

2. Open the file: ldap-config.properties.

3. Make following changes:

grails.plugins.springsecurity.providerNames = ldapAuthProvidergrails.plugins.springsecurity.ldap.context.managerDn = The path of LDAPgrails.plugins.springsecurity.ldap.context.managerPassword = Passwordgrails.plugins.springsecurity.ldap.context.server = ldap://master serveripgrails.plugins.springsecurity.ldap.authorities.ignorePartialResultException = truegrails.plugins.springsecurity.ldap.search.searchSubtree = truegrails.plugins.springsecurity.ldap.search.base = dc=oracledemo,dc=comgrails.plugins.springsecurity.ldap.authorities.groupSearchFilter=member(0)

4. In the application, navigate toMenu > Administration > Settings, and then select LDAPAuthentication.

5. For the Enable LDAP Authentication setting, selectYES.

6. Complete the following settings:

l Server: Enter the IP address for Active Directory (ldap://[ip]:[port]/).

l Base: Enter the base directory to start the search. For example, dc=mycompany,dc=com].

l Enter the appropriateManager DN.

l Enter the appropriateManager Password.

l Retrieve Database Roles: Select whether to retrieve additional roles from the database using theUser/Role many-to-many.

l Retrieve Group Roles: Select, whether to infer roles based on group membership.

l Ignore Partial Result Exception: Select whether to ignore partial result exceptions.

l Search Subtree: Select whether you want to search in subtrees.

l Search Filter: This is the pattern to be used for the user search. For Example, {0} is the user’s DN.



l Group Search Base: Enter the base DN fromwhich the search for group membership should beperformed.

l Group Search Filter: Enter the pattern to be used for the user search. For example, {0} is the user’sDN.

l Group Role Attribute: Enter the ID of the attribute which contains the role name for a group.



7. When you have finished, click Save.

Log SettingsThis section allows you to view the application and set the log levels for eachmodule in theSNYPRapplication. To access the log settings, navigate toMenu > Administration > Settings. Select LogSettings from the left pane.

Application LogsThis option displays the application logs and provides an option to set the application logs to auto update. Ifyou selectYES, an additional option to Disable Auto Update After a specified period is available.



Logging

Setting up Logging to securonix.log FileThe SNYPR application logs both errors and debug statements to a log file. Conveniently namedsecuronix.log, the log file is located in the “/logs” directory.

You can change the location of the securonix.log file to any desired folder.

To specify the location of the log file:

1. Navigate to /WEB-INF/classes.

2. Search for a file named log4j.properties.

3. Open the file with a text editor.

4. To specify the location of the logs file, search for the following line under the # File Appender heading:

log4j.appender.file.file=.../securonix.log

Note: To begin logging to the new location, youmust restart the application.

Changing the log formatBy default, the log file does not include the date on which the log waswritten. This is because of the followingdirective in log4j.properties:

log4j.appender.file.layout.ConversionPattern=%d{ABSOLUTE} %-5p [%c{1}] %m%n

For example, from securonix.log:

09:37:26,744 DEBUG [LoginController] auth. Getting license information…

If you want to change this setting to include the date, use the following format:

log4j.appender.file.layout.ConversionPattern==%d{dd MMM yyyy HH:mm:ss,SSS} %-4r [%t] %-5p %c{1}%x - %m%n



Log LevelsERROR: The ERROR level designates error events that might still allow the application to continue running.

FATAL: The FATAL level designates very severe error events that will presumably lead the application toabort.

OFF: Turn off logging.

WARN: TheWARN level designates potentially harmful situations.

INFO: The INFO level designates informational messages that highlight the progress of the application atcoarse-grained level.

ALL: The ALL level has the lowest possible rank and is intended to turn on all logging.

DEBUG: The DEBUG level designates fine-grained informational events that are most useful to debug anapplication.

TRACE: The TRACE level designates finer-grained informational events than the DEBUG level.

Changing Logging LevelsLogging can be changed for eachmodule within the application. To change the logging levels, perform thefollowing steps:


2. On the left navigation pane, click Log Settings > Logging.

3. Change the log level for the desired module.

4. To save your changes, click Update.

Change Log Levels for Modules

The following modules are available for logging:

l Imports: Logging for User Import and Glossary Import actions.

l Activity Imports: Logging for Activity Import for various connections.

l Policy Engine: Detect Behavioral Analytics, Anomaly Detection

l Web Services: Web application components.

l Work Flow: SOC TeamReview, Activity Outlier Workflow, Access CertificationWorkflow.

l Licensing: Logging for Managing, updating license.

l Views: Users, Resources, Peers, Organizations, Application.

l Run: Access, Activity, Policy violations, Behavior Profiles.

l Reports: Running and rendering reports.

l Configure: All actions available under the configure menu.

l UI Utilities: Analytical Activities, Applications, Dashboard, Incidents, Organizations, Peer, Resource,Detect, Transaction, User, Utility Impl, Token, CommonUI Utilities,Workbench Util.



Log Level Choices

All modules have the same log level choices. The default setting for each, however, is different. Choices are:

l All: All has the lowest possible rank and is intended to turn on all logging.

l Debug: Designates fine-grained informational events that are most useful to debug an application.

l Error: Designates error events that might still allow the application to continue running.

l Fatal: Designates very severe error events that will presumably lead the application to abort.

l Info: Designates informational messages that highlight the progress of the application at a coarse-grained level.

l Off: Off has the highest possible rank and is intended to turn off logging.

l Trace: Designates finer-grained informational events than debug.

l Warn: Designates potentially harmful situations.

Set Log Levels

To set the log levels:

1. From the Select a resource to view logs dropdown list, select a module to view its current Log Level.

2. To change the current log level for a specific module, select an option from the Log Level dropdown.



3. To save your changes, click Update.

Manage LicenseThis section allows you to review your licenses installed with the application. View details about the currentlicenses including number of users and resources licensed, license issue and expiration date and issuerdetails. To manage licenses, navigate toMenu > Administration > Settings. SelectManage License fromthe left pane.



Current License

Installed LicensesThis section displays the installed licenses, which you can uninstall by clicking theUninstall button.

Install/Upgrade LicenseIn the Install/Upgrade License options you can upload a new license and enter a new activation key.

SAML SettingsSecurity AssertionMarkup Language (SAML) is an XML-based, open-standard data format for exchanging



authentication and authorization data between parties, in particular, between an identity provider and aservice provider. SAML settings are related to configuration of single-sign on (SSO), which help reduce theadministrative overhead of distributing multiple authentication tokens to the user.

To configure the SAML settings:

1. Navigate toMenu > Administration > Settings, and then from the left navigation pane, selectSAMLSettings. On the SAML Settings page, you can:

l Generate metadata for the new Service Provider

l Share the metadata for the Service Provider

l Obtain a list of registered Identity Providers

l Create users with Securonix

2. Click theClick here to generate new service provider metadata link.



3. The SAML Current Settings screen appears.

l Entity ID: The Entity ID is a unique identifier for an identity or service provider. This value is includedin the generated metadata.

l Entity Base URL: Base to generate URLs for this server. For example: https://myServer:443/saml-app. Enter the public address fromwhich your server will be accessed.

l Entity Alias: The Alias is an internal mechanism that allows the application to collocate multipleservice providers on one server. The alias entity must be unique.

l Include IDP Discovery: Select this option to include identity provider discovery in the metadata.



l SSOBindings: Select the bindings to use for SSO, which include Post, PAOS, and Artifact. Thebinding, in general, determine how an SAML request and responsemap to protocols for messagingand communication.

l Security Profile: From the dropdown list, select the option you want to use for trust of signature,encryption, and SSL/TLS credentials.

l Signing Key: The key used for digital signatures of SAMLmessages

l Encryption Key: The key used for digital encryption of SAMLmessages

l SSL/TLS Key: The key used to authenticate an instance for SSL/TLS connections

l Sign metadata: Select this option to digitally sign the generated metadata with the specifiedsignature key.

l Sign sent AuthNRequests: If selected, the generated metadata is digitally signed using thespecified signature key.

l Require signed authentication Assertion: If selected, the generated metadata is digitally signedusing the specified signature key.



l Require signed LogoutRequest: If selected, the generated metadata request is digitally signed forlogout requests using the specified signature key.

l Require signed LogoutResponse: If selected, the generated metadata request is digitally signedfor logout responses using the specified signature key.

l Require signed ArtifactResolve: If selected, the generated metadata request is digitally signed forartifact resolution using the specified signature key.

4. To continue, click Generate Metadata. The new Service Provider metadata is generated.

5. TheMetadata Management for SAML screen appears to with the newly-generated Service Providermetadata. Share the metadata with the Service Provider to allow for redirection of request to theapplication.

6. Click the link to obtain a list of registered identity providers. The Metadata Management for SAML screenappears to download the metadata for the Identity Provider.

7. Copy the metadata provided by the SAML provider in the text box to register the new Identity Provider.

8. Click Submit to save the identity provider (IDP) metadata.

SMTP Server SettingsThe application uses the mail server for the following purposes:

l To send email notifications on a violation.

l To send job success/failure notifications.

l To send email notifications on user lifecycle changes (new, updated, and terminated users).

l To send notification emails for case-related issues.

l To receive emails when comments are added to existing cases.



Adding an SMTP Server

To set up a new SMTP server:


2. On the left navigation pane, click SMTP Server Settings.

3. To add a new server, click Add New Mail Server.

4. Use the following steps to configure the General Settings section:



a. Mail Box Name: You can keep the default setting or provide a name of your choice.

b. Host: Enter a host name for the mail server.

c. Port: Enter an outgoing port.

d. From email: Type the name of the email account used for sending email.

e. SSL enabled?: Toggle the YES/NO switch to enable or disable SSL communication.

f. Authentication required: If the mail server requires authentication, selectYES.

g. UserName and Password: If authentication is set to YES, enter the username and password.

6. Use the following steps to configure theMore Settings section:

a. Font name: Select a font type from the dropdown list. The default font is Arial.

b. Font size: Enter the size of the font you want to use in your email notifications. The default font size is2.

c. Batch size: Enter the number of email notifications that are sent in a batch. The default setting is 25.

d. Interval: Enter the number of seconds for retrial. The default is 10.

e. Process In Batch: Choose whether you want to send email notifications in batches. The default isYes.

f. StopWhen Done: Choose whether you want to stop sending email notifications when all of themessages in queue are completed. The default is Yes.

7. When you have finished, click Save. You can also save settings and send a test email, or test the SMTPserver using choices at the bottom of the Mail Server Settings screen:

UI PreferencesFrom this screen, you can enter text to appear on the SNYPR Logon screen. For example, display thecompany privacy policy.



Configure SSL for Secure Data Transferfrom Remote Ingester to HadoopClusterThis section covers the following sections in detail:

l Kafka Brokers: Creating SSL Certificates, Keystores, and Truststores

l Kafka Brokers Enabling SSL Keystore and truststore

l Remote Ingester: Creating SSL Certificates, Keystores and Truststores

l Remote Ingester: Configuring the Properties Files

Kafka Brokers: Creating SSL Certificates,Keystores and TruststoresTo create server certificates in the Hadoop cluster, follow these general steps:

l Create a master keystore to store all the certificates centrally. This is an optional step. Youmay alreadyhave a master keystore in which you can store your CA-signed SSL certificate.

l Create a CA-signed SSL certificate (for signing all server certificates with a trust chain).

Note: Youmay choose to create a self-signed certificate or use certificates that are signed by aCA (Certified Authority), such as Verisign. The signed certificates are required for signing allserver certificates with a trust chain.

l Create server certificates (one for each server, with the subject = the FQDN of the server for easieridentification)

l Create a server keystore (one for each server)

l Import the server certificate into the server keystore

l Create a truststore (the same truststore can be used for all servers)

l Import the SSL certificate trust chain into the truststore

Once you have created and saved an SSL certificate, follow these steps to create a keystore and truststore onyour Kafka brokers in the Hadoop Cluster. These stepsmust be repeated on each Kafka broker (generallythree for Securonix).


SNYPR6.1Administration GuideConfigure SSL for Secure DataTransfer fromRemote Ingester toHadoopCluster

Keystore Description

server.jks The client keystore for access to the Securonix environment. This keystore contains auniquecertificate key pair for each customer,with acommon name that is equal to the server name.Thecertificate is used formutual authentication to access the Securonix environment. This id is used foraccess control in the environment.

truststore.jks The truststore for the Securonix environment. This contains the certificate trust chain for the Securonixinfrastructure. The same truststore is usedby all the servers.

Table 1: Examples of keystore and trust keystore names on Kafka Brokers

Note: The naming conventions used in this section are for suggestion purposes only. Youmay useyour own naming conventions, but ensure you are copying the correct server certificate to eachserver in your cluster.

1. Open your master keystore using a tool of your choice if you have stored your SSL certificate in themaster keystore.

Keystore Explorer is an open source utility that is being used in this example to open the master keystore.You can download KeyStore Explorer at http://keystore-explorer.org.

2. Select the signed certificate. For example, securonix ca. Right click and selectSign > Sign New KeyPair. Enter the key password for the certificate.



3. Accept the defaults, and click OK.

4. Accept the defaults, and enter theName details.



5. The name details include the following, for example:

l CN=

l OU=saas

l O=Securonix

l L=Addison

l ST=TX

l C=US

6. Click OK, and click OK again.

7. Accept the alias. Enter the password and click OK.



8. Create a server keystore to store the server certificate:

l Select file as New.

l Select keystore type as JKS, and click OK. The server keystore is a Java key store.

9. Enter the keystore password and click Ok to save.

10. Enter the name of the keystore. Ensure that you use the same name for all the server certificates. Create anew folder with the server FQDN (Fully Qualified Domain Name) as the name, and save the keystoreinside the folder. Keep the keystore open to import the correct server certificate into it for the next step.



11. Select the master keystore, and select the server certificate to export to a file. Right click, select Export >Export Key Pair

12. Enter the password for the certificate file and click Export.

13. Import the server certificate into the server keystore, and click OK.

l select the correct server keystore.

l select Tools > Import Key Pair

l Select PKCS #12

14. Enter the decryption password associated with the certificate file. Select the server certificate file, and click



Import.

15. Accept the default alias and click OK.

16. Enter the Key Pair entry password and click OK.

17. Save the keystore. The keystore is ready to bemoved to the server.

18. Create keystores for each server in your cluster with the same keystore name, the correct server certificate,and the same password for the keystore and key pair.

19. Create a trust keystore. You are creating a certificate chain of trust with a trust keystore (or truststore).

The truststore contains an SSL certificate chain you are willing to trust when a remote party presents itscertificate. Essentially, a certificate chain is an ordered list of certificates that contains a CA-signedSSL certificate (root SSL certificate) and an intermediate SSL certificate. This combination enables thereceiver to verify that the sender and its intermediate SSL certificate are trustworthy. The chain or pathbegins with the CA-signed SSL certificate, and each certificate in the chain is signed by the entityidentified by the next certificate in the chain. The chain terminates with the root SSL certificate.

l Select file as New.

l Select keystore type as JKS, and click OK.

20. Save the keystore. Select File > Save. Enter the keystore password and selectOK.



21. Enter the name of the keystore. This truststore will be shared with all the servers. Keep the truststore opento import the correct server certificate in the next step.

22. Open the master keystore from the Keystore Explorer. Use your master keystore password.

23. Select the securonix ca certificate. Right click, select Export > Export Certificate Chain.

24. Enter the key password for the securonix ca.

25. Select a file name for the certificate chain to export, and click Export.



26. Select the truststore (truststore.jks) that was created earlier. Select Tools > Import Trusted Certificate.

27. Select the file that was exported, and click Import. SelectOK.

28. Save the truststore. This truststore is ready to move to all servers for SSL configuration.

Kafka Brokers: Enabling SSL Keystore andTruststoreComplete the following actions on each server:

# sudo mkdir /opt/certs



1. Copy the keystore (different for each server), and the truststore (the same for all servers) to:/opt/certs/server.jks

/opt/certs/truststore.jks

chown root:root /opt/certs/truststore.jks

chmod 0440 /opt/certs/truststore.jks

chown root:root /opt/certs/server.jks

chmod 0440 /opt/certs/server.jks

2. Configure Kafka for SSL in the Cloudera Manager:

a. Log in to Cloudera Manager.

b. From theHome page, selectKafka on the left.

c. Set the values in the Kafka service by clicking theConfiguration tab. On the Configuration screen,search for the property that you want to set, for example, ssl.client.auth. Similarly, set the values for theremaining properties as shown in the table.



Property Value

SSLClient Authentication (ssl.client.auth) required

Inter Broker Protocol (security.inter.broker.protocol) Inferred

KafkaBroker AdvancedConfiguration Snippet(Safety Valve) for kafka.properties

num.network.threads=16socket.send.buffer.bytes=1048576socket.receive.buffer.bytes=1048576socket.request.max.bytes=104857600replica.fetch.wait.max.ms=500replica.socket.timeout.ms=30000replica.socket.receive.buffer.bytes=65536replica.high.watermark.checkpoint.interval.ms=5000controller.socket.timeout.ms=30000controller.message.queue.size=10zookeeper.sync.time.ms=2000socket.request.max.bytes=104857600queued.max.requests=16fetch.purgatory.purge.interval.requests=100producer.purgatory.purge.interval.requests=100

Enable TLS/SSL for KafkaBroker (ssl_enabled) checked

KafkaBroker TLS/SSL Server JKSKeystore FileLocation (ssl.keystore.location)

/opt/certs/server.jks

Table 2: [Kafka Properties to Set]



Property Value

KafkaBroker TLS/SSL Server JKSKeystore FilePassword (ssl.keystore.password.generator)

keystore password

KafkaBroker TLS/SSL Server JKSKeystore KeyPassword (ssl.key.password.generator)

key password

KafkaBroker TLS/SSLCertificate Trust Store File(ssl.truststore.location)

/opt/certs/saastruststore.jks

KafkaBroker TLS/SSLCertificate Trust StorePassword (ssl.truststore.password.generator)

truststore password

Table 2: [Kafka Properties to Set] (continued)

d. Stop the Kafka brokers by going to Instances.

e. Select a Kafka broker, and click theActions for Selected dropdown. From this dropdown, clickStop.



f. Deploy the client configuration.

g. Start the Kafka brokers.

Remote Ingester: Creating SSL Certificates,Keystores and TruststoresDepending on the number of Remote Ingesters in your environment, youmust have an SSL certificategenerated for each of them. Each Remote Ingester also uses a truststore for the environment; this truststore isthe same for all the Remote Ingesters.

Keystore Description

ingester.jks The client keystore for access to the Securonix environment. This keystore contains auniquecertificate keypair for each customer,with acommon name that is equal to the Remote Ingester servername.The certificate is used formutual authentication to access the Securonix environment. This id isused for access control in the environment.

truststore.jks The truststore for the Securonix environment. This contains the certificate trust chain for the Securonixinfrastructure. The same truststore is usedby all the Remote Ingesters.

Table 3: Example of keystore and trust keystore names on the Remote Ingester

Creating an Ingester Keystore with New CA-SignedSSL CertificateOnce you have created and saved a CA-signed SSL certificate, follow these steps to create a keystore andtruststore for each Remote Ingester.

1. Open your master keystore using a tool of your choice if you have stored your SSL certificate in themaster keystore.

Keystore Explorer is an open source utility that is being used in this example to open the master keystore.You can download KeyStore Explorer at http://keystore-explorer.org.


SNYPR6.1Administration GuideConfigure SSL fo

6.1 administration guide...10-0-0-90.securonix.com/10.0.0.90:8032 cs_event_enrichment running...

Documents