© 2012 IBM Corporation1
Logging and Tracing
InfoSphere Streams Version 3.0
Jingdong Sun, Warren Acker, and Ginny MurrayInfoSphere Streams
© 2012 IBM Corporation2
Important Disclaimer
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.
WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.
IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
• CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS); OR
• ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENT GOVERNING THE USE OF IBM SOFTWARE.
The information on the new product is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information on the new product is for informational purposes only and may not be incorporated into any contract. The information on the new product is not a commitment, promise, or legal obligation to deliver any material, code or functionality. The development, release, and timing of any features or functionality described for our products remains at our sole discretion.
THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
© 2012 IBM Corporation3
Agenda
Separated Streams logging from tracing
Configurable log/trace file location (Streams Version 2.0.0.4) Dynamic updating of log/trace properties/configs Separating Streams PEC service tracing from PE application
tracing New rolling files to keep stdout/stderr
Enhanced product problem determination guide
© 2012 IBM Corporation4
Separated Streams logging from tracing
Logging – the collection of external diagnostic information, typically used by administrators to correct issues that are within their realm of control
– Totally new to Streams– Target user – Streams administrators
Tracing – the collection of product internal diagnostic information, typically used by a product support or product development team to debug issues encountered in a customer deployment
– Target user – Streams support team and Streams application development support team
© 2012 IBM Corporation5
Separated Streams logging from tracing -- New(updated) instance properties
Instance scope properties for both logging and tracing– LogPath: root directory for Streams to save log and trace files (Streams 2.0.0.4)
Instance scope log properties– LogType: file, syslog– LogLevel: error, warn, info. Default: warn. Dynamically update– LogFileMaxFileNum: Valid when using “file” logging option, the maximum number of
rolling log files that will be stored for each process that logs data. Default: 3 – LogFileMaxFileSize: Valid when using “file” logging, the maximum size of each
rolling log file for each process that logs data. Default: 5 MbNote: LogFileMaxFileNum and LogFileMaxFileSize are updated. For Streams Version 3.0, they are used for logging. But for prior releases, they are used for tracing
Instance scope trace properties– TraceFileMaxFileNum: The maximum number of rolling trace files that will be stored
for each process. For both trace and stdout/stderr files. Default: 3 – TraceFileMaxFileSize: The maximum size of each rolling log file for each process
that logs data. For both trace and stdout/stderr files. Default: 5 Mb– HC.PEC.TraceLevel: For PEC trace level setting– InfrastructureTraceLevel and all XXX.TraceLevel: support off, error, warn, info,
debug, and trace levels. Default: error. Dynamically update
© 2012 IBM Corporation6
Separated Streams logging from tracing -- Streams Console updates
© 2012 IBM Corporation7
Separated Streams logging from tracing -- Streams Console updates
© 2012 IBM Corporation8
Separated Streams logging from tracing -- CLI commands updates
streamtool getlog streamtool cleanlog streamtool viewlog
– New options of –log, --trace– Please check man page for more details
© 2012 IBM Corporation9
Separated Streams logging from tracing -- SPL updates
Deprecated config: logLevel New config: tracing
Deprecated macros: SPLLOG New macros:
– SPLAPPTRC – SPLAPPLOG
Deprecated built-in native function:– <string T> public void log(enum {error, info, debug, trace} logLevel, T
message, T aspect) New built-in native functions:
– <string T> public void appLog(enum {error, warn, info} logLevel, T message, T aspect)
– <string T> public void appTrc(enum {error, warn, info, debug, trace} trcLevel, T message, T aspect)
© 2012 IBM Corporation10
Separated Streams logging from tracing -- Java operator updates Use the standard Java logging apis, java.util.logging and Apache log4j Tracing facility:
– The root logger for java.util.logging will map to the Streams trace facility.– Java log level to Streams Trace Level:
• SEVERE : error; WARNING : warn; INFO : info; CONFIG : info; FINE : debug; FINER : debug; FINEST : trace
– Sample:Logger logger = Logger.getLogger(“com.ibm.streams.operator.samples.sinks.HttpPost”);
Logging facility:– A Logger for the Streams log facility will be added with name
com.ibm.streams.operator.log - and will log messages to the:• the Streams log facility for errors logged with a severity >= CONFIG• the Streams trace facility for errors with a level of < CONFIG.
– Sample:Logger log = Logger.getLogger(“com.ibm.streams.operator.log” + “.” + “com.acme.MyOperator”, myBundle);
Deprecation APIs:– OperatorContext.getLogging(): Replaced by Logger.getLogger(name)– Logging interface and all methods: Replaced by java.util.util.Logger. Aspects
handled by setAspects()– Logging.Level: Enumeration and all values replaced by java.util.logging.Level– AbstractOperator.getLogger(): Replaced by Logger.getLogger(name)
© 2012 IBM Corporation11
Dynamic updating of log/trace properties/configs
Support dynamic updating of Streams service and application log and trace level.
Related updates from Streams Console and IDE CLI:
– streamtool setprop/rmprop• No change from user interface• But some properties can be updated dynamically to Streams Runtime:
LogLevel, InfrastructureTraceLevel, XXX.TraceLevel– streamtool updatepe and streamtool displaype
• New to Streams Version 3.0– streamtool submitjob
• New –config option
© 2012 IBM Corporation12
Dynamic updating of log/trace properties/configs -- Streams Console updates
© 2012 IBM Corporation13
Separating Streams PEC service tracing from PE application tracing
Prior releases:– SPL config logLevel affect both PEC service and PE application tracing– InfrastructureTraceLevel setting does NOT affect PEC service
Streams Version 3.0:– SPL config tracing affect only PE application tracing– InfrastructureTraceLevel and HC.PEC.TraceLevel settings affect PEC service
tracing Comparatives between Streams Version 3.0 and prior releases:
– turn off trace totally (all trace levels to off):• Prior releases: Not supported• Streams Version 3.0: InfrastructureTraceLevel=off, SPL::tracing=off
– all trace levels to "error"• All releases: Default
– turn Streams services trace (not including PEC)• Prior releases: InfrastructureTraceLevel=trace• Streams Version 3.0: InfrastructureTraceLevel=trace, HC.PEC.TraceLevel=off
– Turn on PE application trace, but keep Streams service trace off• Prior releases: Not supported, since PEC and PE trace are managed together• Streams Version 3.0: InfrastructureTraceLevel=off, SPL::tracing=trace
© 2012 IBM Corporation14
New rolling files to keep stdout/stderr
Prior releases:– stdout/stderr messages from a Streams service saved with xxx.out.0 file,
such as sam.out.0 file– The .0 file size is not configured and can keep growing (Issue: potentially
taking whole disk space) Streams Version 3.0
– stdout/stderr messages from a Streams service saved with xxx.stdouterr file, such as sam.stdouterr file
– Support rolling for this file like standard Streams log and trace file– Fix the problem of stdout/stderr message files of Streams Runtime
services and applications taking disk space without control
© 2012 IBM Corporation15
Enhanced product problem determination guide -- section explains techniques, tools, and IBM Support resources
• Use techniques, diagnostic tools, and the new log and trace services to resolve problems.
• Find the IBM Support information you need to diagnose and report problems.
• Contact IBM Support and submit problem determination information.
• Receive continuous updates about new problems, solutions, and IBM support resources.
© 2012 IBM Corporation16
Find solutions to the latest problems for:
• IBM InfoSphere Streams
• InfoSphere Streams Studio
• The Toolkits
Enhanced product problem determination guide --section explains the known problems and solutions
© 2012 IBM Corporation17
Use the IBM InfoSphere Streams Information Center to:
• Over 1,000 runtime log messages for all the Streams components.
• Look up and search for runtime log messages by message ID.
Enhanced product problem determination guide – section enables you to look up and search for the runtime log
messages
© 2012 IBM Corporation18
Log message example:
From Streams.log file that collected by streamtool getlog or by streamtool viewlog –log:
[2012-10-22T09:15:07.976-0400] [streams@daslerjo] [10.6.24.40] [:::HC.MonitorPeHealth] CDISR3500E The service of the processing element container for processing element ID 1 was forced to shut down because the typical shut down operation for the processing element application did not succeed.
[2012-10-22T09:15:08.203-0400] [streams@daslerjo] [10.6.24.40] [:::SAM] CDISR1308I The Streams Application Manager stopped the 1 processing element.
CDISR3500E
The service of the processing element container for processing element ID processing_element_id was forced to shut down because the typical shut down operation for the processing element application did not succeed.
Explanation
The typical shut down process for the processing element application failed. The service was forced to shut down.
User response
Contact the development team that developed the processing element, and ask them to correct the error.
CDISR1308I
The Streams Application Manager stopped the processing_element_id processing element.
Explanation
The processing element was stopped.
User response
This message is for informational purposes only.