jmx bridge for pcf documentation - pivotal softwareby default, the nozzle prefix opentsdb.nozzle...

29
JMX Bridge for PCF ® Documentation Version 1.9 Published: 29 Jan 2019 © 2019 Pivotal Software, Inc. All Rights Reserved.

Upload: others

Post on 20-May-2020

18 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

JMXBridgeforPCF®

Documentation

Version1.9

Published:29Jan2019

©2019PivotalSoftware,Inc.AllRightsReserved.

Page 2: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

234

151620212425

TableofContents

TableofContentsPivotalCloudFoundryJMXBridgeDeployingJMXBridgeUsingJMXBridgeUsingSSLwithaSelf-SignedCertificateinJMXBridgeJMXBridgeResourcesTroubleshootingandUninstallingJMXBridgeApplicationSecurityGroupsReleaseNotesandKnownIssues

©CopyrightPivotalSoftwareInc,2013-present 2 1.9

Page 3: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

PivotalCloudFoundryJMXBridge

ThePivotalCloudFoundry(PCF)JMXBridgecollectsandexposessystemdatafromCloudFoundrycomponentsviaaJMXendpoint.Youcanusethissystemdatatomonitoryourinstallationandassistintroubleshooting.

TheJMXBridgetooliscomposedofthefollowingtwoVMs:

TheJMXprovider

ANozzlefortheLoggregatorFirehose .FormoreinformationabouthowaFirehosenozzleworks,seeNozzles .

ProductSnapshotThefollowingtableprovidesversionandversion-supportinformationaboutPCFJMXBridge.

Element Details

Version v1.9.9

Releasedate February9,2018

CompatibleOpsManagerversion(s) v1.11.3orlater,v1.12.x,v2.0.x,v2.1.x,v2.2.x

CompatibleElasticRuntimeversion(s) v1.11.x,1.12.x

CompatiblePivotalApplicationServiceversion(s) v2.0.x,v2.1.x,v2.2.x

IaaSsupport AWS,Azure,GCP,OpenStack,andvSphere

IPsecsupport? Yes

JMXBridgeUserGuideDeployingJMXBridge

UsingJMXBridge

UsingSSLwithaSelf-SignedCertificateinJMXBridge

Resources

TroubleshootingandUninstallingJMXBridge

ApplicationSecurityGroups

ReleaseNotesandKnownIssues

ViewthesourceforthispageinGitHub

IMPORTANT:ThePivotalCloudFoundry(PCF)JMXBridgetileisdeprecated,andnofurtherdevelopmentwillbemadeagainstthisproduct.

©CopyrightPivotalSoftwareInc,2013-present 3 1.9

Page 4: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

DeployingJMXBridgePagelastupdated:

TheJMXBridgetoolisaJMXextensionforElasticRuntime.FollowtheinstructionsbelowtodeployJMXBridgeusingthePivotalCloudFoundry (PCF)OperationsManager.

Step1:InstalltheJMXBridgeTile

1. DownloadJMXBridge .

2. ImportJMXBridgeintoOpsManagerbyfollowingtheinstructionsforAddingandImportingProducts .

3. OntheInstallationDashboard,clicktheJMXBridgetile.

TheorangebarontheJMXBridgetileindicatesthattheproductrequiresconfiguration.

Step2:AssignAvailabilityZonesandNetworks1. SelectAssignAZsandNetworks.Thissectionshowstheavailabilityzones(AZs)thatyoucreate whenconfiguringOpsManagerDirector.

2. (vSphereandAmazonWebServicesOnly)SelectanAZunderPlacesingletonjobsin.OpsManagerrunsMetricsjobswithasingleinstanceinthisAZ.

3. (vSphereandAmazonWebServicesOnly)SelectoneormoreAZunderBalanceotherjobsin.OpsManagerbalancesinstancesofMetricsjobswithmorethanoneinstanceacrosstheAZsthatyouspecify.

Note:TousetheFirehoseNozzle,youmustinstallElasticRuntime beforeJMXBridge.Startinginv1.8.7,theJMXBridgeproductenforcesthisinstallorderdependency.

©CopyrightPivotalSoftwareInc,2013-present 4 1.9

Page 5: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

4. SelectaNetworkfromthedrop-downmenu.

5. ClickSave.

Step3:ConfigureJMXProvider1. SelectJMXProvider.

2. EnteranewusernameandpasswordintotheJMXProvidercredentialsusernameandpasswordfields.

Note:JMXBridgeusesthedefaultAssignedNetworkifyoudonotselectadifferentnetwork.

Note:Whenyousavethisform,thefollowingverificationerrordisplaysbecausethePCFsecuritygroupblocksICMP.Youcanignorethiserror.

©CopyrightPivotalSoftwareInc,2013-present 5 1.9

Page 6: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

3. Recordthesecredentials.YouusethesetoconnectJMXclientstotheJMXProvider.

(Optional)Step4:DisableorEnableNATSupport1. SelecttheEnableradiobutton.NATsupportisdisabledbydefault.ThisoptionallowsyoutosettheNATIPasthehostIP.Bydefault,theinternalIP

addressoftheJMXProviderVMissetasthehostIP.

2. EntertheNATIPastheExternalIPaddressintheform 0.0.0.0

3. Ifyouhaveenabledordisabled NATSupport ,clickSave.

Note:ToconnecttotheJMXProviderafterinstall,youmustusethespecifiedIPaddress.TheIPaddressdisplayedinthe Status tabalwaysreflectstheinternalIPaddressoftheJMXProviderVM,nottheexternalIPaddress.

©CopyrightPivotalSoftwareInc,2013-present 6 1.9

Page 7: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

(Optional)Step5:DisableorEnabletheNozzlePrefixBydefault,theNozzleprefix opentsdb.nozzle prependstoFirehose-transmittedmetrics,inordertomaintainbackwardcompatibilitywithpriorversionsofJMXBridge.

1. Ifyoudonotrequirebackwardcompatibility,youcanmakethemetricsmorereadablebyclearingtheEnableNozzlePrefixcheckbox.Thisomitsthe opentsdb.nozzle prefix.

2. Ifyouhaveenabledordisabled EnableNozzlePrefix ,clickSave.Enablingordisablingthisfeaturecausestemporaryoverlapofmetricscomingthroughinbothnamingformats.MetricswiththeformermetricnameformatcontinuetoappearintheMBeantreeuntiltheJMXProviderisrestartedortheMBeanstoreneedsadditionalspacetostorenewmetrics.

(Optional)Step6:DisableorEnableSecurityLogging1. SelecttheEnableSecurityLoggingcheckbox.AccesstotheJMXendpointisloggedtoSTDOUTbydefault.Youcanenablethissecurityloggingin

theJMXBridgetileconfigurationbyselectingthischeckbox,ordisableitdeselectingthischeckbox.Securityloggingisenabledbydefault.

©CopyrightPivotalSoftwareInc,2013-present 7 1.9

Page 8: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

2. Ifyouhaveenabledordisabled SecurityLogging ,clickSave.

(Optional)Step7:ConfigureSSL1. SelecttheEnableSSLcheckbox.EnablingSSLrequiresJMXclientstouseSSLtoconnecttotheJMXProvider.IfSSLisnotenabled,JMXclientscan

connecttotheJMXProviderwithoutSSLcredentials.

Note:RelatedlogoutputismadeavailablebyinitiatingaJMXProviderlogsdownloadfromtheJMXBridgetileconfigurationstatustab,thenfetchingthedownloadfromthelogstab.

©CopyrightPivotalSoftwareInc,2013-present 8 1.9

Page 9: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

IfyouselecttheEnableSSLcheckbox,youmustalsoprovideanSSLcertificateandprivatekey.TherearetwowaystoprovideanSSLcertificateandprivatekey:

Ifyouareusingasignedcertificate,pasteanX.509certificateintheCertificatePEMfieldandaPKCS#1privatekeyinthePrivateKeyfield.IfyouwanttouseSSLbutdonotwanttouseasignedcertificate,youmustperformthefollowingactions:

1. Generateaself-signedcertificateontheserver.2. Importtheself-signedcertificatetoatruststoreontheclient.3. StartjConsole,oranothermonitoringtool,withthetruststore.

Formoreinformation,seeUsingSSLwithaSelf-SignedCertificate.

©CopyrightPivotalSoftwareInc,2013-present 9 1.9

Page 10: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

2. AfterprovidinganSSLcertificateandprivatekey,clickSave.

(Optional)Step8:ConfigureErrandsErrandsarescriptsthatOpsManagerrunstoautomatetasks.Bydefault,OpsManagerrunsthepost-installerrandslistedbelowwhenyoudeployPCFJMXBridge.However,youcanpreventaspecificpost-installerrandfromrunningbydeselectingitscheckboxontheErrandspage.

©CopyrightPivotalSoftwareInc,2013-present 10 1.9

Page 11: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

SelectSmoketestsforJMXBridgetocausetheJMXBridgetoverifythefollowing:

IftheFirehoseNozzleisenabled,thattheNozzleisreceivingmetricsIftheFirehoseNozzleisenabled,thattheproductisnotaslowconsumerIfBOSHMetricsareenabled,thattheproductisreceivingappropriatehealthmetrics

(Optional)Step9:ResourceConfigurationTodisabletheFirehoseNozzleorstopreceivingElasticRuntime (includingDiego)metrics,modifytheinstancecountoftheFirehoseNozzlefrom 1

to 0 .

Step10:ApplyChanges1. NavigatetothePCFOpsManagerInstallationDashboard.

2. InthePendingChangesview,clickApplyChangestoinstallJMXBridge.

Note:Iferrorsoccurduringtheinstallduetosmoketests,refertothetroubleshootingdocumentationformoreinformation.

©CopyrightPivotalSoftwareInc,2013-present 11 1.9

Page 12: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

Afterinstallationcompletes,amessageappearsstatingthatthechangeshavebeenapplied.

Step11:ProvideaJMXProviderIPAddressIfyouwanttoconsumeBOSHsystemmetricswithJMXBridge,followthestepsbelow.

FindtheIPAddressoftheJMXProvider1. ClickReturntoProductDashboard.

2. ClicktheJMXBridgetileandselecttheStatustab.

3. RecordtheIPaddressoftheJMXProvider.

Note:InPCFv2.0andlater,BOSHsystemmetricsareavailablethroughtheLoggregatorFirehose.ThismeansJMXBridgeconsumesthembydefaultthroughitsFirehosenozzleandyoudonotneedtocompletethissection.Ifyoudo,youmayreceiveduplicatedata.Formoreinformation,seethev1.9.5sectionofReleaseNotesandKnownIssues.

Note:Afterinstallation,yourJMXclientconnectstothisIPaddressatport44444usingthecredentialsthatyousupplied.AlsoensurethatTCPport44445isopen.

©CopyrightPivotalSoftwareInc,2013-present 12 1.9

Page 13: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

EntertheJMXProviderIPAddress1. ReturntotheInstallationDashboard.ClicktheOpsManagerDirectortileandselectDirectorConfig.

2. IntheJMXProviderIPAddressfield,entertheIPaddressoftheJMXProvider.ClickSave.

Step12:CompleteInstallation1. InthePendingChangesview,clickApplyChanges.

©CopyrightPivotalSoftwareInc,2013-present 13 1.9

Page 14: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

Whencomplete,amessageappearsstatingthatthechangeshavebeenapplied.

2. ClickReturntoProductDashboard.JMXBridgeisnowinstalledandconfigured.

Afterinstallationandconfiguration,metricsforCloudFoundrycomponentsautomaticallyreporttotheJMXendpoint.

ViewthesourceforthispageinGitHub

©CopyrightPivotalSoftwareInc,2013-present 14 1.9

Page 15: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

UsingJMXBridgePagelastupdated:

JMXBridgeisaJavaManagementExtensions(JMX)toolforPivotalApplicationService.Tohelpyoumonitoryourinstallationandassistintroubleshooting,JMXBridgecollectsandexposessystemdatafromPivotalCloudFoundrycomponentsviaaJMXendpoint.JMXBridgeconsumesplatformmetricdatafromtheFirehoseviathe firehose-jmx-nozzle andsystemhealthmetricsdirectlyfromBOSH.

GuidanceonKeyMetricsofMonitoringInterestJMXBridgereportsallplatformmetricdatabeingtransmittedfromPCFviaBOSHandtheFirehose.NotallplatformmetricsareofequalinterestwhenmonitoringPCF.ReferenceMonitoringPivotalCloudFoundry forrecommendationsofkeyindicators.

Example-VirtualMachineMetricsJMXBridgereportsdataforeachvirtualmachine(VM)inadeployment.UsethesetypesofmetricstomonitorthehealthofyourVirtualMachines.

ThefollowingtableshowsthenameoftheVirtualMachinemetric,whatthemetricrepresents,andthemetrictype(datatype).

METRICNAME DEFINITION METRICTYPE(DATATYPE)

system.cpu.sys AmountofCPUspentinsystemprocesses Gauge(Float)

system.cpu.user AmountofCPUspentinuserprocesses Gauge(Float)

system.cpu.wait AmountofCPUspentinwaitingprocesses Gauge(Float)

system.disk.ephemeral.percent PercentageofephemeraldiskusedontheVM Gauge(Float,0-100)

system.disk.ephemeral.inode.percent Percentageofinodesconsumedbytheephemeraldisk Gauge(Float,0-100)

system.disk.persistent.percent PercentageofpersistentdiskusedontheVM Gauge(Float,0-100)

system.disk.persistent.inode.percent Thepercentageofinodesconsumedbythepersistentdisk Gauge(Float,0-100)

system.disk.system.percent PercentageofsystemdiskusedontheVM Gauge(Float,0-100)

system.healthyIndicateswhetheraVMsystemishealthy.`1`meansthesystemishealthy,and`0`meansthesystemisnothealthy

Gauge(Float,0-1)

system.load.1m Amountofloadthesystemisunder,averagedoveroneminute Gauge(Float)

system.mem.percent PercentageofmemoryusedontheVM Gauge(Float)

system.swap.kb AmountofswapusedontheVMinKB Gauge(Float)

system.swap.percent PercentageofswapusedontheVM Gauge(Float,0-100)

ViewthesourceforthispageinGitHub

©CopyrightPivotalSoftwareInc,2013-present 15 1.9

Page 16: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

UsingSSLwithaSelf-SignedCertificateinJMXBridgePagelastupdated:

SecureSocketLayer(SSL)isastandardprotocolforestablishinganencryptedlinkbetweenaserverandaclient.TocommunicateoverSSL,aclientneedstotrusttheSSLcertificateoftheserver.

ThistopicexplainshowtouseSSLwithaself-signedcertificateinJMXBridge(formerlyOpsMetrics).ThisSSLlayersecurestrafficbetweenJMXBridgeandtheuser,andisseparatefromtheSSLlayerconfiguredbetweenElasticRuntime andtherestoftheOpsManagerenvironment.

TherearetwokindsofSSLcertificates:signedandself-signed.

Signed:ACertificateAuthority(CA)signsthecertificate.ACAisatrustedthirdpartythatverifiesyouridentityandcertificaterequest,thensendsyouadigitallysignedcertificateforyoursecureserver.Clientcomputersautomaticallytrustsignedcertificates.Signedcertificatesarealsocalledtrustedcertificates.

Self-signed:Yourownservergeneratesandsignsthecertificate.Clientsdonotautomaticallytrustself-signedcertificates.TocommunicateoverSSLwithaserverprovidingaself-signedcertificate,aclientmustbeexplicitlyconfiguredtotrustthecertificate.

ThefollowingprocedureconfiguresaJMXuserclienttotrustaself-signedcertificatebyimportingthecertificatetoitstruststore,aninternalkeystore.TouseatrustedcertificatesignedbyaCA,youonlyneedtopastetheCertificateandKeyintothefieldsintheOpsManagerJMXBridgetile,asshowninStep1,Option2,below.

Step1:SupplySSLCertificate

Option1:GenerateSelf-SignedCertificateFollowthestepsbelowtogenerateaself-signedcertificateonyourserver:

1. InPivotalOpsManager,clicktheJMXBridgetile.

2. CheckEnableSSL.

3. ClickGenerateSelf-SignedRSACertificate.

Note:CertificatesgeneratedinElasticRuntimearesignedbytheOperationsManagerCertificateAuthority.Theyarenottechnicallyself-signed,buttheyarereferredtoas‘Self-SignedCertificates’intheOpsManagerGUIandthroughoutthisdocumentation.

©CopyrightPivotalSoftwareInc,2013-present 16 1.9

Page 17: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

4. Enteryoursystemandapplicationdomainsinwildcardformat.Optionally,alsoaddanycustomdomainsinwildcardformat.ClickGenerate.

5. Selectandcopythecertificate.

6. Pastethecertificateintoatextfileandsaveasa .cer file,suchas MY-JMX-BRIDGE.cer .

Option2:UseanExistingSelf-SignedCertificate1. InPivotalOpsManager,clicktheJMXBridgetile.

2. CheckEnableSSL.

3. Pasteyourcertificateandprivatekeyintotheappropriateboxes.ThisisyourX.509certificateandPKCS#1privatekey.

©CopyrightPivotalSoftwareInc,2013-present 17 1.9

Page 18: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

Step2:ImporttheSelf-signedCertificatetoaTruststoreFollowthestepsbelowtoimporttheself-signedcertificatetoyourclient:

1. Copyyourcertificatefile MY-JMX-BRIDGE.cer fromyourservertoyourclient.

2. Navigatetotheclientdirectorywhereyoucopiedthesavedcertificate.

3. Use keytool-import toimportthecertificatewithanaliasof ops-metrics-ssl tothetruststore localhost.truststore :

$keytool-import-aliasops-metrics-ssl-fileMY-JMX-BRIDGE.cer-keystorelocalhost.truststore

If localhost.truststore alreadyexists,apasswordpromptappears.Enterthekeystorepasswordthatyourecordedinapreviousstep.If localhost.truststore doesnotexist,youmustcreateapassword.

4. Verifythedetailsoftheimportedcertificate.

Step3:StartaMonitoringToolwiththeTruststoreAfteryouimporttheself-signedcertificateto localhost.truststore ontheclient,configureyourmonitoringtool,suchasJconsole,tousethetruststore.Youdothisfromacommandline,bystartingyourmonitoringtoolwiththelocationandpasswordofthetruststore.

1. Passinthelocationof localhost.truststore toyourmonitoringtoolwiththe javax.net.ssl.trustStore property,anditspasswordwiththejavax.net.ssl.trustStorePassword property.Forexample,youwouldinvokejConsolewith:

$jconsole-J-Djavax.net.ssl.trustStore=/lib/home/jcert/localhost.truststore-J-Djavax.net.ssl.trustStorePassword=KEYSTORE_PASSWORD

2. IntheRemoteProcessfield,enterthefullyqualifiedhostnameoftheMaximusserver,portnumber 44444 .

©CopyrightPivotalSoftwareInc,2013-present 18 1.9

Page 19: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

3. TocompletetheUsernameandPasswordfields,refertotheCredentialstaboftheJMXBridgetileinPivotalOpsManager.Bydefault,thesecredentialsare admin and admin .

YourmonitoringtoolshouldnowcommunicatewithyourserverthroughtheSSLconnection.

ViewthesourceforthispageinGitHub

©CopyrightPivotalSoftwareInc,2013-present 19 1.9

Page 20: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

JMXBridgeResources

ResourceRequirementsThefollowingtableshowsthedefaultresourceandIPrequirementsforinstallingthetile:

Product Resource Instances CPU RAM Ephemeral Persistent StaticIP DynamicIP

JMXBridge JMXProvider 1 2 4GB 8GB 1GB 1 0

JMXBridge JMXFirehoseNozzle 1 2 4GB 8GB 1GB 1 0

JMXBridge SmokeTests 1 4 4GB 8GB 0 1 0

GuidelinesIfyouanticipatealargevolumeofmetricscomingfromtheFirehose,thenscaleupthenumberoftheJMXFirehoseNozzleinstancesaccordingly.

ViewthesourceforthispageinGitHub

©CopyrightPivotalSoftwareInc,2013-present 20 1.9

Page 21: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

TroubleshootingandUninstallingJMXBridgePagelastupdated:

ThistopicdescribeshowtoresolvecommonissueswiththeJMXBridgeforPivotalCloudFoundry(PCF)tileandhowtouninstallthetileifnecessary.

TroubleshootThefollowingsectionsprovidehelpwithtroubleshootingJMXBridgeforPCF.

MissingMetricsfromPCFInstallationorFirehoseIfyoudonotseeexpectedmetricsfromElasticRuntimeintheJMXprovider,verifythatyouinstalledElasticRuntimebeforeJMXBridge.IfyouinstalledJMXBridgefirst,performthefollowingsteps:

1. SSHintothejmx-firehose-nozzleVM.ForinformationabouthowtousetheBOSHCLItoSSHintoaVM,seeAdvancedTroubleshootingwiththeBOSHCLI .

2. Grantsudoaccesstothemachine:

$sudo-i

3. Restartthe jmx-firehose-nozzle job.

$monitrestartjmx-firehose-nozzle

MissingBOSHMetricsIfyoudonotseeexpectedmetricsfromBOSH,trythefollowingsteps:

1. MakesuretheIPaddressinJMXBridge>Status>JMXProvidermatchesthevalueenteredinOpsManagerDirector>DirectorConfig>JMXProviderIPAddress.

2. IftheaddressesdonotmatchandyouseenoBOSHmetricsinthesystem,contactPivotalSupport forhelp.

ValidatingJMXBridgeMBeansIfyoudonotseemetricsfromJMXBridgeinyourthird-partytoolingintegrationasexpected,firsttrythefollowingstepstoquicklydebugwhetherthereisanissuewiththeJMXBridgeproductoriftheissueiswiththetoolingintegration:

1. VerifyJava6+ isinstalled.

2. Run jconsole :

$jconsole

3. SelectRemoteProcessandentertheIPoftheJMXProviderVMwithport 44444 .

4. FillintheusernameandpasswordfortheJMXProviderthatwasenteredduringinstallationofJMXBridge.

5. ClickConnect.

©CopyrightPivotalSoftwareInc,2013-present 21 1.9

Page 22: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

6. AllowInsecureconnectionifSSLwasnotenabled.

YoucannowviewallMBeansemittedbyJMXBridge.

Note:IfyouhaveenabledSSL,seeUsingSSLwithaSelf-SignedCertificateinJMXBridge.

©CopyrightPivotalSoftwareInc,2013-present 22 1.9

Page 23: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

SetUpPortForwardingforJMX

Ifyouareconnectingtojconsolefromalocationdifferentfromtheinstalllocation(forexample,deployedonAWSorGCP),youhavetosetupportforwardingtoaccesstheMBeans.

1. Setupportforwardingononetabofyourconsoleandkeepitopen:

[email protected]

2. Start jconsole inanewtabandsetupthe socksProxyPort totheforwardedport:

jconsole-J-DsocksProxyHost=localhost-J-DsocksProxyPort=7777

3. Navigate jconsole asnormal.

SmokeTestsIferrorsoccurwhenthesmoketestsrun,youcanfindtheerrorsintheChangeLogfortheinstallation.Somecommonfailuresarelistedbelow.

Error internalMetricsAreSent() Fails

Cause TheJMXProviderdidnotreceiveinternalhealthmetricsfromtheJMXFirehoseNozzle.

Solution RestarttheJMXFirehoseNozzleVMandcheckthelogstoverifyitisrunningcorrectly.

Error receivingFirehoseMetrics() Fails

Cause TheJMXFirehoseNozzleisnotreceivingmetricsfromtheFirehose.

SolutionRestarttheJMXFirehoseNozzleVMandcheckthelogstoverifyitisconnectedtotheFirehose.IfyouseealotofreconnectattemptsinthelogsthenyoulikelyneedtoscaleupthenumberofJMXFirehoseNozzleinstancesintheResourceConfigtab.

UninstallTouninstalltheJMXBridgeforPCFtile,seeDeletingaProduct .

ViewthesourceforthispageinGitHub

©CopyrightPivotalSoftwareInc,2013-present 23 1.9

Page 24: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

ApplicationSecurityGroupsPCFapplicationsdonotinteractdirectlywiththePCFJMXBridgetile.Therefore,youdonotneedtocreateApplicationSecurityGroups(ASGs)tointeractwiththebridgefromanexternalapplication.

ViewthesourceforthispageinGitHub

©CopyrightPivotalSoftwareInc,2013-present 24 1.9

Page 25: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

ReleaseNotesandKnownIssues

v1.9.9ReleaseDate:February12,2018

ReleaseNotesStemcellforv1.9.9isnowv3468

Networkconnectionswilltimeoutandreconnectifnonewmetricsarereceivedfor30seconds.

KnownissuesThenewflowofBOSHsystemmetrics(seereleasenotesforv1.9.5formoredetails)cannotbedisabled.Therefore,ifyouarecurrentlyusingthePCFJMXBridgetiletoconsumethem,youmayreceiveduplicatedata.Topreventthis,deleteJMXProviderIPAddressinDirectorConfigofyourOpsManagerDirectortile.

DeletingtheIPaddressmeansthatBOSHsystemmetricswillnolongerbesenttoJMXBridgeusingthedirectconnectionfromtheBOSHDirectortotheJMXProvider.AstheseBOSHsystemmetricsarenowavailableinJMXBridgebydefaultthroughitsFirehosenozzle,breakingthepriordirectconnectionbydeletingtheJMXProviderIPaddresspreventstheduplicationofBOSHmetricsforJMXBridgeconsumers.

v1.9.5ReleaseDate:November15,2017

ReleaseNotesStemcellforv1.9.5isnowv3445

PCFnowforwardsBOSHhealthmetricsgeneratedforallVMsinadeploymenttotheLoggregatorFirehosebydefault.Formoreinformationaboutthisfeatureanditsimplementation,seetheBOSHSystemMetricsForwardersectionintheOverviewoftheLoggregatorSystem .

ThenewflowofBOSHsystemmetricscannotbedisabled.Therefore,ifyouarecurrentlyusingthePCFJMXBridgetileortheBOSHHMForwardertoconsumethem,youmayreceiveduplicatedata.Topreventthis,youcandothefollowing:

StopusingPCFJMXBridgetoconsumeBOSHsystemmetricsoutsideoftheFirehose.SeeKnownIssues.UninstalltheBOSHHMForwarder.

BecauseBOSHSystemmetricsnowcomefromtheFirehose,theirnamespacesaredifferentinPCFJMXBridge.ForanexplanationofhowmetricnamesdifferbetweenPCF2.0andearlierversions,seethefollowingtable.

PCFVersion Explanation

1.12andearlier

ExampleMetric:system.healthy

Description:TheBOSHDirectordeliversthemetricname.Themetricisnestedinthetreestructurebydeploymentname,VMname,VMinstancenumber,andattributesforthatVMinstance.Thesub-nodeofVMinstancenumberisalwaysnamednull.ReferenceImage:

©CopyrightPivotalSoftwareInc,2013-present 25 1.9

Page 26: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

2.0 ExampleMetric:bosh-system-metrics-forwarder.system.healthy

Description:TheFirehosedeliversthemetricname.ThetreeshowstheVMGUIDinsteadoftheVMinstancenumberandthesub-nodeisalwaysempty.ThisnamespacingaffectsallpreviousBOSHhealthmetrics.ReferenceImage:

©CopyrightPivotalSoftwareInc,2013-present 26 1.9

Page 27: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

KnownissuesThenewflowofBOSHsystemmetricscannotbedisabled.Therefore,ifyouarecurrentlyusingthePCFJMXBridgetiletoconsumethem,youmayreceiveduplicatedata.Topreventthis,deleteJMXProviderIPAddressinDirectorConfigofyourOpsManagerDirectortile.

DeletingtheIPaddressmeansthatBOSHsystemmetricswillnolongerbesenttoJMXBridgeusingthedirectconnectionfromtheBOSHDirectortotheJMXProvider.AstheseBOSHsystemmetricsarenowavailableinJMXBridgebydefaultthroughitsFirehosenozzle,breakingthepriordirectconnectionbydeletingtheJMXProviderIPaddresspreventstheduplicationofBOSHmetricsforJMXBridgeconsumers.

v1.9.3

©CopyrightPivotalSoftwareInc,2013-present 27 1.9

Page 28: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

ReleaseDate:October13,2017

ReleaseNotesMaintenanceupdateofthefollowingproductdependencies:

OpenJDKnowv1.8.0.144Golangnowv1.9.1

ToensurecompatibilitywithupcomingversionsofPCF,JMXBridgewillnolongerbestaticallyallocatingIPs.However,onceinstalled,theallocatedIPwillremainthesamethroughfutureupgradesprovidedthetilehasnotbeenuninstalled.AnytoolingthatcurrentlyreliesonobtainingtheallocatedIPthroughtheOpsManager’s GET /api/v0/deployed/products/:product_guid/static_ips endpointcannowusetheGET /api/v0/deployed/products/:product_guid/status endpointtoretrievethisinformation.

Stemcellforv1.9.3remainsv3363

KnownissuesNone

v1.9.2ReleaseDate:August11,2017

ReleaseNotesMaintenanceupdateofthefollowingproductdependencies:

OpenJDKnowv1.8.0.141Golangnowv1.8.3GRPC-ALLnowv1.5.0Guavanowv23.0BouncyCastle-ALLnowv1.57netty-tcnative-boringssl-staticnow2.0.5.Final

JMXBridgereleasenowusesSHA-256checksumsforimprovedsecurity

Updates requires_product_versions inthemetadatatobelessstrictinordertoallowJMXBridgev1.9toalsoworkwiththefuturePCFv1.12

Stemcellforv1.9.2remainsv3363

KnownissuesNone

v1.9.1ReleaseDate:June15,2017

MajorFeaturesJMXBridgev1.9containstwomajorarchitecturechangesinsupportofoverallsecurityimprovements.Nobreakingchangesinthemetricsoutputorformathavebeenintroducedinthiseffort,soconsumptionofbothBOSHandplatformmetricsshouldcontinuetobeseamlessforend-usersuponupgradingfromJMXBridgev1.8toJMXBridgev1.9.

TheformerFirehoseconsumer, opentsdb-firehose-nozzle ,hasbeenupdatedtoanew,gRPCnozzle jmx-firehose-nozzleBOSHmetricdatanowflowsfromOpsManagertoJMXBridgeviathenew jmx-bosh-plugin .TherelatedinstallationfieldinOpsManagerhasbeenrenamedfrom“MetricsIP”to“JMXProviderIPAddress”,anditautomaticallyconfiguresthenewplugin.OpsManagerwillmigrateIPentriesin

©CopyrightPivotalSoftwareInc,2013-present 28 1.9

Page 29: JMX Bridge for PCF Documentation - Pivotal SoftwareBy default, the Nozzle prefix opentsdb.nozzle prepends to Firehose-transmitted metrics, in order to maintain backward compatibility

the“MetricsIP”fieldtothe“JMXProviderIPAddress”fielduponupgradefromPCFv1.10toPCFv1.11

Newoptionalconfigurationto“DisableorEnabletheNozzlePrefix”;enabledbydefault

ThebehavioroftheJMXBridgetoprepend opentsdb.nozzle ontoallnon-BOSHFirehose-transmittedmetricsoutputtedwasanartifactofthepriorFirehosenozzle.Asthiswasreplaced,thisartifactwasalsoeliminated.InordertoavoidbreakingchangestoconsumersofpriorJMXBridgeversions,anewoptionalfeature,enabledbydefault,wascreatedtocontinuetoprependthis opentsdb.nozzle valueontometricspreviouslyoutputtedwithitIfyoudonotrequirebackwardcompatibility,youcanmakethemetricsmorereadablebyclearingtheEnableNozzlePrefixcheckbox.Thisomitsthe opentsdb.nozzle prefix,transmittingthemetricvaluesintheirpcf-emittednameformat.Thisbackwards-compatibilityfeaturemaybedeprecatedinfutureversionsofJMXBridge.Itisstronglyrecommendedthatconsumernamemappingsbeupdatedwhenpossible,andthisfeaturetoprepend opentsdb.nozzle thendisabled.

SupportforFirehosetransmissionoftaggedmetrics

TheFirehoseiscapableoftransmittingmetricswithtags.Whilefewemittedplatformmetricsarecurrentlyusingthiscapability,forthosethatare,JMXBridgev1.9willpassthetagdataalonginthesameformatasseenwhenusing cf nozzleAnexampleofataggedmetricis DopplerServer.listeners.receivedEnvelopes .WhenviewingthismetricviaJMXBridgeor cf nozzle thetransmittedformatis DopplerServer.listeners.receivedEnvelopes[event_type=ContainerMetric,protocol=grpc]

ReleaseNotesJMXBridgev1.9.1istargetedforPCFv1.11.x

JMXBridgeusersupgradingtoPCFv1.11,mustalsoupgradetoJMXBridgev1.9duetoaarchitecture-drivendependencybetweenOpsManagerv1.11andJMXBridgev1.9

Stemcellforv1.9.1isv3363

KnownissuesInstallingJMXBridgev1.9onOpsManagerv1.11.0impactsBOSHmetricsoutputasfollows:

ID informationdoesnotcomethroughJMXBridgeIP Property comesthroughasablankstringinsteadof null inJMXBridge

IfyouwanttoconsumeBOSHmetricsfromPCFv1.11usingJMXBridgev1.9,youmustuseOpsManagerv1.11.3orlater.

PastMinorv1.8.xReleasenotesforv1.8.xreleasescanbefoundhere .

ViewthesourceforthispageinGitHub

©CopyrightPivotalSoftwareInc,2013-present 29 1.9