![Page 1: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/1.jpg)
LVE Stats 2 Training
![Page 2: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/2.jpg)
• Modular, extendable and efficient
architecture
o Can monitor anything
o Customize what is monitored
Use RES mem instead of LVE PMEM
o In memory data analysis - better
performance
o Easy to extend - suspend, notify, etc…
Why LVE Stats v2?
![Page 3: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/3.jpg)
• Better precision
o Only 1 month of data vs 2 months for v1
o Yet, 1 minute intervals vs 1 hour intervals
Why LVE Stats v2?
![Page 4: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/4.jpg)
• Better charts (SVG)
• Include DB Governor statistics
Why LVE Stats v2?
![Page 5: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/5.jpg)
• Faults for CPU/IOo Not ‘faultable’ resources, just throttled
o We count it as a ‘fault’, once per minute, if limit reached
o Easier to identify/see when user hits the limit
Why LVE Stats v2?
![Page 6: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/6.jpg)
• Collects snapshots
o URL
o Process
o Query
Why LVE Stats v2?
![Page 7: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/7.jpg)
• Backwards compatible by default
o By default lve-stats-2.x runs in backward
compatibility mode.
lveinfo and lvechart run as before
Will be changed in ~12 months
• Make new mode default by setting mode=v2 in
/etc/sysconfig/lvestats2
o Or use -m v1|v2 option in lveinfo.
Backwards compatible
![Page 8: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/8.jpg)
• lveinfo/lvechart are backward
compatible
o Old options should work the same
• /var/lve/info populated as before
o Every 5 seconds, used by control panel to
display ‘live’ information
What stayed the same?
![Page 9: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/9.jpg)
• --dbgov -- allows to manipulate DB Governor stats
• -c/--csv [FILE] -- saves data in CSV format
• -j/--json -- json output
• --time-unit -- averages time in output based on the
time unit
• --servers-info for multi-server install, lists servers &
LVE version
• -m v1/v2 -- v1 display output in backward
compatible mode, v2 - new mode
• --blank-value [val] -- substitute instead of
unsupported limit, default -
lveinfo
![Page 10: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/10.jpg)
• -u/--user -- show stats for particular MySQL user
• -l/--limit -- max number of entries in output
• -o/--order-by -- order output by
con/cpu/read/write
• -b/--format -- columns to show
• --csv/--json -- display data in csv or json formats
• --from/--to/--period -- same as lveinfo
lveinfo --dbgov
![Page 11: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/11.jpg)
• Includes DB Governor data on CPU/IO charts
• --format {svg,png} -- allows to use SVG or PNG format, default SVG
What’s new: lvechart
![Page 12: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/12.jpg)
• Top like utility for LVEo will replace lvetop
• Powers new UI
• Includes DB Governor data
• Supports --json only for now (more later, including live/on screen)o Used in the next version UI for cPanel/DA/Plesk
• Takes data from:o /var/lve/cloudlinux_top.json file
o Updated by lvestast v2 every 5 seconds
What’s new: cloudlinux-top
![Page 13: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/13.jpg)
• - hide-mysql - don’t show DB Governor related data
• - u/--username - show data only for specific user,
substring matching
• - d/--domain - show data only for specific domain,
substring matching
• - m/--max - max number of entries to show, default 25
cloudlinux-top options:
![Page 14: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/14.jpg)
• dbgovchart - show DBGovernor charts
• lve-create-db -- creates or re-creates database
for LVE Stats v2.0
• lve-read-snapshot -- reads snapshots for end
user. Can produce data in json, or produce
‘summary’ stats for a particular period
More commands...
![Page 15: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/15.jpg)
• Not on CL5
Storing History: Supported Databases
![Page 16: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/16.jpg)
• Automated upgrade from lve-stats 1.x
once released to production channel
• Automated settings migrations
• Data migrated automatically within first
24 hourso Throttled, so disks are not overloaded
o No support for migrating stats on centralized
storage
Upgrading: Migrating data
![Page 17: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/17.jpg)
• /var/lve/lvestats2.db -- SQLite db
with all the data
• /var/lve/info -- ‘real time’ usage to
show in control panels
• /var/lve/cloudlinux_top.json -- data
for cloudlinux-top command
Important files
![Page 18: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/18.jpg)
• /etc/sysconfig/lvestats2 -- main config file
• /etc/sysconfig/lvestats.config/ -- config
files for Pluginso HistoryCleaner.cfg -- how often to remove
data from db
o SnapshotSaver.cfg -- snapshot settings (how
many, how often)
o StatsNotifier.cfg -- Notifications settings
Config files
![Page 19: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/19.jpg)
• db_type = sqlite
• plugins=/usr/share/lve-stats/plugins
• keep_history_days=30
• server_id=7269a505-3
• mode=v1
...
/etc/sysconfig/lvestats2
![Page 20: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/20.jpg)
• NOTIFY_ADMIN=N
• NOTIFY_RESELLER=N
• NOTIFY_CUSTOMER=N
• NOTIFY_INCLUDE_RESELLER_CUSTOMER=N
• NOTIFY_CPU=Y
• NOTIFY_IO=Y
• NOTIFY_IOPS=Y
• NOTIFY_MEMORY=Y
• NOTIFY_EP=Y
• NOTIFY_NPROC=Y
• NOTIFY_MIN_FAULTS_ADMIN=1
• NOTIFY_MIN_FAULTS_USER=1
• NOTIFY_INTERVAL_ADMIN=12h
• NOTIFY_INTERVAL_USER=12h
StatsNotifer.cfg – Notify when limited
![Page 21: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/21.jpg)
Notification message example
![Page 22: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/22.jpg)
• period_between_incidents=300
• snapshots_per_minute=2
• max_snapshots_per_incident=10
SnapshotSaver.cfg
![Page 23: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/23.jpg)
• Easy way to extend lvestats
• Simple Architecture
• Python based
• http://docs.cloudlinux.com/index.html?creating_a
_plugin.html
Plugins
![Page 24: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/24.jpg)
• Collectors
• Analyzers
• Persistors
• Notifiers
collectors
analyzers
PersistorsNotifiers
Plugin Types
![Page 25: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/25.jpg)
• Generic / always included
• Optional
o LVEDestroyer
o ResMEMCollector
• Custom
More about plugins
![Page 26: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/26.jpg)
• period -- how often to execute, default 5
seconds
• order -- order in the execution sequence,
to decide which plugin goes next
• Any other settings can be set using config
files
Plugin Properties
![Page 27: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/27.jpg)
Name Period Order Default
LVECollector 5 1000 Y
CPUInfoCollector 5 2000 Y
LVEUsernamesCollector 3600 3000 Y
LVEUsageAnalyzer 5 4000 Y
LveUsageAggregator 60 5000 Y
DBGovSaver 5 6000 Y
FileSaver 5 7000 Y
CloudLinuxTopFileSaver 60 8000 Y
Plugins
![Page 28: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/28.jpg)
DBSaver 60 9000 Y
DbUsernamesSaver 3600 10000 Y
DBSaverX60 3600 11000 Y
SnapshotSaver 30 12000 Y
StatsNotifier varied 13000 Y
HistoryCleaner 3600 14000 Y
ResMEMCollector 30 1500 N
LVEDestroyer 5 - N
Plugins
Name Period Order Default
![Page 29: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/29.jpg)
• To enable plugin, copy or link it to /usr/share/lve-
stats/plugins director
• For example to enable ResMEMCollector plugin, do
ln -s
/opt/alt/python27/lib/python2.7/site-packages/lvestat
s/plugins/generic/res_mem_collector.py
/usr/share/lve-stats/plugins/
Adding plugins
![Page 30: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/30.jpg)
# FSize_watcher_collector.py# Example plugin for monitoring file size.# Part 1. Collector import osfrom lvestats.core.plugin import LveStatsPlugin # Key nameCOLLECTOR_KEY = 'FSizeWatcher_fsize'COLLECTOR_KEY_FILENAME = 'FSizeWatcher_fname' class FSize_watcher_collector (LveStatsPlugin):
# this plugin should be first in chain order = 0 # File to monitoring file_to_monitoring = None def __init__(self): pass # Sets configuration to plugin def set_config(self, config): self.file_to_monitoring = config.get('file_to_monitoring', None) pass # Work method def execute(self, lve_data): try: # if monitoring file absent, do nothing if self.file_to_monitoring is None or not os.path.exists(self.file_to_monitoring): return # Get file size stat_info = os.stat(self.file_to_monitoring) fsize = stat_info.st_size # Place file name and file size to server data dictionary lve_data[COLLECTOR_KEY_FILENAME] = self.file_to_monitoring lve_data[COLLECTOR_KEY] = fsize except (OSError, IOError): # file absent or any other error - remove size from dictionary del lve_data[COLLECTOR_KEY]
Example Plugin
![Page 31: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/31.jpg)
• LVE_VERSION LVE version. • stats -- Dictionary, that contains lve id’s as keys and LVEStat class objects as values. Every LVEStat
object contains values of usages and limits taken from /proc/lve/list file for every LVE Id. Dictionary keys – integer lve id, including 0 for “default” LVE. This dictionary is updated on each iteration of lvestats-server (every 5 seconds by default).LVEStat – is a standard server class, it can be imported with the command from lvestats.core.lvestat import LVEStat.The class is described in the file /opt/alt/python27/lib/python2.7/site-packages/lvestats/core/lvestat.py.Here you can find the whole list of data fields and their functions.
• old_stats -- stats content from the previous iteration. Before the first iteration – empty dictionary.• totalHz -- When LVE_VERSION is 4, real CPU frequency in Hz multiplied by number of cores. When
LVE_VERSION > 4, CPU speed is in conventional units and equals to 1000000000 * cores (1 GHz per core).
• procs -- Quantity of CPU/cores.• lve_usages -- Contains accumulated LVE statistics for each 5-seconds interval in current minute. Cleared
each minute.• lve_usage -- Contains aggregated LVE Statistics for “previous” minute to store to database. Overwritten
each minute.
Plugin Data Convention
![Page 32: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/32.jpg)
• Python 2.7 (www.python.org): No comment;
• Sqlite (sqlite.org), MySQL (www.mysql.com), MariaDB (
mariadb.org), PostgreSQL (www.postgresql.org)
• Sqlalchemy (sqlalchemy.org): the Python SQL Toolkit and
Object Relational Mapper;
• Alembic (alembic.zzzcomputing.com): database migration tool
for usage with the SQLAlchemy;
• Numpy (numpy.org): package for scientific computing with
Python;
• Sentry (sentry.io): real-time error tracking;
• Prospector (prospector.landscape.io): python static analysis
tool.
Stack technology
![Page 33: How lve stats2 works for you and your customers](https://reader036.vdocument.in/reader036/viewer/2022062412/587772451a28ab5b568b5f59/html5/thumbnails/33.jpg)
Questions?
Twitter.com/CloudLinuxOS Visit CloudLinux.com