infiniflux feature perf comp_v1
TRANSCRIPT
Performance and Feature Comparison on InfiniFlux, MySQL, MongoDB, Splunk, and Elasticsearch
www.infiniflux.com
Feature - Query Language
2
Product Query Language
InfiniFlux Easy to process statistics as it supports standard SQL
MySQL Easy to process statistics as it supports standard SQL
Elasticsearch Require to enter conditional clauses of SQL queries in the REST API and JSON format
MongoDB Difficult to use it since it requires to enter conditional clauses in the “Find” method
Splunk Easy to process statistics as it supports search processing language (SPL)
Conclusion
Easy and efficient to process unstructured big data with Splunk or Elasticsearch and structured big data with Infiniflux which supports SQL.
For statistical processing, difficult to use Elasticsearch and MongoDB due to complicated query.
Feature - Time Series Query
3
Product Time Series Query
InfiniFluxUse “Duration” clause and hidden column, “_ARRIVAL_TIME”. Able to process high-speed query since input data are partitioned based on input time.
MySQL Not supported
Elasticsearch Not supported
MongoDB Not supported
Splunk Possible to use “Duration” clause for searching
Conclusion For time series query, InfiniFlux and Splunk are easy and efficient to use. In terms of search performance, InfiniFlux is the best for searching time series data.
Feature - Full Text Search
4
Product Full Text Search
InfiniFluxAble to search by using “Search” operator.Able to search data at a high-speed by using the inverted index.
MySQLUse “Like” operator.It may not able to use index based on search patterns, and as a result, slow down in speed.
Elasticsearch Supported
MongoDB Supported
Splunk Supported
ConclusionInfiniFlux is the only available DBMS for full text search at a high-speed against structured data.
Feature - Extended Data Type
5
Product Extended Data Type
InfiniFlux Support IPv6 and IPv4 types and operators (contains and contained)
MySQL Not supported
Elastic search Not supported
MongoDB Not supported
Splunk Support functions such as “cidrmatch”
Conclusion InfiniFlux is the only available DBMS for supporting network data type and related functions.
6
FieldTime of log
creationSource IP
Sourceport
DestinationIP
Destinationport
Protocol type
Log text Status Code Data Size
Field Name arrivaltime srcip srcport dstip dstport protocol eventlog eventcode eventsize
Field Type datetime ipv4 integer ipv4 integer shortvarchar(1024)
short long
Evaluate data input and analyze performance of each product by 100,000,000 records (13GB size) on basic hardware environment
Hardware Specifications
- CentOS 6.6
- Intel(R) Core(TM) i7-4790
CPU @ 3.60GHz (4 core)
- 32GB Memory
- SATA DISK
Test Targets
- InfiniFlux 2.0
- MySQL 5.2
- Splunk 6.2.3
- Elasticsearch 1.5.3
- MongoDB 3.0.3
[DATA]
Performance
7
4334
13848
698
1624
393
0 2000 4000 6000 8000 10000 12000 14000 16000
Elasticsearch
MySQL
Splunk
MongoDB
INFINIFLUX
DATA LOADING TIME (sec)
Performance
ConclusionInfiniFlux ables to load 100,000,000 data records only within 393 seconds, the fastest time ever among the competitors.
8
3
1
85
208
4
0 50 100 150 200 250
Elasticsearch
MySQL
Splunk
MongoDB
INFINIFLUX
COMPLEX SEARCH (sec)
Performance
ConclusionInfiniFlux only takes 4 seconds to perform composite operations.MySQL takes a second to search, which is the fastest, however, it takes a long time to load data.
9
4337
13849
783
1832
394
Elasticsearch
MySQL
Splunk
MongoDB
INFINIFLUX
OVERALL RESULT (sec)
Performance
Conclusion Overall, InfiniFlux ables to conduct 100,000,000 records of data input and composite operations only within 394 seconds.
10
Performance
20.4
17.52
21.6
42.11
4.1
0 5 10 15 20 25 30 35 40 45
Elasticsearch
MySQL
Splunk
MongoDB
INFINIFLUX
STORAGE SIZE (GB)
Conclusion InfiniFlux able to compress 13GB of data including indices into 4.1GB showing compression rate of 77%.
11
Performance
InfiniFlux MongoDB Splunk MySQL Elasticsearch
During time (sec) 389 (00:06:29) 1624 (00:10:16) 698 (00:11:38) 13848 (03:50:48) 4334 (01:12:14)
Insert csv size (GB) 13G
Data size (GB) 4.1G 42.1157G 8.6G 17.52G 20.4G
Compression rate (%) 76.92%Decompressed
(223.97%)33.95%
Decompressed(130.77%)
Decompressed(156.92%)
Memory usage (%) 29.22 73.75 40.78 87.59 89.82
Memory usage (GB) 9.0756 22.9073 12.667 27.20 27.8965
Data search
Text search (2.6M) 2s 213s (00:03:33) 424s (00:07:04) 31s 2s
IP search (2.66M) 1s 212s (00:03:32) 40s 1s 3s
Time search Less than a second 211s (00:03:31) 8s 1s 2s
Statistic
Sum 25s 217s (00:03:37) 435s (00:07:15) 35s 1s
Average 25s 219s (00:03:39) 436s (00:07:16) 46s 4s
Count 17s 218s (00:03:38) 382s (00:06:22) 45s 3s
Complex query 4s 208s 85s 1s 3s
OVERALL RESULT 394s 1832s 783s 13849s 4337s
*For more information on the result, please visit: http://www.infiniflux.com/performance
