c-treeedge iot database - faircom€¦ · node.js rest api multimodel design c-treeedge consists of...

Developer Guide

c-treeEDGE IoT Database

All Rights Reserved ii www.faircom.com

Contents

1. Introduction to c-treeEDGE ............................................................................... 1

2. c-treeEDGE Developer Quick Start ................................................................... 5

2.1 Starting and Stopping c-treeEDGE ...................................................................... 6

3. Supported Plug-ins ............................................................................................ 8

3.1 Plug-ins ................................................................................................................ 8

3.2 HTTP Plug-in ....................................................................................................... 9

3.3 ThingWorx AlwaysOn Plug-in............................................................................. 11

3.4 ThingWorx MQTT Plug-in .................................................................................. 12

3.5 MQTT Plug-in .................................................................................................... 12

3.6 Using Node-RED with MQTT ............................................................................. 14

3.7 Using Node-RED with c-treeEDGE .................................................................... 14

3.8 Using Node.js with c-treeEDGE ......................................................................... 15

3.9 REST API Plug-in .............................................................................................. 15

3.10 OPC UA Plug-in ................................................................................................. 16

4. Tutorials ........................................................................................................... 18

4.1 Tutorial for the ThingWorx AlwaysOn Plug-in ..................................................... 19

4.2 Tutorial for the ThingWorx MQTT Extension ...................................................... 28

4.3 Tutorial for using Node.js with c-treeEDGE ........................................................ 30

4.4 Tutorial for Using Node-RED with the c-treeEDGE MQTT Broker ...................... 34

4.5 Tutorial for Node-RED........................................................................................ 39

4.6 Tutorial for Using Python with c-treeEDGE MQTT ............................................. 48

4.7 c-treeEDGE REST API ...................................................................................... 51

4.8 Tutorial for OPC UA Installation and Configuration ............................................ 52

4.9 Configuring the MQTT Plug-in............................................................................ 55

5. Automatic Timestamps, Aggregation, and Purging ...................................... 57

6. Replication with c-treeEDGE ........................................................................... 58

7. Administrative Tools........................................................................................ 63

8. FairCom License Agreement for c-treeEDGE ................................................ 65

Introduction to c-treeEDGE

All Rights Reserved iii www.faircom.com

9. Index ................................................................................................................. 67

All Rights Reserved 1 www.faircom.com

1. Introduction to c-treeEDGE

Note: For installation instructions, see the c-treeEDGE Installation Guide (https://docs.faircom.com/doc/c-treeEDGE_Installation/).

The c-treeEDGE® IoT Database allows developers to place a fast, transaction-controlled

database on the "edge," near their IoT data. c-treeEDGE places storage close to the sensors,

instead of in the cloud or an on-premises server, so it provides autonomy in situations when

communication is slow or unavailable. In addition, c-treeEDGE can store unfiltered sensor data

with no bandwidth concerns. It allows you to put intelligence in your IoT projects for advanced,

real-time decision processing.

Related Topics:

c-treeEDGE Administration Guide

MQTT

Node.js

REST API

Multimodel Design

c-treeEDGE consists of a collection of connectors (for use with ThingWorx, Node-RED, MQTT,

Node.js, etc.) and a powerful database engine that can be located on the "edge."

This database engine, the c-treeEDGE Server, is based on FairCom's c-treeACE Server. The

unique c-treeACE "multimodel" design allows both navigational and SQL (relational) access to

the same data at the same time. This design allows c-treeEDGE to handle volumes of raw sensor

data at high-speeds and offer sophisticated features such as immediately-consistent ACID

transactions. The relational interfaces provide SQL access to this same data using

industry-standard APIs such as ODBC, JDBC, ADO.NET, etc.

https://docs.faircom.com/doc/c-treeEDGE_Installation/



Streamlined to Suit Your Needs

Every app is different. That means every app has different requirements. Let's face it:

applications running on the edge often have limited resources. They don't need any unnecessary

features. One app may need SQL access for compatibility with third-party applications. Another

may want to use FairCom's famously high-performance non-SQL interfaces for pure speed of

data handling. One app may need to add automatic timestamps and aggregate data; another may

not need these features.

Plug-In Architecture

c-treeEDGE is designed with your unique needs in mind. Many of its features

are placed in plug-in modules which you enable only if you require these exact

features. FairCom does not shoehorn everyone into a "one-size-fits-all" design

that forces you to install features you do not need. c-treeEDGE allows you to

install only the features that are needed with your application. You get

maximum performance from your edge devices with only the minimum features

to deploy and support.

This new plug-in architecture is an important addition to c-treeEDGE and the core c-treeACE

database technology it’s based upon. Not only are the provided c-treeEDGE connectors

(ThingWorx, Node-RED, MQTT, and REST) implemented as plug-ins, but other important

features of c-treeEDGE are as well, such as the http server, aggregation modules, time stamps,

etc. Many future additions will take advantage of this extensible plug-in architecture framework.

Lightweight Yet Powerful

c-treeEDGE is lightweight enough to run on small IoT edge devices yet powerful enough to host

data from thousands of sensors. This allows you to decide where your data is stored and how

much of it (if any) is sent to the cloud or other machines in the enterprise. You can perform

queries and make decisions locally on the edge without the latency of first getting data to the

cloud or machines elsewhere in your environment. c-treeEDGE allows you to save all sensor data

for a site on location and send only aggregate data upstream. If things go wrong, your data

remains available on the edge device waiting for debugging and analysis.



The image below shows the c-treeEDGE architecture and the available connectors.

Using c-treeEDGE as a Persistent Data Store

c-treeEDGE uses a microservices architecture,

so its API is structured as a highly modular

collection of loosely coupled, fine-grained

services and lightweight protocols. Combined

with the tiny footprint of c-treeEDGE (tunable to

under 40MB), this allows it to be used as a

persistent data store in IoT applications residing

on devices and gateways near the "edge" of the

network.

The following connectors are provided to allow

integration into your IoT projects:

ThingWorx "thing" supporting a REST API,

MQTT, and the AlwaysOn protocol

Node-RED Node

Node.js for JavaScript integration

MQTT Broker

REST API supporting both navigational and

relational (SQL) access

OPC UA for industry-standard automation

control



Unlimited Options

c-treeEDGE is a two-part solution:

A connector that allows it to integrate with your application (ThingWorx, Node-RED, and a

REST API are supported)

The c-treeEDGE Server (the database server), is installed on an edge device or gateway.

The connector connects your application to the c-treeEDGE Server. Depending on your host

platform, the extensive set of relational and navigational APIs shown below may be available.

API Availability on All Platforms

Windows x86/x64. Linux x86/x64 Raspbian (ARM) c-treeEDGE packages include all the APIs

shown in the above diagram. Tutorials and instructions are located in c-treeEdge.<platform>\sdk.

Contact FairCom http://www.faircom.com/company#ContactUs if you are interested in any other

APIs or operating systems.

Replication

FairCom offers an advanced replication solution (packaged separately), which can maintain an

up-to-date copy of your data in real-time. The files created by the c-treeEDGE Server are

"replication aware" to simplify the addition of replication. You can learn more about replication in

the chapter titled Replication with c-treeEDGE

(https://docs.faircom.com/doc/c-treeEDGE_DevelopmentGuide/Replicationwithc-treeEDGE.htm)

and in the Replication Manager User Guide (https://docs.faircom.com/doc/ops_manager).

Contact FairCom http://www.faircom.com/company#ContactUs for more information.

http://www.faircom.com/company#ContactUs

https://docs.faircom.com/doc/c-treeEDGE_DevelopmentGuide/Replicationwithc-treeEDGE.htm

https://docs.faircom.com/doc/ops_manager

http://www.faircom.com/company#ContactUs


2. c-treeEDGE Developer Quick Start

This chapter covers installation and tutorials to get you immediately up-and-running with

c-treeEDGE in your projects.

The c-treeEDGE environment requires three components:

1. c-treeEDGE. See the c-treeEDGE Installation Guide (https://docs.faircom.com/doc/c-treeEDGE_Installation/).

2. Plug-in - A plug-in extends c-treeEDGE. Many plug-ins exist. You can use plug-ins supplied by FairCom and you can create your own plug-ins. You may add a plug-in to integrate c-treeEDGE with another system such as ThingWorx AlwaysOn. You may add a new communications protocol, such as MQTT, OPC UA, and REST. You may add a new transformation capability, such as processing Cognex image data. You may add a new processing capability, such as running a machine learning algorithm.

The Supported Plug-ins (page 8) chapter provides information and tutorials for connecting

to a variety of platforms.

3. Tools - Administrative tools are available in GUI and Command-Line form. See the Administrative Tools (page 63) chapter.

The server is packaged together with a set of plug-ins for popular environments (ThingWorx,

Node-RED, MQTT, REST, etc.).

Tutorials

This document contains a set of tutorials to help you get going in a hurry:

Tutorial for ThingWorx AlwaysOn Plug-in (page 19)

Tutorial for the ThingWorx MQTT Extension (page 28)

Tutorial for Using Node.js with c-treeEDGE (page 30)

Tutorial for Using Node-RED with the c-treeEDGE MQTT Broker (page 34)

Tutorial for Using Python with c-treeEDGE MQTT (page 48)

c-treeEDGE REST API (page 51)

Tutorial for OPC UA Installation and Configuration (page 52)

See also:

FairCom MQTT Reference Guide (https://docs.faircom.com/doc/mqtt/)

FairCom REST API Developer's Guide (https://docs.faircom.com/doc/rest_api/)

https://docs.faircom.com/doc/c-treeEDGE_Installation/

https://docs.faircom.com/doc/mqtt/

https://docs.faircom.com/doc/rest_api/

c-treeEDGE Developer Quick Start


2.1 Starting and Stopping c-treeEDGE

There are many options to start and stop c-treeEDGE depending on your platform.

Windows

Starting on Windows

Windows Service (recommended) - Select and start the service via the Windows service

administrator.

As a Console or ToolTray Application - Simply double-click the executable in the

c-treeEDGE-Installation-Folder\server directory.

From the command line - Start the application like any other Windows application by typing

ctreeedge.exe.

Stopping on Windows

By default, c-treeEDGE requires an administrator password to initiate a server shutdown. The

default (and permanent) administrator account is ADMIN with an initial default password of

ADMIN.

Warning: For enhanced security, FairCom strongly recommends changing the default ADMIN password immediately after installation. Refer to the c-treeEDGE Administrator's Guide (https://docs.faircom.com/doc/ctedge_admin/) for details.

The Windows Service does not require the administrator password as it is assumed c-treeEDGE

is running under administrative privileges.

c-treeEDGE Monitor - The monitor tool for Windows provides an option to stop the server.

Windows Service - If you installed the c-treeEDGE database to run as a Windows service,

select and stop the service via the Windows service administrator.

ToolTray - If you ran c-treeEDGE as a ToolTray application, right-click on its ToolTray icon

and select the appropriate shutdown option.

Console - If you ran c-treeEDGE in a console window, close the console application window

via the menu or the standard Windows "X" in the upper corner.

ctstop - This scriptable command line based utility can be used to stop the c-treeEDGE

database.

ctadmn - This interactive command-line administrator utility provides an option to stop the

server.

Note: When connecting via a network, be aware that the TCP/IP port should be allowed by any local firewalls. If you cannot connect with remote clients, you may need to specifically allow the c-treeEDGE database port.

https://docs.faircom.com/doc/ctedge_admin/

c-treeEDGE Developer Quick Start


Unix • Linux • Mac OSX

Starting c-treeEDGE

Unix, Linux, and Mac OS X users can use the included and preconfigured start scripts.

./startedge

This script is provided as a "C" shell script and will work in most Unix shell environments. You are

free to modify and tailor it to suit your local requirements.

From a command line - Start the server process as a background process:

>ctreeedge&

Stopping c-treeEDGE

By default, the c-treeEDGE database requires an administrator password to initiate a server

shutdown. The default (and permanent) administrator account is ADMIN with an initial default

password of ADMIN.

Warning: For enhanced security, FairCom strongly recommends changing the default ADMIN password immediately after installation. Refer to the c-treeEDGE Administrator's Guide (https://docs.faircom.com/doc/ctedge_admin/) for details.

c-treeEDGE Monitor - The monitor tool provides an option to stop the server.

./stopedge - A Unix shell script preconfigured to stop the c-treeEDGE database.

ctstop - This scriptable command line based utility can be used to stop c-treeEDGE.

ctadmn - This interactive command line based administrator utility provides an option to stop

the server.

Note: When connecting via a network, be aware that the TCP/IP port should be allowed by any local firewalls. If you cannot connect with remote clients, you may need to specifically allow the c-treeEDGE database port.



3. Supported Plug-ins

c-treeEDGE integrates with many products and communication protocols, including the MQTT

message queue protocol, ThingWorx, OPC UA protocol, etc.

This chapter provides documentation for the supported tools, protocols, and APIs for accessing

the c-treeEDGE. If you don’t see support for a tool, protocol or API that is needed for your

environment, please contact your nearest FairCom office. c-treeEDGE is designed to be

extended. FairCom can deliver new general use plug-ins as part of the c-treeEDGE. FairCom

Professional Services can also build custom plug-ins for your unique needs. You can build your

own plug-ins and FairCom Support is available to help.

This chapter also provides information explaining how the c-treeEDGE database uses Plug-ins

(page 8) to provide extendable, modular support for for these platforms and it explains how to

configure the HTTP Plug-in (page 9) for secure communication.

Each section in this chapter includes a tutorial for information on how to use the tool or framework.

3.1 Plug-ins

The c-treeEDGE database ships with many plug-ins, which extend its capabilities. They provide

the database with the ability to communicate over HTTP, MQTT, OPC UA, REST, ThingWorx

AlwaysOn, etc.

Each plug-in is independently enabled and configured with a plug-in based architecture. They are

not enabled by default to maximize security. (Plug-ins are secure, but they increase the attack

surface by opening extra network ports and listening across more communication protocols.)

The ctsrvr.cfg configuration file contains a list of plug-ins that run when the database starts. This

file is located in c-treeEDGE-Installation-Folder\server\config.

The example below shows the plug-in section of the ctsrvr.cfg configuration file on a Linux server:

; Plugins

PLUGIN cthttpd;./web/libcthttpd.so

;PLUGIN ctagent;./agent/libctagent.so

;PLUGIN ctthingworx;./thingworx/libctthingworx.so

;PLUGIN ctopc;./opc/libctopc.so

;PLUGIN ctaggregation;./aggregation/libcttimestamp.so

In this example, the first plug-in is cthttpd. This is the HTTP plug-in that enables the IIoT Hub’s

HTTP protocol, web services, and MQTT services. Following the keyword PLUGIN is the name of

the folder containing the plug-in’s configuration file. The configuration file must have the same

name as the plug-in’s folder but with the .json extension. In the example above, the name of the

plug-in folder is cthttpd and this folder contains a configuration file named cthttpd.json.

Supported Plug-ins


All plug-in folders must be located in the c-treeEDGE Server working directory (for example

c-treeEDGE-Installation-Folder\server). You will see the plug-in sub-folders in this location (for

example .\web, .\opc, etc.).

A semicolon must follow the name of the plug-in’s folder.

Following the semicolon is the relative file system path to the plug-in’s executable. The path is

relative to the folder that contains the plug-in.

In the example above, ./web/libcthttpd.so is the relative path to the plug-in’s executable, which is

named libcthttpd.so.

On Microsoft Windows, a plug-in is a Dynamic Linked Library with the extension of .dll. On Linux,

a plug-in is a shared object library with the extension of .so. On MacOS, a plug-in is a dynamic

library with an extension of .dylib.

To disable a plug-in, put a semicolon (;) before the keyword PLUGIN. To enable a plug-in,

remove the semicolon.

Contact FairCom support to request new types of general-use plug-ins. FairCom Professional

services can build custom plug-ins for you. You can also build your own plug-ins with help from

FairCom support.

3.2 HTTP Plug-in

c-treeEDGE includes a web server plug-in that allows HTTP access for the REST API and

browser-based tools. See Configuring the Browser-Based Tools

(https://docs.faircom.com/doc/browser_tools/ConfiguringtheBrowser-BasedTools.htm) in the

Browser-Based Tools (https://docs.faircom.com/doc/browser_tools/) guide.

This HTTP server is pre-configured to listen for HTTP connections on port 8081. If you have a

port conflict, you can change it to any unused port.

Alternatively, and recommended, is to enable secure (HTTPS) connections (see below).

To change the port used by the HTTP server, edit the cthttpd.json file located in the

c-treeEDGE-Installation-Folder\server\config folder:

{

"listening_https_port": 8443,

"ssl_certificate": "./web/fccert.pem",

"document_root": "./web/apps",

"mqtt_enabled": true,

"mqtt_listening_port": 1883,

"mqtt_persistence_enabled": true,

"mqtt_log_enabled": false,

"applications": [

"mqtt;ctMQTT.dll",

"ctree;ctREST.dll"

]

}

https://docs.faircom.com/doc/browser_tools/ConfiguringtheBrowser-BasedTools.htm

https://docs.faircom.com/doc/browser_tools/

Supported Plug-ins


"listening_http_port": 8081 - Sets the port number used by the HTTP server for the

REST API and browser-based tools. 8081 is the default; you can change it to any available port.

(with the exception of the database engine's already two configured ports).

A list of ports is provided in Download and Installation.

Complete Configuration Format

Many additional options can be configured including secure TLS configurations (recommended).

Contact FairCom if you have specific requirements not covered with these options.

{

"access_control_list": "-0.0.0.0/0,+192.168/16",

"auth_domain": "mydomain.com",

"cgi_interpreter": "#!/path/to/php-cgi.exe",

"cgi_pattern": "**.cgi$|**.pl$|**.php$",

"dav_auth_file": "PUT",


"enable_directory_listing": "yes",

"extra_mime_type": ".cpp=plain/text,.java=plain/text",

"global_auth_file": "./web/passFile",

"hide_files_patterns": "secret.txt|even_more_secret.txt",

"hexdump_file": "./web/hexDumpFile",

"index_files": "index.html,index.htm,index.shtml,index.cgi,index.php,index.lp",

"listening_http_port": 8080,


"ssi_pattern": "**.shtml$|**.shtm$",


"url_rewrites": "**.doc$=/path/to/cgi-bin/handle_doc.cgi",

"web_thread_count": 10,





"mqtt_conn_inactive_timeout": 60,

"mqtt_conn_request_timeout": 20,

"mqtt_max_dbconn_count": 30,

"application": [

"mqtt;ctMQTT.dll",

"ctree;ctREST.dll",

"AceMonitor;ctMonitor.dll",

"SQLExplorer;ctSQLExplorer.dll",

"ISAMExplorer;ctISAMExplorer.dll",

"OpsManager;ctMemphis.dll"

],

"linked_ace_server": "FAIRCOMS@localhost"

}


3.3 ThingWorx AlwaysOn Plug-in

The c-treeEDGE ThingWorx

AlwaysOn Plug-in connect c-tree to

the ThingWorx IoT platform, which

allows developers to model complex

applications and dashboards.

c-treeEDGE provides full SQL and

navigational access to your data

persisted on the edge with very little

change to your ThingWorx project.

c-treeEDGE can be added to your project with a simple drag-and-drop operation.

As opposed to other ThingWorx persistence solutions (e.g., JDBC, ADO, SQL), which connect to

databases in the cloud, c-treeEDGE persists data on the edge as well as in the cloud.

With c-treeEDGE integration, it is easy to create the model in the ThingWorx platform based on

existing data from c-treeEDGE. No matter how your data is persisted on the edge (through

c-treeDB Navigational APIs, SQL APIs, REST, MQTT, or OPC UA, etc.), this data is automatically

updated in the platform with the desired frequency, from near real-time to a predefined frequency

(e.g., one value per hour).

The c-treeEDGE AlwaysOn Plug-in is easy and intuitive. You can quickly add c-treeEDGE to

existing ThingWorx applications to locally persist data in the factory and automatically send it to

ThingWorx.

This feature is designed for developers of ThingWorx applications. It requires minimum

experience with ThingWorx Foundation and databases. The plug-in provides an easy way to map

edge-persisted data to a the properties of a "thing."

c-treeEDGE collects data from MQTT and OPC UA at high speed from local devices. Its

ThingWorx AlwaysOn connector uploads these newly inserted records directly into ThingWorx. It

uses the TimeStamp field built into each record to determine which records to upload into

ThingWorx. Integrations with c-treeEDGE can also update and delete records, but the

c-treeEDGE Always On Connector only loads the latest inserts into ThingWorx.

c-treeEDGE also provides a ThingWorx MQTT extension, as described in the c-treeEDGE

Development Guide (https://docs.faircom.com/doc/c-treeEDGE_DevelopmentGuide/). That

extension is useful if you are already using MQTT in your application and do not need the

enhanced integration of the AlwaysOn protocol.

Let's get started - You will find a guide to setup and installation in this chapter in the section

titled ThingWorx AlwaysOn Plug-in - Installation and Configuration (page 19).

https://docs.faircom.com/doc/c-treeEDGE_DevelopmentGuide/

Supported Plug-ins


3.4 ThingWorx MQTT Plug-in

The c-treeEDGE database can

integrate with ThingWorx using an

MQTT Broker.

Note: The ThingWorx MQTT Plug-in is deprecated in favor of the ThingWorx AlwaysOn Plug-in (page

11).

The application can push information

to an MQTT broker and ThingWorx can subscribe to the broker to receive those updates.

c-treeEDGE has an embedded MQTT broker, which can persist data locally on the edge and

forward the same messages to the remote broker. In this way, c-treeEDGE is able to “proxy” the

MQTT communication and persist the data with no impact on the application:

This integration makes it easy to create the model in the ThingWorx platform based on existing

data from c-treeEDGE. No matter how your data is persisted on the edge (through c-treeDB

Navigational APIs, SQL APIs, REST, MQTT, or OPC UA, etc.), this data is automatically updated

in the platform with the desired frequency, from near real-time to a predefined frequency (e.g.,

one value per hour).

The c-treeEDGE extension in ThingWorx Marketplace

(https://marketplace.ptc.com/apps/206336/c-treeedge-iot-database-extension) uses the

MQTT Broker to provide integration with ThingWorx.

Let's get started - A tutorial with documentation is available in the software package in the

sdk\ThingWorx directory. For setup and installation, see ThingWorx MQTT Extension (page 28).

3.5 MQTT Plug-in

The c-treeEDGE database includes

an MQTT Plug-in that transforms the

hub into an MQTT Broker with the

ability to cache MQTT messages in

relational tables for easy integration

with other systems.

This chapter is a brief MQTT

reference to get you started quickly.

For a complete reference, see the book titled MQTT Reference Guide

(https://docs.faircom.com/doc/mqtt/).

For information about enabling and configuring the MQTT plug-in, see Configuring the MQTT

Plug-in (page 55).

For a tutorial about configuring messages and persistence topics: Tutorial for MQTT:

Configuring a Persistence Topic

(https://docs.faircom.com/doc/mqtt/TutorialforMQTT_ConfiguringaPersistenceTopic.ht

m).

https://marketplace.ptc.com/apps/206336/c-treeedge-iot-database-extension


https://docs.faircom.com/doc/mqtt/TutorialforMQTT_ConfiguringaPersistenceTopic.htm


Supported Plug-ins


MQTT, or Message Queuing Telemetry Transport, is a lightweight protocol allowing small pieces

of data to be transferred from machine to machine across networks. MQTT works well even on

low bandwidth, high latency connections, which makes it ideal for the Internet of Things (IoT).

MQTT communication consists of a publisher, a broker, and a subscriber:

A subscriber connects to a broker and registers for a particular topic (a UTF-8 string used by

the broker to filter messages for each connected client).

A publisher connects to a broker and sends a message tagged with a particular topic.

The broker receives the message and sends it to any subscribers who have subscribed to

that topic.

Any number of publishers can be sending messages on one or more topics. Any number of

subscribers can be listening for one or more topics. All messages for a particular topic will be sent

to all clients subscribed for that topic regardless of where they were sent from. The broker,

publisher, and subscriber can be on the same machine or different machines.

The c-treeEDGE MQTT Broker is a generic broker for connecting c-tree as a persistent data store

under an MQTT application. It can be placed on smaller devices and gateways typically found

near the sensors. It is used to persist MQTT message contents on the edge. By locating

c-treeEDGE on the edge, MQTT message contents can be persisted near the sensors where

service is unaffected by the latency of the cloud.

Any of the JSON fields contained in a specified topic can be written automatically to a specified

database table. This table is simultaneously available via a c-treeACE SQL server allowing ISAM

and SQL connections to the persisted fields. Data continues to flow through the c-treeEDGE

Broker normally, going from publisher to subscriber while, at the same time, being stored in a

database. Configure as many tables as you want each one collecting fields from different topics.

This allows full SQL access to some or all fields of data moving through the c-treeEDGE MQTT

Broker with very little change to standard MQTT setups.

MQTT uses System Topics to control its operation. These topics provide a variety of metadata

about the operation of the queue as well as the location of a configuration file. This mechanism

can be used to determine which topics will be persisted.

For more MQTT information, see mqtt.org (http://mqtt.org/).

Let's get started:

You will find important information about MQTT configuration in Configuring the MQTT Plug-in

(page 55).

For an MQTT tutorial explaining messages and persistence topics, see Tutorial for MQTT:

Configuring a Persistence Topic

(https://docs.faircom.com/doc/mqtt/TutorialforMQTT_ConfiguringaPersistenceTopic.htm).

For using MQTT with Node-RED, see the section titled Node-RED Using the c-treeEDGE MQTT

Broker (page 34).

Additional information for Python developers using c-treeEDGE is provided in Python Using

c-treeEDGE MQTT (page 48).

Note: For reference information about the supported MQTT operations, see the FairCom MQTT Reference Guide (https://docs.faircom.com/doc/mqtt/) in the on-line documentation.

http://mqtt.org/



Supported Plug-ins


3.6 Using Node-RED with MQTT

Node-RED is a browser-based,

drag-and-drop editor that greatly

simplifies the creation of applications

by describing flow between hardware

devices, APIs, and online services. It

includes a palette of nodes, which can

be used by simply dragging them into

your application. The palette of nodes

can be easily extended to add new

capabilities. Node-RED is built on Node.js. Its lightweight and efficient event-driven, non-blocking

model makes it ideal to run at the edge of the network on low-cost hardware, such as the

Raspberry Pi.

FairCom c-treeEDGE provides an MQTT broker for use in Node-RED projects. If you are using

MQTT in your project, this is a simple way to add persistence on the edge. c-treeEDGE also

includes a generic node for connecting c-tree as a persistent data store in a Node-RED

application. Once installed, c-treeEDGE is represented in the node palette so you can quickly add

persistence to your Node-RED application.

When using MQTT, simply add the c-treeEDGE MQTT Broker to your application. c-treeEDGE

appears in your palette and you can quickly add persistence to your application.

Let's get started - A tutorial and guide to setup and installation is available in the section titled

Node-RED Using the c-treeEDGE MQTT Broker (page 34).

For more about support for MQTT provided by FairCom, see FairCom MQTT Reference Guide

(https://docs.faircom.com/doc/mqtt/) in the on-line documentation.

3.7 Using Node-RED with c-treeEDGE

FairCom provides a node for

Node-RED, called ctree. The official

name in the code is

node-red-contrib-ctree. In

Node-RED, this node is visible as the

following icon:

This node makes it easy to use Node-RED to read, insert, update, and delete data in the

c-treeEDGE database. It communicates with c-treeEDGE using its REST plug-in.


Supported Plug-ins


The c-treeEDGE Node.js node module for Node-RED can be installed providing a storage node

icon in your Node-RED palette. Drag this storage node into your flows and quickly add

persistence to your application.

Let's get started - A tutorial with documentation is available in the software package in the

Driver\restapi.nodeRED directory. In addition, you will find a guide to setup and installation is

available in the section titled Node.js Client Library for c-treeEDGE (page 30).

For more about the FairCom support for Node.js, see FairCom Node.js Support

(https://docs.faircom.com/doc/node_js/) in the on-line documentation.

3.8 Using Node.js with c-treeEDGE

The c-treeEDGE database provides a

client library for Node.js. This makes it

easy to build web applications and

web services that automate and

process data in the database.

Node.js is an application server used

to create web applications and web

services. It executes JavaScript code.

The c-treeEDGE database client library for Node.js is a rich API that provides full access to data

stored in the hub.

There are also several MQTT libraries that run in Node.js. You can use them to automate and

persist data in the c-treeEDGE database.

For more about the FairCom support for Node.js, see FairCom Node.js Support


Let's get started - A tutorial is available in the section titled Tutorial for Using Node.js with

c-treeEDGE (page 30).

3.9 REST API Plug-in

The c-treeEDGE database provides a

REST plug-in that allows you to

access data and resources in the

database.

The reference guide for the FairCom

REST API is located in the FairCom

REST API Developer's Guide

(https://docs.faircom.com/doc/rest_api/).

https://docs.faircom.com/doc/node_js/



Supported Plug-ins


REST Troubleshooting Tips

Check the REST server port. Most often it will be 8443, but could be 8080 if your server is not set

up for SSL encryption. You can test this by using one of the following URLs in your browser:

No SSL: http://localhost:8080/ctree/api/v1/openapi

With SSL: https://localhost:8443/ctree/api/v1/openapi

Note: The REST port is configured with the cthttpd.json file in the config folder using the

listening_http_port or listening_https_port keyword, as described in HTTP Plug-In

(page 9). If your port is not 8443, you will need to change the URLs in this tutorial accordingly.

Let's get started - A guide to setup and installation is available in the section titled c-treeEDGE

REST API (page 51).

For more about the FairCom REST API, see the FairCom REST API Developer's Guide

(https://docs.faircom.com/doc/rest_api/) in the on-line documentation.

3.10 OPC UA Plug-in

The c-treeEDGE database provides a plug-in that connects to an OPC server, which gathers data

from devices that communicate using the OPC UA protocol.

The OPC (Open Platform Communications) interoperability standard is a machine-to-machine

communication protocol for industrial automation. OPC is the world’s most popular standard for

open automation data connectivity. The OPC Foundation is responsible for the development and

maintenance of this standard (https://opcfoundation.org/).

OPC is a series of specifications developed by industry vendors, end-users and software

developers. These specifications define the interface between clients and servers, as well as

servers and servers, including access to real-time data, monitoring of alarms and events, access

to historical data and other applications.

OPC UA (OPC Unified Architecture) is the main successor to the "classic" OPC introduced in

1996. OPC UA (IEC 62541 published between 2010 and 2012) combines all the separate OPC


Supported Plug-ins


protocols into one specification, which simplifies development. Data exchange with OPC UA is

secure and reliable.

OPC UA is platform-independent so it is easily incorporated into Windows, Linux, Mac, Android,

and other platforms. This is significant for the manufacturing industry where machines and

systems are often running on different platforms.

Architecture

OPC is implemented using a server/client architecture:

The OPC server converts the hardware communication protocol used by a Programmable

Logic Controller (PLC) into the OPC protocol.

The OPC client connects to the hardware and uses the OPC server to get data from—or

send commands to—the hardware.

The plug-in is configured in a JSON file with information on how to connect to an OPC server.

This information includes OPC server URL, namespace, nodes, and frequency.

When the plug-in successfully connects to the OPC server, the c-treeEDGE database

automatically caches data sent to the OPC server. All cached data in the IIoT Hub is available for

delivery to other systems and for local processing for real-time decision making.

Support for Other OPC Standards

c-treeEDGE supports the "open62541" specification, however the OPC client function calls are in

a ctOPCClient.dll, which is a thin layer on top of the OPC client. If you already have your own

certified OPC solution, you can easily replace this layer with your own DLL implementing a short

list of functions. Contact FairCom if you have an interest in this support.

Let's get started - A guide to setup and installation is available in the section titled OPC UA

Installation and Configuration (page 52).


4. Tutorials

c-treeEDGE supports the ThingWorx platform, the Node-RED development environment, and ad

hoc development using MQTT, Node.js, Python, or just about any environment you can think of.

This section provides links to tutorials and additional documentation for the supported tools,

protocols, and APIs for accessing the c-treeEDGE Server. If you don’t see support for a tool,

protocol or API that is needed for your environment, please contact your nearest FairCom office.

c-treeEDGE is very flexible and it might be trivial for us to add this support for your project.


4.1 Tutorial for the ThingWorx AlwaysOn Plug-in

This tutorial demonstrates the basics of installing and using the c-treeEDGE ThingWorx

AlwaysOn Plug-in.

This plug-in built uses the ThingWorx AlwaysOn protocol (https://developer.thingworx.com/sdks

https://developer.thingworx.com/sdks).

c-treeEDGE requires the creation and configuration of a c-treeEDGE "thing" in ThingWorx. Refer

to the ThingWorx website for more information about using ThinkWorx.

You will need to install and start the c-treeEDGE, as described in Download and Installation (https://docs.faircom.com/doc/c-treeEDGE_DevelopmentGuide/DownloadandInstallation.htm) in the c-treeEDGE Developer's Guide.

The following PLUGIN keyword has been added to the c-treeEDGE ctsrvr.cfg file, which is located

in the c-tree server/config folder:

PLUGIN ctthingworx;./thingworx/libctthingworx.so

If you are using the ThingWorx AlwaysOn plug-in, be sure there is no semicolon (;) at the

beginning of that line. The semicolon is present by default and should be removed.

The above example is for Linux. On Windows, the library is called ctthingworx.dll.

The following directory has been added to c-treeEDGE under the server folder:

thingworx

libctthingworx.so - plug-in dynamic library (Linux only)

ctThingWorx.dll - plug-in dynamic library (Windows only)

fccert.pem - self-signed certificate for SSL

ctreeEDGE_entities.xml - required platform entities for the plug-in

c-treeEDGE also adds a new configuration file to the server/config folder: ctthingworx.json. This

file is used to configure the ThingWorx AlwaysOn plug-in.

Entity Setup

This guide assumes an existing ThingWorx platform is available.

ThingWorx entity requirements - The c-treeEDGE ThingWorx plug-in requires some entities

that this step will make available in the ThingWorx platform:

Data shapes: ctEDGE, ctDatabase, ctTable, and ctField

https://developer.thingworx.com/sdks

https://docs.faircom.com/doc/c-treeEDGE_DevelopmentGuide/DownloadandInstallation.htm

Tutorials


Thing Template: CtreeEdgeTemplate

Thing creator: CtreeEdgeThing

These entities are available in ctreeEDGE_entities.xml file provided in the c-treeEDGE package

(in the server/config folder). Please import them into ThingWorx using the option From File and

the type of Entities:

After importing, you should see the following entities in your ThingWorx platform:

c-treeEDGE Plug-in Configuration File

server/config/ctthingworx.json is a JSON setting file in the following format:

{

"thingworxHost": "ThingWorx host",

"thingworxPort": 80,

"thingworxAppkey": "ThingWorx app key",

"thingworxSSLCertificate": "./thingworx/fccert.pem",

"thingworxCACertificate": "CA Certificate",

"thingworxCertType": 2,

"thingworxThings": [

{

"thingName": "ctEdgeThing2",

"thingCreator": "ctEdgeThing"

}

]

Tutorials


}

Where:

thingworxHost: <string> Server host name or IP address where the ThingWorx platform is

running.

thingworxPort: <numeric> Server port where the ThingWorx platform is listening. This

corresponds to the Tomcat “HTTP/1.1 Connector Port.”

thingworxAppkey: <string> Application key generated in the ThingWorx platform used for

authentication.

See ThingWorx Help Application Keys

(http://support.ptc.com/help/thingworx_hc/thingworx_8_hc/en/index.html#page/ThingWorx/He

lp/Security/ApplicationKeys/ApplicationKeys.html).

thingworxSSLCertificate: <string> SSL certificate file name.

thingworxCACertificate: <string> SSL certificate authority name.

thingworxCertType: <string> SSL certificate type.

thingworxThings: <array> An array of "things" in the platform to be created to map to

c-treeEDGE.

• thingName: <string> Name you want assigned to the new “thing” which will be created in

the ThingWorx platform.

• thingCreator: <string> Name of the existing thing creator in the ThingWorx platform. This

will be used as a model to create the new “thing". This came from the

ctreeEDGE_entities.xml file you imported into ThingWorx above.

Example File {

"thingworxHost": "THINGWORX9000",

"thingworxPort": 80,

"thingworxAppkey": "66412e88-8495-4dd7-b409-7d94507e836e",

"thingworxSSLCertificate": "./thingworx/fccert.pem",

"thingworxThings": [

{

"thingName": "ctEdgeThing2",

"thingCreator": "CtreeEdgeThing"

}

]

}

http://support.ptc.com/help/thingworx_hc/thingworx_8_hc/en/index.html#page/ThingWorx/Help/Security/ApplicationKeys/ApplicationKeys.html

http://support.ptc.com/help/thingworx_hc/thingworx_8_hc/en/index.html#page/ThingWorx/Help/Security/ApplicationKeys/ApplicationKeys.html

Tutorials


Loading the c-treeEDGE Thing

When c-treeEDGE starts with the active plug-in set accordingly, it connects to the ThingWorx

platform and creates a new "thing" with the <thingName> set in the ctthingworx.json file based

on the existing <thingCreator>. If there is already a "thing" with this same name in ThingWorx,

it tries to use it. If the template does not match, the plug-in initialization will fail. Check the

c-treeEDGE server\data\CTSTATUS.FCS file, and the various Thingworx log files (in the

C:\ThingworxStorage\logs folder) for error messages.

Note that when the c-treeEDGE server is not running, the following icon (indicated by the mouse

pointer) will show that the "thing" is disconnected.

While disconnected, the following message will be presented:

Tutorials


When the c-treeEDGE is connected, the following icon will show that the "thing" is connected.

While connected, the following message will be presented:

Note that, in certain situations, neither of these icons will be presented. This indicates a problem,

and the log files should be consulted.

Loading c-treeEDGE Schema

When the c-treeEDGE server starts, just after connecting to the ThingWorx platform, the plug-in

loops through all the databases, tables, and fields and populates them in a property in the “thing.”

This property name is ctEdge:

Tutorials


You can navigate and visualize all the databases, tables and fields on this property:

If any schema change happens in c-treeEDGE after the schema has been loaded into the

ThingWorx platform, it will not be automatically reloaded. So, a service is provided in the “thing” to

reload this information. Go to </> Services -> GetCTEdge -> <Execute button>:

Mapping Properties

While navigating through the ctEdge property when getting into the field level, we can set how the

field value from c-treeEDGE persistence should be mapped to a property in the current “thing” at

the platform. Execute the following steps:

1. Click the Edit button (green “pencil in a circle” icon, pictured below) for the ctEdge property and navigate to find the field to be mapped.

Tutorials


2. Click the Edit button for the field record:

3. Check the Sync checkbox. This tells the c-treeEDGE plug-in that ThingWorx should be notified whenever a new record is added to the mapped c-treeEDGE table.

4. If you want to name the new property to be mapped in the platform, enter it in the PropertyName field. If you leave it empty, a new name will be automatically generated. Names should not start with numerical digits or contain periods.

5. If you want to set a minimum interval to wait before new value changes are updated, you can specify it in the Interval field. The default value of 0 means that every value change on the persistence table in c-treeEDGE will be pushed to the platform. If that frequency of updates is too high for the platform, you can use this setting to limit the frequency of updates to the platform. For example, to update no more than once every 10 seconds, set this field to 10:

6. Click the Set button and Done in all the other windows.

7. Click the Save button on the ctEdge edit window.

8. Now run/execute the BindCTEdge service, to create the new properties and get them syncing with c-treeEDGE.

Tutorials


9. Close the ctEdgeThing2 using the “X” button on the left, and then and re-open it. Note that the “minimize” button on the right is NOT adequate.

10. Switch to the Properties and Alerts tab and the new Properties will be there.

With these steps you are able to set all the mapping between the desired field data persisted in

c-treeEDGE and its “thing” in the ThingWorx platform.

Type Mapping

The table below shows the data type mapping between the c-tree fields in c-treeEDGE and

ThingWorx properties:

c-tree type ThingWorx type

CT_BOOL TW_BOOLEAN

CT_CHAR

CT_CHARU

CT_INT2

CT_INT2U

CT_INT4

CT_INT4U

CT_INT8

CT_INT8U

TW_INTEGER

CT_SFLOAT

CT_DFLOAT

CT_EFLOAT

CT_SQLBCD

CT_MONEY

TW_NUMBER

CT_DATE

CT_TIME_MS

CT_TIME

CT_TIMES

CT_TIMES_MS

TW_DATETIME

CT_FSTRING

CT_FPSTRING

CT_F4STRING

CT_STRING

CT_PSTRING

CT_F2STRING

CT_ARRAY

CT_2STRING

CT_4STRING

CT_FUNICODE

CT_F2UNICODE

CT_UNICODE

CT_2UNICODE

TW_STRING

Tutorials


c-tree type ThingWorx type

CT_JSON TW_JSON

Connecting

To enable the mapping between the data persisted in c-treeEDGE and the “thing” in the platform,

execute the server BindCTEdge. Go to </> Services -> BindCTEdge -> <Execute button>:

It loops through all the mapping set in the previous section and execute the following steps:

1. Validate that the mapping property name is set, if it is not, populate it automatically with the following format: <database>_<table>_field.

2. Validate that the mapping property exists in the “thing.” If it is missing, create it automatically.

3. Start a “Record Update Callback” on the persisted table to process the insert of new records and update the property in the platform.

After these steps, no matter how the new data records are inserted in the persisted table, the last

value is supposed to be populated automatically in the mapped property in the platform.

Note 1: ThingWorx may be slow refreshing existing "things" and after executing these steps new properties may not display. Close the "thing" and reopen it to see your newly-created properties.

Note 2: ThingWorx platform error messages are redirected to the c-treeEDGE database CTSTATUS.FCS log. We'll ignore several of these logged messages for now.

Tutorials


Disconnecting

To disable mapping between data persisted in c-treeEDGE and the “thing” in the platform,

execute the server UnbindCTEdge. Go to </> Services -> UnbindCTEdge -> <Execute button>

(you may need to scroll down to the bottom to see the UnbindCTEdge service):

4.2 Tutorial for the ThingWorx MQTT Extension

This tutorial demonstrates the basics of using the c-treeEDGE ThingWorx Extension. This

extension uses MQTT to connect to the ThingWorx platform.

Note: The c-treeEDGE ThingWorx Extension has been deprecated in favor of the ThingWorx AlwaysOn Plug-in (page 19).

The c-treeEDGE database also provides a ThingWorx plug-in using the AlwaysOn protocol, as

described inThingWorx AlwaysOn Plug-in (page 11). The AlwaysOn connector provides

enhanced integration with ThingWorx.

This section explains how to install c-treeEDGE on the ThingWorx platform. ThingWorx allows

enterprise-scale applications to be modeled by dragging and dropping "things."

c-treeEDGE requires creation and configuration of a c-treeEDGE "thing" in ThingWorx. Please

refer to the ThingWorx website and the c-treeEDGE Thing download in the ThingWorx

Marketplace for more information.

You will need to install and start the c-treeEDGE database, as described in Download and Installation.

To install the c-treeEDGE database MQTT extension to ThingWorx, follow these steps:

1. From a web browser, launch ThingWorx.

2. Log into ThingWorx as an administrator.

3. Go to Import/Export > Import.

4. Click Choose File and select ctreeEdgeThingworxExtension.zip. (located in c-treeEDGE.<platform>\sdk\Thingworx)

Tutorials


5. Click Import.

Note: If an "Import Successful" message does not display, contact your ThingWorx System Administrator.

6. Click Yes to refresh Composer after importing the final extension.

7. Confirm that the Extension has been imported properly. Check the Application Log for potential problems.

Configuration

Create a new "thing" and select CtreeEdgeMQTTThingTemplate as a Thing Template.

Once the c-treeEDGE Database "thing" is created, it must be configured to connect to a

c-treeEDGE Database instance.

Within the ThingWorx Composer, open up the c-treeEDGE Database "thing"

Navigate to the Configuration section

Under the connection settings, enter valid information for each of the connection parameters

Click the Add button to set the first property to be persisted on the Edge and enter the following

information:

"Thing" to have property persisted on the Edge.

Property name to be persisted on the Edge.

MQTT Topic mapped to the property.

JSON tag used to send/get the property value.

Number of threads used on the c-treeEDGE for persistence. This number should be

increased only in case the volume of data is expected to be high.

Database where the persistent table will be created/maintained. Enter:

ctreeSQL

Table name for the persistence data.

If c-treeEDGE should automatically add a timestamp to the persisted information.

Configure the MQTT System Topic. It must match the one subscribed to by the c-treeEDGE.

Enter the following information:

ServerName for the MQTT broker.

ServerPort for the MQTT Port number.

UserName for authentication.

UserPassword for authentication.

UseSLL in case the MQTT connection should be SSL.

SystemTopic the c-treeEDGE is subscribing to.

Save the "thing."

Tutorials


4.3 Tutorial for using Node.js with c-treeEDGE

This tutorial demonstrates connecting to Node.js using the c-treeEDGE Node.js module. For

more about the FairCom support for Node.js, see FairCom Node.js Support


Installation

This Node.js module is included in your product download. Look for the <your-API-folder> folder

in the <FairCom-Folder>\drivers folder.

1. Using the command line, create and then change into a new folder.

2. Issue the following command, adapting it to the location of your <Your-API-Folder> folder:

npm install C:path\to\<Your-API-Folder>

Note: You may be required to update to the latest npm version. (For example, many Raspberry Pi Raspbian devices come pre-installed with an old version.) npm is easily updated as followed: > sudo npm install -g npm@latest

Usage

The module currently contains 5 methods:

createTable

deleteTable

createIndex

createRecord

query

Basic usage of the module starts with the following two lines:

var ctree = require("ctree");

ctree.setAuth("localhost", 8443, "admin", "ADMIN", true);

Then you can call any of the 5 methods.

Create Table createTable(db, table, fields, callback)

db: your existing database name

table: your not-existing table name

fields: an array with objects containing the fields: name, type and length

callback: a callback function that will be called with two parameters once the table is created:

err and res. If err is false, the table was created successfully.

Valid field types:

"BOOL"

"TINYINT"


Tutorials


"UTINYINT"

"SMALLINT"

"USMALLINT"

"INTEGER"

"UINTEGER"

"BIGINT"

"UBIGINT"

"MONEY"

"DATE"

"TIME"

"FLOAT"

"DOUBLE"

"TIMESTAMP"

"EFLOAT"

"BINARY"

"CHARS"

"FPSTRING"

"F2STRING"

"F4STRING"

"NUMBER"

"CURRENCY"

"PSTRING"

"VARBINARY"

"LVB"

"VARCHAR"

"LVC"

"NCHAR"

"NVARCHAR"

Example:

ctree.createTable("ctreeSQL", "books", [

{

"name": "id",

"type": "INTEGER",

"length": 4

},

{

"name": "name",

"type": "VARCHAR",

"length": 128

},

{

"name": "author",

"type": "VARCHAR",

"length": 128

Tutorials


},

{

"name": "country",

"type": "VARCHAR",

"length": 48

}

]

, function(err, res){

if (err) {

console.log('error:', err);

} else {

console.log("table created");

}

});

Delete Table deleteTable(db, table, callback)


table: your existing table name

callback: a callback function that will be called with two parameters once the table is deleted:

err and res. If err is false, the table was deleted successfully.

Create Index createIndex(db, table, index, unique, fields, callback)



index: your not-existing index name

unique: true or false depending if you want the index to allow duplicate values

fields: an array with objects containing the fields: name and ascending

callback: a callback function that will be called with two parameters once the index is created:

err and res. If err is false, the index was created successfully.

Example:

ctree.createIndex("ctreeSQL", "books", "myindex", false, [{

"name": "name",

"ascending": true

}], function(err, res){

if (err) {


} else {

console.log('response', res);

}

});

Tutorials


Create Record createRecord( db, table, record, callback)



record: an object containing any or all the fields of your table

callback: a callback function that will be called with two parameters once the record is

created: err and res. If err is false, the record was created successfully.

Example:

ctree.createRecord("ctreeSQL", "books", {

"name": "Moby Dick",

"author": "Herman Melville",

"country": "United States"

}

, function(err, res){

if (err) {


} else {

console.log("record created: ", res);

}

});

Query query( db, table, index, limit, offset, query, callback)



index: your existing index name

limit: the amount of records to return

offset: the amount of records to skip (for paging)

query: for example: {name:{"operator":"=", "value":"Moby Dick"}}

• with operator being one of “=”, “>=”, “<=”, “>” or “<”

callback: a callback function that will be called with two parameters once the query is

executed: err and res. If err is false, the query was executed successfully and the res object

contains the query search result.

Example:

ctree.query("ctreeSQL", "books", "myindex", 100, 0, {name:{"operator":"=", "value":"Moby Dick"}},

function(err, res){

if (err) {


} else {

console.log('response', res);

}

});

Tutorials


4.4 Tutorial for Using Node-RED with the c-treeEDGE

MQTT Broker

This tutorial demonstrates collecting sensor data in Node-RED on a Raspberry Pi and storing the

data in the c-treeEDGE database utilizing MQTT.

Node-RED is a simple way to create applications by dragging together nodes. c-treeEDGE

provides a plug-in so it can be used with Node-RED. Because it supports several IoT operating

systems, c-treeEDGE can run on an edge device or on a PC used as a gateway. For more about

support for MQTT provided by FairCom, see the FairCom MQTT guide in the on-line

documentation.

If your Node-RED project is using MQTT, the easiest way to add edge persistence to your project

is to drop in the c-treeEDGE MQTT Broker.

If you are not using MQTT in your project, you can use the c-treeEDGE Node.js node to add

edge persistence to your project. See Node-RED Node for c-treeEDGE (page 39).

In this section, you will learn how to install Node-RED and c-treeEDGE on a Raspberry Pi. For

more about the Node-RED development environment, see Node-RED later in this document.

You will need to install and start the c-treeEDGE database, as described in Download and Installation.

To use c-treeEDGE with Node-RED, follow these steps:

1. It is good practice to update the operating system on your Raspberry Pi before installing or updating other software:

sudo apt-get update

sudo apt-get dist-upgrade

2. Upgrade Node-RED on the Pi. Note that an earlier version of Node-RED comes pre-installed on the full Raspbian image, however we will use the newer version:

https://nodered.org

Raspberry Pi install/upgrade page for Node-RED:

https://nodered.org/docs/hardware/raspberrypi

The following command works well:

bash <(curl -sL

https://raw.githubusercontent.com/node-red/raspbian-deb-package/mast

er/resources/update-nodejs-and-nodered)

3. Set Node-RED to start on launch:

sudo systemctl enable nodered.service

4. Reboot or start Node-RED manually:

sudo systemctl start nodered.service

5. Optional: Install Firefox on the Pi. In early releases, Node-RED performed better on Firefox than on the built-in browser, but it now works on both.

sudo apt-get install firefox-esr

6. Launch Firefox (or the built-in browser) and go to the following URL to open Node-RED:

Tutorials


http://localhost:1880

Using Node-RED:

1. Install nodes needed for your sensors on the Pi.

This is done inside Node-RED. Using the Hamburger Menu, select Manage Palette and go

to the Install tab. For this example, search for "BME280" to complete the install.

Tutorials


2. Make a Flow that reads your sensors on the Pi.

Note the Inject node on the far left is set to repeat a Boolean True signal. Select a

reasonable value, such as every 10 seconds. The Bme280 sensor will produce JSON output

on any input, so this results in a sensor reading every 10 seconds.

3. Install c-treeEDGE. It comes packaged as a standard zip file. Send it to the Pi and unzip it.

4. Launch c-treeEDGE. Using the terminal, navigate into the c-treeEDGE-Installation-Folder/server folder and launch the server with the following command:

./startedge

5. Change the flow to send data to the c-treeEDGE broker via MQTT.

Drag an MQTT output node onto the flow and connect the sensor output to its input.

6. Double-click the MQTT node to configure it.

Tutorials


7. Enter a string in the Topic field such as Sensor1. This will identify messages being sent from this sensor. Set QoS to 0 and Retain to false.

8. Click the pencil icon to the right of the Server field to configure a new server setup.

9. Give it a name such as ctreeEDGE.

10. Enter localhost in the Server field.

11. Click Add then Done.

12. Your Raspberry Pi should now be broadcasting MQTT messages to c-treeEDGE.

Check Messages

Now let's check to see if the messages are getting to the c-treeEDGE MQTT Broker:

1. In your Node-RED flow, drag an MQTT input node in and configure it to connect to the c-treeEDGE Broker. For the Server field, pick the same MQTT setup you added for the

MQTT output node above. Set the Topic to Sensor1. Set the QoS to 0.

2. Drag in a debug node and place it next to your MQTT output node and connect those two nodes. Deploy and switch to the debug tab and you should see your sensor data coming through every 10 seconds.

Turn on persistence and save your sensor data.

MQTT must be configured as described in Configuring the MQTT Plug-in (page 55).

Persistence is controlled via JSON data published to the c-treeEDGE broker on a predefined

System Topic, ctreeAdministration, that the server watches.

1. Drag the following nodes in to your flow: inject input node

Tutorials


MQTT output node Connect the right side of the inject node to the left side of the MQTT node.

2. Configure the MQTT output node to use the same MQTT server setup created above. Set the

topic to ctreeAdministration.

3. Configure the inject node to use a payload of JSON. Leave repeat turned off so it will only fire when clicked on. Click the JSON editor button (the three dots at the right edge of the payload field) and paste the JSON code shown below into the JSON editor window. Click Done. Change the Name field to "Persistence On" and then click Done again to save the changes to the inject node.

The JSON object describes the database to create and what data to save. By adjusting the

JSON code you can control the database name created, the tags persisted to fields, and the

names and types of those fields.

{

"operation": "CreatePersistenceTopic",

"persistenceTopic": "Sensor1",

"databaseConnectionString": "FAIRCOMS@localhost",

"tableName": "s1db",

"tableAutoTimeStamp": true,

"tableReplicationReady": true,

"mapOfPropertiesToFields": [

{

"jsonPropertyPath": "temperature_F",

"fieldName": "temperature",

"fieldType": "DOUBLE"

},

{

"jsonPropertyPath": "humidity",

"fieldName": "humidity",


},

{

"jsonPropertyPath": "pressure_Hg",

"fieldName": "pressure",


}

]

}

If you want to be able to stop or turn off persisting the data, drag another inject node into the

flow and connect it to the MQTT output node the same way you did the first inject node.

Change the Payload Field to be JSON and open the JSON editor like before and paste the

JSON code shown below into the editor window. Click Done and then change the Name field

to be "Persistence Off". Click Done to save the changes to the node.

{

"operation": "DeletePersistenceTopic",


Tutorials


"persistenceTopic": "Sensor1"

}

Click the button to the left of the "Persistence On" inject node to create the table.

4. As soon as an MQTT message with a matching topic comes into c-treeEDGE, your table will be created in c-treeEDGE-Installation-Folder/server/data/ctree*.dbs/

5. Open c-treeACE SQL Explorer (or any ODBC SQL query tool) on a Windows PC and perform a query on the data:

Select * from Sensor1Table where temperature > 72.0

6. When you are done, click the button to the left of the "Persistence Off" inject node to stop creating a new record for each matching MQTT message.

4.5 Tutorial for Node-RED

This tutorial demonstrates collecting sensor data in Node-RED on Windows and storing the data

in a c-treeEDGE database utilizing the c-treeEDGE Node.

Node-RED is a simple way to create applications by dragging together nodes. c-treeEDGE

provides a plug-in so it can be used with Node-RED. Because it supports several IoT operating

systems, c-treeEDGE can run on an edge device or on a PC used as a gateway.

c-treeEDGE provides a node that can be used in Node-RED projects. This allows c-treeEDGE to

be added to your project simply by dragging the node into the project.

If your Node-RED project uses MQTT, you can use the c-treeEDGE MQTT Broker to add

edge persistence to your project. See Node-RED Using the c-treeEDGE MQTT Broker (page

34).

You will need to install and start the c-treeEDGE database, as described in Downlaod and Installation.

Tutorials


Node.js Installation

Before you can install Node-RED, you must have a working install of Node.js. The use of Node.js

LTS 8.x is recommended. Users of Node.js 6.x and 4.x should ensure they have the latest

updates. Node-RED no longer supports Node.js 0.12.x or 0.10.x.

To check your version of Node.js:

> node -v

The easiest way to install Node-RED is to use the node package manager, npm, that comes with

Node.js. Installing as a global module adds the command node-red to your system path:

> npm install -g --unsafe-perm node-red

Note: To uninstall, use: npm uninstall -g node-red

Running (on Windows)

If you have installed Node-RED as a global npm package, you can use the node-red command:

> node-red

After Node-RED has started, you should be able to access the Node-RED web interface using

http://127.0.0.1:1880/ in your browser.

Installing the Node-RED Node for c-treeEDGE

Note: If Node-RED is running, in a Windows command window, press Ctrl+C to stop it.

The c-treeEDGE Node-RED Node is included with the c-treeEDGE download. Look for it in the

following path:

c-treeEDGE.<platform>\sdk\restapi.nodeRED\node-red-contrib-ctree

Running Node-RED has created a new folder in your %HOMEPATH% folder called .node-red. This

is your userDir folder. Think of it as the home folder for Node-RED configuration for the current

user. You will often see this referred to as ~/.node-red in the Node-RED documentation.

1. In a command window, go to the .node-red folder:

• Linux: cd ~/.node-red

• Windows: cd %HOMEPATH%\.node-red

2. Now install the c-treeEDGE node by typing the following (substituting the path for your installation) at the command prompt:

> npm install C:\path\to\node-red-contrib-ctree

3. Run Node-RED again:

> node-red

Tutorials


You should now see a new ctree node under the Node-RED node section Storage:

Using the Node-RED Node

Drag and Drop the new ctree node into a flow.

Connect an Inject and Debug output node.

Double click the ctree node. This should open up the configuration window:

Tutorials


Click the pencil button next to Add new ctree_configuration. This will open the c-treeEDGE

server connection configuration window:

Enter all the information that applies in your environment and click Add. This will bring you back

to the node configuration window:

Next you have to decide on the action you want to take. You can choose one of: Create, Read,

Update, and Delete.

Finally, you have to select the target of your action. You can choose one of: Table, Index, or

Record.

Create Table

To create a table select create as the action and table as the target.

Click Done.

Tutorials


Open up the Inject node configuration window.

Set the payload to be JSON and open the JSON editor by clicking the three dots. Paste the

following text:

[

{

"name": "id",

"type": "VARCHAR",

"length": 10

},

{

"name": "name",

"type": "VARCHAR",

"length": 128

},

{

"name": "author",

"type": "VARCHAR",

"length": 128

},

{

"name": "country",

"type": "VARCHAR",

"length": 48

}

]

Click Done in the JSON editor window.

Enter a name for your new, currently not-existing table in the topic text field separated by a colon

(":"). The result should look something like this:

Click Done.

Tutorials


Click Deploy:

And finally click the little button in front of the Inject node:

You should now have a new table in your c-treeEDGE database.

Create Record

To create a record, select Create as the action and Record as the target.

Click Done.

Open the Inject node configuration window.

Set the payload to be JSON and open the JSON editor by clicking the three dots. Paste the

following text:

{




}

Default Date and Time Format Note:

The c-treeDB default date and time format is set to:

CTDATE_MDCY

CTTIME_HMST

This results in the following string format:

MM/DD/CCYY h|hh:mm:ss.ttt (24 h) (ttt are milliseconds), hours can be either one or two

digits. (See c-treeDB Date Formats https://docs.faircom.com/doc/ctreedb/38256.htm and

c-treeDB Time Formats https://docs.faircom.com/doc/ctreedb/38254.htm for details and

alternative formats.)

The node expects and returns dates, times, and timestamps in the above string format.


Enter the name for the table in the topic text field.

Click Done.

Click Deploy:


The debug window should show the record ID of the record created.

https://docs.faircom.com/doc/ctreedb/38256.htm


Tutorials


You should now have a new record in your table.

Add Index

Note: If you attempt a query without an index, you get error 14011, Out of memory.

To create an index select create as the action and index as the target.

Click Done.

Open up the Inject node configuration window.

Set the payload to be JSON and open up the JSON editor by clicking the three dots.

Paste the following text:

{"fields":[{"name": "country", "ascending": true}],"unique":false}


Enter the table name AND a name for your new, currently not-existing index in the topic text field

separated by a colon.

For example: books:c_index

Click Done.

Click Deploy.

And finally click on the little button in front of the Inject node:

The debug window should show: "index created"

You should now have a new index in your c-treeEDGE database.

Read Record (Query)

To query records, select read as the action and record as the target.

This is the most complex operation and thus requires the most information.

It needs the following:

table name

the index name

the number of results to return (top)

the number of records to skip

Enter this information in the topic field separated with colons. For example:

library:c_index:100:0

Click Done.

Open the Inject node configuration window.

Tutorials


Set the payload to be JSON and open the JSON editor by clicking the three dots.

Paste the following text:

{

"find": {

"country": {

"operator": "=",

"value": "United States"

}

}

}

Note: Be sure to use an index on the field being searched. Pass in the index as part of the topic.

See the Default Date and Time Format Note earlier in the section about creating records.


Enter the name for the table in the topic text field.

Click Done.

Click Deploy:


The debug window should show the following:

Node-RED Node API:

Create Table

c-tree node fields:

Action: "create"

Target: "table"

Inject node fields:

Payload:

Tutorials


[

{

"name": "<field name>",

"type": "<field type>",

"length": "<field length>"

},

{ ... }

]

Topic: <table name>

Create Record

c-tree node fields:

Action: "create"

Target: "record"

Inject node fields:

Payload:

{




}

Topic: <table name>

REST Default Date and Time Format Note:

The c-treeDB default date and time format is set to:

CTDATE_MDCY

CTTIME_HMST

This results in the following string format:

MM/DD/CCYY h|hh:mm:ss.ttt (24 h) (ttt are milliseconds), hours can be either one or two

digits. (See c-treeDB Date Formats https://docs.faircom.com/doc/ctreedb/38256.htm and

c-treeDB Time Formats https://docs.faircom.com/doc/ctreedb/38254.htm for details and

alternative formats.)

The node expects and returns dates, times, and timestamps in the above string format.

Read Record

c-tree node fields:

Action: "read"

Target: "record"

Inject node fields:



Tutorials


Payload:

{

{

"find": {

"<field name>": {

"operator": "<operator [ >, >=, <=, <, = ]>",

"value": "<field value>"

}

}

}

Topic: <table name>:<index name>:<top>:<skip>

See the REST Default Date and Time Format Note above.

Create Index

c-tree node fields:

Action: "create"

Target: "index"

Inject node fields:

Payload:

{"fields":[{"name": "country", "ascending": true}],"unique":false}

Topic: <table name>:<index name>

For more information:

See https://nodered.org/

4.6 Tutorial for Using Python with c-treeEDGE MQTT

This tutorial demonstrates collecting sensor data in Python on the Raspberry Pi and storing the

data in the c-treeACE database utilizing MQTT.

The Eclipse Paho project provides open-source client implementations of MQTT messaging

protocols. In this tutorial, we will use the Paho MQTT Publisher to communicate with c-treeEDGE

on the Pi. The example uses Python 3 to read a sensor and publish the value via MQTT to

c-treeEDGE. For more information about MQTT, see MQTT Broker later in this document.

You will need to install and start c-treeEDGE database, as described in Download and Installation.

To use MQTT for messaging from Python to c-treeEDGE on a Pi, follow these steps:

1. Install the necessary libs to read the sensors via Python on the Pi.

For example, to use a DS18b20 sensor, the install command is:

sudo apt-get install python3-w1thermsensor

Tutorials


2. Install the Paho MQTT Python publisher on the Pi. It is available from:

https://pypi.org/project/paho-mqtt/

3. Type the following:

pip3 install paho-mqtt

4. Write your Python 3 example program to read sensors and publish to c-treeEDGE on PC via MQTT with the topic Sensor1. The example code below uses Python 3:

import time

import paho.mqtt.client as mqtt

from w1thermsensor import W1ThermSensor

sensor = W1ThermSensor()

client = mqtt.Client()

client.connect("localhost", 1883, 60)

while True:

temp = sensor.get_temperature(W1ThermSensor.DEGREES_F)

jsonMsg = "{\"temperature_F\":%f}" % (temp)

client.publish("Sensor1", jsonMsg)

print("published %s" %(jsonMsg))

time.sleep(10)

Now let's check to see if the messages are getting to the c-treeEDGE MQTT Broker:

Note: MQTT must be configured as described in Setting up MQTT Persistence on the Edge

(page 55).

1. Run the following Python 3 program to subscribe to your published data. You should see your data coming through every 10 seconds:


def on_connect(client, userdata, flags, rc):

print("Connected with result code "+str(rc))

def on_message(client, userdata, msg):

print(msg.topic+" "+str(msg.payload))


client.on_connect = on_connect

client.on_message = on_message


client.subscribe("Sensor1")

Tutorials


client.loop_forever()

2. Run the following Python 3 program to start persisting your sensor data. The program simply packages up JSON data to be published on a specific topic. By adjusting the JSON data you can control the database name created, the tags persisted to fields, and the names and types of those fields.




jsonMsg = "{\"operation\": \"CreatePersistenceTopic\",

\"persistenceTopic\": \"Sensor1\", \"databaseConnectionString\":

\"FAIRCOMS@localhost\", \"tableName\": \"Sensor1Table\",

\"tableAutoTimeStamp\": true, \"tableAutoTimeStampIndex\": true,

\"mapOfPropertiesToFields\": [ { \"jsonPropertyPath\":

\"temperature_F\", \"fieldName\": \"temperature\", \"fieldType\":

\"DOUBLE\"}, {\"jsonPropertyPath\": \"humidity\", \"fieldName\":

\"humidity\", \"fieldType\": \"DOUBLE\"}, {\"jsonPropertyPath\":

\"pressure_Hg\", \"fieldName\": \"pressure\", \"fieldType\":

\"DOUBLE\"}]}"

client.publish("ctreeAdministration", jsonMsg)

print("published %s" %(jsonMsg))

The above JSON message is shown below formatted for readability:

jsonMsg =

"{

"operation": "CreatePersistenceTopic",

"persistenceTopic": "Sensor1",


"tableName": "Sensor1Table",

"tableAutoTimeStamp": true,

"tableAutoTimeStampIndex": true,

"mapOfPropertiesToFields":

[

{

"jsonPropertyPath": "temperature_F",

"fieldName": "temperature",


},

{

"jsonPropertyPath": "humidity",

"fieldName": "humidity",


},

{

"jsonPropertyPath": "pressure_Hg",

Tutorials


"fieldName": "pressure",


}

]

}"

3. As soon as an MQTT message comes into c-treeEDGE with a matching topic, your table will be created in server/data/ctreeSQL.dbs/

4. Open c-treeACE SQL Explorer (or any ODBC SQL query tool) on a PC and perform a query on the data:

Select * from Sensor1Table where temperature > 72.0

4.7 c-treeEDGE REST API

c-treeEDGE provides a REST API, which allows it to be used in applications that do not use

Node-RED, ThingWorx, or MQTT.

Full documentation of this JSON-oriented HTTP-based API is presented in REST API Developer

Guide (https://docs.faircom.com/doc/rest_api/).

Basic database CRUD (Create, Read, Update, Delete) operations are available via this modern

API framework.

Create and Delete Tables

List Tables

Query Tables

Create and Delete Records


Tutorials


Create and Delete Indexes

Get Available Table Indexes

You will need to install and start the c-treeEDGE Server, as described in Installing the c-treeEDGE Server.

A complete tutorial demonstrating this API is available in the FairCom Database Engine REST

API (https://docs.faircom.com/doc/rest_api/) developer guide in the on-line documentation.

4.8 Tutorial for OPC UA Installation and Configuration

Enabling the OPC UA Plug-in

The OPC UA plug-in is enabled by adding the following server configuration in ctsrvr.cfg in the

server/config folder:

Windows:

PLUGIN ctopc;./opc/ctopc.dll

Linux/Unix:

PLUGIN ctopc;./opc/libctopc.so

The plug-in is loaded at server startup, connects to an OPC server, and begins persisting data.

Check CTSTATUS.FCS to see if there was an error starting OPC Plug-in.

Before starting the c-treeEDGE database, make sure the OPC server is running; otherwise, the plug-in will not be able to connect to it.

Configuring the OPC UA Plug-in

The OPC UA plug-in is configured using the ctopc.json file in the opc folder under the

c-treeEDGE server folder. This configuration file is in JSON format:

{

"opcClientDllPath": "./opc/ctOPCClient.dll",

"databases": [

{

"databaseName": "ctreeSQL",

"sqlDatabase": true,

"authFileName": "authFile.set",

"databaseUser": "ADMIN",

"databasePassword": "ADMIN",

"targetTables": [

{

"targetTableName": "OPCtable1",

"purgeHistoryUnit": "day",

"purgeHistoryPeriod": 60,

"opcServerUrl": "opc.tcp://localhost:4840",

"opcServerAuthFileName": "opcAuthFile.set",

"opcServerUser": "",

"opcServerPas": "",

"opcDataCollectionIntervalSeconds": 10,

"targetFields": [

{

"targetFieldName": "node1",


Tutorials


"opcNamespace": 1,

"opcNodeName": "the.answer"

}

]

}

]

}

]

}

Where:

opcClientDllPath: <string> Path to ctOPCClient.dll

databases: <array> Array of c-treeEDGE database connections.

• databaseName: <string> Database name

• sqlDatabase: <boolean> Indicates if table should be included in SQL. Possible values

are: true or false. If you are using a SQL API, set the value to true.

• authFileName: <string> c-tree Secure authentication file (.set)

• databaseUser: <string> c-treeEDGE user name.

• databasePassword: <string> c-treeEDGE user password.

• targetTables: <array> Array of c-treeEDGE tables where data will be persisted.

targetTableName: <string> Table name for persisted data.

purgeHistoryUnit: <string> Time unit for purging data.

Supported values: "none", "minute", "hour", "day", "month", "year"

purgeHistoryPeriod: <numeric>Time value for purging data.

opcServerUrl: <string> OPC server URL used to connect to your OPC server.

opcServerAuthFileName: <string> OPC Secure authentication file (.set)

opcServerUser: <string> OPC User Name

opcServerPas: <string> OPC User password

opcDataCollectionIntervalSeconds: <numeric> Interval value in seconds between

reads. A value of 0 (zero) means that data will be persisted each time data is changed.

targetFields: <array> Array of c-treeEDGE field names to map.

targetFieldName: <string> Field name.

opcNamespace: <numeric> OPC namespace name.

opcNodeName: <string> or <numeric> OPC node name.

Example: {

"databases": [

{

"databaseName": "ctreeSQL",

"sqlDatabase": true,

"databaseUser": "ADMIN",

"databasePassword": "ADMIN",

"targetTables": [

{


"purgeHistoryUnit": "day",




"targetFields": [

Tutorials


{


"opcNamespace": 1,

"opcNodeName": "the.answer"

} ]

},

{


"purgeHistoryUnit": "month",




"targetFields": [

{


"opcNamespace": 1,

"opcNodeName": 51034

},

{


"opcNamespace": 0,

"opcNodeName": 119

} ]

} ]

} ]

}

OPC Test Server

A test directory is provided under the opc directory in the Raspbian, and Linux, and Windows

x86/x64 versions of the package. This test directory contains an example of the OPC server and

OPC client. It is an open-source implementation for Windows 64, Linux 64, and Raspberry Pi (it

can be downloaded at https://open62541.org/). No setting is needed. Just execute server_ctt

before starting c-treeEDGE.

Raspbian Example

For example, in the Raspbian version, execute ./server_ctt & in the following folder:

Tutorials


server/opc/test/open62541_v0.3.0/Raspbian.ARM.32bit

Windows Example

In Windows, double-click the following file:

\server\opc\test\open62541_v0.3.0\Windows.x86.64bit\server_ctt.exe

OPC Client Implementation Example

In the same folder indicated in the examples above, is a program called client. It is an OPC Client

implementation example that connects to server_ctt, reads some data, and changes a node

value (node “the.answer”). To run it, execute it as described above for server_ctt.

4.9 Configuring the MQTT Plug-in

These instructions assume you have downloaded c-treeEDGE database and installed it on the same machine as the application. See Download and Installation. They also assume you have

followed the directions elsewhere in this document to configure your environment.

Enabling the MQTT Plug-in

For security reasons, plug-ins are not enabled by default.

The MQTT plug-in is enabled by adding the following information to the server configuration in the

file named ctsrvr.cfg in the server/config folder:

Windows:

PLUGIN cthttpd;./web/cthttpd.dll

Linux/Unix:

PLUGIN cthttpd;./web/libcthttpd.so

The plug-in is loaded at server startup and starts listening for MQTT messages and starts

sending messages in response to previously cached data and new data collected by the

database.

MQTT Plug-in Configuration

The MQTT plug-in is configured using the cthttpd.json file in the \server\config\ folder. This file is

in JSON format. It contains all the HTTP configuration settings. The MQTT settings are controlled

by the properties starting with mqtt_.

{








Tutorials


"applications": [

"mqtt;ctMQTT.dll",

"ctree;ctREST.dll"

]

}

If you want to run the HTTP server but you do not want to run MQTT, set mqtt_enabled to

false.

"mqtt_enabled": false

You can change the TCP/IP port on which MQTT is listening by setting

"mqtt_listening_port" to another port number, such as 2020.

"mqtt_listening_port": 2020

If you want to use the c-treeEDGE database to broker MQTT messages but you do not want to

cache data from those messages, set "mqtt_persistence_enabled" to false.

"mqtt_enabled": false

You can turn on MQTT logging by setting "mqtt_log_enabled" to true.

"mqtt_log_enabled": true


5. Automatic Timestamps, Aggregation, and

Purging

The FairCom Database Engine supports multiple data automation features to quickly and

efficiently process data streams. These features can be used independently or combined for

optimizing data capture.

Automatically stamp data records with the current date and time as they are captured.

Automatically aggregate selected columns for advanced data analytics.

Automatically purge data as it ages to minimize storage retention on devices with limited

storage capacity.

These features are ideal for quickly capturing streams of data, automatically timestamping the

data, automatically aggregating the data, and automatically retaining desired data and purging old

data.

You have extensive declarative control over each of these features, which makes it easy to

enable or disable timestamping, or control the time period before purging data, or controlling

aggregation time windows for average, min, max, etc.

For more information about support for these features, see the Automatic Data Aggregation

(https://docs.faircom.com/doc/aggregation/) guide in the on-line documentation.

https://docs.faircom.com/doc/aggregation/


6. Replication with c-treeEDGE

In the IoT it is very common to stream data to the cloud for storage and other processing, such as

running reports. c-treeEDGE allows you to store and process data on the edge device, saving the

time it takes to move data to the cloud. It also has the benefit replicating data to the cloud.

Cloud Replication

Replicating from the edge to the cloud consists of three pieces:

1. A source server on the edge collecting data into one or more tables.

2. A target server in the cloud receiving all the data changes from the source server.

3. A replication process that connects to both and moves the data between them.

Replication differs from streaming in that, if a connectivity outage occurs, replication will pick up

where it left off when the outage is over and no data will be skipped. With streaming, data

generated during the outage most often is lost. Replication also moves data in the same

transaction chunks that were originally added to the source server, preserving atomicity on the

target server.

You can configure your replication using a graphical tool, the Replication Manager. The

configuration is stored in the Memphis database and propagated throughout the system. To

simplify setup, c-treeEDGE and the files it creates are replication-aware.

For full documentation see the Replication Manager Guide (https://docs.faircom.com/doc/ops_manager) in the FairCom online documentation.

Source Server

Install a c-treeEDGE database on the source machine. See the c-treeEDGE Administration Guide

(https://docs.faircom.com/doc/ctedge_admin/) for detailed steps on installing and configuring a

server. To enable replication, you must do either of the following:

Specify REPLICATE in the MQTT persist JSON data (see examples in Node-RED Using the

c-treeEDGE MQTT Broker (page 34) and Python and c-treeEDGE MQTT (page 48)).

OR

Add the following line to the ctsrvr.cfg file in the source server's directory, replacing "<table

name>" with the name of the table(s) you wish to replicate:

REPLICATE ./ctreeSQL.dbs/<table name>.dat

If you wish to replicate more than one table, you can add multiple lines or use wildcards such as

"*.dat". The REPLICATE keyword identifies tables that will be replicated for the source server.

The path is relative to the server's data folder. Tables created via MQTT persistence will be

located in data/ctreeSQL.dbs/ by default.



Replication with c-treeEDGE


Add REPL_SRLSEG_ALLOW_UNQKEY YES to the ctsrvr.cfg file. All tables to be replicated must

have a unique index. This keyword allows simple tables to be replicated by using our built in

identity index.

Connect the source server to the Replication Manager server by editing the

/server/config/ctagent.json file. Change the "memphis_host": "localhost", line by replacing

"localhost" with the IP of the target server.

Start the source server.

Target Server

Install a c-treeACE Server on the target machine. If this server is on the same machine as the

source server, you will have to change its name. Look for the SERVER_NAME keyword in the

ctsrvr.cfg file in the server folder. Make sure this name is unique (we sum up the ASCII equivalent

of the name, so SRV1 and 1SRV will be identical, which will cause problems).

Replication Manager

Open a browser window and point it to this URL:

https://<target server IP>:8443/ReplicationManager/

You should see the Replication Manager app load. Verify that both servers appear.



Drag a line between the c-treeEDGE source server and the c-treeACE target server. You will be

warned that the source server must push the changes, this is normal for replication from a

c-treeEDGE server.

Create a new publication with the table from the source server.



If you want all data to stay on the target server, make sure Ignore Partition is checked under

Advanced Options.

Subscribe to the publication just created.

Right-click on the publication line and select Deploy.

You should see the table be created on the target server and the data moved over.



Ports:

If there is a VPN or firewall between any of the machines you will have to provide network access

from the machine the Replication Agent is running on to both the target and source server. The

port you will need to open is the ISAM (NoSQL) port each server is using. The port the server

uses is based on the server name. For the default name of FAIRCOMS, the port will be 5597. For

other server names, check the ctstatus.fcs file in the servers "data" directory for the port in use by

searching for a line similar to:

"Server using TCP/IP Socket Port Number: 5597"

See the Replication Manager Guide (https://docs.faircom.com/doc/ops_manager) for more about

installing, configuring, and using replication.



7. Administrative Tools

FairCom provides a suite of graphical tools and a set of

command-line tools to assist with creating and maintaining tables

and monitoring the c-treeEDGE.

Note: See the following for information about configuring your system to use those tools: - Configuring the Browser-Based Tools (https://docs.faircom.com/doc/browser_tools/ConfiguringtheBrowser-BasedTools.htm) - Command-Line Tools Redistributables Folder (https://docs.faircom.com/doc/cmdline_edge/78253.htm)

Browser-Based Graphical Tools

FairCom provides a set of tools, featuring a graphical user interface (GUI), for monitoring

operation and exploring your FairCom ISAM and SQL databases.

https://docs.faircom.com/doc/browser_tools/ConfiguringtheBrowser-BasedTools.htm

https://docs.faircom.com/doc/cmdline_edge/78253.htm

Administrative Tools


Because these tools are browser-based, they can run on any system that has a web browser and

a network connection.

The following tools are provided:

c-treeACE SQL Explorer

c-treeACE ISAM Explorer

Ace Monitor

To learn more about these tools, see Connecting to the Browser-Based Tools

(https://docs.faircom.com/doc/browser_tools/ConnectingtotheBrowser-BasedTools.htm) in

FairCom Browser-Based Tools (https://docs.faircom.com/doc/browser_tools/).

Command-Line Tools

The c-treeEDGE database ships with utilities for developers and administrators to use to ensure

the integrity of their database. These utilities include a set of command-line utilities, which can be

run for a command-line prompt or from a script.

To learn more, see Command-Line Tools (https://docs.faircom.com/doc/cmdline_edge).

https://docs.faircom.com/doc/browser_tools/ConnectingtotheBrowser-BasedTools.htm

https://docs.faircom.com/doc/browser_tools/

https://docs.faircom.com/doc/cmdline_edge


8. FairCom License Agreement for

c-treeEDGE

The use of this software is governed by a license agreement. This agreement sets forth the terms

that govern the development use of this software.

This agreement sets forth the terms that govern development use of this software. It does NOT

address the terms under which you may deploy the applications created with the software.

Deployment is only authorized under the terms of a separate, signed deployment amendment.

Click here to see the licenses:

Developers License Agreement

(https://docs.faircom.com/license/V2_EDGE_Development_190607.pdf)

EDGE Extensions License

(https://docs.faircom.com/license/V2_EDGE_Extensions_190607.pdf)

https://docs.faircom.com/license/V2_EDGE_Development_190607.pdf

https://docs.faircom.com/license/V2_EDGE_Extensions_190607.pdf

Copyright Notice

Copyright © 1992-2020 FairCom USA. All rights reserved.

No part of this publication may be stored in a retrieval system, or transmitted in any form or by any means, electronic,

mechanical, photocopying, recording or otherwise without the prior written permission of FairCom USA. Printed in the

United States of America.

Information in this document is subject to change without notice.

Trademarks

c-treeACE, c-treeRTG, c-treeEDGE, c-treeAMS, c-tree Plus, c-tree, r-tree, FairCom, and FairCom’s circular disc logo are

trademarks of FairCom USA, registered in the United States and other countries.

The following are third-party trademarks: AMD and AMD Opteron are trademarks of Advanced Micro Devices, Inc.

Macintosh, Mac, Mac OS, and Xcode are trademarks of Apple Inc., registered in the U.S. and other countries.

Embarcadero, the Embarcadero Technologies logos and all other Embarcadero Technologies product or service names

are trademarks, service marks, and/or registered trademarks of Embarcadero Technologies, Inc. and are protected by the

laws of the United States and other countries. Business Objects and the Business Objects logo, BusinessObjects, Crystal

Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned

herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd.

Business Objects is an SAP company. HP and HP-UX are registered trademarks of the Hewlett-Packard Company. AIX,

IBM, POWER6, POWER7, and pSeries are trademarks or registered trademarks of International Business Machines

Corporation in the United States, other countries, or both. Intel, Intel Core, Itanium, Pentium and Xeon are trademarks or

registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Microsoft, the .NET

logo, the Windows logo, Access, Excel, SQL Server, Visual Basic, Visual C++, Visual C#, Visual Studio, Windows,

Windows Server, and Windows Vista are either registered trademarks or trademarks of Microsoft Corporation in the

United States and/or other countries. Novell and SUSE are registered trademarks of Novell, Inc. in the United States and

other countries. Oracle and Java are registered trademarks of Oracle and/or its affiliates. QNX and Neutrino are

registered trademarks of QNX Software Systems Ltd. in certain jurisdictions. CentOS, Red Hat, and the Shadow Man logo

are registered trademarks of Red Hat, Inc. in the United States and other countries, used with permission. UNIX and

UnixWare are registered trademarks of The Open Group in the United States and other countries. Linux is a trademark of

Linus Torvalds in the United States, other countries, or both. Python and PyCon are trademarks or registered trademarks

of the Python Software Foundation. OpenServer is a trademark or registered trademark of Xinuos, Inc. in the U.S.A. and

other countries. Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the United States and other

countries.

Btrieve is a registered trademark of Actian Corporation.

ACUCOBOL-GT, Micro Focus, RM/COBOL, and Visual COBOL are trademarks or registered trademarks of Micro Focus

(IP) Limited or its subsidiaries in the United Kingdom, United States and other countries.

isCOBOL and Veryant are trademarks or registered trademarks of Veryant in the United States and other countries.

All other trademarks, trade names, company names, product names, and registered trademarks are the property of their

respective holders.

Portions Copyright © 1991-2016 Unicode, Inc. All rights reserved.

Portions Copyright © 1998-2016 The OpenSSL Project. All rights reserved. This product includes software developed by

the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/).

Portions Copyright © 1995-1998 Eric Young ([email protected]). All rights reserved. This product includes cryptographic

software written by Eric Young ([email protected]). This product includes software written by Tim Hudson

([email protected]).

Portions © 1987-2020 Dharma Systems, Inc. All rights reserved. This software or web site utilizes or contains material

that is © 1994-2007 DUNDAS DATA VISUALIZATION, INC. and its licensors, all rights reserved.

Portions Copyright © 1995-2013 Jean-loup Gailly and Mark Adler.

6/3/2020


9. Index

A Administrative Tools..............................................63 API diagram ........................................................... 1 APIs, additional ...................................................... 1 Automatic Timestamps, Aggregation, and

Purging .............................................................57

C Configuring the MQTT Plug-in ...............................55 Copyright Notice ................................................. lxvi c-treeEDGE Developer Quick Start ........................ 5 c-treeEDGE REST API .........................................51

E edge ...................................................................... 1 environments ......................................................... 1

F FairCom License Agreement for c-treeEDGE ........65

G gateway ................................................................. 1

H HTTP Plug-in ......................................................... 9

I installation ............................................................. 5 Introduction to c-treeEDGE .................................... 1

J JSON................................................................ 1, 48

M microserver ............................................................ 1 MQTT ......................................................... 1, 34, 48 MQTT Plug-in .......................................................12 multimodel ............................................................. 1

N navigational ........................................................... 1 Node.js .................................................................39 Node-RED ...................................................... 34, 39 NoSQL .................................................................. 1

O OPC UA Plug-in ....................................................16

P Plug-ins ................................................................. 8 Python/MQTT, tutorial ...........................................48

R relational ................................................................ 1 Replication with c-treeEDGE .................................58

REST API Plug-in ................................................ 15

S Starting and Stopping c-treeEDGE ......................... 6 Supported Plug-ins................................................. 8

T ThingWorx ........................................................... 28 ThingWorx AlwaysOn Plug-in ............................... 11 ThingWorx MQTT Plug-in ..................................... 12 topic ....................................................................... 1 Tutorial for Node-RED .......................................... 39 Tutorial for OPC UA Installation and

Configuration .................................................... 52 Tutorial for the ThingWorx AlwaysOn Plug-in........ 19 Tutorial for the ThingWorx MQTT Extension ......... 28 Tutorial for using Node.js with c-treeEDGE ........... 30 Tutorial for Using Node-RED with the

c-treeEDGE MQTT Broker ................................ 34 Tutorial for Using Python with c-treeEDGE

MQTT .............................................................. 48 Tutorials ............................................................... 18

U Using Node.js with c-treeEDGE ............................ 15 Using Node-RED with c-treeEDGE ....................... 14 Using Node-RED with MQTT ............................... 14

c-treeedge iot database - faircom€¦ · node.js rest api multimodel design c-treeedge consists of...

Documents