TerkinData C++


Tip

You might want to read this document on our documentation space, all inline links will be working there.

Introduction

TerkinData C++ is a convenient library for handling sensor readings. It helps to decouple the sensor reading domain from the telemetry domain in a typical data logger application. While providing a generic interface, it can serialize measurement values to CSV, x-www-form-urlencoded and JSON formats.

The library solves some important obstacles usually encountered when doing telemetry:

  • When serializing to CSV, the order of fields is important. Therefore, an ordered list of field names is required for proper operation. When serializing a CSV data line and some sensor values are missing due to sensor defects or other errors, this must not have an impact on the CSV data line. Optionally, the CSV header line can be prefixed with an arbitrary string on serialization.
  • Some sensors like the DHT33 (RHT04) digital humidity/temperature sensor deliver two values in a single reading cycle: Humidity and temperature. The data channeling from reading the sensor to telemetry submission has to account for that. As with the DS18B20 digital thermometer or other 1-Wire devices, multiple sensors of the same type might be attached, the data channeling also has to account for that. This is achieved by one level of indirection between the sensor reading and the telemetry domain by introducing a convenient mapping between lowlevel sensor values and the designated highlevel telemetry field names, which is resolved and applied on serialization.
  • A timestamp string in arbitrary format can be attached to a single measurement.

It relies on a reasonable STL implementation as it uses the map and vector data containers and its corresponding iterators under the hood.

Environment

TerkinData C++ compiles with avr-g++ 4.9.1 (AVR) and g++ 5.3.0 (MacPorts) as well as xtensa-lx106-elf-g++ 4.8.2 (xtensa-lx106, crosstool-NG 1.20.0) on Mac OS X. It currently does not compile with clang++ 3.4. Patches are welcome!

The example programs csv_basic.cpp, urlencoded_basic.cpp and json_basic.cpp have been tested successfully at runtime on x86_64 (Mac OS X).

It is currently alpha quality as we didn’t hunt down potential memory leaks yet. It has also not been run on embedded MCU hardware yet, as this is just a design draft by now. Helping hands on that are very welcome!

Synopsis

Setup

#include <TerkinData.h>
using namespace TerkinData;
using namespace TerkinUtil;

void DataManager::setup() {

    // List of field names
    this->field_names  = new DataHeader({"time", "temperature", "humidity", "rssi", "voltage"});

    // Map names of lowlevel sensor values to highlevel telemetry data fields
    (*this->sensor_field_mapping)[string("dht.0.temp")]   = string("temperature");
    (*this->sensor_field_mapping)[string("dht.0.hum")]    = string("humidity");

    // Optionally prefix CSV header line with string
    this->csv_header_prefix = "## ";

    // Optionally set float serialization precision
    this->float_precision = 3;

}

DataManager *datamgr = new DataManager();

Collect

// Data container for one reading cycle
Measurement *measurement = new Measurement();

// Set timestamp (any format)
measurement.time = "2017-01-11T14:48:56Z";

// Collect sensor readings
measurement->data["dht.0.temp"]  = 42.42f;
measurement->data["dht.0.hum"]   = 84.84f;
measurement->data["rssi"]        = 72;
measurement->data["voltage"]     = 3.843f;

Serialize

CSV

// Serialize data into CSV format
std::string data_header = datamgr->csv_header();
std::string data_record = datamgr->csv_data(*measurement);

// Use as character array:
// - data_header.c_str()
// - data_record.c_str()

// Free memory
delete measurement;

x-www-form-urlencoded

// Serialize data into x-www-form-urlencoded format
std::string data_record = datamgr->urlencode_data(*measurement);

// Use as character array:
// - data_record.c_str()

// Free memory
delete measurement;

JSON

Note

To use the fine ArduinoJson library for JSON serialization, please compile using -DHAVE_ARDUINO_JSON.

// Serialize data into JSON format
std::string data_record = datamgr->json_data(*measurement);

// Use as character array:
// - data_record.c_str()

// Free memory
delete measurement;

Examples

CSV

cd examples
make csv

======================
TerkinData CSV example
======================

-- Test single reading (complete)
header: ## time,weight,temperature-outside,humidity-outside,temperature-inside,voltage
data:   2017-03-17T02:48:15Z,85.000,42.420,84.840,33.330,3.843

-- Test single reading (incomplete: "weight" and "temperature-inside" missing)
header: ## time,weight,temperature-outside,humidity-outside,temperature-inside,voltage
data:   2017-03-17T02:48:15Z,,42.420,84.840,,3.843

See also

Full source of csv_basic.cpp.

x-www-form-urlencoded

cd examples
make urlencoded

========================================
TerkinData x-www-form-urlencoded example
========================================

-- Test single reading (complete)
data:   time=2017%2D03%2D17T03%3A03%3A57Z&weight=85%2E00&temperature-outside=42%2E42&humidity-outside=84%2E84&temperature-inside=33%2E33&voltage=3%2E84

-- Test single reading (incomplete: "weight" and "temperature-inside" missing)
data:   time=2017%2D03%2D17T03%3A03%3A57Z&temperature-outside=42%2E42&humidity-outside=84%2E84&voltage=3%2E84

See also

Full source of urlencoded_basic.cpp.

JSON

cd examples
make json

=======================
TerkinData JSON example
=======================

-- Test single reading (complete)
data:   {"time":"2017-03-17T03:07:25Z","weight":85.00,"temperature-outside":42.42,"humidity-outside":84.84,"temperature-inside":33.33,"voltage":3.84}

-- Test single reading (incomplete)
data:   {"time":"2017-03-17T03:07:25Z","temperature-outside":42.42,"humidity-outside":84.84,"voltage":3.84}

See also

Full source of json_basic.cpp.

Download

TerkinData C++

Please download a recent version from GitHub:

Dependencies

ArduinoSTL

ArduinoSTL 1.0.2 by Mike Matera

ArduinoJson

ArduinoJson by Benoît Blanchon

This library is optional, but obviously required when serializing to JSON.

Todo

Todo


Etymology

Terkin

Epitomizes the greatest, the most immenense, the supreme and the paramount one.
Basically, the fundamental individual.