.. include:: ../_resources.rst
.. _data-acquisition:
################
Data acquisition
################
.. contents::
:local:
:depth: 1
----
************
Introduction
************
We offer various ways of sending data to our data collection platform ``swarm.hiveeyes.org``.
To successfully publish data to the platform, you should get familiar with the addressing scheme.
We call it the »quadruple hierarchy strategy«.
.. _topology-spec-0.1-rationale:
Addressing
==========
The topology hierarchy is made up of a minimum of four identifiers describing the core structure::
realm / network / gateway / node
The topology identifiers are specified as:
- "realm" is the designated root realm. You should prefix the topic name
with this label when opting in for all features of the telemetry platform.
For Hiveeyes, this should be called ``hiveeyes``.
- "network" is your personal realm, it designates the **owner**.
Choose anything you like or use an
`Online UUID Generator `_
to gain maximum uniqueness.
- "gateway" is your gateway identifier, it designates a sensor node **location**.
Choose anything you like.
This does not have to be very unique, so you might use labels
having the names of sites. While you are the owner of this
namespace hierarchy, remember these labels might be visible on
the collaborative ether, though.
So the best thing would be to give kind of nicknames to your
sites which don't identify their location.
- "node" is your **node identifier**. Choose anything you like. This usually
gets transmitted from a sensor device.
In the following examples, this topology address will be encoded into the variable ``CHANNEL``.
***************
Getting started
***************
We prepared some examples for transmitting measurement values to the "testdrive"
channel of our community data collection platform. Feel free to try it on your own,
the system currently offers the HTTP and MQTT transport protocols.
The accepted payload formats are JSON, x-www-form-urlencoded and CSV.
.. _daq-mqtt:
MQTT
====
.. highlight:: bash
Publish telemetry data to the MQTT bus using ``swarm.hiveeyes.org`` as hostname and ``hiveeyes`` as realm,
as seen in this example::
# Get hold of a MQTT client of your choice
aptitude install mosquitto-clients
# Define the target address
export BROKER=swarm.hiveeyes.org
export CHANNEL=hiveeyes/testdrive/area-42/node-1
# Publish a single measurement sample
echo '{"temperature": 42.84, "humidity": 83}' | mosquitto_pub -h $BROKER -t $CHANNEL/data.json -l
.. _daq-http:
HTTP
====
.. highlight:: bash
Please use ``swarm.hiveeyes.org`` as hostname and ``hiveeyes`` as realm, giving
a base URI of ``https://swarm.hiveeyes.org/api/hiveeyes``.
.. tip::
As sensor node hardware like the GPRSbee doesn't do TLS, there's an additional
endpoint for unencrypted communication. In this case, just use::
export HTTPURI=http://swarm.hiveeyes.org/api-notls
Use HTTP for submitting telemetry data using the HTTP client of your choice.
This example uses HTTPie_, a command line http client that will make you smile.
::
# A. Get hold of a HTTP client of your choice
aptitude install httpie
# B. Define the target address
export HTTPURI=https://swarm.hiveeyes.org/api
export CHANNEL=hiveeyes/testdrive/area-42/node-1
# C. Publish a single measurement sample
# C.1 Post as application/json
http POST $HTTPURI/$CHANNEL/data temperature:=42.84 humidity:=83
# C.2 Post as application/x-www-form-urlencoded
http --form POST $HTTPURI/$CHANNEL/data temperature:=42.84 humidity:=83
HTTP CSV
========
Single readings
---------------
::
# 1. Define the target address
export HTTPURI=https://swarm.hiveeyes.org/api
export CHANNEL=hiveeyes/testdrive/area-42/node-1
# 2. Register field names once
echo '## weight, temperature, humidity, voltage' | http POST $HTTPURI/$CHANNEL/data Content-Type:text/csv
# 3. Send readings
echo '58.697, 19.6, 56.1, 4.13' | http POST $HTTPURI/$CHANNEL/data Content-Type:text/csv
Bulk mode
---------
``cat data.csv``::
## time, weight, temperature, humidity, voltage
2016-08-14T22:02:06Z, 58.697, 19.6, 56.1, 4.13
2016-08-14T22:22:06Z, 58.663, 19.4, 58.3, 4.13
2016-08-14T22:42:06Z, 58.601, 19.1, 57.7, 4.12
Send readings::
cat data.csv | http POST $HTTPURI/$CHANNEL/data Content-Type:text/csv
**************
Advanced usage
**************
For more detailed information, please have a look at
- :ref:`How to send telemetry data using MQTT ` and
- :ref:`How to send telemetry data using HTTP `
----
.. todo::
- Add more convenience by adding appropriate Javascript widgets.
*****************************
Client libraries and programs
*****************************
- For a generic Python library, visit :ref:`daq-python`.
- For a generic PHP library, visit :ref:`daq-php`.
- There is also :ref:`beradio-python `, a serial-to-mqtt
forwarder used as gateway for RF telemetry by :ref:`hiveeyes-one`.
.. tip::
For feeding a sawtooth signal right from your shell without having any
hardware in place, take a look at the :ref:`kotori:sawtooth-signal` page.
.. note::
For selecting a client library and communication style
of your choice, we recommend taking a look at the
:ref:`Kotori data acquisition handbook `.