Chapter 4 ■ Data transformation
Notice we see the four rows we inserted, and the correct enumerated values have been chosen. I should
also note at this point that there is a data type called enum, which you can use to store the strings for the
values in the table itself. We will see this option in action in Chapter 5 ; however, if you are familiar with using
enum, I encourage you to change the table and trigger shown previously to use the enum instead of a character
string.
Now that you have seen several examples of annotation, let’s discuss a more complex application of
aggregating data from multiple sensors or nodes as well as the type of aggregation operations you may want
to implement in your IOT solutions.
Aggregation
Aggregating data for your IOT solution can be a more complex operation than annotating the data.
Aggregation can take several forms. The easiest form to implement is aggregating data from multiple
sensors. Similar in concept but quite a bit different in implementation is aggregating data from multiple
nodes (other data collectors). Finally, the more complex form of aggregation is performing operations on
sets of data such as statistical or counting functions.
In this section, we explore each of these forms of aggregation from a high level. Since the
implementation of each can be complex, I present general overviews rather than specific code solutions.
That said, we will see some of these aggregation forms implemented in Chapter 8. Thus, the following
sections discuss strategies, practices, and general descriptions of each form.
Data from Multiple Sensors
It is normal for IOT solutions to collect data from multiple sources or sensors. Sometimes the sensors are
used to observe different things (events, objects, and so on), but most times several sources or sensors are
used to observe a single thing or subject. That is, you may have a data collection node in the form of an
Arduino board reading data from several sensors monitoring something. Or perhaps you may have a single
Arduino reading many sensors that monitor several subjects.
The question then is how do you store the data? Should you store the sensor values as a set like you’ve
seen in the earlier examples in this chapter, or should you store the values separately? There are two
approaches: storing data based on sensor timing and storing data based on specific time periods.
Sensor-Driven Data
Data that is generated based on the timing of sensor reads means the data is stored at intervals dictated by
the availability of the data. Further, the data is used (displayed, tabulated or mined for information, and so
on) independently when the data is collected. That is, the time when the data is used has no meaning for
the data.
While it is natural to store a set of sensor values that correspond to a single subject, if the sensors
produce values at different intervals, you may have to consider storing the values one at a time rather than
waiting for all sensors to report. In fact, it is not unusual for different sensors to be read at different intervals.
This may be because of the nature of what you are observing and where the timing of the observation can
help produce knowledge.
For example, consider the plant-monitoring solution. You may want to read temperature once every
hour (or even more frequently) because outdoor temperature can change quickly in some areas in an hour,
or you may want to read temperature every few hours for indoor, controlled environments. However, for
soil moisture, you may want to read the values twice a day since soil moisture may not change as quickly,
especially for controlled climates.