142 W.P. Worzel
2.2.3 Conceptual Data Flow Design
Data Flow Design is usually used as a documentation tool that explicates the flow
of data and processing within a system, often within a database. However, it can
be used to describe the flow of data in a distributed application as it describes how
and when data is distributed, what processing is done in parallel and how the data is
joined after processing.
An example of a Conceptual Data Flow Design diagram is shown in Fig. 1.
Here the flow of data is shown by the “pipes” connecting computational actions
and the tagged nodes show the processing of relations. Each of the blue ‘M’ labels
denote a map process and the red ‘R’ labels denote a reduce operation. While this
is a graphical diagram, the same information can be rendered in a table format in
a concise form that is easy to process. In Fog Lifter these design elements can be
connected from sources through processors to sinks where the results are stored.
Implicit in this is that a result from one analysis may be made available through the
Registry for other analyses (Fig. 3 ).
2.2.4 Security and Privacy
Security and privacy issues are already a major topic in the emerging world of the
IoT/ IoE. A key assumption in Fog Lifter is that sharing of data and computing
resources must be an act of permission, not one of omission. To implement this,
a transactional approach via blockchain technology is used. The use of data, or
processors must either be certified by a transaction permitting the use to a specific
entity or paid for by that entity and the owners control whether they are willing to
share with any given user. Details about such approaches may be found in Wood
( 2015 ) and Brody and Pureswaran (2015).
Fig. 3 Data workflow design (from Nathan 2013 )