Many big data projects have been developed over the past 15 years, and developments continue today. However, their formalism is different and specific to the implementation. A common formal framework is missing for phrasing analytic queries on an abstract level, independent of the specifics of any system. This paper proposes HIFUN, high-level functional formalism, aiming to study and express analytic queries in a common abstract way that can be encoded and evaluated in a particular environment.

The paper describes the consequent functional theory, algebra, and features of HIFUN. The roles are similar to the relational theory, algebra, and structured query language (SQL) of the relational world, but are much more flexible and adequately able to express analytic queries for the various kinds of data put into existing systems. Attributes are defined as functions. Analytic queries are defined on this functional base. The analysis context is a directed graph showing the correspondence among datasets and query attributes. The introduced operations on functions lead to a functional algebra. The query language over the analysis context is based on functional expressions of this algebra. In fact, the authors describe how to encode HIFUN queries as a MapReduce job and in SQL, and vice versa. This can be applied to “platforms relying on the MapReduce model of computation to evaluate HIFUN queries (for example, Hadoop or Spark).”

For efficiency, rewriting queries and creating hierarchies of partially summed data is an important theme of the paper, and “SQL group-by queries can be abstracted as HIFUN queries.”

Further investigations are needed in the following directions:

The extension/generalization of HIFUN to subsume all features of the relational algebra (renaming, union, and difference, for example);

Defining HIFUN encodings in big data frameworks different from the MapReduce model; and

How the study described can be accommodated to dynamically changing input datasets, for instance, by incrementally taking into account the differences.

The structure of the paper is consistent and the style is clear. The authors demonstrate their intent, and a running example throughout the paper helps introduce the concepts.

I recommend the paper for architects and developers of big data systems, as well as for those who are interested in developing a common abstract language and development tool for big data systems.