Plot of the described mechanisms on the Big Data Blueprint

Selecting data sources on information content and data qualities

Analysing the cleaned, aggregated and filtered datasets

Data governance activities and application functions layers for maintaining qualities of the datasets

Processing activities for filtering, cleaning and transformation of datasets

Application functions and activities for securing datasets and storage functionalities

Not part of a data architecture but the input of the architecture, it is the raw material of a data pipe and therefore an essential part

Storage of primary and temporary datasets for usage in higher layers of the architecture

Usage of the processed data in work processes in the consumer organisation

Visualisation of the data or information for interpretation by analists and users

Functionality for analysing the transformed and standardized data in a data analytics tool.

Management of clusters in a distributed or cloud based infrastructure

Big data sets can be voluminous so compressing has advantages for storage and/or processing

Coordination of automated tasks for example in batch processes and mapreduce.

Register function for the governance of data sets in relation to consumers, business owners and qualities.

Transformation and transfer functionality from data sources to storage, processing and usage.

Processing of data for transformation, cleaning and filtering of data

Portal functionality to give users and administrator access to the other logical functionalities in a Big Data environment.

Retrieval and querying function on the transformed and standardized datasets

Management function for the various resources in a Big Data environment

Administration and registration of security concepts in a Big Data environment.

Serialization and deserialization of proprietary datasets to standardized stored datasets

Storage function of big data sets.

Visualisation of query and analytics results to data scientists and end users.

Functionality to design workflows to automatically perform transformations and cleaning of datasets.