Diseño e implementación de una arquitectura MapReduce para aplicaciones Big Data mediante síntesis de alto nivel

Abstract

This Master’s thesis comprises the design and implementation of a hardware platform that integrates critical algorithms of Big Data application processing on a Xilinx Zynq series SoC, improving significantly it’s efficiency in comparison to software implementations. Specifically, a MapReduce platform has been designed using the classic WordCount application to establish a generic architecture that allows for easy IP block substitution to modify the application into another, if it complies with the MapReduce outline. This work starts presenting key aspects of the Big Data concept and techniques for massive data analysis in modern systems. Through this, we cover the MapReduce programming model, for large data processing and generation through parallelization on clusters. Following up, the used Xilinx Zynq Design Methodology is explained, that allows for the design of flexible and configurable platforms, leading finally into the tools used in this thesis, among which we emphasize the Xilinx Vivado Design Suite for SoC FPGA. Lastly, the designed prototype is put through a validation phase to estimate the performance parameters of the system, i.e. latency, throughput and utilization of the MapReduce system. Based on these results, we conclude that the designed platform is a good solution with respect to massive data analysis for Big Data applications, due to its high computing speed, low power consumption and utilization and ultimately due to its flexibility to be adapted for other MapReduce applications of similar characteristics.