At the end of the course, the students will have an overview of the challenges associated with storing and processing data at large scale. They will know how to use Big Data software tools to efficiently store and process large amounts of data, including tools that can operate in realtime.
The ability to process large amount of data is key to both industry and research today. As computing systems are getting larger, they generate more data that need to be analyzed to extract knowledge.
Data management infrastructures are growing fast, leading to the creation of large data centers and federations of data centers. Suitable software infrastructures should be used to store and process data in this context. Big Data software systems are build to take advantage of large set of distributed resources to efficiently process massive amounts of data while being able to cope with failures that are frequent at such a scale.
In addition to the amount of data to be processed, the other main challenge that such Big Data systems need to deal with is time. For some use cases, the earlier the results of a data analysis is obtained, the more valuable the result is. Some Big Data systems especially target stream processing where data are processed in realtime.
Through lectures and practical sessions, this course provides an overview of the software systems that are used to store and process data at large scale. The following topics will be covered:
Fundamentals of DBMS, concurrent programming (threads)
P = Presentation of research papers and/or graded labs
E = Written exam
N1 = 30% P + 70% E1
N2 = E2
The exam is given in english only
The course exists in the following branches:
Course ID : WMM9MO72
You can find this course among all other courses.
Dean, Jeffrey, and Sanjay Ghemawat. "MapReduce: simplified data processing on large clusters." Communications of the ACM 51.1 (2008): 107-113.
Zaharia, Matei, et al. "Apache spark: a unified engine for big data processing." Communications of the ACM 59.11 (2016): 56-65.
Murray, Derek G., et al. "Naiad: a timely dataflow system." Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles. ACM, 2013.
Lakshman, Avinash, and Prashant Malik. "Cassandra: a decentralized structured storage system." ACM SIGOPS Operating Systems Review 44.2 (2010): 35-40.
Date of update September 21, 2022