Study Programmes 2017-2018
WARNING : 2016-2017 version of the course specifications
Introduction to stochastic processes, Markov chains
Duration :
20h Th, 10h Pr, 10h Mon. WS
Number of credits :
Bachelor in computer science5
Master in data science (120 ECTS)5
Master of science in computer science and engineering (120 ECTS)5
Bachelor in mathematics4
Lecturer :
Céline Esser, Pierre Geurts
Coordinator :
Pierre Geurts
Language(s) of instruction :
French language
Organisation and examination :
Teaching in the second semester
Units courses prerequisite and corequisite :
Prerequisite or corequisite units are presented within each program
Learning unit contents :
Markov chains in discrete time (definition, classification of states, absorption time, strong Markov property, recurrence and transience, invariant distributions, convergence to equilibrium). Markov chains in continuous time (Q-matrices and exponential, Poisson process, life and death processes, properties of Markov chains in continuous time, classification of states, recurrence and transience, invariant distribution, convergence to equilibrium ).  Queues (Kendall notation, occupancy rates, performance metrics, file M / M / m). Other applications (Markov Chain Monte Carlo, Hidden Markov Models)

Learning outcomes of the learning unit :
After the course, students will master the main properties of most classical stochastic processes.
Prerequisite knowledge and skills :
Basic probability theory. Elementary notions of calculus and linear algebra. Understanding of R and/or Matlab.
Planned learning activities and teaching methods :
In addition to the traditional classroom course, the course includes 10 hours traditional exercise sessions (10h Pr,  ex cathedra).
Students from Montefiore will also have 30 hours of personal research work (30h TD). This work will be carried out in groups, in ways yet to be determined (responsible : Prof. Pierre Geurts)
Mode of delivery (face-to-face ; distance-learning) :
Recommended or required readings :
Partial course notes (including exercise sets) will be made available through MyULg. 
Bibliography - Norris, James R. (1998). Markov chains. Cambridge University Press. - Ross, Sheldon (2006). Introduction to probability models. Academic Press.
Assessment methods and criteria :
Montefiore students : the final grade will be a weighted average of two grades : 
- that obtained after a written exam held in June  (concerning both theory and exercises) 
- the grade obtained after evaluation of a project
Work placement(s) :
Organizational remarks :
Contacts :
Yvik Swan Université de Liège Département de Mathématique,   Grande Traverse, 12, Sart Tilman, B-4000 Liège