MATH2707: Markov Chains II

• [Calculus I (Maths Hons) (MATH1081) OR Calculus I (MATH1061)] AND [Linear Algebra I (Maths Hons) (MATH1091) OR Linear Algebra I (MATH1071)] AND Probability I (MATH1597) AND Analysis I (MATH1051) [the latter may be a co-requisite].

• Analysis I (MATH1051) unless taken before.

• None

• To introduce and develop the concept of a Markov chain, as a fundamental type of stochastic process, and to study key features of Markov models using probabilistic tools such as generating functions.

• Markov property
• Stationary distributions
• Classification of states
• Hitting probabilities and expected hitting times
• Convergence to equilibrium
• Applications to random walks
• Generating functions
• Further topics to be chosen from: Gibbs sampler, mixing and card shuffling, stochastic epidemics, discrete renewal theory, Markov decision processes

Subject-specific Knowledge:

• By the end of this module students will:
• Be able to solve seen and unseen problems involving Markov chains.
• Have a knowledge and understanding of this subject demonstrated through an ability to identify stationary distributions, classify states, and compute hitting probabilities and expected hitting times, and a working knowledge of generating functions and their computational and theoretical power.
• Reproduce theoretical mathematics concerning Markov chains to a level appropriate to Level 2, including key definitions and theorems.

Subject-specific Skills:

• Students will have enhanced mathematical skills in the following areas: intuition for key features of probabilistic systems that evolve in time.

Key Skills:

• Students will have basic mathematical skills in the following areas: problem solving, modelling, computation.

• Lectures demonstrate what is required to be learned and the application of the theory to practical examples.
• Problem classes show how to solve example problems in an ideal way, revealing also the thought processes behind such solutions.
• Tutorials provide active problem-solving engagement and immediate feedback to the learning process.
• Formative assessments provide feedback to guide students in the correct development of their knowledge and skills in preparation for the summative assessment.
• The end-of-year examination assesses the knowledge acquired and the ability to solve predictable and unpredictable problems.

If you have a question about Durham's modular degree programmes, please visit our FAQ webpages, Help page or our glossary of terms. If you have a question about modular programmes that is not covered by the FAQ, or a query about the on-line Undergraduate Module Handbook, please contact us.

Prospective Students: If you have a query about a specific module or degree programme, please Ask Us.

Current Students: Please contact your department.