Stav dette: Markov chains: models, algorithms and applications