PGM-Introduction

Probabilistic Graphical Models

Specifying a joint distribution over many variables is intractable, especially when some random variable has many states, even continuous(of course it can be continuous). Therefore we use PGM which decomposes a complex distribution into smaller structures. For example, the following figure shows one possible graph for medical diagnosis.

There are 2 perspectives to interpret the graph. One is the graph is a representation of a set of independencis, one is the graph is a skeleton that breaks up the distribution into smaller factors, each of which has a smaller space of possibilities, as shown in the figure above. It turns out that the 2 perspectives are equivalent. By factoring the joint distribution, much less parameters are needed to specify the distribution. For example, assume each variable $F$, $H$, $M$, $C$ takes states of yes/no, and $S$ takes 4 states of spring, summer, fall, winter, then the joint distribution needs $2\times 2\times 2\times 2\times 4-1=63$ nonredundant parameters to be specified(because the sum over all entries muct sum to 1, so when 63 entries are determined, the rest one is fixed), while after factorization, the required number of parameters is 3+4+4+4+2=17 for $P(S),~P(F|S),~P(H|S)$, $P(C|H,F),~P(M|F)$ respectively.

3 components-representation, inference and learning-are critical components in constructing an intelligent system. PGM did all of this 3 perspectives. It declares a graph-based representation that encodes our world. It use the representation to answer queries like $P(F|S,M)$(Inference). It can be learned by combining expert knowledge(like som main dependencies) and accumulated data.

Overview and Roadmap

Chapter 3	Bayesian Network Representation
Chapter 4	Markov Network and its unification with Bayesian Network, Conditional Random Fields
Chapter 5	Deeper into the representation of the parameters in PGM
Chapter 6	PGM evolving with time
Chapter 7	Look into models that have continuous variables
Chapter 8	Exponential Family
Chapter 9	Exact Inference(Computationally Intractable)
Chapter 10	Alternative view of Exact Inference
Chapter 11	Approximate Inference(Less cost compared with Exact Inference)
Chapter 12	A very different approximate inference method: Particle-based method
Chapter 13
Chapter 14	Inference in continuous and hybrid (continuous/discrete) networks
Chapter 15	Special-purpose methods for the particular settings of networks that model dynamical systems.
Chapter 16	Fundamental concepts underlying the general task of learning models from data
Chapter 17	Learning parameters for a Bayesian network with a given structure, from fully observable data
Chapter 18	The harder problem of learning both Bayesian network structure and the parameters, still from fully observed data
Chapter 19	Bayesian network learning task in a setting where we have access only to partial observations of the relevant variables
Chapter 20	Learning Markov networks from data, which is significantly harder than the corresponding problem for Bayesian networks
Chapter 21	Causal model
Chapter 22	Utility functions
Chapter 23	Influence diagrams which extend Bayesian networks by introducing actions and utilities

A reader's guide