[Bayesian] Bayesian Deep Learning - Measure theory

본 자료는 edwith 최성준님이 강의하신 Bayesian Deep Learning 강의를 참고하였다. 

 

핵심 키워드

$Measure theory$, $Measure$ , $Set\ function$, $Sigma\ field$, $Measurable\ space$

 

Measure Theory

---

($e.g.$ 몸무게, 나이 등 )

 

set function : a function assigning a number of a set ( $e.g.$ cardinality , length , area )

    $set$을 2차원 공간이라고 친다면, 그 공간 사이에서 원을 그렸을 경우 원 안의 면적을 재는 것.

 

 

$\sigma$-field $\mathcal{B}$ : a collection of subsetsof $U$ such that ( $axioms$ )

    1. $\emptyset \in \mathcal{B} $ ( empty set is included. )

    2. $B \in \mathcal{B} \Rightarrow B^{c} \in \mathcal{B} $ ( closed under set complement. )

    3. $B_{i} \in \mathcal{B} \Rightarrow \cup^{\infty}_{i=1} B_{i} \in \mathcal{B} $ ( closed under countable union. )

 

 

 

properties of $\sigma$-field $\mathcal{B}$

---

   1. $U \in \mathcal{B}$ ( entire set is included. )

 

   2. $B_{i} \in \mathcal{B} \Rightarrow \cap^{\infty}_{i=1} \in \mathcal{B} $ ( closed under countable intersection )

 

   3. $2^{U}$ is a $\sigma$-field.

 

   4. $\mathcal{B}$ is either finite or uncountable, never denumerable.

 

   5. $\mathcal{B}$ and $\mathcal{C}$ are $\sigma$-fields $\Rightarrow \mathcal{B} \cup \mathcal{C}$ is a $\sigma$-field but $\mathcal{B} \cup \mathcal{C} $ is not.

        $\cdot\ \mathcal{B}= \{ \emptyset, \{a\}, \{b,c\}, \{a, b, c \}\} $

        $\cdot\ \mathcal{C}= \{ \emptyset, \{a,b\}, \{c\}, \{a, b, c \}\} $

        $\cdot\ \mathcal{B} \cap \mathcal{C} = \{ \emptyset, \{a, b, c \}\}$ ( this is $\sigma$-field ) 

        $\cdot\ \mathcal{B} \cup \mathcal{C} = \{ \emptyset, \{a\},\{c\}, \{a, b\}, \{b,c\} \{a, b, c\}\} $

            (this is not a $\sigma$-field as $\{a, c \} = \{a\} \cap \{c\} $ is not included. )

        $\cdot\ \sigma(\mathcal{C})$ is called the $\sigma$-field generated by $\mathcal{C}$

 

 

$Bayesian$ 을 설명하는데 왜 $set\ Theory$와 $measure\ Theory$를 설명하는가?

measure 가 probability 이기 때문. 어떠한 element가 $\sigma$-field 안에 있지 않는다면 measure가 될 수 없다. $\sigma$-field 가 정의되어 있지 않는다면 $set$ 자체는 존재하지만 그렇게 만들어진 $\sigma$-field안에 각각의 element들이 정의되어 있지 않기 때문에 element들의 measure은 알수가 없다.

단, 정의되어있지 않다고 해서 0은 아니며 안되는 것일뿐이다.

 

 

$\cdot\ $ A $set\ U$ and a $\sigma$-field of subsets of  $U$  form a measurable space $(U,\mathcal{B})$.

 

$\cdot\ $ A measure $\mu$ defined on a measurable space $(U,\mathcal{B})$ is a set func. $\mu : \mathcal{B} \rightarrow [0,\infty]$ such that

      1. $\mu(\emptyset)$ = 0

      2. For disjoint $B_{i}$ and $B_{j} \Rightarrow \mu(\cup^{\infty}_{i=1}B_{i}) = \Sigma^{\infty}_{i=1}\mu(B_{i})$ (countable additivity)

 

$\cdot\ $Probability is a measure such that $\mu(U)$ = 1, $i.e.$, normalized measure.

 

$\cdot\ $A measurable space $(U,\mathcal{B})$ and a measure $\mu$ defined on it together form a measure space

  $(U,\mathcal{B}, \mu)$