[Bayesian] Bayesian Deep Learning - Measure theory

본 자료는 edwith 최성준님이 강의하신 Bayesian Deep Learning 강의를 참고하였다. 

 

핵심 키워드

$Measuretheory$, $Measure$ , $Setfunction$, $Sigmafield$, $Measurablespace$

 

Measure Theory

---

($e.g.$ 몸무게, 나이 등 )

 

set function : a function assigning a number of a set ( $e.g.$ cardinality , length , area )

    $set$을 2차원 공간이라고 친다면, 그 공간 사이에서 원을 그렸을 경우 원 안의 면적을 재는 것.

 

 

$\sigma$-field $\mathcal{B}$ : a collection of subsetsof $U$ such that ( $axioms$ )

    1. $\mathrm{\varnothing }\in \mathcal{B}$ ( empty set is included. )

    2. $B\in \mathcal{B}\Rightarrow B^c\in \mathcal{B}$ ( closed under set complement. )

    3. $B_i\in \mathcal{B}\Rightarrow {\cup }_{i=1}^{\mathrm{\infty }}{B}_i\in \mathcal{B}$ ( closed under countable union. )

 

 

 

properties of $\sigma$-field $\mathcal{B}$

---

   1. $U\in \mathcal{B}$ ( entire set is included. )

 

   2. $B_i\in \mathcal{B}\Rightarrow {\cap }_{i=1}^{\mathrm{\infty }}\in \mathcal{B}$ ( closed under countable intersection )

 

   3. $2^U$ is a $\sigma$-field.

 

   4. $\mathcal{B}$ is either finite or uncountable, never denumerable.

 

   5. $\mathcal{B}$ and $\mathcal{C}$ are $\sigma$-fields $\Rightarrow \mathcal{B}\cup \mathcal{C}$ is a $\sigma$-field but $\mathcal{B}\cup \mathcal{C}$ is not.

        $\cdot \mathcal{B}=\{\mathrm{\varnothing },\{a\},\{b,c\},\{a,b,c\}\}$

        $\cdot \mathcal{C}=\{\mathrm{\varnothing },\{a,b\},\{c\},\{a,b,c\}\}$

        $\cdot \mathcal{B}\cap \mathcal{C}=\{\mathrm{\varnothing },\{a,b,c\}\}$ ( this is $\sigma$-field ) 

        $\cdot \mathcal{B}\cup \mathcal{C}=\{\mathrm{\varnothing },\{a\},\{c\},\{a,b\},\{b,c\}\{a,b,c\}\}$

            (this is not a $\sigma$-field as $\{a,c\}=\{a\}\cap \{c\}$ is not included. )

        $\cdot \sigma (\mathcal{C})$ is called the $\sigma$-field generated by $\mathcal{C}$

 

 

$Bayesian$ 을 설명하는데 왜 $setTheory$와 $measureTheory$를 설명하는가?

measure 가 probability 이기 때문. 어떠한 element가 $\sigma$-field 안에 있지 않는다면 measure가 될 수 없다. $\sigma$-field 가 정의되어 있지 않는다면 $set$ 자체는 존재하지만 그렇게 만들어진 $\sigma$-field안에 각각의 element들이 정의되어 있지 않기 때문에 element들의 measure은 알수가 없다.

단, 정의되어있지 않다고 해서 0은 아니며 안되는 것일뿐이다.

 

 

$\cdot$ A $setU$ and a $\sigma$-field of subsets of  $U$  form a measurable space $(U,\mathcal{B})$.

 

$\cdot$ A measure $\mu$ defined on a measurable space $(U,\mathcal{B})$ is a set func. $\mu :\mathcal{B}\to [0,\mathrm{\infty }]$ such that

      1. $\mu (\mathrm{\varnothing })$ = 0

      2. For disjoint $B_i$ and $B_j\Rightarrow \mu ({\cup }_{i=1}^{\mathrm{\infty }}{B}_i)={\mathrm{\Sigma }}_{i=1}^{\mathrm{\infty }}\mu (B_i)$ (countable additivity)

 

$\cdot$Probability is a measure such that $\mu (U)$ = 1, $i.e.$, normalized measure.

 

$\cdot$A measurable space $(U,\mathcal{B})$ and a measure $\mu$ defined on it together form a measure space

  $(U,\mathcal{B},\mu )$