Intro
Admin Mid-Term Evaluation open remember to come by office hours once Assignment #2 graded Third Order Model, each state an ASCII character : TEST DATA:
THE WHY DO YOU NOT GODFATHER? AND WE ARENOT ON THE SIDE!
WOUNDS ARE NOT THOSE YET KEPT THE YOU ADD BIRD WIN
MAX: I SEE YOU REMEMBER HOW IT WAS TAKEN OUT. I'M HEARD IN
SPEAK WITH HIM IN THOSE CLOTHES IN THAT GROUND DOWN
LAND OUT A PATRIOT. RAIN IS FALLING
First Order Model, each state a word. TEST DATA:
HORRIBLE SOUND THAT MUST BUZZ IN JOEY: NO PAUL. I ABOUT NOT.
WONDER IN THE CUT AREN'T NIGHTLY IN SOMETIME.
'EM YAKKIN'. IF SUCH SHIT RID WANT GODDAMN CAMEL?
YOU'D AFFORD SHOES MATTER CAN'T HEAR. IF BEER CAME TINGLY.
NOW? STRAIGHT, IN PI REPRESENTED BY MARKET./
First-Order Model, Single Word, Full Corpus, No Smoothing LET ME UP, HUH?
TED: IT'S DIFFERENT. YOU'RE NOT YOUR RIGHT THERE. TRACY ENID FLICK.
TRACY: I GOT ANY CAVALRY! IN IT. AND A HE WAS. I SAW HER VOICE, AS FAR
I CAN READ ABOUT OUR OLD FRIEND.
KAY: WHO PUT IT IN THE OTHER KIDS LIKE HE'S DIABETIC, HE'S GOT THE FLEA
MARKET WOMEN: WALLACE, SO IT'S REALLY CHAPS MY TOP FOR GOD MAKE A
WALL, FOURTH BY ME TOO.
THANK YOU. TOM? HAGEN: WHAT ARE THOSE STUDENTS - LIKE HORSES, AND, GOD
WILLING, WITH MY YEARBOOK?
Pattern ~ Words Markov Order ~ 1
Smoothed ~ false
Lines ~ 5
Started from random given
Trained on the Terminator
ANCHORWOMAN:NO, MUST AT AND ANYTHING.
TERMINATOR:FREEZING.
SARAH:YES.EVERYTHING.?
TRAXLER:DIDN'T FOR BUT SEEN AND GET LIKE OR FALAFEL THESE BE THEM?
SARAH.I KID MONICA WE'T ON!
FirstFirstFirstFirst Order Model Order Model Order Model Order Model (Words one State) (Words one State) (Words one State) (Words one State):::: 1. Single Script “Friends4”
DOWN NOW.WELL,ALAN.THERE'S GETTINGNO.THEY THAN THE
MILLNERS,PERSONALLY COULD CHASE AND COMPLAININGVIKING.
MONICA:HI.
ALL:MMM.
ROSS:SO HOW'D IT OUT.
PHOEBE:OH,NO!I-"SOMETIMES YOU!
ROSS:WHO-WHOWANTS FAIR? YOURSELF?
CHANDLER:HEY,YOURSELF?
CHANDLER:HEY,YOURSELF?
Dictionary file posted Use it or not. Recap Defining an HMM HMM is characterized by N, number of states in the model S, {S_1,S_2,S_3} state at time t is q_t M the number of distinct observation symbols per state V = {v_1,V_2,V_3} State transition probability A={a_ij} a_ij = P(q_t+1 = S_j | q_t = S_i) 1<= i, j <= N if all states can reach all states then a_ij > 0 for all i,m The Observation symbol probability distribution in state j B = {b_j(k)} b_j(k) = P(v_k at t | q_t = S_j) 1<= j <=N , 1<=k <=M The initial state distribution pi = {pi_i} pi_i = P(q_1=S_i) 1<= i <=N Example of how to use it generatively Model can be completely determined by M, N, A, B, and pi New Material Motivate with a house and motion sensor example 3 problems from Rabiner The 3 basic HMM problems |
Pasted Graphic
|
Problem 1 is the evaluation problem It is also considered the scoring problem Choosing among multiple models Problem 2 is an attempt to uncover the hidden variable or find the correct state sequence "correct" is not accurate -> some optimality criterion there are several possible Problem 3 is the learning problem "training" the HMM based on some observed data Solving Problem #1 |
Pasted Graphic
|
|
Pasted Graphic
|
|
Pasted Graphic
|
|
Pasted Graphic
|
|
Pasted Graphic
|
A more tractable version is called the Forward-Backward procedure |
Pasted Graphic
|
|
Pasted Graphic
|
|
Pasted Graphic
|
|
Pasted Graphic
|
|
Pasted Graphic
|
|
Pasted Graphic
|
So this only describes the "forward variable" It is sufficient for problem 1 but we will want to use a backward variable for the other problems |
Pasted Graphic
|
|
Pasted Graphic
|
|
Pasted Graphic
|
|
Pasted Graphic
|