MLE for given HMM and Observation Sequence

Question

We are given an MDP with 3 states: Red, Green, Blue (RGB). 

Emission probabilities:  

R --> 1 (p=1/3), 2 (p=1/3), 3 (p=1/3)

G --> 1 (p=1/3), 2 (p=1/3), 3 (p=1/3)

B --> 1 (p=1/3), 2 (p=1/3), 3 (p=1/3)

Transition Matrix

R --> R (p=1/3), G (p=1/3), B (p=1/3)

G --> G (p=1/2), B (p=1/2)

B --> B (p=1)

These are the observations: 1, 2, 3, 1, 1, 2, 1, 2, 3, 1, 1, 2, 2, 1, 3, 2, 2, 1, 1, 2, 1, 1, 1, 2, 1

What is the most likely explanation for this observation sequence?
Initial state is R.

Answer 1

Since all states have the same emission probabilities, the observations do not change which state path is best.

So we only compare the transition probabilities. As stated, initial state is fixed, so:
s1 = R

From R we can go to R (p=1/3), G (p=1/3), B (p=1/3), which means that all are equal for the first move. But after we reach B, then we will stay in B as B -> B (p = 1). 

Thus, the most likely state sequence is R followed by B for all remaining observations, because moving to B immediately avoids any further transition probability loss. 

So path is: R,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B

The full probability including the observations for this path is P = (1/3)²⁵ * 1/3 = (1/3)²⁶ 
The transitional probability for this path will be 1/3 * 1 * 1 *1 ... * 1 = 1/3

Answer 2

Since all emission probabilities are equal(1/3), the observations do not affect the choice of states. Therefore, we only need to maximize the transition probabilities. 

Starting from R, we compare options:

• Staying in R gives a probability 1/3 at each step
• Going to G gives at most 1/2 per step
• Going to B allows staying in B with probability 1 (since B = 1)

Thus, the optimal strategy is to move to B and remain there, because multiplying by 1 preserves the highest probability.

MLE for this observation:

R B B B B B B B B B B B B B B B B B B B B B B B B B 

Answer 3

We have a system where emissions are completely uninformative: every state (R, G, B) produces 1, 2, or 3 with equal probability $1/3$. This means the observation sequence does not help us distinguish between states at all. So the entire problem is driven only by the transition structure and the initial condition S1​=R.

We should focus on the transition rules, because that is where all the “meaning” is in this model:

• From R, you can go to R, G, or B with equal probability $1/3$. So R is a flexible starting state.
• From G, you either stay in G with probability $1/2$ or move to B with probability $1/2$. Importantly, once you leave G, you can never go back to R.
• From B, you stay in B forever with probability 1. So B is an absorbing “final lock” state.

So the structure is essentially directional: R → G → B, with increasing restriction as you move forward.

Since emissions give no information, the best explanation is the hidden path that has the highest transition probability structure.

That means the system prefers:

• Staying in a state rather than switching unnecessarily
• Using higher-probability self-transitions when available
• Avoiding early entry into the absorbing state B unless it is strongly supported.

The process starts in R (given), remains in R for some time, then eventually transitions into G. Once in G, it tends to stay there for a while due to the relatively strong self-loop probability. After that, it may transition into B, after which it becomes permanently stuck.

Even though the observed numbers look random, the hidden system behaves like a machine that gradually “loses flexibility”:

• R = free exploration stage
• G = semi-stable stage
• B = locked final stage

So the most likely explanation is not driven by the observations, but by the fact that the system naturally drifts forward and becomes more constrained over time.
A path that begins in R, spends a meaningful amount of time in R, then moves into G where it may linger, and eventually transitions into B, after which it remains fixed there.

Answer 4

Because all states have the same emission probabilities:

P(1∣R)=P(1∣G)=P(1∣B)=13

Because and same for observations 2 and 3, the observation sequence does not help distinguish between R, G, B. So, the MLE depends only on the transition probabilities. 

Initial state: q₁ = R , the possible next states are:

 R->R = 1/3,

R->G = 1/3,

R->B = 1/3.

But once we reach B:

B->B = 1

So the best strategy is to move to B as early as possible, because after that there is no transition penalty. Therefore: q₁ =R. Then choose: R->B

After that: B->B->B->B-> .......

For 25 observations, this gives 25 hidden states. Probability:

P = (1/3)²⁵ ⋅ (1/3) = (1/3)²⁶ 

Final answer: R, B, B, B, B, ...

Answer 5

Hi, 

Since all emission probabilities are equal, the observations do not help distinguish states.

So we only maximize the transition probability.

Initial state is R.

Best choice:

R -> B has probability 1/3
B -> B has probability 1 every time after that

So the most likely hidden-state sequence is:

R, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B

Why?

Its transition probability is:

(1/3) * 1 * 1 * … * 1 = 1/3

Any sequence that stays in R or goes through G has smaller probability because it keeps multiplying by 1/3 or 1/2.

Final answer:

R, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B

Answer 6

To find the most likely sequence of states in the observation, let's look at the following three cases:

Case 1: R -> B

First, let's assume the sequence begins with transition from Red to Blue. Then, the probability that the hidden sequence is RBBB....B is (1/3)*1=1/3

Case 2: R -> R

Assuming that we first transition to state R, the total probability that the next is state is any of the three possible values (RGB) is (1/3)*(1/3)=1/9. The likelihood of this case is already smaller than Case 1, thus we can move on.

Case 3: R -> G

Now, assume we start by transitioning to state G. Then the total probability of the next state being either G or B is (1/3)*(1/2)=1/6. Again, this value is already smaller that the probability of Case 1. Therefore, it's safe to say that the most likely sequence hidden behind this observation is RBBB...B, or, in other words, R followed by 24 Bs.

Answer 7

Each state emits 1, 2, or 3 with probability 1/3.  

So the observations do not help us choose between R, G, and B.

The only useful part is the transition matrix.

Initial state is fixed as R.

From R, we can go to:

R with probability 1/3  

G with probability 1/3  

B with probability 1/3  

From G, we can go to:

G with probability 1/2  

B with probability 1/2  

From B, we can only stay in B:

B → B with probability 1

Since B is absorbing, once we enter B, all next transitions have probability 1.

So the best choice is to move from R to B as early as possible.

The observation sequence has length 25, so we need 25 hidden states.

The most likely hidden sequence is:

R B B B B B B B B B B B B B B B B B B B B B B B B

Its probability is:

Emission part = (1/3)^25  

Transition part = (1/3) × 1^23 = 1/3  

So the full probability is:

(1/3)^25 × (1/3) = (1/3)^26

This is better than staying in R, because staying in R adds many transitions with probability 1/3.

So the explanation is to start in R, move to B at the next step, and stay in B until the end.

Answer 8

Since emission probabilities are all equal, all states are equally likely given the observations. However, when we look at transition matrix, we can see that transitioning to B keeps us in B forever. Logically, it is better to go from initial R to B as early as possible , as multiplication by 1 avoids further probability reduction. So the best path is R,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B,B with probability (1/3)^26.

Answer 9

Hello,

Here is my solution according to the given initial state is fixed:

Start = R

Emission probabilities:

For R, G, and B, all observations 1, 2, 3 have probability 1/3.

So the emissions are exactly the same in every state.

That means the observation sequence does not help us distinguish between states.

It only contributes the same factor (1/3) at each step no matter which state we choose.

So to find the most likely explanation, we only need to maximize the transition probability.

Transition rules:

From R:
R -> R = 1/3
R -> G = 1/3
R -> B = 1/3

From G:
G -> G = 1/2
G -> B = 1/2

From B:
B -> B = 1

Important observation:

B is absorbing, because once we enter B, we stay in B with probability 1.

Since 1 is larger than 1/2 and 1/3, the best strategy is to reach B as early as possible.

We start at R.

At the next step, the best choice is:

R -> B with probability 1/3

After that:

B -> B -> B -> ... -> B

all with probability 1

So the most likely hidden-state sequence is:

R, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B

There are 25 observations, so there are 25 states in the sequence.

Step 1 is R, and steps 2 to 25 are all B.

Why this is best:

If we go from R to G first, then we still need another step to possibly reach B, so the path gets an extra factor 1/2.

Example:

R -> G -> B gives transition probability

(1/3)(1/2) = 1/6

which is smaller than direct

R -> B = 1/3

If we stay in R first:

R -> R -> B gives

(1/3)(1/3) = 1/9

which is also smaller.

So entering B immediately is optimal.

Final answer:

The most likely explanation is:

R, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B

Because emissions are identical in all states, the observations do not affect the choice.

The best path is simply the one with the highest transition probability, and that is to move to B as soon as possible and stay there.

Answer 10

Hello Professor,

The observation sequence is:

1, 2, 3, 1, 1, 2, 1, 2, 3, 1, 1, 2, 2, 1, 3, 2, 2, 1, 1, 2, 1, 1, 1, 2, 1

There are 25 observations, so we need 25 hidden states.

The initial state is fixed as R.

The emission probabilities are the same for every state:

R emits 1, 2, and 3 with probability 1/3 each.

G emits 1, 2, and 3 with probability 1/3 each.

B emits 1, 2, and 3 with probability 1/3 each.

Since all states have the same emission probabilities, the observations do not help us choose between R, G, and B. No matter what hidden-state sequence we choose, the emission part of the probability is the same.

So we only need to maximize the transition probability.

The transition rules are:

From R:

R to R = 1/3
R to G = 1/3
R to B = 1/3

From G:

G to G = 1/2
G to B = 1/2

From B:

B to B = 1

Since B is absorbing, once we enter B, we stay in B with probability 1. That means the best choice is to move from R to B as early as possible.

We start at R, so the first hidden state is R.

For the second hidden state, choose B:

R to B = 1/3

After that, every remaining transition is:

B to B = 1

So the most likely hidden-state sequence is:

R, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B

The transition probability for this path is:

(1/3)(1)(1)(1)…(1) = 1/3

The emission probability is:

(1/3)^25

So the full probability is:

(1/3)^25 * (1/3) = (1/3)^26

Any path that stays in R or goes through G has a smaller transition probability, because it keeps multiplying by 1/3 or 1/2.

Final answer:

R, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B

So the most likely explanation is to start in R, move to B immediately, and then stay in B for the rest of the sequence