Newsletter optimization using HMM

Question

You are trying to sell a product by using an email newsletter (delivered weekly).  A visitor can be interested in the product due to 2 different interests.  If they are interested due to interest 1 (for example “soccer”), they may click a link you have in the newsletter.  If they are interested due to interest 2 (for example “tennis”), they may click a different link.  Here is the probability table that shows the likelihood of viewer clicking these two links:

Interest	Link 1	Link 2
No Interest	0.01	0.05
Interest 1	0.3	0.6
Interest 2	0.4	0.8
Interest 1 and Interest 2	0.8	0.95

We do not know the initial state of the visitor.  If the visitor has no interest in interest 1 or interest 2, she a 10% probability of developing either of those interests (independently) during the week. When she has one of the interests, they have a 20% probability of developing the other interest as well.  Once an interest is developed, that interest is never lost.

1. Draw and explain an HMM to model this problem.
2. You make the following observations in 8 weeks of newsletter delivers:  

No Click, Link 1, Link 1, No Click, Link 1, Link 1 and Link 2, Link 1 and Link 2, Link 2, Link 1 and Link What is the best explanation for the interest level of the visitor during those 8 weeks? 

Comments

How should the Markov state be drawn for the "No interests" state? For "Interest 1", I see that there is a 0.2 probability of moving to the "Both" state and a 0.8 probability of moving to the "Both" state. However, for the "No interest" state, since developing either interest is independent of each other, does it stay in "No interest" with 0.9 probability with the 0.10 probability divided between the two states? Alternatively, should we just draw the graph to go to "Interest 1" by default with 0.10 probability?

Answer 1

Here is my solution... it looks like there is a 12000 character limit to post so I have to convert it to pictures. Here is the link to the solution google sheet.

Google Sheet for Solution