The observation percentages from the HMM seem low because the model has to account for all possible combinations. Even with only eight words, the number of sentences, many of which may not be grammatically correct, is huge. That is what this probability tells you: the likelihood of a given sequence out of all possible sequences. Also, some of the probabilities provided in the .hmm file seem pretty arbitrary. That is why the HMM does not always display realistic results and must be optimized to make it worthwhile. For example:

(“robots do kids play chess”) = 
   .1512% even though it is grammatically incorrect

P(“chess eat play kids”) = 0% 

Running the Code: (in windows)

C:\[your active directory] recognize sentence.hmm example1.obs
0.027
0.0288
0.0 

Each line reports P(O | lambda) for that HMM

The statepath helps to determine the meaning of the sentence. For example, if the sentence begins with an auxiliary, it is almost certainly a question, and if an auxiliary comes before a predicate, it is probably a tense modifier. However, one of the pitfalls is that the English language does not rely on auxiliaries alone to make questions. Often a question can only be determined by taking the context into account.

Running the Code: (in windows)

C:\[your active directory] statepath sentence.hmm example1.obs
0.027 SUBJECT PREDICATE OBJECT
0.0288 SUBJECT PREDICATE OBJECT
0 

Each line again corresponds to a data set: In this case, the program outputs the "optimal" state path, i.e., the path with the highest probability P(O, I | lambda) and, if the probability is greater than zero, the state sequence. For example, the above code indicates that the probability of the first optimal path is 2.7%, and the tokens associated with the optimal state sequence is "SUBJECT", "PREDICATE", "OBJECT". For the third input, since the probability of the obvervation is 0.0, no optimal path can be computed.

Optimizing the HMM for a zero probability observation would result in zero change because the optimization depends on existing values of state transitions. Therefore you must always optimize for a sentence that has P(“sentence”)>0

Running the Code: (in windows)

C:\[your active directory] optimize sentence.hmm example2.obs sentence-opti.hmm
0.000588 0.037037 

For each data set, optimize prints out P(O | lambda) for the old HMM before optimization, and P(O | lambda) for the new HMM after optimization.

To make an HMM that can also parse adverbs, you must add an adverb state. Since in this case, the sample adverbs only come after the predicate and always end the sentence, it acts as an end state for the sentence. The present tense, however is a subset of predicate, increasing the probability of a predicate after an auxiliary will optimize the HMM for recognizing present tense verbs. A sample .hmm file might be:

4 8 4

SUBJECT AUXILIARY PREDICATE OBJECT ADVERB

kids robots do can play eat chess food well fast

a:

0.0 0.4 0.6 0.0 0.0

0.7 0.0 0.3 0.0 0.0

0.0 0.0 0.0 0.7 0.3

0.0 0.0 0.0 1.0 0.0

0.0 0.0 0.0 0.0 0.0

b:

0.5 0.4 0.0 0.0 0.0 0.0 0.05 0.05 0.0 0.0

0.0 0.0 0.5 0.5 0.0 0.0 0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0 0.5 0.5 0.0 0.0 0.0 0.0

0.1 0.2 0.0 0.0 0.0 0.0 0.3 0.4 0.0 0.0

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.5

pi:

0.6 0.3 0.1 0.0 0.0 

All Files

• Source Code

Executables

• recognize.exe
• statepath.exe
• optimize.exe

.HMM's and .OBS Files

• sentence.hmm
• example1.obs
• example2.obs

Source Files

• Hidden Markov Model Class
• Observations Class
• recognize.cpp
• statepath.cpp
• optimize.cpp