A speaker's emotional state is conveyed by acoustic and prosodic factors, as well as the words they choose and the gestures they use. We are studying several different contexts in which emotional state is important to determine: 1) an automatic tutoring system, in which students studying physics may be confident or uncertain, frustrated, or angry, and should receive appropriate handling for that state; 2) speech in varied public settings, where speakers may be perceived as charismatic or not, providing some indication of the likely success of speakers' attempts to gain political power; and 3) recorded interviews in which speakers may be telling the truth or not. In each case, our focus is on identifying prosodic and acoustic as well as lexical cues to these different speaker states, so that we may develop systems which automatically distinguish between, e.g., confidence and uncertainty, frustration and satisfaction, charimatic and non-charismatic speech, and deceptive and non-deceptive speech. In this talk we present current results from these studies.