Our research focuses on developing human-aware cognitive models for robots in order to provide adaptive personalized assistance to humans in work environments or at homes. At the low-level of such models, we focus on robot cognition abilities, such as human presence detection, hand and head gesture recognition, attention detection and action recognition in the context. Whereas at the high-level, these low-level signals are used to anticipate human mental states and intentions to better understand their needs for more reliable, natural and adaptive collaboration. This is done through our novel stochastic decision-making and policy selection algorithms, adapting to a human’s changing contextual behaviors, preferences, and characteristics like their collaborated task skills.