Higher levels of physical activity are known to benefit aspects of brain health across the lifespan. However, the role of sedentary behavior (SB) is less well understood. In this review we summarize and discuss evidence on the role of SB on brain health (including cognitive performance, structural or functional brain measures, and dementia risk) for different age groups, critically compare assessment approaches to capture SB, and offer insights into emerging opportunities to assess SB via digital technologies. Across the lifespan, specific characteristics of SB (particularly whether they are cognitively active or cognitively passive) potentially act as moderators influencing the associations between SB and specific brain health outcomes. We outline challenges and opportunities for future research aiming to provide more robust empirical evidence on these observations.