The COVID-19 pandemic placed significant psychological strain on healthcare workers. Our study tracked health care workers in the Czech Republic throughout the pandemic to examine the impact of stressors on psychological distress over time and explore gender differences in these associations. We studied health care workers from the Czech arm of the international COVID-19 HEROES Study who took part in all three waves of data collection in 2020, 2021 and 2022 (n = 264). We employed a path model to examine the relationships among six stressors (low trust in workplace, assignment of new tasks, prioritization of patients, experience of death due to COVID-19, experience of discrimination or violence, contact with COVID-19 patients), psychological distress (measured by General Health Questionnaire), and covariates (age, gender, occupation). We incorporated autoregressive paths and interactions to assess the longitudinal impact of stressors. Prior levels of distress predicted subsequent distress, with significant carry-over effects observed between each wave. Stressors had direct association to distress in the same wave of data collection (2020 and 2021), but not in subsequent waves, and stressors in 2022 did not have a significant direct effect to distress. Men reported lower distress in 2020 and 2021, but not in 2022. Furthermore, no significant interactions between stressors and gender were found. Age and occupation were not related to distress levels. There is a need for adaptable mental health support that addresses current stressors through targeted interventions, while also providing ongoing monitoring beyond crises, particularly for individuals with high distress levels.