A healthy lifestyle with sufficient physical activity (PA) can contribute to weight management. Yet, many people do not maintain a healthy lifestyle. To explain PA, we propose a model that incorporates the Theory of Planned Behavior (TPB) with weight-related self-stigma. We recruited 325 young adults to complete questionnaires regarding their physical activities, weight-related self-stigma, and TPB factors. We used structural equation modeling to examine the model fit and the path invariance across weight groups. The model showed excellent model fit, but path invariance was not supported. Weight-related self-stigma significantly explained the perceived behavioral control, behavioral intention, and engagement of PA. People without overweight and people with overweight have different considerations for PA. Weight-related self-stigma is important for PA as well. To promote a healthy lifestyle, healthcare providers should provide different suggestions or interventions that suit their patients' weight-related concerns.