This thesis book brings together the research findings from four interconnected papers, each contributing to the field of statistical modeling and optimization. The central theme focuses on developing and comparing estimation strategies, optimization algorithms, gamma regression models, and estimation methods for high-dimensional data, showing a coherent progression of ideas and methodologies. The first paper focuses on the comparison of different optimization algorithms in constructing approximate optimal designs. It evaluates both gradient-based and gradient-free methods, including Multiplicative Algorithm, Simulated Annealing, and Nelder-Mead with the barrier method, across different models including, cubic model, quartic model, a practical chemistry model and models with two and three design variables. The study highlights the strengths and weaknesses of these methods through iteration and simulation under various scenarios, providing a comprehensive comparison of their performance. After constructing the optimal design, the focus transitions to strategies for estimating regression coefficients. The second and third papers address this by presenting improved estimation techniques for gamma regression models, which are crucial in areas such as life testing, cancer forecasting, and quality control. These papers introduce novel estimation methods, including Stein-type shrinkage estimators, preliminary test esti- mators, and penalty estimators like LASSO and Ridge Regression. The asymptotic distributional bias and asymptotic quadratic risk of the shrinkage estimators are derived analytically. The papers also explore the performance of the shrinkage estimators when it is suspected that the parameters may be restricted to a subspace of the parameter space. Comprehensive Monte Carlo simulations are conducted to evaluate the proposed estimators, revealing their superiority over traditional maximum likelihood estimators. The effectiveness of the proposed estimators is demonstrated in a real-world appli- cation involving prostate cancer data. The fourth paper extends the exploration to high-dimensional data, addressing the challenge of estimating regression coefficients in data envelopment analysis (DEA). It investigates the efficacy of LASSO, Elastic Net, Adaptive LASSO, and Ridge Regression estimators in the context of DEA. Through extensive simulation studies, the paper assesses these methods in terms of bias and mean squared error under various scenarios, including different levels of correlation among variables and sample sizes. The practical application of these methods is demonstrated using data from the Swedish electricity distribution sector. Collectively, these studies contribute to the advancement of statistical methodologies by offering robust optimization and estimation techniques applicable to a wide range of scientific and practical problems, from optimal design construction to efficiency analysis in high-dimensional settings.