Owing to the widespread application of optimal designs, we are motivated to compare different methods of constructing optimum designs. In this regard, some optimization algorithms are considered to construct approximate optimal designs. Some of these methods are gradient based while some others are gradient free. The studied methods include a class of multiplicative algorithms, simulated annealing and Nelder-Mead with the barrier method. Optimization algorithms are explored through iterations for deterministic methods and simulations for stochastic methods, depending on the approach used. These algorithms are investigated across various models, including the quadratic model, cubic model, quartic model, a practical model in chemistry, and models with two and three design variables. A comprehensive behavioral analysis of optimization algorithms, focusing on iterative and simulative approaches, with key findings from these methods are highlighted. Additionally, the strengths and weaknesses of the methods are analyzed, and a comparison between them is performed. Based on our research, the multiplicative algorithm is the best choice if the gradient is available due to its faster and more accurate performance than Nelder-Mead and simulated annealing and ease of implementation. Gradient-free methods like Nelder-Mead and simulated annealing should be used if finding gradient is difficult or impossible. Although simulated annealing is slower and component-sensitive, it is more accurate. What makes one algorithm better than another depends on the needs of the optimization problem.

Gamma regression is applied in several areas such as life testing, forecasting cancer incidences, genomics, rainfall prediction, experimental designs, and quality control. Gamma regression models allow for a monotone and no constant hazard in survival models. Owing to the broad applicability of gamma regression, we propose some novel and improved methods to estimate the coefficients of gamma regression model. We combine the unrestricted maximum likelihood (ML) estimators and the estimators that are restricted by linear hypothesis, and we present Stein-type shrinkage estimators (SEs). We then develop an asymptotic theory for SEs and obtain their asymptotic quadratic risks. In addition, we conduct Monte Carlo simulations to study the performance of the estimators in terms of their simulated relative efficiencies. It is evident from our studies that the proposed SEs outperform the usual ML estimators. Furthermore, some tabular and graphical representations are given as proofs of our assertions. This study is finally ended by appraising the performance of our estimators for a real prostate cancer data. 

Owing to the broad applicability of gamma regression, we propose some improved estimators based on the preliminary test and Stein-type strategies to estimate the unknown parameters in a gamma regression model. These estimators are considered when it is suspected that the parameters may be restricted to a subspace of the parameter space. Two penalty estimators such as LASSO and ridge regression are also presented. An asymptotic theory for the preliminary test and Stein-type estimators is developed, and asymptotic distributional bias and asymptotic quadratic risk of the proposed estimators are obtained. Comprehensive Monte-Carlo simulation experiments are conducted. Comparisons are then made based on simulated relative efficiency to clarify the performance of the proposed estimators. Practitioners are recommended to use the positive-part Stein-type estimator since its performance is robust irrespective of the reliability of the subspace information. A real data on prostate cancer is considered to illustrate the performance of the proposed estimators. 

This paper explores various methods for estimating the regression coefficients of contextual variables in high-dimensional data using data envelopment analysis (DEA). DEA is a nonparametric method commonly used to evaluate production efficiency in decision-making units (DMUs). This study investigates the efficacy of LASSO, Elastic Net (EN), Adaptive LASSO (ALASSO), and Ridge Regression (RR) estimators for estimation in DEA. The proposed methodology is based on a data generating process (DGP) that accommodates both noise and inefficiency terms aligned with the stochastic frontier model. This methodology can be applied to both single and multiple-output cases. The performance of the penalty methods is assessed in terms of bias and mean squared error (MSE) of the estimators through an extensive Monte Carlo simulation studies. Then the comparison between penalty estimators are done considering different scenarios. The simulation findings indicate that increasing the correlation among variables generally leads to lower biases in penalty estimators. In terms of MSE, the MSE values of penalty estimators decrease as the sample size increases. ALASSO demonstrates the lowest bias in most scenarios, while RR exhibits superior performance in minimizing MSE, especially with higher correlations. The study also applies these methods to real-world data from the Swedish Energy Markets Inspectorate (SEMI), evaluating 144 electricity distribution system operators.