Author ORCID Identifier


Defense Date


Document Type


Degree Name

Doctor of Philosophy


Systems Modeling and Analysis

First Advisor

Dr. David J. Edwards

Second Advisor

Dr. D'Arcy P. Mays

Third Advisor

Dr. Jason Merrick

Fourth Advisor

Dr. Yanjun Qian


Experiments are widely used across multiple disciplines to uncover information about a system or processes. Experimental design is a statistical technique devoted to the methodology of selecting the appropriate samples to aid in the subsequent analysis. We research three open problems in experimental designs regarding calibration, sequential experimentation, and model selection. First, we focus on calibration; the impact of experimental design choice on the performance of statistical calibration is largely unknown. We investigate the performance of several experimental designs with regards to inverse prediction via a comprehensive simulation study. Specifically, we compare several design types including traditional response surface designs, algorithmically generated variance optimal designs, and space-filling designs. Next, we address sequential experimentation; uncertainty remains in optimal design techniques regarding the best way to allocate a given set of runs. The focus on maximizing information in optimal design has emphasized the running of a comprehensive large design all at one time, with or without replication. In practice, it may be better to first run a small screening design to identify important factors followed by an additional design building off knowledge gained in the first phase. We use simulations to compare the performance of D-optimal screening designs with follow up runs selected by Bayesian D-optimal augmentation against the performance of a nonsequential D-optimal design. Lastly, we explore model selection; there currently is not a suitable method available to incorporate pure error into model selection procedures when analyzing screening designs that achieves high power without the trade-off of high false discovery rates. To counteract the lack of noncentrality in the partial F-test denominator contributing to larger partial F-tests with pure error, we consider early stopping methods including Bonferroni adjusted p-values and our proposed forward selection method that incorporates a lack of fit test after each model selection step. Additionally, we develop a model selection method that incorporates pure error using lack of fit tests with LASSO penalized regression. We examine various model selection techniques using a simulation study and propose a strategy for incorporating pure error in model selection procedures that keeps false discovery rates in check.


© The Author

Is Part Of

VCU University Archives

Is Part Of

VCU Theses and Dissertations

Date of Submission