Author ORCID Identifier
0000-0002-4374-2763
Defense Date
2026
Document Type
Dissertation
Degree Name
Doctor of Philosophy
Department
Electrical & Computer Engineering
First Advisor
Dr. Sherif Abdelwahed
Second Advisor
Dr. Robert H. Klenke
Third Advisor
Dr. Stefano Iannucci
Fourth Advisor
Dr. Carl Elks
Fifth Advisor
Dr. Milos Manic
Abstract
The increasing complexity of urban systems has accelerated the development of smart cities, which consist of interconnected subsystems such as transportation, buildings, and healthcare. Significant research efforts have focused on improving individual sectors within smart cities. However, current approaches often fail to capture multi-domain coupling adequately, therefore lack scalable coordination mechanisms, and do not incorporate the hierarchical decision decomposition needed for real-world city systems. The lack of cross-domain integration limits consideration of shared variables, such as energy consumption, emissions, traffic flow, queue length, system efficiency, and occupancy, in the behavior of other systems in the same environment. This results in ineffective resource allocation, conflicting management strategies, and reduced system resilience, as decisions made in one domain may not align with the broader objectives of the smart city. Such disintegration in smart city operations ultimately hinders scalability, adaptability, and overall effectiveness. This dissertation presents a multi-level multi-objective deep reinforcement learning (DRL) framework for modeling and optimizing key smart city subsystems using a systematic, modular approach. Each subsystem (intelligent transportation, smart buildings, and smart healthcare) is conceptualized as a domain-specific reinforcement learning environment and formulated as a multi-level, multi-criterion optimization problem. To create a cohesive optimization framework for the entire system, subsystems at different levels communicate through upper-level coordination signals that shape domain-local reward structures and operational constraints. This enables inter-agent cooperation and coordinated policy learning across multiple areas of the smart city. This dissertation makes several significant contributions: (1) It introduces a unified multi-level, multi-objective DRL framework aimed at optimizing the coordinated operations of smart cities, with applications spanning three distinct urban subsystems: buildings, transportation, and healthcare. (2) It presents a phase-based, two-timescale training procedure enhanced by phase-wise convergence diagnostics, which effectively addresses the non-stationarity inherent in hierarchical multi-agent systems. (3) A perturbation-based sensitivity analysis framework is developed to assess how upper-level coordination variables influence lower-level control actions, thereby facilitating the identification of a minimal sufficient set of coordination signals. (4) The dissertation features a city-level coordinator that learns adaptive domain-priority weights in real-time, which allows for a dynamic balance of performance across various domains while preventing the systematic neglect of any single subsystem. (5) Furthermore, a three-level hierarchical extension is proposed, introducing a meta-coordinator that adaptively allocates training resources across domains and enforces city-wide performance standards, leading to substantial enhancements in coordination quality compared to the two-level baseline. Experimental findings indicate that the proposed hierarchical controllers consistently outperform rule-based baselines in operational efficiency, convergence stability, and coordination responsiveness across all three urban subsystems. Within the coordinated multi-domain framework, the city-level coordinator attains a balanced city score of \(0.662\) while learning interpretable domain-priority trajectories that maintain the modularity of the domain-local controllers. The three-level extension enhances the city evaluation score by \(16.6\%\) and elevates the final-window city reward by \(46.5\%\), demonstrating that meta-level adaptive resource scheduling significantly improves outcomes compared to the two-level baseline. Furthermore, a perturbation-based sensitivity analysis reveals that fewer than three shared coordination variables account for over \(90\%\) of the total coordination influence in each domain, offering valuable insights for developing communication-efficient smart-city coordination. This hierarchical architecture enhances scalable decision-making, accelerates convergence, and maximizes system-level returns across distinct smart city domains.
Rights
© The Author
Is Part Of
VCU University Archives
Is Part Of
VCU Theses and Dissertations
Date of Submission
5-7-2026
Included in
Artificial Intelligence and Robotics Commons, Controls and Control Theory Commons, Other Electrical and Computer Engineering Commons