Reinforcement learning for safety-critical tasks requires policies that are both high-performing and safe throughout the learning process. While model-predictive shielding is a promising approach, existing methods are often computationally intractable for the high-dimensional, nonlinear systems where deep RL excels, as they typically rely on a patchwork of local models. We introduce RAMPS, a scalable shielding framework that overcomes this limitation by leveraging a learned, linear representation of the environment's dynamics. This model can range from a linear regression in the original state space to a more complex operator learned in a high-dimensional feature space. The key is that this linear structure enables a robust, look-ahead safety technique based on a multi-step Control Barrier Function (CBF). By moving beyond myopic one-step formulations, RAMPS accounts for model error and control delays to provide reliable, real-time interventions. The resulting framework is minimally invasive, computationally efficient, and built upon robust control-theoretic foundations. Our experiments demonstrate that RAMPS significantly reduces safety violations compared to existing safe RL methods while maintaining high task performance in complex control environments.

Multi-objective reinforcement learning (MORL) aims to optimize policies in environments with multiple, often conflicting objectives. While a single, preference-conditioned policy offers the most flexible and efficient solution, existing methods often struggle to cover the entire spectrum of optimal trade-offs. This is frequently due to two underlying challenges: destructive gradient interference between conflicting objectives and representational mode collapse, where the policy fails to produce diverse behaviors. In this work, we introduce D3PO, a novel algorithm that trains a single preference conditioned policy to directly address these issues. Our framework features a decomposed optimization process to encourage stable credit assignment and a scaled diversity regularizer to explicitly encourage a robust mapping from preferences to policies. Empirical evaluations across standard MORL benchmarks show that D3PO discovers more comprehensive and higher-quality Pareto fronts, establishing a new state-of-the-art in terms of hypervolume and expected utility, particularly in complex and many-objective environments.

Reinforcement learning for safety-critical tasks requires constructing a policy that prioritizes taking safe actions while optimizing performance. Moreover, in many applications it is important to maintain safety during training, not just at the end of the learning process. Model-predictive shielding using weakest preconditions is a promising framework that helps maintain safety during training and deployment, but current techniques are limited to low-dimensional state spaces (4 features) and struggle to scale to higher dimensional environments with complex environment dynamics. In this paper we present SPARKD, a highly scalable framework for model-predictive shielding which works by linearizing non-linear dynamics using a lifted representation of the state space. Our framework leverages the Koopman Operator theory to learn basis functions which augument the state space to capture highly non-linear transition models. The use of the lifted space by SPARKD allows for effective safety calculation using convex optimization and efficient safety analysis for shielding. Our experiments show that SPARKD is capable of learning performant policies with fewer safety violations than existing safe RL techniques.

Logical specifications have been shown to help reinforcement learning algorithms in achieving complex tasks. However, when a task is under-specified, agents might fail to learn useful policies. In this work, we explore the possibility of improving coarse-grained logical specifications via an exploration-guided strategy. We propose AutoSpec, a framework that searches for a logical specification refinement whose satisfaction implies satisfaction of the original specification, but which provides additional guidance therefore making it easier for reinforcement learning algorithms to learn useful policies. AutoSpec is applicable to reinforcement learning tasks specified via the SpectRL specification logic. We exploit the compositional nature of specifications written in SpectRL, and design four refinement procedures that modify the abstract graph of the specification by either refining its existing edge specifications or by introducing new edge specifications. We prove that all four procedures maintain specification soundness, i.e. any trajectory satisfying the refined specification also satisfies the original. We then show how AutoSpec can be integrated with existing reinforcement learning algorithms for learning policies from logical specifications. Our experiments demonstrate that AutoSpec yields promising improvements in terms of the complexity of control tasks that can be solved, when refined logical specifications produced by AutoSpec are utilized.

The research community lacks a middle ground between StarCraft II’s full game and mini-games. In the full game, sprawling state-action space renders reward signals sparse and noisy, but in mini-games, simple agents saturate performance with little genuine strategy. This yawning complexity gap hinders steady curriculum design and prevents many academic groups from experimenting with modern RL algorithms under realistic compute budgets. To fill this gap, we present the Two-Bridge Map, the first entry in an open-source benchmark series we purposely engineered as an intermediate benchmark to sit between these extremes. By disabling economy mechanics such as resource collection, base building, and fog-of-war, the environment isolates two core wargaming skills: long-range navigation and micro-combat. Two-Bridge ships as a lightweight, Gym-compatible wrapper on top of PySC2. Because it is a directly interactable environment rather than a replay-driven pipeline, researchers can train and evaluate any RL algorithm immediately---no downloading, filtering, or preprocessing of Blizzard replays required. Preliminary experiments show that agents learn coherent manoeuvring and engagement behaviours without imposing full-game computational costs. By open-sourcing the maps, wrappers, and reference scripts, we invite researchers to adopt the Two-Bridge Map Series as a standard benchmark.

As societies worldwide strive to reduce carbon footprints and transition toward cleaner energy sources, grid-integrated district energy systems (DES) emerge as a pivotal player in achieving these objectives. The escalating complexity of DES necessitates adaptive, synergistic, and hierarchical control of heterogeneous systems to achieve common energy and cost conservation goals. Prior research highlights several challenges of model-based control techniques for DES, such as limited access to computational tools, prolonged durations to digital twin development, and the complexities associated with control design. In contrast, model-free control methodologies appear as a viable alternative. As a response, our study explores a reinforcement learning-based (RL) supervisory control to minimize the operational costs in a university campus DES. To enhance overall system efficiency, we utilize resource flexibility to improve DES operations by responding to fluctuations in utility prices. In this paper, we demonstrate the toolchain, and virtual testbed development, engineer a suitable RL reward, along with the learning from challenges. From the case study, the RL agent showcases a significant 32% net operational cost savings and a 13% peak demand reduction compared to the conventional thermal load following control. This research signifies the potential of RL-based control systems in optimizing the performance of complex DES and multi-energy systems involving several control points.

Colorization of grayscale images is a severely ill-posed inverse problem among computer vision tasks. We present a novel end-to-end deep learning method for the automatic colorization of grayscale images. Past methods employ multiple deep networks, use auxiliary information, and/or are trained on massive datasets to understand the semantic transfer of colors. The proposed method is a 38-layer deep convolutional residual network that utilizes the CIELAB color space to reduce the problem’s solution space. The network comprises 16 residual blocks, each with 128 convolutional filters to address the ill-posedness of colorization, followed by 4 convolutional blocks to reconstruct the image. Experiments under challenging heterogeneous scenarios and using the Imagenet, Intel, and MirFlickr datasets show significant generalization when assessed visually and against PSNR, SSIM, and PIQE. The proposed method is relatively simpler (16 million parameters), faster (15 images/sec), and resource-efficient (just 50000 training images) when compared to the state-of-the-art

Accurately forecasting energy consumption for buildings has become increasingly important over the years owing to the increasing prices of energy. A good forecast gives an understanding of how much the expected load (demand) of the building would be in the coming days and months. This could be used in further planning of energy usage within the building. This also becomes important due to the dynamic nature of energy rates. With an accurate forecast, one could also aim for spot trading by which the energy is bought and sold at different rates in a daily fashion. We target short-term and medium-term demand forecasting for buildings. Data Sampling is an integral part of training time-series models. The temporal horizon along with the patterns captured contribute to the model learning and thus its forecasts. When the data is aplenty with more than one value per day, the traditional sliding window method is unable to forecast for short-term forecasts without the actual truth values because of its continuous nature. The forecasts deviate very quickly and become unusable. In this paper, we present a novel data sampling technique called Discrete Sequencing. This samples data sequences in a lagged fashion which looks at a much larger temporal horizon with a smaller sequence size. We demonstrate the efficacy of our sampling technique by testing the forecasts on three different neural network architectures.

The popularity of 360° videos has grown immensely in the last few years. One probable reason is the availability of low-cost devices and ease in capturing them. Additionally, users have shown interest in this particular type of media due to its inherent feature of being immersive, which is completely absent in traditional 2D videos. Nowadays such powerful 360° videos have many applications such as generating various content-specific videos (gaming, knowledge, travel, sports, educational, etc.), during surgeries by medical professionals, in autonomous vehicles, etc. A typical 360° video when seen through a Head Mounted Display (HMD) gives an immersive feeling, where the viewer perceives standing within the real environment in a virtual platform. Similar to real life, at any point in time, the viewer can view only a particular region and not the entire 360° content. Viewers adopts physical movement for exploring the total 360° content. However, due to the large volume of 360° media, it faces challenges during transmission. Adaptive compression techniques have been incorporated in this regard, which is in accordance with the viewing behaviour of a viewer. Therefore, with the growing popularity and usage of 360° media, the adaptive compression methodologies are in development. One important factor in adaptive compression is the estimation of the natural field-of-view (FOV) of a viewer watching 360° content using a HMD. The FOV estimation task becomes more challenging due to the spatial displacement of the viewer with respect to the dynamically changing video content. In this work, we propose a model to estimate the FOV of a user viewing a 360° video using an HMD. This task is popularly known as the Virtual Cinematography. The proposed FOVSelectionNet is primarily based on a reinforcement learning framework. In addition to this, saliency estimation is proved to be a very powerful indicator for attention modelling. Therefore, in this proposed network we utilise a saliency indicator for driving the reward function of the reinforcement learning framework. Experiments are performed on the benchmark Pano2Vid 360° dataset, and the results are observed to be similar to human exploration