Offline Reinforcement Learning (RL) addresses the challenge of learning optimal policies from pre-collected data, making it a promising approach for real-world applications where online interactions with an environment are costly or impractical. We propose an offline RL method named Quantile-Guided Diffusion Policy (qGDP), which trains a quantile network to label the training dataset and uses these labeled samples to train the diffusion model and generate new samples with the trained model according to classifier-free guidance. qGDP can adjust the preference of sample generation between imitating and improving behavioral policies by adjusting the input condition and changing the guidance scale without retraining the model, which will significantly reduce the cost of tuning the algorithm. qGDP exhibits exceptional generalization capabilities and allows easy adjustment of action generation preferences without model retraining, reducing computational costs. Experimental results on the D4RL dataset demonstrate state-of-the-art performance and computational efficiency compared to other diffusion-based methods.

Prediction of couriers’ delivery timely rates in advance is essential to the logistics industry, enabling companies to take preemptive measures to ensure the normal operation of delivery services. This becomes even more critical during anomaly conditions like the epidemic outbreak, during which couriers’ delivery timely rate will decline markedly and fl uctuates signifi cantly. Existing studies pay less attention to the logistics scenario. Moreover, many works focusing on prediction tasks in anomaly scenarios fail to explicitly model abnormal events, e.g., treating external factors equally with other features, resulting in great information loss. Further, since some anomalous events occur infrequently, traditional data-driven methods perform poorly in these scenarios. To deal with them, we propose a deep spatial-temporal attention model, named DeepSTA. To be specific, to avoid information loss, we design an anomaly spatio-temporal learning module that employs a recurrent neural network to model incident information. Additionally, we utilize Node2vec to model correlations between road districts, and adopt graph neural networks and long short-term memory to capture the spatial-temporal dependencies of couriers. To tackle the issue of insuffi cient training data in abnormal circumstances, we proposean anomaly pattern attention module that adopts a memory network for couriers’ anomaly feature patterns storage via attention mechanisms. The experiments on real-world logistics datasets during the COVID-19 outbreak in 2022 show the model outperforms the best baselines by 12.11% in MAE and 13.71% in MSE, demonstrating its superior performance over multiple competitive baselines.

The road network is considered the skeleton of any district region; it determines the efficiency and productivity of the district in that it acts like blood vessels transporting people, goods, and information. The relationship between road networks and economic development is an important research topic. However, in many poor districts in China, sparse data on the road network evolution prohibits the analysis of public policy. We demonstrate how advancements in satellite imagery and machine learning (ML) can help recognize the road network development and infer the causal impact of road network on regional economic development.

Diffusion models have demonstrated their efficacy in generating images, videos, and point clouds. To control the output of these models, guidance has been used in several ways. Classifier guidance and classifier-free guidance have been used to reinforce input conditioning, while reconstruction guidance has been used for coherent video generation. This presentation focuses on the research conducted on controllable diffusion models. It also explores their applications, such as text-image generation, action generation, and trajectory prediction. Finally, it attempts to discuss possible application scenarios within our research track. We are looking forward to your valuable suggestions.

Developing smart cities is vital for ensuring sustainable development and improving human well-being. One critical aspect of building smart cities is designing intelligent methods to address various decision-making problems that arise in urban areas. As machine learning techniques continue to advance rapidly, a growing body of research has been focused on utilizing these methods to achieve intelligent urban decision making. First, I will introduce recent advances in intelligent urban decisions, following the taxonomy from three aspects, including tasks, methods and goals. Then I will briefly discuss my own opinions on the possibility to apply recently popular LLM+RL method for decision making in urban scenarios. I hope this survey can help researchers in related fields understand the recent advances made in existing works, and inspire novel applications of machine learning in smart cities.

Pedestrian dynamics refers to the underlying law governing the motion of pedestrians, with important position in crowd simulation. Despite its prevalent use, the commonly employed model, Social Force, is experientially derived and falls short in accurately portraying pedestrian movement. Extracting pedestrian dynamics equations directly from movement trajectory data is key to improving our grasp of pedestrian mobility and level of realism of crowd simulator. Yet, prevailing AI for Science methods struggle with the considerable stochasticity and heterogeneity inherent in the trajectory data. To tackle these challenges, we propose a fusion of Co-teaching and Mixture-of-Experts methodologies. Two synthetic experiments have preliminarily demonstrated the feasibility of this approach.

To meet the increasing demand for mobile traffic in the 5G era, the deployment density and quantity of base stations (BSs) in wireless access networks have significantly increased. However, this has exacerbated the issue of energy consumption in mobile communication networks and exacerbates the magnitude of the energy saving issue in base stations. Consequently, the complexity of the co-coverage relationships among cells has greatly increased, making it more challenging to optimize the coordination among BSs. Moreover, directly processing large-scale datasets is not feasibledue to memory constraints and other limitations. In light of the large-sca le BS energy-saving scenarios in 4G and 5G, we propose a combined approach that utilizes data-driven graph neural networks (GNNs) and reinforcement learning. First, we employ spectral clustering to partition the large-scale dataset into more manageable subsets of small-scale subgraphs. Subsequently, we use GNN encoders to encode the complex inter-cell coverage relationships within each small-scale subgraph. To address the problem of coordinated optimization among subgraphs, we employ the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm. Finally, we address the information loss caused by data partitioning by transferring nodes between different subgraphs, enabling global energy optimization. Our approach achieves a significant improvement in energy savings, with an over 60% enhancement in energy efficiency in a real dataset consisting of tens of thousands of cells.

With the rapid advancement of communication technology and the exponential growth of mobile users, the joint optimization of base station (BS) deployment and antenna configuration in large-scale networks improving network coverage quality and throughput has become increasingly important. In particular, large-scale BS cooperative optimization has emerged as a highly significant topic. The mobile network consists of multiple BSs, and the location of the BSs and various parameters of antennas can be adjusted to provide high-quality communication services. The location of the BS and the setting of the antenna parameters have a crucial impact on the performance of the entire network. Traditional methods for network optimization encounter significant challenges when confronted with the intricate and volatile nature of communication environments in the real world. These conventional approaches often exhibit noteworthy limitations, such as inefficiency in achieving optimal performance, subpar precision in modeling intricate network behaviors, and an inability to scale gracefully when applied to expansive and intricate scenarios. To overcome these challenges, we propose a hierarchical reinforcement learning approach that decomposes the problem into BS location optimization as well as antenna parameter optimization. The upper-layer strategy is employed to address the problem of BS deployment, and the output action is the deployment information of BSs. The lower-layer strategy concentrates on the BS antenna optimization problem, and the output action is the antenna parameter configuration of the established BS. In both upper-layer strategy and lower-layer strategy in hierarchical reinforcement learning, we design an Enhanced Multi-Agent Proximal Policy Optimization with Representation Learning (EMAAPPO-RL), which leverages the power of the UNet network to extract information from multiple intelligent agents, and a representation learning mechanism is used to explore the impact of BS deployment and configuration on coverage and throughput. The results demonstrate significant improvements in signal coverage rate and network throughput compared to the competing methods.

Social network simulation plays a crucial role in addressing various challenges within social science. It offers extensive applications such as state prediction, phenomena explanation, and policy-making support, among others. Due to the formidable human-like capabilities exhibited by large language models (LLMs) in sensing, reasoning, and behaving, we can utilize these qualities to construct the social network simulation system. In order to simulate the process of information propagation on the online social network and , we have designed a message propagation simulation framework among the system. This framework is not only close to reailty, but also can provide the necessary information for the prompt engineering of the LLM.

Sustainable development is crucial for humanity and natural environment. Accurate measurement of sustainable development outcomes, such as economic indicators, is important for research and policy. Traditional approaches to measure such outcomes typically depend on manually collecting data such as household survey, which are costly and time-consuming. Moreover, such data is lacked in many areas of greatest need. For example, 39 of 59 African countries conducted fewer than two surveys from 2000 to 2010, hampering efforts to target intervention to these areas. In contrast, satellite and street view images are increasingly available and easy to access, which shed light on an alternative path to measure outcomes with such data through image processing and machine learning methods. In this talk, we review recent studies that make use of satellite and street view images to understand and promote sustainable development. Specifically, we focus on approaches that leverages artificial intelligence methods to extract information from images. Hopefully this talk can help inspire new ideas in this direction.

The extensively deployed traffic cameras present an opportunity to reconstuct vehicle trajectories at an urban scale. However, the majority of these cameras are situated at intersections, meaning that vehicles are only sporadically observed. This poses challenges for the reconstruction of detailed spatiotemporal vehicle trajectories. To address this issue, we employ a Seq2Seq model in conjunction with an Nerual-ODE-Bayes approach to establish a continuous-time vehicle trajectory reconstruction system.