With the increasing penetration of renewable energy sources such as photovoltaic and wind power, carbon emissions are reduced on one hand, but on the other hand, the grid experiences heightened volatility and uncertainty. Virtual Power Plants (VPPs), by aggregating small-scale distributed renewable energy sources, effectively mitigate this uncertainty. VPPs strategically dispatch aggregated resources and engage in profitable electricity transactions through bidding in the electricity market. However, bidding strategies are impacted by uncertainties in future market prices, uncertainties associated with renewable energy generation, and uncertainties in user loads. Designing trading strategies that are more resilient to these uncertainties poses a significant challenge. This paper proposes Uncertainty-Aware Reinforcement Learning for Bidding Strategies to address this challenge. Initially, an ensemble-based approach is employed to assess uncertainty, followed by incorporating this uncertainty assessment into the modeling of states and actions in the reinforcement learning algorithm. We evaluate our proposed approach using real-world data from Shandong. This work is ongoing, and constructive suggestions are welcome.

With the rise of cloud computing, the Internet of Things, and distributed networks, the dispersion and diversity of resources are increasing. Innovative methods are required to ensure the safe, efficient, and fair utilization of these resources. The effective management and authorization of distributed resources have become a crucial research issue. This paper proposes a comprehensive distributed resource authorization framework based on the project development of data component flow networks, aiming to provide scalable and customizable solutions for various application scenarios. The system adopts the decentralized, tamper proof, and security features of blockchain technology, while effectively addressing performance and flexibility issues through the scalability of side chain technology. The results of this study have broad application potential, not only for digital asset management, but also for expanding to fields such as real estate leasing. The system design has brought breakthrough innovation to distributed resource management, laying a solid foundation for building a more secure and efficient digital society.

Considering the swift advancement in communication technology and the exponential increase in mobile user numbers, the joint optimization of base station (BS) deployment and antenna configuration within large network infrastructures has become critically important. The domain of large-scale BS cooperative optimization is highly significant in current research, especially given the challenges of spectrum scarcity and energy inefficiency in mobile communication systems. Strategic placement of multiple BSs and the adjustment of antenna parameters are crucial for ensuring high-quality communication services amid growing demands for higher data rates and ubiquitous connectivity. However, traditional network optimization methods, including heuristic approaches and classic reinforcement learning, are challenged by the complex, dynamic nature of contemporary communication environments. These methods typically fall short in adaptability, scalability, and in accurately representing the stochastic and nonlinear nature of wireless networks, often requiring extensive training data not feasible in real-time scenarios. This highlights the need for innovative approaches that can dynamically adapt to changing network conditions and user demands.To address the identified challenges, the HMAAPPO-RL framework introduces a hierarchical approach, dividing the optimization into BS deployment and antenna parameter refinement. The first layer focus on BS deployment effectively manages spectrum resources and energy efficiency, crucial for dynamic environments. The second layer's antenna parameter refinement ensures high-quality service in fluctuating demand areas, enhancing adaptability. The integration of the UNet architecture is a key innovation, allowing for the processing of complex data sets and handling the dynamics of modern networks. This capability is essential for accurate network representation and understanding. Furthermore, the incorporation of representation learning in the framework marks a significant advancement. This feature enables the prediction of configuration impacts on network performance, leading to more effective optimization strategies and improved network throughput and signal coverage. Overall, the HMAAPPO-RL framework's blend of hierarchical reinforcement learning, UNet architecture, and representation learning adeptly tackles mobile communication network optimization challenges, offering a notable improvement in the field.

To accommodate the explosive wireless traffics, massive multiple-input multiple-output (MIMO) is recognized as a cornerstone technology for advancing next-generation communication systems. Within massive MIMO cellular networks, the implementation of coordinated beamforming (CB), involving the collaborative design of beamformers across multiple base stations (BSs), proves to be an effective strategy for enhancing overall network performance. This study delves into the challenge of maximizing the sum rate in a massive MIMO mobile cellular network, where each cell's multi-antenna BS serves multiple mobile users concurrently through downlink beamforming. While existing optimization-based CB algorithms offer solutions that are nearly optimal, their reliance on real-time and global channel state information (CSI), coupled with high computational complexity, renders them impractical for deployment in dynamic mobile cellular networks. To address this limitation, we introduce a novel approach, a distributed reinforcement learning-based coordinated beamforming (Distributed RLCB) framework. This framework empowers each BS to determine beamformers using only local CSI and historical information from other BSs. Furthermore, the proposed approach significantly reduces computational complexity. Extensive numerical simulations validate the effectiveness of our approach, demonstrating comparable performance to centralized iterative optimization algorithms while offering lower computational demands and reduced information requirements.

Modeling information diffusion on social networks can effectively guide the prediction and control of information propagation, the study of social influence and public opinion, and the optimization of network structure. Existing information diffusion prediction methods can predict critical attributes, such as information diffusion paths, by modeling social network structure and user behavior. However, they are either only modeled at the macro level (predicting the number of affected users) or the micro level (predicting users' affective states), or ignore the inherent heterogeneity of users. In this paper, we propose Multi-Scale Activity Network（MSA-Net） to capture topological information for different scales and to predict the users who will be affected at a specific time in the future, taking the role of the users in the information diffusion into account. Specifically, we achieve more accurate prediction by modeling three scales or levels: micro-scale (user level), meso-scale (intra-community level), and macro-scale (inter-community level). Then, we introduce the user activity level for each user by using out-degree and in-degree to model the individual differences of users as a information transmitter and a receiver.

With the rapid advancement in autonomous vehicle (AV) technology, ensuring the safety and reliability of these systems through comprehensive testing has become crucial. Due to the high costs and challenges of conducting extensive real-world tests, AV developers are increasingly focusing on simulation testing. In this context, generating naturalistic and effective simulation scenarios is a key issue in both research and industry. In this talk, we introduce our approach using reward-guided diffusion methods. Initially, we train a diffusion model on a real-world driving dataset to generate realistic vehicle trajectories, taking into account the road network and other traffic participants. Then, during the sampling process, we introduce an RL-based guidance model to refine the generated results.

Traffic simulation platforms play a crucial role in various urban research endeavors, encompassing the analysis, control, and optimization of traffic. However, currently available simulators such as SUMO, CityFlow, and CBEngine have limitations in terms of simulation efficiency, preventing them from facilitating city-scale simulations involving millions of agents traversing hundreds of thousands of roads. The inefficiency of these simulators poses a significant challenge when it comes to implementing more advanced optimization algorithms like reinforcement learning, which require a more significant number of training samples compared to traditional algorithms. In this study, we introduce Simulet, a GPU-accelerated traffic simulation platform explicitly designed for large-scale scenarios. Leveraging the parallel nature of GPU architecture, Simulet can simulate millions of agents within milliseconds, demonstrating exceptional simulation speed. Additionally, Simulet offers versatility by supporting multiple optimization scenarios, including traffic signal control, congestion pricing, tidal lane optimization, one-way lane optimization, and dynamic traffic assignment. The platform provides a user-friendly Python API, enabling effortless interaction with Simulet's functionalities.

Generating mobile traffic in urban contexts is important for industrial and academic researches. However, existing solutions show weakness in capturing the spatial-temporal dynamics between mobile traffic and urban environments. In this paper, we propose a Knowledge-Guided Conditional Diffusion model (KG-Diff) for controllable mobile traffic generation. We construct an Urban Knowledge Graph (UKG) to fully characterize the features of urban environment, which incorporates the relations of different entities in the city. Conditioning on the UKG, we design the denoising network of diffusion model with a Temporal Extraction (TE) module and a Spatial Connection (SC) module. The two modules capture the correlations of mobile traffic and environment features via a frequency attention mechanism and spatial graph learning scheme, so that environmental features can make a strong controllability on the generated mobile traffic. Extensive experiments on two real-world datasets show that the proposed framework not only improves generation fidelity by up to 19%, but also enhances the controllability to generate specific patterns, with a gain of up to 13%.

Limited data access has been a longstanding barrier to data-driven research and development in the networked systems community. In this work, we investigate whether and how the diffusion model can be utilized to incentivize data sharing. We identify key fidelity and scalability metrics in the existing GAN-based methods, and recognize the pivotal challenges these workloads present to current GAN approaches in terms of fidelity (e.g., long-term dependencies, complex multi-dimensional relationships, mode collapse) and privacy . By integrating domain-specific insights with the latest advancements in machine learning and privacy, we determine the design choices to address these challenges. Drawing from these insights, we have crafted an end-to-end framework called PacketDiff to generate packet traces for specified APPs.

Recommender systems play a crucial role in shaping users' content consumption experiences. However, concerns have been raised about the potential formation of filter bubbles and the entrenchment of user interests within narrow content domains. In this study, we aim to shed light on the existence of filter trenches, where user exposure is concentrated onto narrow aspects of a given interest, rather than covering the plurality of categories within that interest. After validating the presence of this phenomenon by analyzing the dynamics of item depth exposure on short-video platform Kuaishou, we propose a recommender model that considers interest depth based on explicit user feedback. This model aims to prevent users from falling into a "filter trench" by avoiding a feedback loop posed by popularity bias. By considering interest diversity and proportionality, the proposed model aims to balance intra-interest plurality while retaining a user's variety of unique interests. Through this research, we aim to shed light on the existence of filter trenches and propose an improved recommender system design that prioritizes plurality within interests. By mitigating the potentially harmful aspects of filter trenches in recommender systems, our proposed model strives to promote balanced content consumption and mitigate risks to vulnerable online populations.

As the deployment of 5G accelerates, its impact on 4G networks becomes increasingly significant. Most of the existing measurement studies focus on the performance evaluation of NSA-5G or on the changes in behavior patterns of 5G users/mobile terminals affected by 4G. This research aims to identify the basic rules of 5G base station deployment from existing datasets. We envision the ultimate deployment form of 5G networks under the current development model and explore the impact of new 5G base station deployments on the existing network.

With the deployment of 5G networks, the huge power consumption caused by them accounts for a significant proportion of urban residents' electricity consumption.Therefore, the impact of 5G network power consumption on urban power grids has become a challenging issue, and will become increasingly severe with the intensive deployment of 5G base stations.Although mobile network operators have adopted a series of optimization methods to alleviate this problem, the effect is still limited.We will comprehensively analyze the measurement data of 5G networks in large cities from 2022 to 2023, model the intensive deployment of 5G networks and their power consumption, quantitatively estimate the power consumption of 5G networks in large cities in the future, and study practical and effective methods to reduce their impact on the power grid.

Offline Reinforcement Learning (RL) addresses the challenge of learning optimal policies from pre-collected data, making it a promising approach for real-world applications where online interactions with an environment are costly or impractical. We propose an offline RL method named Quantile-Guided Diffusion Policy (qGDP), which trains a quantile network to label the training dataset and uses these labeled samples to train the diffusion model and generate new samples with the trained model according to classifier-free guidance. qGDP can adjust the preference of sample generation between imitating and improving behavioral policies by adjusting the input condition and changing the guidance scale without retraining the model, which will significantly reduce the cost of tuning the algorithm. qGDP exhibits exceptional generalization capabilities and allows easy adjustment of action generation preferences without model retraining, reducing computational costs. Experimental results on the D4RL dataset demonstrate state-of-the-art performance and computational efficiency compared to other diffusion-based methods.

Prediction of couriers’ delivery timely rates in advance is essential to the logistics industry, enabling companies to take preemptive measures to ensure the normal operation of delivery services. This becomes even more critical during anomaly conditions like the epidemic outbreak, during which couriers’ delivery timely rate will decline markedly and fl uctuates signifi cantly. Existing studies pay less attention to the logistics scenario. Moreover, many works focusing on prediction tasks in anomaly scenarios fail to explicitly model abnormal events, e.g., treating external factors equally with other features, resulting in great information loss. Further, since some anomalous events occur infrequently, traditional data-driven methods perform poorly in these scenarios. To deal with them, we propose a deep spatial-temporal attention model, named DeepSTA. To be specific, to avoid information loss, we design an anomaly spatio-temporal learning module that employs a recurrent neural network to model incident information. Additionally, we utilize Node2vec to model correlations between road districts, and adopt graph neural networks and long short-term memory to capture the spatial-temporal dependencies of couriers. To tackle the issue of insuffi cient training data in abnormal circumstances, we proposean anomaly pattern attention module that adopts a memory network for couriers’ anomaly feature patterns storage via attention mechanisms. The experiments on real-world logistics datasets during the COVID-19 outbreak in 2022 show the model outperforms the best baselines by 12.11% in MAE and 13.71% in MSE, demonstrating its superior performance over multiple competitive baselines.

The road network is considered the skeleton of any district region; it determines the efficiency and productivity of the district in that it acts like blood vessels transporting people, goods, and information. The relationship between road networks and economic development is an important research topic. However, in many poor districts in China, sparse data on the road network evolution prohibits the analysis of public policy. We demonstrate how advancements in satellite imagery and machine learning (ML) can help recognize the road network development and infer the causal impact of road network on regional economic development.

Diffusion models have demonstrated their efficacy in generating images, videos, and point clouds. To control the output of these models, guidance has been used in several ways. Classifier guidance and classifier-free guidance have been used to reinforce input conditioning, while reconstruction guidance has been used for coherent video generation. This presentation focuses on the research conducted on controllable diffusion models. It also explores their applications, such as text-image generation, action generation, and trajectory prediction. Finally, it attempts to discuss possible application scenarios within our research track. We are looking forward to your valuable suggestions.

Developing smart cities is vital for ensuring sustainable development and improving human well-being. One critical aspect of building smart cities is designing intelligent methods to address various decision-making problems that arise in urban areas. As machine learning techniques continue to advance rapidly, a growing body of research has been focused on utilizing these methods to achieve intelligent urban decision making. First, I will introduce recent advances in intelligent urban decisions, following the taxonomy from three aspects, including tasks, methods and goals. Then I will briefly discuss my own opinions on the possibility to apply recently popular LLM+RL method for decision making in urban scenarios. I hope this survey can help researchers in related fields understand the recent advances made in existing works, and inspire novel applications of machine learning in smart cities.

Pedestrian dynamics refers to the underlying law governing the motion of pedestrians, with important position in crowd simulation. Despite its prevalent use, the commonly employed model, Social Force, is experientially derived and falls short in accurately portraying pedestrian movement. Extracting pedestrian dynamics equations directly from movement trajectory data is key to improving our grasp of pedestrian mobility and level of realism of crowd simulator. Yet, prevailing AI for Science methods struggle with the considerable stochasticity and heterogeneity inherent in the trajectory data. To tackle these challenges, we propose a fusion of Co-teaching and Mixture-of-Experts methodologies. Two synthetic experiments have preliminarily demonstrated the feasibility of this approach.

To meet the increasing demand for mobile traffic in the 5G era, the deployment density and quantity of base stations (BSs) in wireless access networks have significantly increased. However, this has exacerbated the issue of energy consumption in mobile communication networks and exacerbates the magnitude of the energy saving issue in base stations. Consequently, the complexity of the co-coverage relationships among cells has greatly increased, making it more challenging to optimize the coordination among BSs. Moreover, directly processing large-scale datasets is not feasibledue to memory constraints and other limitations. In light of the large-sca le BS energy-saving scenarios in 4G and 5G, we propose a combined approach that utilizes data-driven graph neural networks (GNNs) and reinforcement learning. First, we employ spectral clustering to partition the large-scale dataset into more manageable subsets of small-scale subgraphs. Subsequently, we use GNN encoders to encode the complex inter-cell coverage relationships within each small-scale subgraph. To address the problem of coordinated optimization among subgraphs, we employ the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm. Finally, we address the information loss caused by data partitioning by transferring nodes between different subgraphs, enabling global energy optimization. Our approach achieves a significant improvement in energy savings, with an over 60% enhancement in energy efficiency in a real dataset consisting of tens of thousands of cells.

With the rapid advancement of communication technology and the exponential growth of mobile users, the joint optimization of base station (BS) deployment and antenna configuration in large-scale networks improving network coverage quality and throughput has become increasingly important. In particular, large-scale BS cooperative optimization has emerged as a highly significant topic. The mobile network consists of multiple BSs, and the location of the BSs and various parameters of antennas can be adjusted to provide high-quality communication services. The location of the BS and the setting of the antenna parameters have a crucial impact on the performance of the entire network. Traditional methods for network optimization encounter significant challenges when confronted with the intricate and volatile nature of communication environments in the real world. These conventional approaches often exhibit noteworthy limitations, such as inefficiency in achieving optimal performance, subpar precision in modeling intricate network behaviors, and an inability to scale gracefully when applied to expansive and intricate scenarios. To overcome these challenges, we propose a hierarchical reinforcement learning approach that decomposes the problem into BS location optimization as well as antenna parameter optimization. The upper-layer strategy is employed to address the problem of BS deployment, and the output action is the deployment information of BSs. The lower-layer strategy concentrates on the BS antenna optimization problem, and the output action is the antenna parameter configuration of the established BS. In both upper-layer strategy and lower-layer strategy in hierarchical reinforcement learning, we design an Enhanced Multi-Agent Proximal Policy Optimization with Representation Learning (EMAAPPO-RL), which leverages the power of the UNet network to extract information from multiple intelligent agents, and a representation learning mechanism is used to explore the impact of BS deployment and configuration on coverage and throughput. The results demonstrate significant improvements in signal coverage rate and network throughput compared to the competing methods.

Social network simulation plays a crucial role in addressing various challenges within social science. It offers extensive applications such as state prediction, phenomena explanation, and policy-making support, among others. Due to the formidable human-like capabilities exhibited by large language models (LLMs) in sensing, reasoning, and behaving, we can utilize these qualities to construct the social network simulation system. In order to simulate the process of information propagation on the online social network and , we have designed a message propagation simulation framework among the system. This framework is not only close to reailty, but also can provide the necessary information for the prompt engineering of the LLM.

Sustainable development is crucial for humanity and natural environment. Accurate measurement of sustainable development outcomes, such as economic indicators, is important for research and policy. Traditional approaches to measure such outcomes typically depend on manually collecting data such as household survey, which are costly and time-consuming. Moreover, such data is lacked in many areas of greatest need. For example, 39 of 59 African countries conducted fewer than two surveys from 2000 to 2010, hampering efforts to target intervention to these areas. In contrast, satellite and street view images are increasingly available and easy to access, which shed light on an alternative path to measure outcomes with such data through image processing and machine learning methods. In this talk, we review recent studies that make use of satellite and street view images to understand and promote sustainable development. Specifically, we focus on approaches that leverages artificial intelligence methods to extract information from images. Hopefully this talk can help inspire new ideas in this direction.

The extensively deployed traffic cameras present an opportunity to reconstuct vehicle trajectories at an urban scale. However, the majority of these cameras are situated at intersections, meaning that vehicles are only sporadically observed. This poses challenges for the reconstruction of detailed spatiotemporal vehicle trajectories. To address this issue, we employ a Seq2Seq model in conjunction with an Nerual-ODE-Bayes approach to establish a continuous-time vehicle trajectory reconstruction system.