Calendar

Presenter:  Xiaotian Li
Advisor:  Lijun Yin
When: Friday, July 26, 1 pm
Where: EB G11 

Title: Toward Dynamic, Multimodal, Interactive, and Generative AI for Facial Behavior Analysis

Abstract: Facial behavior analysis has evolved significantly with the introduction of dynamic, multi-modal, interactive, and generative AI technologies. This prospectus outlines recent advancements in these areas, aiming to deepen our understanding of human emotions and intentions through advanced computational techniques. It explores the integration of various modalities—such as RGB facial expressions, 3D depth maps, thermal maps, physiological data, language semantics, and contextual cues—emphasizing their collective role in comprehensive behavioral analysis. By focusing on real-time adaptability, this research investigates temporal context-aware frameworks that dynamically respond to user inputs, enhancing accuracy and applicability across multiple domains. Additionally, the role of generative AI models in simulating and analyzing facial recognition is examined. Key areas in machine learning, such as attention mechanisms, self-supervised learning, semi-supervised learning, contrastive learning, multi-modal feature fusion, and domain adaptation, play critical roles in enhancing the robustness and accuracy of these analyses. This research aims to highlight the transformative impact of these technologies on facial behavior analysis, paving the way for deeper insights into enhanced human behavior systems.

Presenter:  Xiang Zhang
Advisor:  Lijun Yin
When: Monday, July 29, 2 pm
Where: EB G11 

Title: Exploring Multimodal learning on Face behavior understanding and identity recognition

Abstract: The human face is a complex and highly individualistic feature, with subtle expressions and distinct characteristics vital for accurate interpretation and recognition. This prospectus investigates the integration of multimodal learning approaches to enhance face behavior understanding and identity recognition. Among various methods for understanding face behavior, face expression recognition and face action unit detection are the most extensively studied. Multimodal learning com- bines information from various sources, such as visual, textual, and 3D data, to improve the performance and robustness of deep learning models. We design and evaluate several multimodal architectures and fusion strategies, highlighting their effectiveness in different scenarios, i.e., face expression recognition, face action unit detection, and face recognition. The results demonstrate that multimodal models outperform unimodal counterparts in both face behavior analysis and identity recognition tasks, offering significant advancements in the development of intelligent systems for security, human-computer interaction, and entertainment. Our findings underscore the importance of incorporating modalities to capture the complexity of human facial behavior and identity cues.

Presenter:  Aniruddha Rakshit
Advisor:  Jayson Boubin

When: Tuesday, August 13 @ 10am

Where: EB P03

Title: RPE: Automatic Right-sizing for Complex Edge Deployments

Abstract: Edge deployments across remote environments like crop fields, forests, smart cities etc. require a diverse set of hardware and software. To monitor these remote environments, fully autonomous vehicles like Unmanned Aerial Vehicles (UAVs) need to run with the help of edge devices, comprising different levels of computational resources. Ensuring the accuracy and correctness of edge deployments is crucial yet challenging because of heterogeneous behavior of sensors, edge devices and testbeds. Moreover, building efficient testbeds for diverse sensors, compute platforms, and network manipulation is tedious for researchers, leading to over-provisioning and cost increases. My research focuses on edge deployments which balances the cost of IoT deployments along with performance in an optimized manner. I developed Righteous, an automatic edge deployment tool which treats resources like CPU, RAM and Network as hyperparameters. I designed Informed Pareto Simulated Annealing (iPSA) to find feasible and near optimal configurations which also maintain user defined goals. Righteous surpasses the state-of-the-art task scheduling problems with a certain amount of time. We use Righteous in conjunction with the PROWESS testbed to optimize a drone swarm deployment workload. Our results demonstrate that Righteous finds up to 3.5X cheaper configurations than leading hyperparameter tuning and resource allocation techniques, and does so up to 76.3X faster.