This project seeks to improve the robustness of wireless sensing and networking technologies through a reality-aware wireless architecture that blends networking and sensing. Robust perception and high-bandwidth networking benefit innovations across a diverse spectrum of high-impact areas including mixed-reality, robotics, and automated vehicles. For example, the use of such techniques to enhance driver assistance systems or automated vehicles has the potential to save numerous lives. In addition to disseminating results through scholarly publication, the project will engage the wireless and automotive industry to facilitate the technology transfer. The project also includes a set of integrated education and broadening participation activities to engage and retain students from underrepresented groups through internship programs, educational and outreach activities at each participating institution.

As wireless sensing and networking technologies make significant strides in today’s world, applications such as automated driving or augmented reality are increasingly involving rich sensing of the environment with unprecedented network requirements. Existing approaches that strictly separate the network stack and the perception component face challenges in providing robust perception and high-bandwidth networking. To address this, this project develops and studies a reality-aware wireless architecture that blends networking and sensing components, rather than isolating them. This approach exploits sensor information and scene geometry to provide improved and more predictable wireless network performance. It also uses information received over the network to aid perception functions such as object recognition and point correspondence. The team first explores the design space of network architectures for blending perception and communication by designing low-energy tags and visual signaling strategies. The team then develops Simultaneous Localization and Mapping(SLAM) algorithms that blends conventional strategies with network information to enhance robustness. It also designs geometric matching techniques to enhance object association in images with network information. At the network and link layers, the system will exploit knowledge about physical obstacles and the surrounding geometry obtained from camera views and other sensors to provide more predictable and seamless high-bandwidth coverage. The outcomes from the thrusts are integrated into a Reality-Aware Network(RAN) architecture that exploits information about the environment gathered via sensors. The architecture is implemented and evaluated in indoor and outdoor experiments, culminating in a validation on the Platform for Advanced Wireless Research (PAWR) COSMOS testbed.

Personnel

Principal Investigators

Graduate Students

Former Students

Research Direction

The major research objectives of this project are to:

  1. Design and study a network architecture that can blend visual perception and wireless communication to increase overall wireless system performance;

  2. Design low-energy, visual signaling strategies, hardware markers or ‘tags’ and communication protocols to improve wireless device lifetime;

  3. Develop improved Simultaneous Localization and Mapping (SLAM) algorithms that blend conventional computer vision strategies with tag recognition and decoding;

  4. Implement a reality-aware network architecture prototype hat integrates the results from objectives 1-3 and lets applications address and communicate with perceived nodes;

  5. Experimentally evaluate the reality-aware network architecture through controlled lab setups and integration with the NSF PAWR COSMOS testbed.

Publications

Dataset

Please see our Vi-Fi Dataset (Raw Data), Synchronized Data (Google Drive), Synchronized Data (OneDrive) & DATA.md

Educational and Outreach Activities

The major goals of the broadening impact plans of this project are to:

  1. Execute a broadening participation in computing (BPC) plan that implements intervention mechanisms across the three participating institutions to improve research and training of women in engineering and computing;

  2. Involve K-12 and undergraduate students in research;

  3. Design and integrate learning modules on perception, communication and networking topics derived from this research, into courses taught by the PIs in their respective institutions.

Broadening Participation in Computing

Funding

This work has been supported by the National Science Foundation (NSF) under the grant of CNS Core: Medium: Collaborative: Reality Aware Networks (CNS-1901355, 1910170, 1901133)