Abstract

Robot autonomy has imprecision in both perception and action. Human assistance can correct this inaccuracy by teaching a robot to make corrective adjustments. The Microsoft HoloLens 2 is an Augmented Reality headset that allows users to view and manipulate holograms in three dimensions, making it well-suited for intuitive Human-Robot Interaction. However, its tracking capabilities are poorly tailored for fine robot control. Our team proposes a human-robot collaborative workspace, as well as a collection of AR Human-Robot Interfaces designed to facilitate human-assisted robotic manipulation. Additionally, we designed a compound manipulation and perception task to showcase our proposed platform's ability to assist robot autonomy and propose a user study to evaluate its efficacy.

Contributions

• Robotic Control & Vision Sub-team Leader
• Robust Multi-camera fusion system for real-time robot and object pose detection using AprilTags
• Coordinate Space colocation with HoloLens V-SLAM system
• Inverse Kinematics of Robot Arm in conjunction w/ AR interface control
• Stack: Python, C++, C#
• Tags: ROS, OpenCV, AprilTags, Azure Spatial Anchors, Unity , HoloLens