The underwater domain is a harsh environment in which human divers operate. As the use of robots to facilitate human tasks in unstructured environments has been researched in the space domain, an underwater robotic assistant could help human divers perform a number of complicated tasks. An underwater robotic assistant could help a human diver by illuminating work areas, fetching tools from the surface, or monitoring the diver’s activity for abnormal behavior. However, there are a number of domain specific issues that make Underwater Human-Robot Interaction (UHRI) different than Human-Robot Interaction (HRI) in terrestrial or space environments. The first challenge we are addressing in this research is to develop a method to process both optical imagery and high-frequency sonar imagery in order to accurately assess the environment and interact with the human diver.