THIS PROPOSAL ADDRESSES TWO OF THE TOP TECHNICAL CHALLENGES IN THE NASA RTA (ROBOTICS TELEROBOTICS AUTONOMY SYSTEM) ROADMAP FROM A NONTRADITIONAL ANGLE. FOR A HIGH DEXTERITY ROBOT THAT IS EQUIPPED WITH MULTIPLE VISUAL AND HAPTIC SENSORS SUCH AS THE ROBONAUT 2 THE TWO RTA TECHNICAL CHALLENGES OBJECT RECOGNITION AND POSE ESTIMATION AND FUSING VISION TACTILE AND FORCE CONTROL FOR MANIPULATION ARE HIGHLY CORRELATED. IN MANY REAL WORLD SITUATIONS WHERE OBJECTS ARE OCCLUDED 3D PERCEPTION ITSELF IS NOT SUFFICIENT FOR ACCURATE OBJECT RECOGNITION AND POSE ESTIMATION. ONLY BY INCORPORATING MANIPULATION AND HAPTIC FEEDBACK INTO OBJECT RECOGNITION AND IMPLEMENTING A CLOSEDLOOP CONTROL USING HANDEYE COORDINATION CAN A ROBOT ACHIEVE HUMAN LIKE PERCEPTION AND MANIPULATION. INSTEAD OF HAVING AN INDEPENDENT OBJECT RECOGNITION SYSTEM I PROPOSE A MORE INTEGRATED APPROACH IN WHICH CONTINUOUS INFORMATION ACQUIRED FROM MULTIPLE SENSORS AND ROBOT ACTIONS ARE FUSED TO ACHIEVE BETTER RECOGNITION AND UNDERSTANDING OF THE ENVIRONMENT.