From Haptics

Research/Teleoperation

Jump to: navigation, search

Contents

Overview

Copyright: Intuitive Surgical, Inc.
Copyright:
Intuitive Surgical, Inc.

This work aims to develop methods to acquire, display, and determine the utility of haptic feedback to operators of robot-assisted minimally invasive surgical (RMIS) systems. While RMIS systems have been used in clinical settings for over 5 years and provide numerous advantages over traditional surgical techniques, none have the ability to provide sensitive haptic (force or tactile) feedback to the surgeon. This greatly reduces the transparency and intuitiveness of these RMIS systems.

We are taking a multifaceted approach at incorporating haptic feedback in RMIS systems that involves sensing, control, sensory substitution, and environmental estimation:

Sensory substitution of haptic information

Get the Flash Player to see this player.
Graphical Force Feedback


Sensory substitution involves using alternative sensory methods (e.g. auditory or visual) to display haptic information to operators during teleoperation. While the information presented via sensory substitution may not be as rich as true haptic feedback, it provides a cost-effective, practical way of providing haptic feedback during surgical teleoperation.

Control methods for haptic feedback without force sensors

While force sensors allow the teleoperator to sense the amount of force applied to the surgical environment, they are not always practical due to design constraints like cost, weight, and biocompatibility. It is possible to provide haptic feedback without force sensors by estimating the amount of force applied to the environment, but methods of implementing these estimation techniques are not yet completely realized.

Degrees of freedom of haptic feedback: control issues and human performance

While the need for force feedback in surgical teleoperators is clear, providing force feedback in every direction or degree of freedom of the teleoperator is not always feasible due to cost, size, etc. In many teleoperators, partial force feedback, or force feedback in only a few direction, may be possible, however, the stability of teleoperators with partial force feedback has not been shown. Additionally, little is known about how partial force feedback affects operator performance.

Estimating tissue properties during robot-assisted surgery

Understanding the properties of the surgical environment is crucial for clinical success. Surgeons spend years learning how to properly handle and manipulate tissue to optimize the surgical outcome and diagnose disease. RMIS systems have the potential to automatically estimate tissue properties during surgery and provide on-line clinical diagnosis and feedback to the surgeon.

Virtual Fixtures for teleoperators and cooperative manipulators

RMIS systems have potential to augment physicians’ surgical abilities with the use of virtual fixtures. Virtual fixtures can guide an operator along a preferred path or keep an operator out of an undesired area. Thus virtual fixtures can be used to train surgeons to perform complex tasks and/or provide an additional safety mechanism during the surgical procedure.

People

  • PI: Allison Okamura
  • Lawton Verner
  • Tomonori Yamamoto
  • Jim Gwilliam
  • Tricia Gibo
  • Undergraduate researcher: Rahul Agarwal, Kamini Balaji, Alex Vacharat
  • Clinical: David Yuh and Li-Ming Su
  • Previous: Mohsen Mahvash, Pannadda Marayong, Jake Abbott, Carol Reiley, Takintope Akinbiyi, Sunipa Saha

Publications

Journal Publications

  1. C. E Reiley, T. Akinbiyi, D. Burschka, D. C. Chang, A. M. Okamura, and D. D. Yuh. Effects of visual force feedback on robot-assisted surgical task performance. Journal of Thoracic and Cardiovascular Surgery, 2007. In press.
  2. M. Mahvash and A. M. Okamura. Friction compensation for enhancing transparency of a teleoperator with compliant transmission. IEEE Transactions on Robotics, 2007. In press.
    {pdf}
  3. J. J. Abbott and A. M. Okamura, "Pseudo-admittance Bilateral Telemanipulation with Guidance Virtual Fixtures", The International Journal of Robotics Research, Vol 26, pp. 865-884, 2007. http://articleworks.cadmus.com/doc/761015
  4. J. J. Abbott and A. M. Okamura, "Stable Forbidden-Region Virtual Fixtures for Bilateral Telemanipulation," ASME Journal of Dynamic Systems, Measurement, and Control, Vol. 128, pp. 53-64, 2006.
    {pdf}
  5. J. J. Abbott and A. M. Okamura, "Effects of Position Quantization and Sampling Rate on Virtual Wall Passivity, IEEE Transactions on Robotics, Vol. 21, No. 5, pp. 952 - 964, 2005.
    {pdf}
  6. M. Kitagawa, D. Dokko, A. M. Okamura, and D. D. Yuh, "Effect of Sensory Substitution on Suture Manipulation Forces for Robotic Surgical Systems," Journal of Thoracic and Cardiovascular Surgery, Vol. 129, No. 1, pp. 151-158, 2005.
    {pdf}
  7. A. M. Okamura, "Methods for Haptic Feedback in Teleoperated Robot-Assisted Surgery," Industrial Robot, Vol. 31, No. 6., pp. 499-508, 2004.
    {pdf}
  8. P. Marayong and A. M. Okamura, "Speed-Accuracy Characteristics of Human-Machine Cooperative Manipulation Using Virtual Fixtures with Variable Admittance," Human Factors, Vol. 46, No. 3, pp. 518-532, 2004.
    {pdf}
  9. A. Bettini, P. Marayong, S. Lang, A. M. Okamura, and G. D. Hager, "Vision Assisted Control for Manipulation Using Virtual Fixtures," IEEE Transactions on Robotics, Vol. 20, No. 6, pp. 953-966, 2004.
    {pdf}
  10. B. T. Bethea, A. M. Okamura, M. Kitagawa, T. P. Fitton, S. M. Cattaneo, V. L. Gott, W. A. Baumgartner, and D. D. Yuh, "Application of Haptic Feedback to Robotic Surgery," Journal of Laparoendoscopic and Advanced Surgical Techniques, Vol. 14, No. 3, pp. 191-195, 2004.
    {pdf}


Conference Publications

  1. M. Mahvash, J. Gwilliam, R. Agarwal, B. Vagvolgyi, L. Su, D.D. Yuh, and A.M. Okamura. "Force-Feedback Surgical Teleoperator: Controller Design and Palpation Experiments", 16th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, 2008, pp. 465-471.
    {pdf}
  2. M. Mahvash and A. M. Okamura, “Enhancing Transparency of a Position-Exchange Tele- operator”, Second Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (World Haptics), 2007. pp. 470-475.
    {pdf}
  3. L. N. Verner and A. M. Okamura, “Effects of Translational and Gripping Force Feedback are Decoupled in a 4-Degree-of-Freedom Telemanipulator”, Second Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (World Haptics), 2007. pp. 286-291.
    {pdf}
  4. T. Yamamoto and A. M. Okamura, “Evaluation of Human Performance with Kinematic and Haptic Errors” , Second Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (World Haptics), 2007. pp. 78-83.
    {pdf}
  5. M. Mahvash and A. M. Okamura, "Friction Compensation for a Force-Feedback Teleoperator with Compliant Transmission" , 45th IEEE Conference on Decision and Control, December 13-15, San Diego, California USA. pp. 4508 - 4513.
    {pdf}
  6. T. Akinbiyi, C. E. Reiley, S. Saha, D. Burschka, C. J. Hasser, D. D. Yuh, and A. M. Okamura. "Dynamic Augmented Reality for Sensory Substitution in Robot-Assisted Surgical Systems," 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2006, pp. 567-570.
    {pdf}
  7. H. C. Lin, P. Marayong, K. Mills, R. Karam, P. Kazanzides, A. M. Okamura, and G. D. Hager, "Portability and Applicability of Virtual Fixtures Across Medical and Manufacturing Tasks," IEEE International Conference on Robotics and Automation, 2006, pp. 225-340.
    {pdf}
  8. M. Mahvash, A. M. Okamura, "Friction Compensation for a Force-Feedback Telerobotic System," IEEE International Conference on Robotics and Automation, 2006, pp. 3268-3273.
    {pdf}
  9. L. N. Verner and A. M. Okamura, "Sensor/Actuator Asymmetries in Telemanipulators: Implications of Partial Force Feedback," 14th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, 2006, pp. 309-314.
    {pdf}
  10. P. Marayong, G. D. Hager, and A. M. Okamura. "Effect of Hand Dynamics on Virtual Fixtures for Compliant Human-Machine Interfaces," 14th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, 2006, pp. 109-115.
    {pdf}
  11. S. Misra and A. M. Okamura, "Environment Parameter Estimation During Bilateral Telemanipulation," 14th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, 2006, pp. 301 - 307.
    {pdf}
  12. J. J. Abbott and A. M. Okamura, "Pseudo-admittance Bilateral Telemanipulation with Guidance Virtual Fixtures," 14th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, 2006, pp. 169-175.
    {pdf}
  13. J. J. Abbott and A. M. Okamura, "A Sufficient Condition for Passive Virtual Walls with Quantization Effects," Proceedings of the ASME Dynamic Systems and Control Division: International Mechanical Engineering Congress and Exposition, 2004, pp. 1065-1073.
    {pdf}
  14. S. O. Chang and A. M. Okamura, "Impedance-Reflecting Teleoperation with a Real-Time Evolving Neural Network Controller," IEEE/RSJ International Conference on Intelligent Robots and Systems, 2004, pp. 2241-2246.
    {pdf}
  15. I. Emeagwali, P. Marayong, J. J. Abbott, and A. M. Okamura, "Performance Analysis of Steady-Hand Telemanipulation versus Cooperative Manipulation," 12th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, 2004, pp. 316-322.
    {pdf}
  16. W. Semere, M. Kitagawa, and A. M. Okamura, "Teleoperation with Sensor/Actuator Asymmetry: Task Performance with Partial Force Feedback," 12th Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, 2004, pp. 121-127.
    {pdf}
  17. J. J. Abbott, G. D. Hager and A. M. Okamura, "Steady-Hand Teleoperation with Virtual Fixtures," 12th IEEE International Workshop on Robot and Human Interactive Communication (RO-MAN 2003), pp. 145-151.
    {pdf}
  18. J. J. Abbott and A. M. Okamura, "Analysis of Virtual Fixture Contact Stability for Telemanipulation," IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003, pp. 2699-2706.
    {pdf}
  19. J. J. Abbott and A. M. Okamura, "Virtual Fixture Architectures for Telemanipulation." 2003 IEEE International Conference on Robotics and Automation, pp. 2798-2805.
    {pdf}
  20. P. Marayong, M. Li, A. M. Okamura, and G. D. Hager, "Spatial Motion Constraints: Theory and Demonstrations for Robot Guidance using Virtual Fixtures," 2003 IEEE International Conference on Robotics and Automation, pp. 1954-1959.
    {pdf}
  21. M. Kitagawa, A. M. Okamura, B. T. Bethea, V. L. Gott, and W. A. Baumgartner, "Analysis of Suture Manipulation Forces for Teleoperation with Force Feedback," Proceedings of the Fifth International Conference on Medical Image Computing and Computer Assisted Intervention -- MICCAI 2002, Lecture Notes in Computer Science (Vol. 2488), T. Dohi and R. Kikinis, Eds., 2002, pp. 155-162.
    {pdf}

Support

This work is supported by National Institutes of Health (R01 EB002004), Whitaker Foundation (RG-02-911), National Science Foundation (IIS-0347464 and EEC-9731478), a NDSEG Graduate Research Fellowship, and a NSF Graduate Research Fellowship.

Retrieved from "http://haptics.lcsr.jhu.edu/Research/Teleoperation"
This page was last modified 14:22, 3 September 2008.