Gowri Shankar Giri
Yaser Maddahi
Yaser Maddahi
2 and
Kourosh Zareinia
Kourosh Zareinia
Department of Mechanical and Industrial Engineering, Ryerson University, Toronto, ON M5B 2K3, Canada Department of Research and Development, Tactile Robotics, Winnipeg, MB R3T 6A8, Canada Author to whom correspondence should be addressed. Robotics 2021, 10(1), 29; https://doi.org/10.3390/robotics10010029Submission received: 19 November 2020 / Revised: 26 January 2021 / Accepted: 1 February 2021 / Published: 5 February 2021
Recent technological development has led to the invention of different designs of haptic devices, electromechanical devices that mediate communication between the user and the computer and allow users to manipulate objects in a virtual environment while receiving tactile feedback. The main criteria behind providing an interactive interface are to generate kinesthetic feedback and relay information actively from the haptic device. Sensors and feedback control apparatus are of paramount importance in designing and manufacturing a haptic device. In general, haptic technology can be implemented in different applications such as gaming, teleoperation, medical surgeries, augmented reality (AR), and virtual reality (VR) devices. This paper classifies the application of haptic devices based on the construction and functionality in various fields, followed by addressing major limitations related to haptics technology and discussing prospects of this technology.
Haptics or haptic technology is defined as the technology of applying touch sensation while interacting with a physical or virtual environment [1]. Physical interaction may be performed at a distance, called teleoperation and the virtual environment could be conducted through a computer-based program. Over recent years, the development of haptic devices has exponentially grown, thanks to the rapid development in technology [2]. With many kinds of information unexploited and the necessity to respond quickly to actions, the importance of haptic devices has rocketed [3,4]. Designing, testing, and manufacturing haptic devices require multidisciplinary knowledge from computer science, programming, and coding to electromechanical design, human factors, and ergonomics [5]. Haptic devices enable a user to interact with computer-generated environments and create a sense of realness and tangibility [1]. This type of interaction is made possible by the actuators of the haptic device mechanism and is called haptic feedback [6].
On the application front, haptic interfaces are employed in different areas from the entertainment industry to specialized medical devices, wearable gloves, and surgical procedures [2]. Telemanipulators, exoskeletal devices, advanced prosthetics, physical rehabilitation, intelligent assistance devices, and near-field robotics are other applications that benefit from haptic technology [7]. Recently, haptic devices have also been implemented in computerized forensics such as 3D facial reconstruction, radiological cross sections, and analysis procedures [8]. For example, giant automobile manufacturer BMW has implemented volume control using gestures [7]. A realistic manufacturing plan for a 5-axis CNC milling process in a multi-sensory virtual environment with visual, haptic, and aural feedback was proposed in [9]. Haptic devices operate based on haptic feedback provided in the form of force and/or torque from objects in a real, teleoperated, or computer-generated environment through a Human–Machine Interface (HMI) [10,11]. Owing to the bidirectional and symmetrical interaction capabilities of haptic devices, the use of haptic apparatus has been productive while collaborating with computer systems to provide real-time feedback on a remote environment [5,12].
Human haptics, machine haptics, and computer haptics are three different areas of haptics technology [3]. When an object is touched by an operator, interaction forces are imposed on their skin, and consequently the sensory systems convey information to the brain and haptic perception is generated. In response, the brain provides commands that activate the muscles, resulting in hand or arm movement. This principle is called the human haptic system [3,13]. Specifically, human haptics relies on kinesthetic information and tactile information [3,14]. Machine haptics is defined as the use of machines to replace human touch autonomously or through telerobotics or haptic interfaces [3,13]. Measurement of positions or contact forces from any part of the human body, computation of information, and display of position and forces to the user are the basic operations performed in machine haptics [3,6,13]. Computer haptics has become prominent over the years and it is related to creating and rendering a sense of touch and feel of virtual objects to the user with the help of algorithms and software architectures [3]. Computer haptics deals with the creation of forces and torques, and sense of touch, while computer graphics deals with haptic rendering and visual rendering [13]. Computer graphics are incapable of providing manual feedback [6]. The interaction with the virtual environment is constructed utilizing joysticks, robotic arms, and actuation systems [2,6]. There exist various types of feedback including force, vibrotactile, and electrotactile feedback systems. The majority of the haptic devices operate based on force feedback and vibrotactile feedback [2,6,15].
Research related to the haptic interface may be classified as (i) studies carried out on technologies providing haptic stimuli and (ii) studies on how users perceive the haptic stimuli. The significance of human perception of haptic stimuli and the effect of emotional and psychological aspects on haptic feedback were addressed in [16]. For example, to create a sense of tactile realness, the joysticks used in the gaming industry provide vibrations in highly tense circumstances [6]. The efficiency, performance, and advancement of haptic interfaces depend on the type of feedback, maneuverability, and manipulability of the end-effector, haptic stimulation, and actuator technology [16]. One of the drawbacks in the design of haptic interfaces is limited workspace and low dexterity. For instance, in Minimally Invasive Surgery (MIS), disturbed hand–eye coordination, reduced perception of depth, and substandard haptic feedback are common limitations faced by surgeons [17]. Factors such as how the device responds, feels, or interacts should also be considered in the design phase for efficient operation [18]. Novel forms of communication, cooperation, and integration between humans and robots have become possible because of the wearability of robotic devices. Wearable haptics is gaining popularity in the robotics world [15], as they enable companies like Apple and Nintendo to improve user experience (UX) with the help of high-fidelity inertial actuators [19]. However, the major challenge of commercializing wearable haptic devices used in AR, medical procedures, and rehabilitation purposes remains intact and to be exploited [15].
Various applications of haptic devices have been studied including some works with a focus on the use of haptic devices in just a particular application. The main contribution of this review paper is in categorizing haptic devices used in research, industry, and medical fields based on their applications. This paper provides information on how the application, in which a haptic device might be used, can change the design of the haptic device and its features to match the requirements of that particular application. For example, while a wearable haptic device might be very useful and important in one application, an anchored desktop haptic device might be much more useful in another application. These facts are shown based on the evidences in literature and how popular a type of haptic device is in each application. Various types of haptic devices used in different applications are discussed in detail, and the importance of haptic devices in each discipline is signified. Existing challenges in the process of designing, customizing, and fabricating haptic devices are also dissected in different applications. The paper also uses several examples to highlight the importance of wearable haptic devices and the compromises that should be made in the process of manufacturing. Last, the paper addresses multiple challenges of implementing haptic devices in real-time applications, followed by highlighting their scope of application.
Sensing and manipulation of objects through touch can be defined as haptics. Haptic devices are used to give feedback on the movements/force generated by users [19]. With this feedback, many operations that are visually impaired can be carried out [18,19]. Haptic devices are also called as input–output devices because they provide feedback to the system [5]. The construction of haptic devices involves the combination of various concepts from different streams such as engineering, computer science, human perception, physics, and information technology [18]. Figure 1 depicts a number of commercial haptic devices with different types of mechanisms in the design. Basically, haptic devices are comprised of actuators (such as electric motors), interface devices, and sensors. Haptic devices can provide the sense of touch to the operators while manipulating a virtual object or an actual object remotely through a teleoperation system. Haptic devices provide tactile and/or kinesthetic feedback, and in some cases, thermal feedback as well [20]. Tactile feedback, also known as cutaneous feedback, can be defined as feedback obtained from various mechanoreceptors attached to the outer surface of the user’s body, generally on the skin [20,21].
Kinesthetic feedback is a different kind of feedback, which also plays a significant role in the field of haptics. Kinesthetic feedback can also be described as proprioception, and it refers to awareness or sense of touch created from muscle tensions with the help of sensory receptors. Unlike tactile feedback, the sensory receptors are implemented in muscled and joints, and not on the surface of the user’s body [15,20,22]. Interaction with a haptic device is usually a two-way interaction by the operator. The operator moves the haptic device, and this motion is sensed by the sensors (e.g, motor encoders) and used to provide motion commands (e.g., velocity commands for a teleoperated robot). The interaction force between the robotic arm can be sensed by force/torque sensors and sent back to the haptic device to be regenerated by actuators of the haptic device and be applied to the operator’s hand. This force feedback is an additional link connecting the user to the task environment and can potentially improve telepresence. For many years, the development of haptic devices has focused on acquiring information and manipulations of objects with the help of touch by machines and humans. The interaction between machine and human in real, teleoperated, virtual, or artificial environments has also been the subject of research [22]. Table 1 shows popular haptic devices.
Haptic devices find various applications in different fields such as medical training, rehabilitation, teleoperated robotic surgeries. The following section discusses different types of haptic devices used for micromanipulation such as medical procedures and surgery, wearable technology, as well as for the tasks conducted through teleoperation. Figure 2 depicts a number of high-end commercial haptic devices having more degrees of freedom force feedback than the haptic devices in Figure 1, used for different purposes including medicine and surgery.
Various fields such as electronics, microscopy, surgery, biology, and material sciences require micromanipulation systems that can perform tasks like sensing, processing, stiffness, and conductivity testing [23]. A 4-degrees of freedom (DoFs) hybrid parallel flexure mechanism-based device was designed for micromanipulation in which a master–slave manipulator is controlled using multi-DoF piezo actuators with the capability of providing haptic feedback. The platform consists of a planar 3-PRR manipulator (three rotary joints) and a 1-DoF bridge mechanism. The system can be employed for the assembly of microelectromechanical systems (MEMS) or a micro-teleoperated contact with biological cells [23]. Inadequate performance of the system presented for small-scale tasks in [23] prompted the creation of a new manual, bilateral cell injection device that uses a null displacement active force sensor coupled with a haptic interface with negligible effective inertia to carry out manual injection in biological samples was introduced [24].
A haptic teleoperation control scheme to carry out micromanipulation tasks was proposed in [25], in which a particular mechanical design was considered for the haptic device architecture that enables an operator to perform a range of micromanipulation tasks. The study emphasized the importance of haptic devices in teleoperation, micromanipulation, and nanoworld. The design of a haptic teleoperation control scheme, which is capable of controlling the actions of the human user and enables monitoring of items at the microscale level, was also discussed [26]. A bilateral telemanipulation system for controlling paramagnetic microparticles was also developed. The platform consists of a pantograph haptic interface and an electromagnetic system with four electromagnetic coils that enables a user to control the position of magnetic beads [27]. The design and control of a teleoperated robotic system, consisting of a 3-DoF robotic wrist and a spherical five-bar mechanism, was explained in [28]. The design of the system was based on motion data gathered using a simulated microanastomosis operation. The platform can be used for microsurgical operations and dextrous micromanipulation tasks [28].
To name a few tokens, some of the teleoperated industrial robots used for surgical applications are as follows:
Mitsubishi RV-1a (Mitsubishi Electric Corp., Tokyo, Japan)—6 DoFs MIRO (DLR, Köln, Germany)—7 DoFs KUKA KR 6/2 (KUKA AG, Augsburg, Germany)—6 DoFs PUMA (Unimation Inc.)—6 DoFs [29] Mitsubishi MELFA 6SL—6 DoFs [30] Mitsubishi PA-10 [31]—7 DoFs Mitsubishi MELFA RV-E2—6 DoFs [29] Hexapod Physik Instrumente (Physik Instrumente, GmbH and Co. KG, Karlsruhe, Deutschland)—6 DoFs [32] Mitsubishi PA-10—6 DoFs [29] Mitsubishi PA-10—7 DoFs [31] Rockwell Samsung AS2 (Rockwell Samsung Automation Inc., Seoul, Korea)—6 DoFs [30]Nowadays, most technologies used for healthcare training are haptic simulators [33]. Training students and novice doctors and technicians in healthcare practice and education training have begun to utilize computer-based simulation systems because of their capability in providing real-time visualizations [34]. These systems enable the users to interact with a virtual environment that is similar to the real world. Training healthcare providers, including surgeons and physicians [35] and dentists [36], is of importance and requires state-of-the-art innovations such as the implementation of haptic devices. Haptic devices add a sense of touch when the user interacts with the virtual environment [37]. A haptic dental procedure simulator called HapTEL was designed to allow dental students to learn dental drilling, cavity preparation for tooth restoration [38].
DenTeach is a portable and compact vibrotactile platform that was developed to facilitate fully remote and physical distancing-aware teaching and learning in dentistry [36]. This platform helps dental schools adapt to the COVID-19 pandemic by allowing dental students and instructors to learn and teach practical dental tasks from home or a remote location, and in turn, helps to limit the spread and transmission of the novel coronavirus. DenTeach platform consists of an instructor workstation (DT-Performer), a student workstation (DT-Student), advanced wireless networking technology, and cloud-based data storage and retrieval [36]. The platform procedurally synchronizes the instructor and the student with real-time video, audio, feel (haptics), and posture (VAFP). DenTeach offers three modes: teaching, shadowing, and practice. Teaching mode provides each student with haptic feedback from the instructor workstation (inside the lab or a remote place), and shadowing mode enables the student to download augmented videos and start watching, feeling, and repeating the tasks before entering the practice mode. In the practice mode, students use the DenTeach to conduct delicate dental tasks and evaluate their performance skills automatically evaluated in terms of key performance indices.
An extensive review of simulation of palpation procedures, different techniques used for palpation, and various approaches used in medical systems with the help of haptic feedback is explained in [39]. While comparing different palpations, the multi-finger palpation was found commonly preferred over single-finger palpation due to covering multiple contacts at the same time [39]. Figure 3 presents the list of simulators presently used in surgery, medicine, and dentistry training.
The application of haptic devices in surgical operations like stitching, palpation, dental procedures, endoscopy, laparoscopy, and orthopedics was explained in [37]. An external suture environment was developed, called SutureHap, that is a simulator to replicate the sensations in the medical rooms and offices. Omnihaptic device was used for suture training [21]. They also discussed a popular medical simulation system for dental training system as well as the development of an oral implantation simulator that is able to store data collected from trainees and may be used for rehearsal and medical education [21].
Haptic technology has a growing importance in surgical operations especially MIS that is commonly used in conjunction with a robotic manipulator in a bilateral teleoperation fashion [40,41]. A teleoperation system was proposed for use in MIS: a modified 6 degrees-of-freedom (DoFs) Denso VP-6242G with a serial mechanism and a PHANToM Premium 1.0 kinesthetic device (see Figure 1). In this system, the serial manipulator was employed at the slave site and the haptic device was at the master site [42]. The authors also suggested that grippers can also be added for grasping surgical items. An adjustable, immersive, and configurable platform for optometry training simulation was proposed, involving head-mounted displays, AR interfaces, and a multi-point haptic device [43]. In this platform, preoperative planning and virtual training system were developed based on force feedback. The platform involves an Omega 6 haptic device, an immersive workbench, and a CHAI3D software toolkit. Using this system, the preoperative planning data are transferred, and surgical simulation are carried out by a surgeon to perform osteotomy procedure, learn, and improve their surgical skills [44]. The construction of Pneumatic Artificial Muscles (PAMs) was proposed [45] that have flexible, inflatable membranes and they exhibit orthotropic material behavior. PAMs can be formed conveniently and are light; therefore, in another application, they are of interest for rehabilitation purposes because of their functioning as locomotion devices [45]. The importance of haptic detection between the surgical instrument and human organ and tissue in virtual surgery was presented in their paper [46]. Navigation in surgery was possible with the help of tactile and force feedback between the surgical instrument and the human organ [46]. An application of Omega 7 haptic devices in neuroArm surgical system is depicted in Figure 4. The platform uses two haptic devices to transfer the sense of touch to both hands of a surgeon.
A haptic intracorporeal palpation was proposed that Uses a cable-driven robotic system that includes a remote sensing strategy [47]. The platform employs teleoperated cable-driven parallel manipulator that is a new, simple, and cost-effective approach for restoring haptic sensation during the performance of intracorporeal palpation. The conducted tests showed evidence of reasonable accuracy in estimating the amount of force. In another work, the authors integrated a 7-DoF master device into the da Vinci Research Kit and conducted tissue grasping, palpation, and incision tasks using robot-assisted surgery by experienced surgeons, surgical residents, and non-surgeons [48]. Statistical analysis showed that haptic feedback improves key surgical outcomes for tasks requiring a pronounced cognitive burden for the surgeon; however, possible longer task completion times were observed.
Over the years, many industries have started to design and develop haptic devices with portability and wearability in mind. These wearable haptic devices enable better communication, cooperation, and integration between humans and interfaces [15] (Pacchierotti et al., 2017). In this section, a number of wearable haptic devices are presented. Note that a majority of the listed devices are still in the research phase and still need further improvement to be commercialized and be adopted by clinicians in healthcare.
Many industries incorporate pick and place operations, and the main actions to be considered are grasping and manipulation for efficient manufacturing and profitable production rates. The shape and weight of the objects to be held are measured using cutaneous feedback derived from the fingertip contact pressure and kinesthetic feedback of finger positions. The currently used VR systems cannot provide a realistic haptic experience and are normally large, mechanically complex, and their workspace is limited. Grabity is a wearable haptic device designed to simulate kinesthetic pad opposition grip forces and weight for grasping virtual objects in VR. The arrangement of movement, that may affect the amount of grabity, enables precision grasping and strong grasping force feedback by means of a brake. In addition, two tiny actuators aids in creating a virtual force that is tangential in the direction of each finger pad. Grabity provides vibrotactile feedback during contact, high stiffness force feedback during grasping, and weight force feedback during lifting [49].
LinkTouch is a wearable haptic device that can represent the force vector sensation at the fingerpad. This device is distinct because it consists of an inverted five-bar linkage. The device consists of two DC motors that drive the cranks of the five-bar linkage mechanism. These two motors are mounted at the sides of the distal phalanx. The direction of rotation of motors determines the outcome of the perception. When the rotation of the motors is in the same direction, the definition of contact point coordinates takes place. On the other hand, when the rotation of the motors is in the opposite direction, the generation of pressure occurs. In this way, the fingerpad is able to produce 2-DoF force feedback. Besides, the device can also represent the transitional state from a non-contact condition to a contact condition [50]. HapThimble is a device that can produce tactile, pseudoforce, and vibrotactile feedback from the users’ fingertips. It is similar to physical buttons and the amount of force is measured. With the help of force–depth curves, all kinds of haptic feedback rendered were analyzed and used for efficient operation [51].
A novel wearable haptic device called MagTics is introduced and tested for positive results. Unlike conventional devices, MagTics eliminates huge power consumption problems to produce sufficient force. MagTics allows for a thin form factor and supreme flexibility in the haptic interface. The interface works based on magnetically actuated bidirectional tactile pixels shortly known as taxels. Henceforth, rich haptic feedback is achieved in a mobile setting using an interface of this kind [52]. Hapballoon is a novel device that can be worn on the fingertips. This device can present three types of sensations: force, warmth, and vibration. Feedback is generated from the devices when the inflated balloon encounters individual devices [53].
FinGar stands for “Finger Glove for Augmented Reality”, and it can be described as a wearable haptic device that combines electrical and mechanical stimulation. The skin sensory mechanoreceptors are stimulated, and tactile feedback is generated for virtual objects in AR. The device is mounted on the fingers and with the help of a DC motor, high-frequency vibration and shear deformation are produced to the whole finger. The device is usually attached to the middle finger, index finger, and thumb. Unlike other conventional devices, it is lightweight, has a simple mechanism, and in no way hinders the natural movement of the hand. The characteristics above mentioned can be attributed to any wearable haptic device used in a VR system. The principle behind FinGar is the type of stimulation it employs based on the application in need. Electrical stimulation is used to provide pressure and low-frequency vibration whereas, mechanical stimulation is used to provide high-frequency and skin deformation. The stimulations can be differentiated based on the type of activities the mechanoreceptors in our skin carry out [54].
The proposed device is capable of providing kinesthetic force feedback which is paramount for haptic feedback and interface. Touchscreens gap the bridge between the spatial and cognitive gap between input and visual display. Conventional touchscreens provide a visual response that is triggered by physical interactions. Many works have begun to provide physical feedback and not disconnect the user from the virtual space. This has been possible because of kinesthetic haptic feedback and physical feedback. Several works have focused on controlling the magnitude of friction force between the user and the device. The proposed device uses a mechatronic design to implement a haptic interface with the help of a steering wheel. This steering wheel provides kinesthetic force feedback on a large-format touchscreen. This device serves the purpose of presenting the haptic constraints such as area-of-effect fields or paths. This device can be used in various applications such as wall and maze applications, gaming, touchscreen accessibility, and upper limb stroke rehabilitation therapy or physiotherapy. This device focuses on enforcing elements with the virtual display. The sensing devices used in the system are nylon flexures to provide uniform stiffness and haptic feedback [55].
A novel wearable device is designed with three pairs of micromotors with belts attached to each pair of motors. The device can be worn on the user’s fingers. When the motors rotate in opposite directions, the device sends feedback tgo the user based on the tension and vibration created by the belts. This device can be used to sense force or vibration on texture and also aids in edge detection of a surface [56]. Multi-modal haptic feedback is becoming an attraction in the haptic world and a novel wearable device called PATCH was developed. PATCH stands for Pump-Actuated Thermal Compression Haptic device and the device can read thermal and compression cues off the user’s skin. Water under different temperature conditions is utilized by the device to provide thermal or compression readings. The PATCH system can be rated high in terms of wearability [57]. Cybergrasp is a novel 7-DoF device that is wearable on the human arm. The device is structured using mechanical links and human joints, unlike conventional haptic devices that just contain mechanical links. Such a biomedical design allows the device to the users’ actions quite easily. The 7-DoF module consists of a 3-DoF wrist arrangement, 3-DoF shoulder arrangement, and 1-DoF elbow arrangement. These arrangements adapt easily to the motions of the human arm. Furthermore, these arrangements can be operated individually. This device can be operated in three modes: active, passive, and restrained, and finds application in teleoperation, VR, and medical rehabilitation [58].
Haptic rendering and visual rendering are fundamental components of developing a virtual haptic system. Haptic rendering refers to determining the haptic force, and visual rendering enables us to visually display the interaction of a virtual object [59]. A VR or AR system is normally constructed with head-mounted displays such as goggles, accelerometers, and loudspeakers, and is based on these elements to generate a computer-based environment for an operator to interact with the environment [60,61]. Besides, audio and video, skin-based sensory systems are underdeveloped in VR and AR technology [59,60,62], which were proved to provide an enhanced experience in several applications such as entertainment and medicine. Wireless modes of operation have become possible with the help of small, lightweight epidermal VR systems. These systems are thin, soft architectures that can be mounted onto the skin and programmed for the operation of haptic operations [60,62]. Three VR devices, namely, LeapMotion that tracks the motion of the fingertips, hand avatar that mimics the motion of the hand, and fingertips wearable tactile device that provides pressure stimuli were proposed. The interaction between the operator and a virtual environment [63] may be enabled by rendering based on textures and pressure differences [63]. A novel interaction VR device called SlingDrone was proposed [64] that can provide force feedback with visualized trajectory planning. This device employs a micro-quadrotor for control and interaction and provides application in 3D printing technology [64]. A wearable VR device called the ThirdHand, can be attached to the wrist of the user and act as additional support. The device provides constant force feedback [65]. In wearable technologies, the haptic modality is discussed based on (i) tactile feedback to address the tactile perception from the skin, such as vibrations and (ii) kinesthetic feedback to address the kinesthetic perception of our muscular effort. This helps clinicians and therapists to assess the performance of a patient during treatment sessions using the data received based on two main factors: skin and muscle.
Commercial cutaneous wearable devices such as Apple Watch and Samsung Gear Live were reviewed [66]. The hRing is a novel wearable haptic device that uses cutaneous feedback and can be used on the proximal finger phalanx. The device consists of two servo motors and a belt that is placed in contact with the user’s finger [67]. The design and development of a 3-RRS fingertip device, which is a wearable skin stretch device were discussed. The device consists of a static upper body and a movable end-effector. The device also consists of three small servo motors which are supported by the upper body which is located on the nail side of the finger. The end-effector is assembled in a way to contact the finger pulp. The end-effector makes contact with surfaces by enabling movement and rotation towards the user’s fingertip [68]. Commodity electromyography (EMG) armbands are popular in the gaming industry. They provide a new platform for human–computer interfaces [67]. The user interface of EMG with the help of kinesthetic haptic sensory feedback was reviewed in detail [69]. The characterisation of Novint Falcon as a robot manipulator to provide feedback, and also create viable kinematic and dynamic models [70]. A new haptic technology called Po2 was developed. The device utilizes gesture-based illusive tactile sensations in gaming platforms. It consists of two vibrating actuators and provides tactile motion. The device is able to sense movements and vibrations between hands [71].
Several surgical robotic systems have been developed to provide enhanced dexterity, improved accuracy, and better ergonomics. Surgical robotic systems and teleoperation systems have been developed in recent years to overcome workspace constraints, to improve dexterity, accuracy, and provide enhanced ergonomics [72]. A master–slave teleoperation system to carry out surgical operations and manipulations. The master device consists of a pair of haptic devices and the slave device consists of multiple arms [73]. Two franka emika robot arms were presented to serve as a twin master–slave system. The system was designed to carry out haptic-guided teleoperation. The objective was to study the interaction forces between the master and the slave [72]. A Cable-Driven Parallel Robot teleoperation system, consisting of a master CDPR and a slave CDPR, was proposed. The master and slave CDPRs were connected through a wireless channel, and the components of the haptic force were realized using admittance control. With this system, gait training was carried out and reduction of stress on the body and legs was achieved [74]. The design and development of various teleoperation systems, with two stylus arrangements for CombX haptic device, was proposed to provide force outputs. This stylus arrangement can be implemented in telesurgical systems. Examples of haptic devices are Cobotic hand-controller, DELTA-R, CU parallel haptic device, and VirtuaPower [73]. The significance of implementing fingertip devices for pick and place operation was explained in [75]. The work proposed the importance of multipoint, multi-contact haptic feedback. Fingertip devices realize forces on human fingertips, showing that wearable devices are promising for robotic manipulations systems in bilateral teleoperation [75].
An fMRI-compatible haptic device was proposed to study and investigate neural mechanisms for precision grasp control. An electromagnetic actuation system was used to control the haptic interface [76]. The design and development of a haptic teleoperation system using soft micro-fingers. In this system, the microfingers act as the slave, and they are maneuvered using a haptic interface. These micro-fingers integrated with artificial muscles are used to transmit tactile information. The proposed teleoperation system can be used for rehabilitation purposes [77]. The design and fabrication of a teleoperation device called MiniOct. MiniOct was designed particularly for the continuum teleoperation of manipulators. The prototype was proposed and tested for the haptic response, kinesthetic feedback quality [78]. A pick and place teleoperation system was designed, and experiments were conducted. Participants of the experiment micro-manipulated cotton strips to mimic microsurgical operations. The experiment was conducted with three haptic devices: sigma 7, neuroARMPLUSHD, and a master manipulator [79]. The evaluation of haptic devices and end users in telerobotic microsurgery were discussed [79]. The design and operation of teleoperated mobile robots: The mobile robots consist of a 6-DoF haptic device and electromyography (EMG) signals sensor to receive force feedback. Using this hybrid mechanism, mobile robots are operated synchronously, and obstacle avoidance is achieved [80].
The development of haptic devices has been exponential; however, the implementation of haptic technology in various fields has faced numerous challenges. Some of these challenges are discussed in this section.
Design complexity, quality of feedback, and the safety of operation are some of the factors that require improvements [14]. For instance, the inability of teleoperation systems incorporating haptic devices to handle nuclear waste due to the complexity of the task, the unpredictable nature of hazardous materials are still the areas that need enhancements as discussed earlier. Furthermore, the regulatory standards in the nuclear industry are stringent and against the incorporation of autonomous robotic systems [72]. The design of a haptic interface is a multidisciplinary task that is very complex due to its multi-criteria and often overlapping, functional, and performance specifications [10]. Design and development of a haptic device that is acceptable for industrial applications and provides high-quality sense of touch in terms of force and tactile feedback is still an open research area.
In medical applications, teleoperation is considered impractical by some researchers [17], while other researchers have found haptic technology very useful [79]. Patients’ safety, reproduction of realistic haptic feedback, affordability, probe control, and feedback training are some of the challenges that need to be addressed before haptic devices can be implemented in medical procedures. Virtual simulators play a significant role in medical training, and ongoing research has shown evidence of many possible challenges. A virtual simulator used in medical training comprises of a haptic device, medical tools, and a virtual environment with a virtual patient (task environment) [37]. Presently, most simulators employ haptic devices with 3-DoF or 6-DoF. Most training simulators are equipped with 3-DoF haptic devices such as Falcon as they are affordable. Phantom Omni (Touch) offers 6-DoF of positional and rotational feedback with 3-DoF of force rendering. Other devices with high-quality haptic feedback such as Phantom Desktop (Touch X) or Phantom Premium are more expensive, which can be considered as one of the main challenges in the acceptance of the system [40]. In robotics surgery, there are many research studies focused on implementation of haptic feedback in tele-surgical robotic systems [35] and showed its potential and benefits [81]. Challenges in medical applications include safety of the system in terms of stability, quality of the force feedback and transparency, regulatory approvals, economical considerations, and challenges related to the environment in which the haptic device is going to be used, such as MRI compatibility [82]. Haptic devices used in surgical environments provided insufficient realism and future doctors were not well trained for complex surgeries. The haptic devices did not provide enough force feedback, and the benefits of these devices are still not documented properly [19]. Haptic devices are still far from being used in medical communities. Implementation of haptic devices in the medical field is a subject of controversy until now and the absence of or limited haptic feedback is one of the reasons inhibiting the growth of haptic devices. Cutaneous-based haptic devices have large variations in design, frequency response, spatial field, and tactile feedback. These variations are not suitable for commercial applications unlike kinesthetic interfaces [1].
The important aspects of VR systems are immersion, interaction, and imagination. Currently, available VR systems provide visual realism and auditory feedback to some extent. However, they provide insufficient haptic feedback, by which humans can understand the virtual world. Lack of high-quality haptic feedback is one of the limitations of haptic devices in VR applications [83]. Human–Computer Interaction (HCI) is a key element that defines the performance of haptic devices in VR applications. Human user, interface device, and virtual environment synthesized by computer are the three factors that contribute to HCI [84]. Over the years, the major challenges of haptic devices have been simulation and sensing of interactive objects in the computer synthesized world. In free space, the haptic device must be capable of free motion and not exerting large resistance on the operator’s hand. In a constrained virtual world, the range of the device inhibits sufficient contact with objects in the virtual world [62,83,84]. A good haptic device requires the following criteria: (1) stiff solid virtual objects, (2) unsaturated virtual constraints, and (3) ample free space. Promising advancements have been made to overcome these challenges by designing haptic devices with low inertia, adjustable impedance range, high sensing resolution for tracking, and adequate workspace for task simulation [83].
Two factors are required for a better teleoperated robotic system: transparency and stability. Transparency can be defined as the extent to which the remote environment and telepresence can be created. Stability can be defined as the amount of haptic information the sensors can relay back to the user. To balance both stability and transparency, tactile feedback and force feedback can be used in multi-modal platforms. However, different kinds of feedback are still under research and most of the prototypes still require in vivo validation. Therefore, the research community is still looking to commercialize teleoperated robotic systems with haptic feedback [85].
Teleoperated robotic systems are completely void of physical contact between the surgeon and the patient. Therefore, surgeons rely on the sensory information they receive from the workstation. Visual feedback is provided using high-definition (HD) cameras in 2D or 3D and there has been tremendous advancements in recent years in development of high resolution cameras. Auditory feedback is provided by the microphones and speakers with high fidelity. However, providing the sense of touch for surgeons lacks behind the other two. This part relates to the complexity of providing haptic feedback for surgeons. Although there has been many research studies focusing on providing realistic haptic feedback in teleoperated surgical robotic systems, the quality of the haptic feedback is still not acceptable among the medical community, which in part relates to the stability of the operation and complexity of the medical procedures [85]. Haptic feedback must relay information to the surgeon to avoid tissue injury. Until now, the control of interactions between the robot, master, and the remote environment has been insignificant [77,85]. Cutaneous feedback is stable but less transparent. Force feedback is more transparent but less stable and can cause possible tissue damage. There has been significant efforts in recent years for improving the quality of haptic feedback for better differentiating between the hard and soft contacts with tissue [86]. Without the help of improved haptic sensations, teleoperated robotic surgeries will continue to only be a subject of research [76,85].
The reproduction of realistic workstations can be considered as another challenge. A high level of visual realism is an aspect to be considered for the development of simulators. Many factors such as forces implemented on the objects, detection of the probe in the virtual environment, and the scale of the virtual environment should be accounted for to produce realistic workstations. In many cases, the force applied by the user is not equal to the output force incident on the virtual objects. Calibration of force feedback is one of the major challenges when virtual simulators are incorporated with haptic devices [40]. Furthermore, in some cases, operators have found it difficult to detect the probe in the virtual environment. Research has been carried out to overcome these challenges. Researchers have combined 3-DoF haptic devices with external modules to provide additional DoF to carry out medical procedures. Moreover, computer graphics have been developed to improve simulations. Recently developed simulation engines are PhysX, Havok, and Bullet [37,40,42]. Surgeons practice a particular kind of surgery more than 100 times to minimize error. Patient safety is one of the significant aspects to be considered for the implementation of haptic devices in medical training. Patient safety mainly depends on error minimization and minimal bleeding. Furthermore, other factors such as financial, psychological, technical, and organizational should be considered. Until now, only a mere representation of the real workstation has been achieved. Haptic simulators that can record sessions provide in-depth feedback must be developed to increase their usage in medical practices [40].
The design, modeling, and fabrication of tactile displays and cutaneous receptors have been challenging, resulting in the widespread use of kinesthetic interfaces [87]. The following challenges may be considered as potential future work in the field:
Improper sensory feedback is recognized as one of the reasons for prosthesis rejection that affects the performance of the system is noises and disturbances are not removed properly.
A common disadvantage of the implementation of haptic devices is the limitation in workspace and space constraint [31], which is particularly investigated during the performance of surgical operations [70]. The significance of the workspace and the idea of multiple contact points in a haptic interface, that requires more research and developments in the future, may lead to the increase of manipulability and dexterity of the operator and may increase the performance of the operation [75]. Due to the kinematic structure of robotic arms, unlike exoskeletal devices, workspace is restricted. Exoskeletons are wearable and hence provide a larger workspace. There exist some solutions such as cutaneous haptic devices that are compact and wearable but are not precise as kinesthetic devices. Kinesthetic devices are preferred over cutaneous devices although they have overall stability issues; however, more research is required to prove [75].
In addition, the application of collaborative mechanisms in teleoperation fashion could be of importance when dextrous motion is required. The solution of using collaborative robots was studied in [72] and the lack of force feedback at the master side was recognized as one of the main issues in using collaborative robots that need to be addressed by more research.
The discrepancies occurring due to improper feedback, high contact speeds, stiff environment setups in cable-driven teleoperation systems require more enhancements [74].
Some haptic devices are heavy and operators find them difficult to operate [75,76,77,78]. The disadvantage of different kinds of haptic devices highlights the need for more research and development to provide high fidelity haptic feedback for users [15]. Table 1 shows some haptic devices developed by different companies, but there are many more emerging every year.
Innovative ideas and inventions are constantly evolving, and this could be owed to the technological advancements happening in the engineering world. Haptic devices have become imperative in not just the engineering world, but also in different other disciplines. The introduction of haptic devices has enabled the possibility of noteworthy interaction between hardware devices and users. Many applications, especially medical training, have attracted a lot of interest in implementing haptic technology. The medical field has benefited due to the handful of haptic devices available. However, the medicinal world is still skeptical about the usage of these devices in surgeries and training. From this paper, it can be inferred that haptic devices are beginning to replace conventional devices. The ease of operation of haptic devices has been highlighted in this paper. However, lack of awareness and expenses associated with the installation of haptic devices prove to be a limitation for their use in different applications. In conclusion, more research on the technical aspects of the haptic devices is required, and the awareness about these devices needs to increase in order to increase the rate of adopting this technology. To investigate the technical aspect, our future work will focus on classifying the haptic devices in terms of their linkage configuration, the actuation and sensory systems used, and their mechanism and solutions to address kinematic challenges such as redundancy. Besides, we are working on manipulability and dexterity analysis of the studied haptic devices and are categorizing the devices based on manipulability indices such as isotropy index.
Conceptualization, K.Z. and Y.M.; methodology, all authors; resources, K.Z.; data, G.S.G.; writing—original draft preparation, G.S.G.; writing—review and editing, all authors; visualization, all authors; supervision, K.Z.; funding acquisition, K.Z. All authors have read and agreed to the published version of the manuscript.
This research was funded by the Natural Sciences and Engineering Research Council of Canada (NSERC) under Discovery Grant 2019-05562 and Ryerson Dean of Engineering and Architectural Science Research Fund (DRF).
