DESCRIPTION

[0037] The present invention relates to virtual reality simulations and more particularly to computer simulations involving the control of a graphical image, such as a graphical image that is a graphical representation of a hand or a portion of a hand. Although the process is illustrated at least partly in the context of controlling a graphical hand, the present invention can be used in other simulation and computer interactive processes and/or to control other graphical images and should not be limited to the examples provided herein.

[0038] FIG. 1 is a schematic illustration of a simulation system 100 according to the invention. The simulation system 100 is capable of generting a virtual reality environment. A display 105 provides a graphical environment 110 to a user. Within the graphical environment 110 is a graphical image 115 . The graphical image 115 may be, for example, a cursor or other graphical object, the position, movement, and/or shape of which is controllable. For example, the graphical image 115 may be a pointer cursor, a character in a game, a surgical instrument, a view from the end of a surgical instrument, a representative portion of the user, or the like. Also within the graphical environment is a graphical object 120 such as a ball, as shown, or any other graphical representation including another graphical image that may be controlled by the user or by another user. A controller 125 in communication with the display 105 is capable of generating and/or controlling the graphical environment 110 , for example by executing program code including an application program related to the simulation. A user object 130 is manipulatable by a user, and the manipulation of the user object 130 controls the position, orientation, shape and/or other characteristic of the graphical image 115 within the graphical environment 110 , for example by directly correlating a position of the user object 130 with a displayed position of the graphical image 115 or by correlating a position of the user object 130 with a rate of movement of the graphical image 115 . Either the entire user object 130 may be manipulatable by the user or a portion of the user object 130 may be manipulatable relative to another portion of the user object 130 . For example, the user object may be a surface that is engaged by one or more hands of a user, such as a joystick, a mouse, a mouse housing, a stylus, a knob, an elongated rigid or flexible member, an instrumented glove, or the like and may be moveable in from one to six degrees of freedom.

[0039] Optionally, haptic feedback may be provided to the user to increase the realism of the virtual reality environment. For example, when a predetermined event occurs within the graphical environment 110 , such as an interaction of the graphical image 115 with the graphical object 120 , the controller 125 may cause an actuator 135 to output a haptic sensation to the user. In the version shown, the actuator 135 outputs the haptic sensation to the user object 130 through which the sensation is provided to the user. The actuator 135 and the user object 130 may be part of a haptic interface device 140 . The actuator 135 may be positioned in the haptic interface device 140 to apply a force to the user object 130 or to a portion of the user object. For example, the haptic interface device 140 may comprise a user object 130 , such as a mouse housing, having an actuator 135 within the user object 130 , such as a vibrating motor within the mouse housing, or the haptic interface device may comprise a user object 130 , such as a mouse, that is mechanically linked to an actuator 135 . Alternatively, the actuator 135 and the user object 130 may be separate structures, and the actuator 135 may provide a haptic sensation directly to the user, as shown by the phantom arrow in FIG. 1 .

[0040] The actuator 135 may provide the haptic sensation actively or passively. For example, the actuator 135 may comprise one or more motors coupled to the user object 130 to apply a force to the user or to the user object 130 in one or more degrees of freedom. Alternatively or additionally, the actuator 135 may comprise one or more braking mechanisms coupled to the user object to inhibit movement of the user or the user object 130 in one or more degrees of freedom. By haptic sensation it is meant any sensation provided to the user that is related to the user's sense of touch. For example, the haptic sensation may comprise kinesthetic force feedback and/or tactile feedback. By kinesthetic force feedback it is meant any active or passive force applied to the user to simulate a force that would be experienced in the graphical environment 110 , such as a grounded force applied to the user or the user object 130 to simulate a force experienced by at least a portion of the graphical image 115 . For example, if the graphical image 115 is positioned against a surface, a barrier or an obstruction, the actuator 135 may output a force against the user object 130 preventing or retarding movement of the user or the user object 130 in the direction of the barrier or obstruction. By tactile feedback it is meant any active or passive force applied to the user to provide the user with a tactile indication of a predetermined occurrence within the graphical environment 110 . For example, a vibration, click, pop, or the like may be output to the user when the graphical image 115 interacts with a graphical object 120 . Additionally, tactile feedback may comprise a tactile sensation applied to approximate or give the illusion of a kinesthetic force. For example, by varying the frequency and/or the amplitude of an applied vibration, variations in surface textures of different graphical objects can be simulated or by providing a series of clicks when a graphical image penetrates an object, resistance to the penetration can be simulated. For example, in one version a kinesthetic force sensation, such as a spring force, may be applied to the user whenever the graphical image 115 engages the graphical object 120 to simulate a selectively deformable surface. Alternatively or additionally, a tactile sensation, such as a pop, may be applied to the user when the graphical image 115 is moved across a surface of the graphical object 120 to simulate a texture of the graphical object 120 .

[0041] The controller 125 may be a computer 150 , or the like, such as the computer shown in FIG. 2 . In one version, the computer 150 may comprise a processor and may be able to execute program code. For example, the computer may be a personal computer or workstation, such as a PC compatible computer or Macintosh personal computer, or a Sun or Silicon Graphics workstation. The computer 150 may be operable under the Windows™, MacOS, Unix, or MS-DOS operating system or similar. Alternatively, the computer 150 can be one of a variety of home video game console systems commonly connected to a television set or other display, such as systems available from Nintendo, Sega, or Sony. In other embodiments, the computer 150 can be a “set top box” which can be used, for example, to provide interactive television functions to users, or a “network-” or “internet-computer” which allows users to interact with a local or global network using standard connections and protocols such as used for the Internet and World Wide Web. The computer 150 may include a host microprocessor, random access memory (RAM), read only memory (ROM), input/output (I/O) circuitry, and/or other components of computers well-known to those skilled in the art. The computer 150 may implement an application program with which a user is interacting via peripherals, such as haptic interface device 140 and/or user object 130 . For example, the application program can be a simulation program, such as an interactive digital mockup of a designed feature, a medical procedure simulation program, a game, etc. Specifically, the application program may be a computer aided design or other graphic design program, an operating system, a video game, a word processor or spreadsheet, a Web page or browser that implements, for example, HTML or VRML instructions, a scientific analysis program, or other application program that may or may not utilize haptic feedback. Herein, for simplicity, operating systems such as Windows™, MS-DOS, MacOS, Linux, Be, etc. are also referred to as “application programs.” The application program may comprise an interactive graphical environment, such as a graphical user interface (GUI) to allow the user to input information to the program. Typically, the application provides images to be displayed on a display screen 155 and/or outputs other feedback, such as auditory signals. The computer 150 is capable of generating a graphical environment 110 , which can be a graphical user interface, game, simulation, such as those described above, or other visual environment. The computer 150 displays graphical objects 120 , such as graphical representations and graphical images, or “computer objects,” which are not physical objects, but are logical software unit collections of data and/or procedures that may be displayed as images by the computer on display screen 155 , as is well known to those skilled in the art. The application program checks for input signals received from the electronics and sensors of the user object 130 , and outputs force values and/or commands to be converted into haptic output for the actuator 135 . Suitable software drivers which interface such simulation software with computer input/output ( 110 ) devices are available from Immersion Corporation of San Jose, Calif. Display screen 155 can be included in the computer and can be a standard display screen (LCD, CRT, flat panel, etc.), 3-D goggles, or any other visual output device.

[0042] In one version of the simulation system 100 , the user object 130 comprises an instrumented glove 160 . Within or on the instrumented glove 160 are one or more sensors that are capable of detecting a manipulation of the glove. A signal indicative of the detected manipulation is provided to the computer 150 , optionally through glove sensor interface 165 , to control the position, orientation, and/or shape of the graphical image 115 , which may be for example a graphical hand 170 as shown in the version of FIG. 2 . 10

[0043] The position of the instrumented glove 160 may be used to control the position of the graphical hand 170 in the graphical environment 110 . The position of the instrumented glove 160 may be detected by one or more position sensors adapted to detect the position of the instrumented glove 160 in one, two, or three dimensions. The position sensor may include a grounded link connected to the instrumented glove 160 . Alternatively, the position sensor may detect the position of the instrumented glove 160 in space, without being physically connected to a reference. For example in one version, the instrumented glove 160 comprises a Polhemus™ or Ascension™ electromagnetic position sensor to detect the three dimensional position of the instrumented glove 160 in space. The position sensor generates a signal related to the position of the instrumented glove 160 and the signal is provided to the computer 150 . The computer 150 then controls the display of the graphical hand 170 in proportion to the signal. In one version, the displayed position of the graphical hand 170 is directly related to the position of the instrumented glove 160 .

[0044] The orientation of the instrumented glove 160 may alternatively or additionally be used to control the graphical hand 170 . An orientation sensor may be provided to detect the absolute or relative rotation of the instrumented glove 160 about one, two, or three orthogonal axes. As with the position sensor, the orientation sensor may be grounded or may be able to detect rotation in space. A signal related to the orientation of the instrumented glove 160 is then provided to the computer 150 which uses the signal to correspondingly control the display of the graphical hand 170 . Accordingly, the rotation of the instrumented glove 160 about an axis results in a displayed rotation of the graphical hand 170 about an axis, for example a parallel axis. In one version, a single sensor may be used to detect both position and orientation. For example, a Polhemus™ or Ascension™ sensor may be used to detect the position of the instrumented glove 160 in six degrees of freedom. The computer 150 may then use the six degree of freedom signal to control the displayed position and orientation of the graphical hand 170 .

[0045] Alternatively or additionally, the shape of the graphical hand 170 (or other graphical image 115 ) may be controlled by a user manipulating the instrumented glove 160 . For example, one or more joint angle sensors may be provided to sense rotation about a particular joint in the hand (or other body part). The computer 150 may then control the display of the graphical hand 160 in relation to the sensed motion within the hand to, for example, show a corresponding movement of the graphical hand 170 . In this way, the shape of the graphical hand 170 can be controlled to in response to manipulation of the instrumented glove 160 by the user. For example, a simulation may comprise the display of the graphical hand 170 to simulate the movement of the user's hand, such as by showing the graphical hand 170 closing and/or grasping when the user closes his or her hand or makes a grasping motion. One or more joint angle sensors 175 may be positioned to detect the movement of a finger of the user. In another version, the movement of a plurality of fingers may be detected. In a relatively simple version, a single digital or analog sensor detects either an open condition or a closed condition of the user's hand, and the computer 150 correspondingly displays the graphical hand 170 either as being open or as being closed or grasping an object in the graphical environment 110 . In another version, the joint angle position sensor may comprise an analog sensor that provides a variable signal by which the display of the graphical hand 170 may be controlled. The joint angle sensor may comprise one or more of a stain gage, a fiber optic sensor, a potentiometer, or the like.

[0046] In one version, the instrumented glove 160 may comprise both a position sensor and one or more joint angle sensors. For example, the instrumented glove 160 may comprise a CyberGlove™ available from Virtual Technologies, Inc. in Palo Alto, Calif., and described in U.S. Pat. Nos. 5,047,952 and 5,280,265, both of which are incorporated herein by reference in their entireties. In this version, individual joint angle sensors 175 comprise two long, flexible strain gages mounted back to back. The strain gage assemblies reside in guiding pockets sewn over a particular joint. When the joint is flexed, one of the strain gages of the corresponding pair of gages is in tension, while the other strain gage is in compression. Each pair of two strain gages comprise the two legs of a half bridge of a common Wheatstone bridge configuration. An analog multiplexer is used to select which of the half bridge voltages is to be sampled by an analog-to-digital converter. The maximum strain experienced by each gage is adjusted by varying the thickness and elastic modulus of the backing to which the gages are mounted. The backing is selected to maximize the signal output without significantly reducing the fatigue life of a gage.

[0047] In use, a user contacts the user object 130 to interact with the graphical environment 110 . In the version shown in FIG. 2 , the user dons the instrumented glove 160 and moves all or a portion of his or her hand to control the graphical hand 170 which mimics the motion of the user's hand. For example, the user may move-his or her hand to the left in order to cause the graphical hand 170 to be rendered so as to appear to touch the graphical object 120 . In additional, the user may slightly close and appropriately move his or her hand to make the graphical hand 170 appear to grasp the graphical object 120 .

[0048] The realism of the simulation can be increased by providing an actuator 135 adapted to provide one or more haptic sensations to the user during the user's interaction with the graphical environment 110 . The actuator may either provide the haptic sensation directly to the user or may apply the haptic sensation to the user through the user object, for example by applying a force to the surface of the instrumented glove 160 . This allows the user to not only visualize the graphical hand 170 contacting the graphical object 120 , but also to receive an indication through the user's sense of touch that the object has been contacted, thereby providing a more immersive experience. The actuator 135 may comprise a palm forcing mechanism 180 for providing a haptic sensation to the palm of the hand, as shown in phantom in FIG. 2 . It has been discovered that by providing a haptic sensation to the palm, the user's perception of realistic interaction with a graphical object 120 is enhanced. For example, a haptic sensation may be provided to the palm in coordination with the graphical hand 160 grasping the graphical object 120 to simulate an actual grasping of an object. Accordingly, in the version of FIG. 2 , the computer 150 controls the output of a haptic sensation to the user's palm by providing a signal, optionally though actuator interface 185 , to cause the palm forcing mechanism to be actuated.

[0049] The actuator 135 may include the palm forcing mechanism 180 , and optionally may additionally be able to provide a haptic sensation to other portions of the user and may include additional actuators. In one version, the haptic sensation is delivered essentially only to the palm. It has been discovered that during some simulations, such as a power grasping simulation, a haptic sensation in the palm is perceived by the user as a realistic sensation. Accordingly, by providing an actuator that delivers a haptic sensation to the palm, realistic haptic feedback for many simulations may be provided in an easily implementable and inexpensive version. In other versions, the palm haptic sensations may be combined with haptic sensations delivered to other portions of the user, as will be described below.

[0050] FIG. 3 shows how the electrical and mechanical signals propagate through an embodiment of the simulation system 100 . In this version, an actuating mechanism, such as a DC servo motor causes the palm forcing mechanism 180 to apply a force to the palm. The computer 150 , or other controller or signal processor, sends a digital value representing the desired actuation level control signal to the digital-to-analog converter 190 . The analog output of the digital-to-analog converter 190 may then be amplified by a variable gain amplifier 195 to produce an analog voltage activation signal. This voltage is placed across the servo motor, driving the motor a desired amount. The voltage signal may alternately be converted to a current activation signal for driving the motor at a desired torque. The joint angle sensors 175 generate an analog signal related to the relative angle of rotation of a joint. The signals from the joint angle sensors 175 are passed through an analog-to-digital converter 200 to provide digitized values to the computer as a physical state signal. In the graphical environment 110 , the physical state signal may then cause motion in a corresponding graphical hand 170 . If the graphical hand 170 or a portion of the graphical hand 170 contacts a graphical object 120 in a predetermined manner, a haptic sensation is output to the palm of the user. For example, in one version, the computer 150 calculates the force to be applied to the palm using data related to the graphical object's shape and/or compliance. The computer 150 then causes an activation signal to be sent to the palm forcing mechanism to convey haptic information about that virtual force. The computer 150 , digital-to-analog converter 190 , analog-to-digital converter 200 , bus 205 and variable gain amplifier 195 may be elements of a signal processor.

[0051] The palm forcing mechanism 180 may be designed to be comfortably positionable in or near the palm of the user. The palm forcing mechanism 180 may be incorporated into the user object 130 or may be a separate mechanism. For example, in one version, the palm forcing mechanism 180 is an independent unit that is adapted to contact the palm of a user. The palm forcing mechanism 180 may either directly contact the palm of the user or may contact the palm through at least a portion of the user object 130 . For example, when used with an instrumented glove 160 , the palm forcing mechanism 180 may be positioned on the outside of the instrumented glove 160 . The palm forcing mechanism 180 may be held in place by a strap that extends around the hand or it may be otherwise attached to the hand, such as by adhesive tape. Alternatively, the palm forcing mechanism 180 may be positioned within the instrumented glove 160 . In addition, the palm forcing unit may include a rigid plate secured to the hand that serves as a grounding member against which the palm forcing mechanism 180 can exert a force. In another version, the palm forcing mechanism 180 may be grounded and the user may place his or her hand on or in proximity to the palm forcing mechanism 180 during use.

[0052] A version of a palm forcing mechanism 180 is shown in FIG. 4A . A user's hand 210 is shown in a cross-section taken along section A-A of FIG. 3 (thumb not shown), and a palm forcing mechanism 180 is positioned within or near the palm 215 of the user. The palm forcing mechanism 180 comprises a force applying member 220 including a surface 225 that is adapted to contact and/or exert a force against the palm 215 when actuated. In this version, the force applying member 220 comprises a deformable member 230 , such as strip of metal, sufficiently stiff to retain a slightly bowed shape, as shown in FIG. 4A . A first end 235 of the deformable member 230 is fixed, for example by being fixed against translation relative to a guide member 240 . A second end 245 of the deformable member 230 is connected to a distal end of a tendon 250 which is slidably disposed within the guide member 240 . The tendon 250 extends through the guide member 240 and is connected at its proximal end to an actuating mechanism 255 . The actuating mechanism 255 is able to exert a pulling force on the tendon 250 , that is it is able to exert a force in the direction shown by arrow 260 . When the actuating mechanism 250 pulls on the tendon 250 , a force is applied to the second end 245 of the deformable member 230 causing the second end 245 to be brought nearer the first end 235 , thereby further bowing the deformable member 230 .

[0053] As more force is applied, the deformable member 230 contacts the palm 215 , and the user experiences a contact sensation, as shown in FIG. 4B . Thus, when the actuating mechanism 255 is controlled by the computer 150 , the application of the contact sensation may be coordinated with a contact event in an ongoing simulation. As a graphical hand 170 , for example, grasps a graphical object 120 , the user may simultaneously visualize the grasp and feel the sensation in his or her palm by actuation of the palm forcing mechanism 180 . Optionally, the actuating mechanism 255 may be capable of continuing to exert a pulling force on the tendon 250 , as shown in FIG. 4 C, to apply a stronger force on the palm 215 . In this way, the simulation system 100 may be able to provide a contact sensation related to the strength of the user's grasping of the graphical object 120 or to simulate a weight or inertia of the graphical object 120 .

[0054] In one version, the deformable member 230 may comprise a leaf spring that is biased into a configuration substantially as shown FIG. 4A . In this version, the tendon 250 may be a flexible cable or wire, sufficiently rigid in tension to transmit the pulling force from the actuating mechanism 255 to the second end 245 of the deformable member 230 but sufficiently flexible to pass from the first end 235 to the actuating mechanism 255 along a non-linear path. For example, the tendon 250 may comprise Dacron™ or Kevlar™ slidably disposed in one or more fixed tubular casings, as described in U.S. Pat. No. 5,631,861 which is incorporated herein by reference in its entirety. Alternatively, the tendon 250 may comprise metallic material or may be a composite. The bias of the leaf spring is used to return the deformable member 230 to a non-contacting position when the pulling force is removed by the actuating member 255 . This simplifies the control process associated with applying a contact sensation. For example, the computer 150 may provide a “pull” signal to drive the actuating mechanism 255 when a contact sensation is desired, and may remove the “pull” signal, resulting in no driving of the actuating mechanism 255 when there is no contact sensation desired, the bias of the spring acting as an active member returning the palm forcing mechanism 180 to the condition of FIG. 4A . In an alternative version, the tendon 250 may be rigid in both tension and compression and the actuating mechanism 255 may drive the tendon 250 in both directions to controllably bow or unbow the deformable member 230 .

[0055] FIGS. 5A and 5B show exploded and assembled views, respectively, of a version of a palm forcing mechanism 180 comprising a biased deformable member 230 and a flexible tendon 250 . This version includes a housing 265 which serves to fix the first end 235 of the deformable member 230 to the guide member 240 . The housing includes a top surface 270 which is shaped to be positionable adjacent the palm 215 of a user. The top surface 270 may either be spaced from the palm 215 or may rest against the palm 215 . It has been discovered that the continuous contact of the top surface 270 against the palm 215 does not affect the user's perception of a contact sensation. The housing 265 may be made of rigid metal, plastic, and/or ceramic material. The housing 265 includes openings 275 in a side wall for receiving prongs 280 extending from the first end 235 of the deformable member 230 . The openings 275 include walls 277 that abut against the prongs 280 to limit translation of the deformable member 230 and to allow the deformable member 230 to bow. The prongs 280 may be rotatable within the openings 275 to facilitate the bowing. A connecting member 285 connects the second end 245 of the deformable member 230 to the tendon 250 . The connecting member 285 comprises eyelets 290 which are received around extending prongs 295 of the second end 245 . The prongs 295 also extend into longitudinally extending slots 300 in the housing 265 . Another eyelet 305 is connected to the end of the tendon 250 by suitable means, such as by gluing, soldering, or wrapping the tendon around the eyelet 305 . Accordingly, as the tendon 250 is pulled, the connecting member 285 pulls on the prongs 295 which then slide within slot 300 . This action causes the deformable member 230 to bow, as discussed above. When the bowing is sufficient to cause the surface 225 of the deformable member 230 to extend above the top surface 270 of the housing 265 , a contact sensation is applied to the palm 215 .

[0056] Another version of the palm forcing mechanism 180 is shown in FIG. 6A . In this version, the deformable member 230 comprises a pivoted member comprising a first bar 310 and a second bar 315 pivotally connected at a hinge 320 . The second bar 315 is also pivotally connected to the guide member 240 , or to a housing, for example at a second hinge 325 . The tendon 250 is attached to the first bar 310 , and as the tendon 250 is pulled, the hinge 320 is forced upwardly and it or another portion of the deformable member 230 contacts the palm 215 . In another version, a strap or the like (not shown) and/or a rigid plate is positioned below the distal end of the first bar 310 to inhibit the distal end from being pulled downwardly, thereby increasing the force applied to the palm 215 by the deformable member 230 . Alternatively or additionally, a housing similar to the one shown in FIGS. 5A and 5B may provide a base against which the deformable member 230 may be forced toward the palm 215 .

[0057] FIG. 6B shows another version of a palm forcing mechanism 180 . This version is shown with a deformable strip 340 , similar to the leaf spring discussed above, as the deformable member 230 but may be used with pivotally connected bars, such as those shown in FIG. 6A . The version of FIG. 6B includes a motor 350 receiving a rotatably driven shaft 355 . The shaft 355 includes interior threads that engage threads on a rod 345 that is connected to an end 347 of the deformable strip 340 . As the motor 350 rotates shaft 355 in a first direction, the rod 345 is retracted into the shaft 355 and the end 347 of the deformable strip 340 is moved toward the motor 350 . The motor 350 is non-rotatably fixed to the other end 348 of the deformable strip 340 , for example by rigid member 360 . Accordingly, movement of the first end 347 toward the motor results in movement of the first end 347 toward the second end 348 and a corresponding bowing of the deformable strip 346 . As shown the motor 350 is positioned within the palm forcing mechanism 180 . Alternatively, the motor 350 may be located remotely, and the rod 345 may pass through a lumen in the rigid member 360 , for example. The actuating mechanism 255 may alternatively be a solenoid or voice coil, or the like.

[0058] FIG. 6C shows a version of the palm forcing mechanism 180 similar to the version shown in FIG. 4A . However, in the version of FIG. 6 C, the actuating mechanism 255 is positioned on the hand of the user, for example by being fixed to the backside of the hand, as shown. This version is advantageous in that the tendon 250 may be contained within the haptic interface device 140 reducing the likelihood of entanglement of the tendon 250 with wires or cables that may be present. FIG. 6C also shows a version of the actuating mechanism comprising a motor 365 which rotatably drives a pulley 370 on which is wound the tendon 250 .

[0059] FIG. 6D shows a version of the palm forcing mechanism 180 where a contact sensation is applied to the palm by pushing on the tendon 250 rather than by pulling on the tendon 250 . In this version, the tendon 250 may be sufficiently rigid in compression to adequately apply a force to the palm forcing mechanism 180 . In this version, the deformable member 230 is inverted and includes a contact member 380 on its inner surface. The contact member 380 includes a surface 385 adapted to engage the palm 215 when the palm forcing mechanism 180 is actuated. To actuate the palm forcing mechanism 180 the tendon 250 is pushed toward the second end 245 of the deformable member 230 as shown by arrow 387 . This forces the second end 245 away from the first end 235 and straightens the deformable member 230 thereby forcing the contact member 380 upward and against the palm 215 . An opening 390 may be provided within the contact member 380 to allow the tendon 250 to pass therethrough in all operative positions of the contact member 380 . Alternatively, the version of FIG. 6D can be used with a flexible tendon 250 . In this version, a the deformable member may be biased so that the ends are biased apart. Tension applied to the tendon 250 overcomes the bias and release of the tension actuates a contact sensation.

[0060] FIG. 6E shows a self-contained version of the palm forcing mechanism 180 with an interior actuating mechanism 255 . The actuating mechanism 255 is on a lower member 395 which is held in place by a housing and/or a strap. The actuating member 255 drives a piston 400 upwardly and against the deformable member 230 to force the deformable member against the palm 215 . Alternatively, the members may be separated from one another and the actuating mechanism 255 may displace the two members to contact the palm 255 . The actuating mechanism may be a solenoid or voice coil actuator or may comprise a rotating threaded shaft that receives a rod attached to the deformable member, as discussed above.

[0061] In one version of the palm forcing mechanism 180 , the tendon 250 may be used to directly apply the force to the palm 215 . For example, the guide member 240 , a version of which is shown in cross-section in FIG. 6 F, may comprise a channel 405 that is shaped to direct the tendon 250 toward the palm 215 . Optionally, the distal end of the tendon 250 may include a force transmitting cap 410 to distribute the force to applied to the palm 215 . As a force is applied in the direction of arrow 415 , the tendon contacts the palm 215 . The cap 410 may include an extension 420 that prevents the cap 410 and the tendon 250 from being retracted into the channel 405 .

[0062] FIG. 6G shows another version of the palm forcing mechanism 180 . In this version, a plate 430 is position below the palm 215 . The plate includes a contacting portion 435 extending toward the palm 215 . The tendon 250 is connected to the plate 430 at an end near the guide member 240 . By pulling on the tendon 250 the plate 430 is moved toward the palm 215 and the contacting member 435 applies a force to the palm 215 . FIG. 6G shows the actuating mechanism 255 on the backside of the hand. Alternatively, a remote actuating mechanism may be used.

[0063] FIG. 6H shows a strap 440 that may be used to secure any of the disclosed palm forcing mechanisms 180 or portions thereof to the hand 210 . The strap 440 may extend around the hand and may have ends that are attachable to one another. For example, as shown, the strap ends are attached by a hook and loop type fastener 450 . Other suitable attachments may also be used. Alternatively, the ends may be separately fastened to the hand, such as by adhesive, or may be fastened to a plate or the like on the back of the hand 210 . The strap 440 may include one or more openings through which elements, such as the tendon 250 , the guide member 240 , or electronic components, may pass.

[0064] The actuator 135 may be designed to reduce its operating space. For example, when applying a force to the palm 215 , or to other portions of the body, it may be undesirable to have the force receiving portion of the body burdened with a large actuator 130 that hinders the movement of the user or reduces the realism of the simulation. In one version of the ivention, the space requirements of the actuator 135 may be reduced by a mechanism that allows the force applied to the user to be in a different direction than the actuated force. For example, in the embodiments of FIGS. 4 A- 4 C, 5 A- 5 B, 6 A- 6 D, and 6 G, the actuators 135 reduce their footprints by providing a force to the user in a direction substantially orthogonal to the direction of the application of an actuating force. This application of a substantially orthogonal force is advantageous in that the space occupied by the actuator 130 may be reduced allowing for the actuating mechanism 255 to be, for example, remotely located and allowing for the tendon 250 to be less obstructively directed away from the portion of the body to which the force is being applied. The other embodiments may also be designed to reduce the amount of position of space they occupy.

[0065] In another version of an actuator 135 that allows for the application of an actuating force in a direction different than, and optionally substantially orthogonal to, the direction of an applied force to the user, a tendon 250 may be used to initiate a cam action that applies a force to the user. Examples of cams are shown in FIGS. 7A though 7 D. In the version of FIG. 7 A, the palm forcing mechanism 180 comprises a lower member 460 and an upper member 465 which includes a surface 470 adapted to apply a force to the palm 215 . Pivotally and eccentrically attached to the lower member 460 is a cam 475 . The tendon 250 is attached to the cam 475 so that a force on the tendon 250 in the direction of arrow 480 causes the cam 475 to rotate and separate the lower member 460 from the upper member 465 . Since the lower member 460 is grounded, for example by strap 440 , the upper member 465 is forced upwardly and the surface 470 applies a contact sensation to the palm 215 . A linear cam is shown in the version of FIG. 7B . The tendon 250 is connected to a block than includes a cam surface 500 . Above the block 485 is a force applying member 490 that has a lower cam surface 500 and an upper surface 495 for contacting the palm 215 . When the block 485 is pulled by the tendon 250 the cam surfaces 500 cause the force applying member 490 to move upwardly and contact the palm 215 . the guide member 240 may include an extension 510 to guide the movement of the force applying member 490 . In the version of FIG. 7 C, the lower member 460 comprises a threaded rod 515 . A nut 520 having internal threads is received on the rod 515 . The tendon 250 is attached to the outer surface of the nut 520 so that a force in the direction of the arrow 480 causes the nut to rotate about the rod axis, thereby moving the nut 520 upward and applying a force to the palm 215 . The force may be applied by a surface 525 on the nut, or an upper member may be attached to the nut 520 . Alternatively, the rod and nut may be replaced by a telescoping arrangement 530 where rotation of a pulley 535 results in extension of telescoping members, as shown in FIG. 7D . Alternatively, a turn-buckle type arrangement may be used.

[0066] FIG. 8 shows a pneumatic version of the palm forcing mechanism 180 . An inflatable air bladder 540 is positioned under the palm 215 . A pump 545 or other source of pressurized air is connected to the air bladder 540 by way of a controllable valve 550 . In one version, the computer 150 controls the opening and closing of the valve 550 to selectively allow for inflation of the air bladder 540 . Alternatively, the computer 150 may control the operation of the pump 545 . In one version, the air bladder 540 may include a spring mechanism that biases the air bladder 540 into the deflated position shown in FIG. 8 and the valve 550 may be a one-way valve when in the closed condition so that the force on the palm 215 may be removed when the valve is closed. In another version, one or more openings may be provided in the bladder 540 that allow the escape of air. When the valve is open the volumetric flow rate of the inflow of fluid may be greater than the volumetric flow rate of the escape of fluid to achieve a desired inflation of the air bladder 540 .

[0067] Any of the above palm forcing mechanism 180 may be modified to apply multipoint contact of the palm 215 . For example, FIG. 9 shows a version of the palm forcing mechanism 180 where the deformable member 230 comprises a first portion 555 and a second portion 560 that contact the palm 215 at different locations when the tendon 250 is pulled a sufficient amount. Optionally, the tendon 250 may pass through openings 565 in the portions to increase the stability of the deformable member 230 . The multi-point contacting ability can provide a realistic contact sensation to the user when the graphical object 120 to be grasped has a non-continuous surface. In a more advanced version, the two portions may be separately actuated by two separate tendons. This allows for multiple points to be independently contacted by the palm forcing mechanism 180 .

[0068] The palm forcing mechanism 180 may be used to apply a tactile sensation, such as a vibration, to the palm. For example, in the version shown in FIG. 4 A, the actuating mechanism 255 may apply an oscillating or cyclical pulling force on the tendon 250 to vibrate the palm. In one version the force applying member 220 may cyclically contact the palm 215 . In another version, the force applying member 220 may be in continuous contact with the palm 215 and the haptic information may be provided through the tactile sensations where the magnitude of the force applied to the palm is varied. Any of the above disclosed embodiments may be vibrated in this manner. Additionally, the palm forcing mechanism 180 may comprises a rotating eccentric mass, as described in U.S. Pat. No. 6,088,017 which is incorporated herein by reference in its entirety.

[0069] The actuating mechanism 255 may comprise a servo motor, solenoid, or voice coil as described above or may comprise other actuators known in the art, such as piezoelectric, shape memory alloy, pneumatic, hydraulic, and vapor pressure actuators. Other known electrical, electromagnetic, electromechanical or the like actuators may alternatively or additionally be used.

[0070] Any of the above described forcing mechanisms may be adapted to apply a force to a portion of a user's body other than the palm. For example, as shown in FIG. 10, a force applying member 220 may comprise a deformable member 230 actuated by a tendon 250 to apply a haptic sensation to the tip of a finger. A low profile version of the mechanism may be worn under or over an instrumented glove 160 to provide realistic haptic sensations to the user's finger.

[0071] The palm forcing mechanism 180 may be used in coordination with other forcing mechanisms. For example, the palm forcing mechanism 180 may be used with a device capable of provide haptic feedback to one or more fingers, as shown in FIG. 11 . In this version, the haptic interface 140 comprises a finger forcing mechanism 570 and the palm forcing mechanism 180 . The finger forcing mechanism 570 may comprise a tip portion 575 adapted to contact a portion of a user's finger. The tip portion 575 is connected to a tendon 250 ′. Tendon 250 ′ may be pulled by an actuating mechanism (not shown) to exert a force on the finger. A force-augmenting structure 580 may also be provided to provide more realistic forces to the finger tip. In one particular version, the palm forcing mechanism 180 is used in conjunction with a CyberGrasp™ device available from Virtual Technologies, Inc. and described in U.S. Pat. Nos. 5,631,861 and 6,042,555, both of which are incorporated herein by reference in their entireties. The finger forcing mechanism 570 is worn on the hand and applies computer-controlled force feedback to one or more, preferably each, of the fingers. The haptic interface 140 may be advantageously used to simulate the interaction of the graphical hand 170 and a graphical object 120 . An instrumented glove 160 is worn to control the graphical hand 170 . The user uses his or her hand to grasp the graphical object 120 with the graphical hand 170 . The computer transmits force commands to the actuating mechanisms associated with the finger forcing mechanism 570 so the user may “feel” the graphical object 120 in his or her fingertips. The computer also transmits force commands to the palm forcing mechanism 180 so that the user may also “feel” the graphical object 120 in his or her palm. Thus, both precision grasps, which primarily use the finger tips, and power grasps, where an object is held against a user's palm, may be simulated. The palm forcing mechanism 180 in this version may be held in place by strap 440 which extends around the hand and is attached to a palm plate used to secure the finger forcing mechanism 570 to the hand. In one version, the palm forcing mechanism 180 and the finger forcing mechanism 570 may use the same force generating and force transmitting system, for example one or more DC motors with spools for respectively winding the tendon 250 and the tendon 250 ′. The more electrical current that is send to the motors, the more torque is applied to the spools to force the tendons 250 , 250 ′. In another version, the palm forcing mechanism 180 may be used with the CyberTouch™ device available from Virtual Technologies, Inc., and described in U.S. Pat. No. 6,088,017 which is incorporated herein by reference in its entirety. Additionally or alternatively, the palm sensing mechanism 180 may be used with the CyberForce™ device available from Virtual Technologies, Inc. and described in U.S. Pat. Nos. 5,631,861 and 6,042,555 and in U.S. Provisional Patent Application No. 60/191,047 filed on Mar. 21, 2000, all of which are incorporated herein by reference in their entireties. In this version, the haptic interface 140 comprises a finger forcing mechanism 570 , a palm forcing mechanism 180 and a grounded force applying member attachable to the user at the wrist, for example. The grounded force applying member may be capable of applying forces in from one to six degrees of freedom and may also be capable of detecting movement in from one to six degrees of freedom.

[0072] Another version of the simulation system 100 according to the present invention comprises a user object 130 capable of detecting the position of the hand of a user, but that does not have to be worn like a glove. Instead, the user may place his or her hand in contact with the user object 130 in order to interact with the graphical environment 110 . For example, as shown in FIG. 12 , the user object 130 may comprise a mouse 600 which is manipulatable in at least a planar workspace 605 by a user. This version is convenient and inexpensive to implement while still providing an advantageous virtual reality experience to the user. U.S. Pat. Nos. 6,211,861, 6,100,874, 6,166,723, U.S. patent application Ser. No. 09/585,741 filed on Jun. 2, 2000, and U.S. Provisional Patent Application No. 60/224,584 filed on Oct. 11, 2000 describe versions of haptic mice and are incorporated herein by reference in their entireties.

[0073] The mouse 600 of FIG. 12 adapted to control the graphical image 115 in the graphical environment 110 . Either the entire mouse 600 may serve as the actuatable user object 130 or the housing 615 of the mouse 600 may serve as the actuatable user object 130 . As the mouse 600 is manipulated by a user, its position is detected and the manipulation is communicated to the computer 150 to, for example, control the positioning of the graphical image 115 on a computer screen 155 . Mouse 600 is an object that may be grasped or gripped or otherwise contacted by the hand to be manipulated by a user. By grasp in this context it is meant that users may releasably engage a portion of the object in some fashion, such as by hand, with their fingertips, etc. In the described embodiment, mouse 600 is shaped so that a user's fingers or hand may comfortably grasp the object and move it in the provided degrees of freedom in physical space. For example, a user can move mouse 600 to provide planar two-dimensional input to a computer system to correspondingly move the graphical image 115 , such as a graphical hand. In addition, mouse 600 may includes one or more buttons 620 a, 620 b, 620 c, 620 d, 620 e to allow the user to provide additional commands to the computer 150 , as will be described below. Typically, the mouse 600 is a smooth- or angular-shaped compact unit that is designed to fit under a user's hand, fingers, and/or palm, but can also be implemented as a grip, finger cradle, cylinder, sphere, planar object, etc. or may assume the shape or contour of a portion of a person's body.

[0074] In the version of FIG. 12 , the mouse 600 rests on a ground surface 605 such as a tabletop, mousepad, or a platform. A user grasps the mouse 600 and moves the mouse 600 in a planar workspace on the surface 605 as indicated by arrows 630 . Mouse 600 may be moved relative to the ground surface 205 , and in one version may be picked up and placed in a different location. In another version, the mouse 600 is linked to the ground surface 205 . A frictional ball and roller assembly (not shown in FIG. 12 ) can in some embodiments be provided on the underside of the mouse 600 to translate the planar motion of the mouse 600 into electrical position signals, which are sent to the computer 150 over a bus 205 as is well known to those skilled in the art. In other embodiments, different mechanisms and/or electronics can be used to convert mouse motion to position or motion signals received by the computer 150 , as described below. Mouse 600 may be a relative device, in which its sensor detect a change in position of the mouse, allowing the mouse 600 to be moved over any surface at any location. Alternatively, an absolute mouse may also be used, in which the absolute position of the mouse 600 is known with reference to a particular predefined workspace. The bus 205 , which communicates signals between mouse 600 and computer 150 may also provide power to the mouse 600 . Components such as actuator 135 may require power that can be supplied from a conventional serial port or through an interface such as a USB or Firewire bus. In other embodiments, signals can be sent between mouse 600 and computer 150 by wireless transmission/reception. In some embodiments, the power for the actuator can be supplemented or solely supplied by a power storage device provided on the mouse 600 , such as a capacitor or one or more batteries. Some embodiments of such are disclosed in U.S. Pat. No. 5,691,898, which is incorporated herein by reference in its entirety.

[0075] Mouse 600 may include or be acted on by an actuator 135 which is operative to produce forces on the mouse 600 and thereby provide haptic sensations to the user. The mouse 600 may be either a tactile mouse or a kinesthetic force feedback mouse, or both. In one version a tactile mouse comprises, for example, an actuator 135 positioned within the mouse 600 and outputs a force to the housing 215 of the mouse 600 . This version is particularly useful in providing tactile sensations, such as vibrations, to the user. In one version, the actuator 135 comprises a grounded link that is connected to the mouse 600 to provide kinesthetic force feedback to the mouse 600 in two or more degrees of freedom, for example by forcing the mouse 600 in the direction of arrows 230 . This version is particularly useful in kinesthetically simulating contours and the feel of objects. Each of these versions will be described herein below.

[0076] FIG. 13 is a side cross-sectional view of a version of the mouse 600 of FIG. 12 where the mouse is a tactile mouse 650 . Tactile mouse 650 includes one or more actuators 135 for imparting haptic feedback such as tactile sensations to the user of the tactile mouse 650 . The actuator 135 outputs forces on the tactile mouse 650 which the user is able to feel. The embodiment of FIG. 13 is intended to provide inertial forces rather than contact forces; contact forces are described with respect to FIG. 14 . In some embodiments, two or more actuators 135 can provide inertial forces or contact forces, or one actuator 135 can provide inertial forces, while a different actuator 135 can provide contact forces.

[0077] Tactile mouse 650 includes a housing 615 , a sensing system 655 , and a tactile actuator assembly 660 . Housing 615 is shaped to fit the user's hand like a standard mouse while the user moves the tactile mouse 650 in the planar degrees of freedom and manipulates the buttons 620 a - 620 e. Other housing shapes can be provided in many different embodiments.

[0078] Sensing system 655 detects the position of the tactile mouse 650 in its planar degrees of freedom, e.g. along the X and Y axes. In the described embodiment, sensing system 655 includes any one of known sensing technologies. For example, in the version shown, a standard mouse ball 665 for providing directional input to the computer 150 . Ball 665 is a sphere that extends partially out the bottom surface of the tactile mouse 650 and rolls in a direction corresponding to the motion of the tactile mouse 650 on a planar surface 605 . For example, when the tactile mouse 650 is moved in a direction indicated by arrow 670 (y direction), the ball rotates in place in a direction shown by arrow 675 . The ball motion can be tracked by a cylindrical roller 680 , or the like, which is coupled to a sensor 685 for detecting the motion of the mouse 600 . A similar roller and sensor can be used for the x-direction which is perpendicular to the y-axis. Other types of mechanisms and/or electronics for detecting planar motion of the tactile mouse 650 can be used in other embodiments. In some embodiments, high frequency tactile sensations can be applied by the actuator that cause a mouse ball 665 to slip with respect to the frictionally engaged rollers. In another version, an optical sensor that has no moving mouse ball component may be used. A suitable optical mouse technology is made by Agilent of Palo Alto, Calif. and can be advantageously combined with the tactile sensation technologies described herein, where the optical sensor detects motion of the mouse relative to the planar support surface by optically taking and storing a number of images of the surface and comparing those images over time to determine if the mouse has moved. For example, the IFeel™ mouse device from Logitech Corporation uses this type of sensor.

[0079] Buttons 620 a - 620 e can be selected by the user as a “command gesture” when the user wishes to input a command signal to the computer 150 . The user pushes a button 620 down (in the degree of freedom of the button approximately along axis z) to provide a command to the computer 150 . The command signal, when received by the computer 150 , can manipulate the graphical environment in a variety of ways. In one embodiment, an electrical lead can be made to contact a sensing lead as with any mechanical switch to determine a simple on or off state of the button. An optical switch or other type of digital sensor can alternatively be provided to detect a button press. In a different continuous-range button embodiment, a sensor can be used to detect the precise position of one or more of the buttons 620 a - 620 e in its range of motion (degree of freedom). In some embodiments, one or more of the buttons 620 a - 620 e can be provided with force feedback (instead of or in addition to the tactile feedback from actuator 135 ), as described in copending U.S. patent application Ser. No. 09/235,132, filed on Feb. 18, 1999 and which is incorporated herein by reference in its entirety. In one version, the buttons 620 may be used to control the shape of a graphical image 115 , such as a graphical hand 170 , as will be described below.

[0080] The tactile actuator assembly 660 may include an actuator assembly including an actuating mechanism 690 , such as a motor, a flexure mechanism (“flexure”) 695 , and an inertial mass 700 coupled to the actuating mechanism 690 by the flexure 695 . The inertial mass 700 is moved in a linear direction by the actuating mechanism 690 , for example approximately in the z-axis 705 which is approximately perpendicular the planar workspace of the mouse 600 in the x- and y-axes, e.g. the mouse's position or motion is sensed in the x-y plane. The tactile actuator 660 is coupled to the housing 615 of the tactile mouse 650 such that inertial forces caused by the motion of the inertial mass 700 are applied to the housing 615 of the tactile mouse 650 with respect to the inertial mass, thereby conveying haptic feedback such as tactile sensations to the user of the tactile mouse 650 who is contacting the housing 615 . Thus, the actuating mechanism 690 need not directly output forces to the user or to a user-manipulatable object, but instead the moving mass creates an inertial force that is indirectly transmitted to the user. Thus, the inertial mass is used as a grounding reference for tactile sensations, and the housing 615 may serve as a palm forcing mechanism 180 . Alternatively, the actuating mechanism 690 may directly apply the forces or may be coupled to a rotating eccentric mass.

[0081] One version of the tactile mouse 650 provides linear output forces using a rotary actuator, i.e. an actuator outputting a rotary force (torque). In the current actuator market, rotary actuators such as rotary DC motors are among the most inexpensive types of actuators that still allow high bandwidth operation (when driven with signals through, for example, an H-bridge type amplifier). These types of motors can also be made very small and output high magnitude forces for their size. Thus, actuating mechanism 690 may be a DC motor, but can be other types of rotary actuators in other embodiments. For example, a moving magnet actuator can be used instead of a DC motor; such an actuator is described in detail in copending patent application No. 60/133,208, incorporated herein by reference. Other types of actuators can also be used, such as a stepper motor controlled with pulse width modulation of an applied voltage, a pneumatic/hydraulic actuator, a torquer (motor with limited angular range), shape memory alloy material (wire, plate, etc.), a piezo-electric actuator, etc. The tactile mouse 650 in the version shown in FIG. 13 makes use of low cost flexure as a mechanical transmission to convert a rotary actuator force to a linear force that is used to move the inertial mass, and to also amplify the forces to allow more compelling haptic sensations. Versions of the flexure are described in U.S. patent application Ser. No. 09/585,741. In the described embodiment of FIG. 13 , tactile actuator 660 has a stationary portion which is coupled to a part of the housing 615 (and thus stationary only with respect to the portion of the mouse housing to which it is coupled), for example by being coupled to bottom portion 710 of the housing 615 . A rotating shaft of the actuating mechanism 690 is coupled to the moving portion of the assembly that includes the inertial mass 700 and at least part of the flexure 695 , where the inertial mass moves linearly approximately along the Z-axis. The actuating mechanism 690 is operative to oscillate the inertial mass 700 (or itself in some embodiments) quickly along an axis which is approximately parallel to the Z axis. Thus, forces produced by the oscillation of the inertial mass 700 are transmitted to the housing 615 through the tactile actuator 660 and felt by the user.

[0082] Alternatively, directed inertial forces can be output along the X and Y axes in the planar workspace of the device and can be compensated for to prevent or reduce interference with the user's control of the device. One method to compensate is to actively filter imparted jitter in that workspace, as disclosed in U.S. Pat. No. 6,020,876 which is incorporated herein by reference in its entirety. The x and y directed tactile sensations may also provide advantageous and authentic virtual reality related tactile sensations.

[0083] One way to direct an inertial force is to directly output a linear force, e.g., a linear moving voice coil actuator or a linear moving-magnet actuator can be used, which are suitable for high bandwidth actuation. These embodiments are described in greater detail in U.S. Pat. No. 6,211,861 which is incorporated herein by reference in its entirety. These embodiments allow for high fidelity control of force sensations in both the frequency and magnitude domains, and also allow the forces to be directed along a desired axis and allows for crisp tactile sensations that can be independently modulated in magnitude and frequency.

[0084] FIG. 14 is a side elevational view of another version of a tactile mouse 650 ′. In this version, the linear motion provided by the tactile actuator 660 is used to drive a portion of the housing 615 (or other member) that is in direct contact with the user's hand (finger, palm, etc.). The tactile actuator 660 of this version includes an actuating mechanism 690 , flexure 695 , and inertial mass similar to the version of FIG. 13 (except that the actuating mechanism and flexure of FIG. 14 are shown rotated approximately 90 degrees with respect to FIG. 13 ). The tactile mouse 650 ′ of FIG. 14 includes a moving cover portion 720 which can be part of the housing 615 . Cover portion 720 is coupled to the rest of the housing 615 by a hinge allowing their respective motion, such as a mechanical hinge, a flexure, rubber bellows, or other type of hinge. Cover portion 720 may thus rotate about an axis B of the hinge. In other embodiments, the hinge can allow linear or sliding motion rather than rotary motion between cover and housing portions. In the embodiment shown, the cover portion 720 extends in the y-direction from about the mid-point of the mouse housing to near the back end of the tactile mouse 650 ′. In other embodiments, the cover portion 720 can cover larger or smaller areas. Various embodiments of such a moveable cover portion are described in copending patent application Ser. No. 09/253,132 which is incorporated herein by reference in its entirety. The cover portion 720 is rotatably coupled to a link 725 , and the link 725 is rotatably coupled at its other end to the linear moving portion the flexure 695 . Thus, as the member of the flexure 695 is moved along the z-axis, this motion is transmitted to the cover portion 720 through the link 725 , where the rotational couplings of the link allow the cover portion 720 to move about axis B of the hinge. The actuating mechanism 690 can drive the flexure 695 up on the z-axis, which causes the cover portion 720 to move up to, for example, the dashed position shown.

[0085] The user feels the force of the cover portion 720 against his or her hand (such as the palm) as a contact force (as opposed to an inertial force). When the cover portion is oscillated, the user can feel a vibration-like force. Accordingly, the cover portion 720 may be used as a palm forcing mechanism 180 to simulate a contact sensation at the palm. The cover portion can also be used to designate 3-D elevations in a graphical environment. In some embodiments, the configuration described can inherently provide an inertial force as well as the contact force if an inertial mass is moved as described above in addition to the contact portion. In other embodiments, a different “contact member” (e.g. a member that is physically contacted by the user) can be moved instead of the cover portion 320 but in a similar fashion, such as one or more of the mouse buttons 620 a - 620 e or other buttons, tabs, mouse wheels, or dials. Furthermore, in some embodiments multiple actuator assemblies can be used to drive a cover portion and one or more buttons 620 a - 620 e or other controls of the tactile mouse 250 ′. Furthermore, in some embodiments, one actuator assembly can be used to move a cover