Properties of Mechatronic System for Hand Rehabilitation

The article describes an innovative mechatronic device for hand rehabilitation, which enables diagnostics, comprehensive exercises and reporting the rehabilitation results of individual fingers of people who have lost their full efficiency as a result of past illnesses (i.a. stroke) and orthopedic injuries. The basic purpose of the device is to provide controlled, active exercises of the individual fingers, to widen the range of their movements, and to increase their precision of movement. The developed mechatronic device works with original software for PCs containing a diagnostic module, reporting module and a set of virtual reality exercises using biofeedback. The device uses auditory and visual biofeedback, and electromyography (EMG).


INTRODUCTION
Cerebral stroke affects about 15 million people worldwide each year.Among stroke survivors, 1/3 of them suffers from a large functional deficit, mainly in a form of spasticity.It is one of the painful consequences of stroke and is manifested by excessive and incorrect muscle tension, among others located within the upper limb.The contracted muscles prevent the full range of movements and also cause pain.Symptoms that indicate spasticity within the upper limb are, for example, an excessively clenched fist or a strong, unnatural flexion of the wrist or elbow.It makes it difficult or impossible to perform everyday tasks requiring manual handling (holding a cup of tea, dressing up, phone operations).It is estimated that spasticity develops in 20-40% of patients within three months of the onset of stroke.However, the consequences of stroke can be effectively worked on.In many cases, spasticity is not an irreversible process.Due to appropriate exercises, effective and modern methods of rehabilitation as well as patient motivation, it is possible to regain the lost efficiency [4,21].
The current level of technology allows to introduce new medical and rehabilitation solutions.One of them is the functional feedback therapy, for example, used in the upper limb rehabilitation, whereby patients during exercise can watch the effects of their exercise in a graphical form on a computer screen or on a mobile device.This type of therapy is also called biofeedback [16,19,22].
One example of a device for the rehabilitation of the upper limb with special attention to hand improvement is the product of Kinestica called Bimeo PRO.The device consists of a detachable ball halves, two electronic measuring modules attached to the wrist and elbow, and the base to which the entire ball or its two halves is attached.The device provides hands-on exercises based on the most commonly used daily activities performed by a hand.The rehabilitation process is carried out in conjunction with the performed tasks shown on the monitor.
Another example is a mechatronic device called the System Pablo for the rehabilitation of people with hand dysfunctions .The device consists of a universal mechatronic mandrel and a set of auxiliary passive external devices in which the mandrel is placed.The mandrel is a device used for the evaluation and functional training of the fingers, hands and arms.It provides the ability to measure the strength in the fingers and the range of movements, combined with the ability to exercise movement and audiovisual training displayed on the screen.An auxiliary passive module, in conjunction with a spindle, is designed for training the wrist and elbow joint.Exercises are based on the ball on which the patient's open hand rests.
The last solution, which uses EMG biofeedback, is a mechatronic device that supports the movement of the fingers of the hand, based on the external exoskeleton.In the exoskeleton of the hand developed by Prof. K.Y.Tong's team, each finger of this exoskeleton has one degree of freedom and is driven by electric motors.The control is carried out with EMG signals from forearm muscles.
In the case of the two first solutions (Bimeo PRO, System Pablo), these are devices for hand rehabilitation, with a special focus on wrist and fingers in a limited range.The last of these solutions is a typical example of an exoskeleton, where an extensive module responsible for finger movement is controlled by EMG signals [7,14,18,20].In this type of solution, the priority is the ability to carry out the movement, whereas the secondary aspect is the rehabilitation process in terms of the possibility of realizing the assumed motion of the thumb in combination with the virtual reality, EMG and reporting the obtained results [3,9,13,23].

SYSTEM CONCEPT
The device is designed for the rehabilitation of the upper limb with particular regard to the ability to exercise individual fingers, widening the range of their movements, and to increase their precision of movement [1,6,10,11,15].The proposed solution associated with exercising the range of movements, other for the thumb and other for the rest of the fingers, is dictated by the need for higher precision and maneuvers of the thumb, compared to other fingers, which in turn is connected with daily activities people deal with today (including operating smartphones).In addition, the solution provides the ability to exercise precision movements with a widely understood biofeedback significantly affecting the speed of convalescence.
The mechatronic device is designed for the rehabilitation of the right hand and includes an input module connected to electromyographic sensors via an electronic module and a computer equipped with a monitor, mouse and keyboard (Fig. 1).
The input module contains a base that has a bulge in the front.On both sides of the base there is a length regulator.The aforementioned soft, spongy bulge is a support for the wrist of the rehabilitated hand.By using the length adjusters it is possible to change the distance of the bulge from the main module, so that the device can be adjusted to the anthropometric dimensions of the user's hand.
On the base, in its rear part, there is a shapematched head for a rehabilitated hand placed on it.The head is spherical with lateral protuberance and upper flattening.
On the side surface of the input module head there are four grooves with buttons located therein.In addition, the next fifth of the buttons for the thumb is located at the top on the opposite side of the head relative to the other four strain gauges.Under each of the buttons in the input module head the is a built-in strain gauge.
Behind the four lateral finger buttons there are four further grooves in which four rotary wheels are located.On the upper surface of the head, at the top of it, there is a joystick.The joystick was developed for the rehabilitation of the thumb (Fig. 2).The conceptual design of the mechatronic hand rehabilitation device was developed using CAD computer software in Autodesk Inventor.
The rehabilitation of a patient suffering from dysfunction of the upper limb having the ability to perform the movement of the injured hand, but to a limited degree, is carried out in such a way that the rehabilitated arm is placed on a base whose distance from the head is adjustable depending on the size of the hand.The wrist is attached to the bulge and all fingers are placed on buttons.Electromyographic sensors are placed on the patient's forearm.
In the first stage, the force of finger press is practiced.The patient compresses, in a strictly defined way, the buttons under which the strain gauges are located in the input module head.Information about forearm muscles is recorded by electromyographic sensors and via the electronics module transferred to the computer.Muscle reactions are imaged on a computer monitor.
The second stage involves practicing a range of motion of the four fingers that the patient places on the swivel castors and performs reciprocating movements that are performed on four parallel tracks.
In the third stage, the range of the thumb's movements is practiced.The patient places his/her thumb on the joystick and performs tasks displayed on the computer monitor.
In the fourth step, the patient connects and coordinates both the force exercises and the range of finger movements with the thumb.The patient imitates the displayed trajectory of motion in the form of a predefined vertical axis range in which it is intended to stay or perform a task of avoiding obstacles by squeezing the buttons on the head or by performing reciprocating motion of four fingers placed on the swivel castors and any thumb movement on the joystick.
The main module of the mechatronic hand rehabilitation device cooperates with the EMG sensor, which allows to measure the level of forearm muscles tension responsible for finger activity.This sensor has leads in the form of conductors and biomedical electrodes that are placed on the user's skin within the examined muscle.
The advantage of using a mechatronic device for the hand rehabilitation is the ability to perform rehabilitation based on the performance of the exercises shown on the monitor and the patient's ability to observe how the body reacts to controlled movements.The patient obtains the ability to observe, analyze and learn the correct muscle response, which results in the realization of the desired movement by an arbitrarily chosen finger.Information is displayed on the monitor.Additionally, a set of feedback-support games has been developed in order to properly respond to reactions that are important in the rehabilitation process.

SOFTWARE
Software of the developed mechatronic device is intended for PC and was written in Python programming language.This is a high-level programming language with a large program library to generate the code [5,8].Python features include: objectivity, transparency and clarity of source code.When writing code, Atom was used as a universal and powerful text editor for developers.The developed solution includes exercises and a biofeedback game using data taken from sensors located within the hand rehabilitation device.The program communicates with the device via a USB port.
The first part of the program deals with the import of libraries and classes that are contained in them.The most important libraries used in the program are: • PyQt4 -allows to create a graphical interface of the program; • Matplotlib -a library for advanced graphs, NumPy; • pySerial -allows data transmission to external devices via serial port; • Pygame -a library for creating simple games.Then a number of global variables used later in the application were declared [12,17,24].The Main_window() class is responsible for displaying the main menu of the program as soon as it is started.From this level of the program it is possible to go to exercises or biofeedback games, get help on connecting the hardware part of the system to the computer and exit the program (Fig. 3).This was done based on the PyQt4 library.
If the Arduino module is properly connected to the computer, the user will be moved to the settings window, otherwise a message, indicating that the error has occurred, will appear.The seri-al_ports() function checks if external devices are connected to the computer that require the USB port to be emulated as a COM port.The Error_ Arduino() class displays information about an Arduino problem in the form of a simple dialog box.If there are problems with communication with Arduino or incorrect sensor data readings, user has the opportunity to get support with connecting sensors and Arduino properly.
Prior to rehabilitation, users have the option to set exercises parameters, which allows them to adjust the training to individual needs, such as the level of dysfunction.There is also a selection of the sensor used in the exercise (Fig. 4).Opening the next list results in displaying the available COM ports from which the one connected to the Arduino should be chosen (usually only one port is available).This is accomplished through the previously described functions serial_ports().
Options_1() class is responsible for displaying the settings window for biofeedback exercises (Fig. 4).The interface was created using PyQt4 and widgets such as QComboBox and QSpinBox, which allow to choose from the drop-down list i.a. a type of exercise.Each parameter change causes the suitable method to run.Most of them make an assignment to the global variable that corresponds to the made selection.The port_selection() method opens a user-selected serial port, so that Arduino data can be transmitted.It uses the commands from the pySerial library.Method start_exercise() checks using conditional "if statement" whether all the parameters have been set.If so, then a window with the appropriate exercise is started.Otherwise, user receives an error message.
Using the QTextEdit widget derived from the PyQt4 library, there are presented tips for setting the range of exercise limits for individual sensors.These values are determined experimentally and the differences between the sensors are associated with their measuring range [2].
During biofeedback exercises a graph is generated from the data obtained from the selected sensor.It is updated in real time.In the place where the graph is drawn, there is also an exercise pattern in the form of a colored trajectory.The task of the patient is to perform such movements or muscles tension to produce a graph within the prescribed range (Fig. 5).
The Graph_window() class is started by the appropriate method of the class Option_1() after clicking the "Start Exercise" button in the "Biofeedback Settings" window.The PyQt4 library was used to create the interface, but the graph widget is displayed by calling the CustomFigCanvas() class.The "Biofeedback Exercises" window displays a pink trajectory (if the user chooses this type of exercise), within which the patient should try to place a trace of the blue graph that is generated in real time from the data obtained from the selected sensor (Fig. 5).
At the end of the training it is possible to obtain the final report, which contains the result of the exercise (Fig. 6).It contains information about exercise settings, the date and time of the  The software also includes a biofeedback game.A user-controlled object is a blue ball that moves in a vertical axis.If the value obtained from the selected sensor is less than the threshold value set in the game, the ball drops.The appropriate reaction of the patient causes the threshold to be exceeded and the ball moves upwards.The patient's task is to perform appropriate, controlled muscle tension or make moves with individual fingers.
There are also obstacles in the game.They move from right to left of the window.The purpose of the gamer is to avoid obstacles (Fig. 7).
Before starting the game, user also has the opportunity to choose the used sensor, set the threshold and difficulty of the level.After the game is over, caused by a ball hitting the obstacle, a window showing the summary of the game appears (Fig. 8 and Fig. 9).
One of the added advantages of the application is its versatility, which is characterized by compatibility with all sensors included in the presented mechatronic device and an open source form that allows for the expansion of the system and the use of alternative sensors.

PROTOTYPE DEVICE
The conceptual design of the mechatronic device for the hand rehabilitation was developed using CAD computer software in Autodesk Inventor.Based on CAD files, the designed parts of the device were made using 3D printing technology.The complicated shape of the components and the FDM method used were the factors that determined the introduction of numerous support structures to prevent surface collapse during printing.Subsequently, unnecessary print elements were removed.Then the outside elements of the components were covered with a spray putty to achieve a uniform, smooth surface.After finishing surface treatment, the parts of the machine were painted.The next stage in the prototype construction was the assembly of the sensors and the assembly of the entire structure (Fig. 10).
The basic components of the developed device are:  The used joystick model is the Thumb Joystick from SparkFun.It allows to detect motion in two orthogonal directions.This is achieved by using two potentiometers in the module, one for each axis.In addition, pressing the knob activates the button.
The EMG sensor and the Arduino Leonardo microcontroller, to which the pressure sensors, the detector and the joystick are attached, are placed in a black box that is an electronics module.The task of the Arduino microcontroller is to read measurements from individual sensors and transfer them via a USB port to a PC.
Individual components of the device casing were made by 3D printing.Pressure sensors and joysticks are attached to the hollows of the device.The cost of materials and components used to build the prototype of the device was more than 150 euro.

DEVICE TESTING
The prototype of the mechatronic device for the hand rehabilitation has been tested for its ergonomic adjustability to the shape of the hand.During using pressure sensors, the hand rests optimally on the device and the sensors are located under the fingertips according to the assumptions made during the design process.
Adjustable base with a soft bulge create a comfortable wrist support.The design of the swivel castors and joystick in the body of the device allows to use their full range without changing the position of the palm.As in the case of the use of pressure sensors, the hand is naturally laid and the use of the device is not a nuisance.Fig. 11 shows an example of EMG electrodes placement on the patient's forearm during exercise using an EMG biofeedback module.In addition, Fig. 12 shows an exemplary hand practicing the range of movements of individual fingers.
The test of the device also verified the correct functioning of individual sensors in the designed system.In the case of pressure sensors, only one of them is used, because their parameters and capabilities are the same.To make the comparison of the results possible, the same settings for biofeedback exercises were applied (Fig. 13).Exercises were performed by a person with no upper limb dysfunction.
As it can be concluded from the presented graphs, the force sensor, the EMG sensor and the joystick all allow to perform the exercises correctly (Fig. 14, Fig. 15, Fig. 16).In the case of the pressure force sensor, the generated graph was controlled accordance with the exercises dedicated for the thumb training.During a smooth change in pressure there are no significant spikes in the measured values.The sensor accurately reproduces the pressure exerted on its surface (Fig. 14).
Then the exercises were performed with the preset settings using the EMG sensor.The visualization of the obtained results proved to be a bit more difficult in comparison to the analysis of the force sensor also for the thumb.The obtained graph is mostly located within the preset trajectory, however, the value jumps during the muscle tone's changes are considerable and often uncontrolled.Moreover, the accuracy of the EMG sensor is lower in comparison to the force sensor.The biomedical electrodes were located within one of the forearm muscles during the test (Fig. 15).
In the case of using the joystick, the exercise was also performed correctly.The problems that occurred concerned only the presentation of the exercise of the executed movement that turned out to be not very smooth (Fig. 16).Software tests are designed to verify the correctness of the exercise and biofeedback games, as well as the reports generated on them.To run applications on PC it is necessary to install Python 2.7 and libraries from the official Python site.During the tests, a pressure sensor was used the index finger.The exercises "Constant limits" and "Trajectory 2" were executed and the biofeedback game was tested.
During the first test in the "Settings -biofeedback exercises" window, there was selected the "Constant limits" exercise type, where the upper limit of the exercise range was set to 700, the lower limit to 500, while the exercise length was 30 seconds.Then the exercise was initiated.The graph was generated smoothly, without delays, and changes in pressure exerted on the sensor were reflected in the result.The exercise parameters displayed in the exercise window are adequate for the selected options (Fig. 17).
After the training terminated, the report was generated.The information contained is appropriate and the result reflects the correctness of the exercise (Fig. 18).
During the next biofeedback training test, the type of exercise "Trajectory 2" was selected, the upper limit of the exercise range was 550, the lower 350, while the exercise time was set to 20 seconds.Then the training began.As in the previous case, the graph was drawn smoothly and the set values of the parameters corresponded to those displayed in the exercise window (Fig. 19).
The report was generated based on the obtained waveform (Fig. 20).The presented parameter values are compatible with the determined settings.The result of the exercise is also relevant to the content of the obtained graph within the trajectory.
The last test concerned the biofeedback game.The setting window allowed to choose the average difficulty of the level and set a threshold value to 700, after that operation the game is started.Both the ball and the obstacles move smoothly.Exceeding the threshold value by the value   obtained from the sensor results in directing the controlled object upwards.The result displayed in the upper left corner of the window corresponds to the current state of the game (Fig. 21).
After scoring 16 points, the ball hit an obstacle, which resulted in ending the game.The "Game Summary" window was displayed and the "Save Result" button was pressed.A game report that included the result and information about the selected settings appeared (Fig. 22).

CONCLUSIONS
1.The article describes the innovative mechatronic device for the rehabilitation of all fingers of the hand, which enables the implementation of functional therapy with biofeedback.
The conceptual design as well as the prototype of the device along with the tests have been described.2. The developed system consists of a hardware part containing a device for hand rehabilitation with sensors, an EMG module and the Arduino microcontroller, as well as an authoring software for PCs that offers exercises and biofeedback games, also using electromyography.
The biofeedback training increases patient's motivation for further exercises and allows for faster recovery.The software allows to customize trainings to various types of upper limb disorders (including stroke) and levels of dysfunction by changing the exercise parameters.At the end of the exercise, a report is generated with a score indicating the correctness of the task.Results can also be added to the training diary so that it can be graphed using external software and the progress over several training sessions can be assessed.3. The carried-out and presented tests of the mechatronic device for hand rehabilitation allowed to verify the correctness of its operation.The created prototype confirms that the designed shape of the device is adjusted to the dimensions of the hand and provides the availability of sensors for individual fingers.All sensors included in the device allow for the proper biofeedback training.The software testing was also successful.Changes to some parameters, such as setting exercise limits or biofeedback difficulty levels, allow to adjust an exercise to a particular condition and dysfunction.Reports are generated correctly and contain the result of the exercise, which is a measure of its correctness.
4. In addition, ease of use of the software and small dimensions of the hardware make it possible to use the system outside of rehabilitation centers and to do exercises without direct supervision of a physiotherapist.A physiotherapist, in turn, has the ability to supervise the recovery process on the basis of sent reports or tele-rehabilitation. 5.The way of preparing the software itself, along with the way of reading sensor data makes it possible to use other additional sensors so that the system can be expanded in the future.The designed system, especially the device for hand rehabilitation, is an innovative solution, unprecedented in the field of rehabilitation and therapy of the upper limb.

Fig. 1 .
Fig. 1.Visualization of a biofeedback exercise bench containing the described device

Fig. 2 .
Fig. 2. A CAD model showing the main unit of the device

Fig. 3 .
Fig. 3. Main menu of the program

Fig. 10 .Fig. 12 .Fig. 11 .
Fig. 10.The prototype of the device for the hand rehabilitation with electronic module

.
Fig. 17.The biofeedback exercise performed during the first test