The BionicMobileAssistant moves autonomously in space and – thanks to a neural network – can recognise objects independently, grasp them adaptively and work on them together with humans.
(video: Festo)
BionicMobileAssistant, an autonomous helper in assembly and intralogistics
Artificial intelligence plays a central role here. The BionicMobileAssistant is a prototype of a robot system that moves independently in three dimensions and can identify objects, grip adaptively and work together with humans.
The entire system, which has been developed in collaboration with ETH Zurich, has a modular design and consists of three subsystems:
- mobile robot;
- electric robot arm;
- BionicSoftHand 2.0.
The pneumatic gripper, which was first presented by Festo in 2019, is inspired by the human hand.
The BionicSoftHand 2.0: based on the human hand
The human hand – with its unique combination of force, dexterity and fine motor skills – is a true miracle of nature. To enable the BionicSoftHand 2.0 to carry out the movements of the human hand realistically, compact valve technology, sensors, electronics and mechanical components are integrated in the tightest of spaces.
The fingers and the opposable thumb are made up of flexible bellows structures with air chambers, surrounded by a firm yet yielding knitted fabric. This makes the hand light, adaptive and sensitive, yet capable of exerting strong forces. The pneumatic fingers are actuated by a compact valve terminal with piezo valves, mounted directly on the hand.
Further development of the BionicSoftHand with optimised action radius
In order to extend the manoeuvrability of the thumb and index finger compared with the first version of the BionicSoftHand, the developers greatly increased the lateral swivel range of both fingers.
As a result, they now work well together and grip with great precision.
Thanks to a 3D-printed wrist with two degrees of freedom, the hand can also move both back and forth as well as to the left and to the right. This means that gripping with a tight radius is also possible.
Gripper with sensitive fingers
To increase the stability of the fingers, the air chambers now each contain two structural elements that act as bones. For each finger, a bending sensor with two segments determines the positions of the fingertips. The hand also wears a glove with tactile force sensors on the fingertips, the palm and the outer sides of the robot hand. This allows it to sense the nature of the item to be gripped, and to adapt its gripping force to the particular item – just as people do.
Object detection using a neural network
In addition to the tactile sensors, the hand has a depth camera on the inside of the wrist so it can visually detect the objects.
Using it, the robot hand can detect and grip a range of objects, even if these are partially covered.
Once the hand has been correctly trained, it can use the collected data to assess the objects and distinguish good from bad, for example.
The information is processed by a neural network, which has been trained in advance with the aid of data augmentation.
Mobile robot application with electric arm
The BionicSoftHand 2.0 is combined with a mobile ballbot and a lightweight, electric robot arm – the DynaArm. The DynaArm can carry out fast and dynamic movements thanks to its lightweight design with highly integrated drive modules that weigh just one kilo.
Mobile use and stand-alone energy supply
For the ballbot, the developers chose a sophisticated drive concept: the robot balances on a ball.
This means that the BionicMobileAssistant can move freely in all directions.
All the system’s energy supply is on board: the battery for the arm and robot is located in the body; the compressed air cartridge for the pneumatic hand is installed in the upper arm.
The robot is thus not only mobile, but can also work autonomously.
The algorithms stored on the master computer also control the system’s autonomous movements. The robot orients itself independently in three dimensions using two cameras.
Diverse range of application options
The system would be perfect for use as a direct assistant for people, for example as a service robot, as a helping hand in assembly or to assist workers in carrying out ergonomically strenuous or monotonous tasks.
It could also be used in environments where people cannot work because of hazards or restricted accessibility.
This could include maintenance or repair work, data measurements or visual inspections. It is also possible to imagine mobile robots carrying out the simplest of tasks in areas where there is an increased risk of infection or which staff cannot access due to infections. For example, a possible scenario in the future could be a robot bringing drinks and food to the table in restaurants or delivering medicines to hospital patients or people requiring care in old people’s homes.
Hand in hand with people
Thanks to the modular concept, the BionicSoftHand 2.0 can also be quickly mounted on other robot arms and is easy to commission. Combined with the BionicCobot or the BionicSoftArm, both also bionic concepts from Festo, the gripper can, for example, form a completely pneumatic robot system that can work hand in hand with people thanks to its inherent flexibility and compliance.
