kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Learning-based Control for 4D Printing and Soft Robotics
KTH, School of Industrial Engineering and Management (ITM), Production Engineering, Sustainable Production Systems.ORCID iD: 0000-0001-9221-0918
2022 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Exploiting novel sensors and actuators made of flexible and smart materials becomes a new trend in robotics research. The studies on the design, production, and control of the new type of robots motivate the research fields of soft robots and 4D printed robots. 3D Printing (3DP) is an additive manufacturing technology that is widely used in printing flexible materials to fabricate soft robots. 4D Printing (4DP) combines 3DP technologies with smart materials to produce transformable devices. 4DP first prints structures with specifically designed responsive materials. When external stimuli such as temperature, voltage, or magnetic field are applied to the printed structure, it changes shape in a programmable way. The shape morphing property of 4DP makes it a novel approach to the actuators of robots.

The employment of these special materials empowers these new robots with better compliance and adaptability to the working environment. However, compared with the rigid counterparts, they also have complex dynamic properties such as substantial non-linearity and time-variance. These factors make the precise modeling and robust control of these new robots challenging and thus hinder their potential applications. Focusing on soft robotic systems enabled by 3DP and 4DP approaches, this dissertation studies both traditional and Machine Learning (ML)-based approaches to the modeling, perception, and control of soft, non-linear, and time-variant robotic systems. The main contributions of this dissertation are:

  • The scheme of Closed-Loop (CL) controlled 4DP (CL4DP) using temperature stimulated Shape Memory Polymer (SMP) is designed and validated numerically and experimentally. The feedback control system increases the precision and robustness of the shape morphing process of 4D printed SMP. Applications of CL4DP are explored.
  • Data-driven model identification methods are applied to learn the dynamic model of the shape morphing process of CL4DP and the learned model has good quality to support model-based control design. Model-free and adaptive Reinforcement Learning (RL) controllers are developed to deal with the non-linearity and time variance of 4D printed actuators. To improve the stability and quick adaptability, a concise basis function set is selected instead of blindly using Deep Neural Networks (DNNs).
  • A quadruped robot enabled by soft actuators and its simulation model are developed. The computation efficiency and model accuracy of the simulator are studied and optimized by comparing different simulation methods such as Finite Element Method (FEM) and lumped parameter method.
  • The optimal walking gait pattern of a soft-legged quadruped robot is found by grid parameter search and RL with a physics based simulation model. To speed up the RL training process, modeling tricks are used to reduce the simulation time of the model and curriculum learning is used to reduce the learning time.
  • A soft sensor made by printable conductive materials and 3DP is designed and optimally calibrated to estimate the shape of a pneumatically driven soft actuator. The geometry of the soft sensor is optimally designed for the best linearity, hysteresis and drift properties. The online estimation is based on a linear regression model learned from experimental data.
  • A pneumatically driven soft gripper is developed by 3DP, the printable soft sensor, and pole-placement control methods. The operation of the gripper does not require an external image feedback system to measure its shape, which is estimated by the integrated soft sensor. The position feedback by the soft sensor and the controller by the pole-placement method enable the soft gripper to perform complex tasks with high precision.
Abstract [sv]

Användande av nya sensorer och aktuatorer av flexibla och smarta material har blivit en ny trend inom robotikforskning. Studier om design, produktion och styrning av den nya typen av robotar motiverar förskningen om mjuka robotar och 4D-printade robotar. 3D Printing (3DP) är en additiv tillverkningsteknik som används i stor utsträckning vid utskrift av flexibla material för att tillverka mjuka robotar. 4D Printing (4DP) kombinerar 3DP-teknik med smarta material för att producera transformerbara enheter. 4DP skriver först ut strukturer med specifikt designade responsiva material. När yttre stimuli som temperatur, spänning eller magnetfält appliceras på den utskrivna strukturen ändrar den form på ett programmerbart sätt. Omformningsegenskapen hos 4DP skapar ett nytt sätt att aktuera robotar.

Användningen av dessa specialmaterial ger dessa nya robotar bättre följsamhet och anpassningsförmåga till sin arbetsmiljö. Men jämfört med de stela motsvarigheterna har de också komplexa dynamiska egenskaper såsom betydande icke-linjäritet och tidsvarians. Dessa faktorer gör den nogranna modelleringen och robusta kontrollen av dessa nya robotar utmanande och hindrar därmed deras potentiella tillämpningar. Med fokus på mjuka robotsystem som möjliggörs av 3DP- och 4DP-metoder, studerar denna avhandling både traditionella och Machine Learning (ML)-baserade metoder för modellering, perception och kontroll av mjuka, icke-linjära och tidsvarierande robotsystem. De viktigaste bidragen från denna avhandling är:

  • En metod för Closed-Loop (CL) reglerad 4DP (CL4DP) med temperaturstimulerad Shape Memory Polymer (SMP) har utvecklats och validerats både numeriskt och experimentellt. Reglersystemet ökar precisionen och robustheten i omformningsegenskapen för 4D-utskrivet SMP. Tillämpningar av CL4DP utforskas.
  • Metoder för datadriven modellidentifiering tillämpas för att lära sig den dynamiska modellen av omformningsprocessen för CL4DP och den inlärda modellen är lämplig modellbaserad reglering. Modellfria och adaptiva Reinforcement Learning-regulatorer (RL) har utvecklats för att hantera icke-linjäriteten och tidsvariationen hos 4D-utskrivna aktuatorer. För att förbättra stabiliteten och snabb anpassningsförmåga väljs en mindre uppsättning basfunktioner istället för att blint använda Deep Neural Networks (DNN).
  • En fyrbent robot med mjuka aktuatorer och dess simuleringsmodell utvecklas. Beräknings effektiviteten och modell noggrannheten hos simulatorn studeras och optimeras genom att jämföra olika simuleringsmetoder såsom Finite Element Method (FEM) och lumped parameter method.
  • Det optimala gångmönstret för en mjukbent fyrbensrobot hittas genom rutnäts-sökning och RL med en fysikbaserad simuleringsmodell. För att effektivisera RL-tränings-processen används modelleringsknep för att minska simuleringstiden för modellen och curriculum learning används för att minska inlärningstiden.
  • En mjuk sensor gjord av utskrivbara elektriskt ledande material och 3DP är designad och optimalt kalibrerad för att uppskatta formen på ett pneumatiskt driven mjuk aktuator. Den mjuka sensorns geometri är optimalt utformad för bästa linjäritet, hysteres och driftegenskaper. Online-uppskattningen är baserad på en linjär regressionsmodell som lärts från experimentella data.
  • En pneumatiskt driven mjukgripare är utvecklad av 3DP, den utskrivbara mjuka sensorn och polplacerings reglering. Griparens funktion kräver inte ett externt bildåterkopplingssystem för att mäta dess form, vilket istället uppskattas av den integrerade mjuka sensorn. Positionsåterkopplingen från den mjuka sensorn och regulatorn genom polplaceringmetoden gör att den mjuka griparen kan utföra komplexa uppgifter med hög precision.
Place, publisher, year, edition, pages
Stockholm: Kungliga tekniska högskolan, 2022. , p. 67
Series
TRITA-ITM-AVL ; 2022:32
Keywords [en]
3D Printing, 4D Printing, Soft Robots, Machine Learning, Reinforcement Learning, Control
National Category
Robotics and automation Control Engineering Production Engineering, Human Work Science and Ergonomics
Research subject
Production Engineering
Identifiers
URN: urn:nbn:se:kth:diva-319489ISBN: 978-91-8040-379-5 (electronic)OAI: oai:DiVA.org:kth-319489DiVA, id: diva2:1701901
Public defence
2022-11-11, Gladan/ https://kth-se.zoom.us/j/8822145866, Brinellvägen 83, Stockholm, 09:00 (English)
Opponent
Supervisors
Available from: 2022-10-12 Created: 2022-10-07 Last updated: 2025-02-05Bibliographically approved
List of papers
1. Feedback control for the precise shape morphing of 4D printed shape memory polymer
Open this publication in new window or tab >>Feedback control for the precise shape morphing of 4D printed shape memory polymer
Show others...
2021 (English)In: IEEE Transactions on Industrial Electronics, ISSN 0278-0046, E-ISSN 1557-9948, Vol. 68, no 12, p. 12698-12707Article in journal (Refereed) Published
Abstract [en]

Four-dimensional printing (4DP) is a newly emerged technology that uses smart materials for additive manufacturing and thus enables shape and/or property change upon stimulus after the printing process. Present study on 4DP has been focused on open loop stimulus, which can hardly ensure high shape precision and predictable final states. In this paper, a new closed loop 4DP (CL4DP) process supplementing 4D printed actuation with closed loop control methods is proposed. Image feedback is used for enhancing the conventional open loop 4DP morphing process and a controller is implemented to regulate the intensity of the stimulus accordingly in real-time. To achieve precise control, a nonlinear affine system model is built by model identification with measurement data to describe the dynamic shape recovery process of the 4D printed Shape Memory Polymer (SMP). Precise shape control is achieved and the effects of controller parameters on the precision of CL4DP are studied. Traditionally, SMP has a discrete number of selected steady states. With CL4DP, such steady states can be continuous and arbitrary.

Place, publisher, year, edition, pages
IEEE, 2021
Keywords
closed loop control, 4D printing, shape memory polymer
National Category
Control Engineering
Research subject
Industrial Information and Control Systems
Identifiers
urn:nbn:se:kth:diva-287569 (URN)10.1109/TIE.2020.3040668 (DOI)000692884200102 ()2-s2.0-85097951198 (Scopus ID)
Projects
4D Printing
Funder
Swedish Research Council, 2017-04550XPRES - Initiative for excellence in production research
Note

QC 20250401

Available from: 2020-12-16 Created: 2020-12-16 Last updated: 2025-04-01Bibliographically approved
2. Optimal shape morphing control of 4D printed shape memory polymer based on reinforcement learning
Open this publication in new window or tab >>Optimal shape morphing control of 4D printed shape memory polymer based on reinforcement learning
Show others...
2022 (English)In: Robotics and Computer-Integrated Manufacturing, ISSN 0736-5845, E-ISSN 1879-2537, Vol. 73, article id 102209Article in journal (Refereed) Published
Abstract [en]

4D printing technology, as a new generation of Additive Manufacturing methods, enables printed objectsto further change their shapes or other properties upon external stimuli. One main category of 4D printingresearch is 4D printed thermal Shape Memory Polymer (SMP). Its morphing process has large time delay, isnonlinear time variant, and susceptible to unpredictable disturbances. Reaching an arbitrary position with highprecision is an active research question. This paper applies the Reinforcement Learning (RL) method to developan optimal control method to perform closed loop control of the SMP actuation. Precise and prompt shapemorphing is achieved compared with previous control methods using a PI controller. The training efforts of RLare further reduced by simplifying the optimal control policy using the structural property of the prior trainedresults. Customized protective visors against COVID-19 are fabricated using the proposed control method.

Place, publisher, year, edition, pages
Elsevier, 2022
Keywords
4D printing; Shape Memory Polymer; Closed loop control; Reinforcement learning
National Category
Control Engineering Production Engineering, Human Work Science and Ergonomics Robotics and automation
Research subject
Electrical Engineering; Production Engineering
Identifiers
urn:nbn:se:kth:diva-298887 (URN)10.1016/j.rcim.2021.102209 (DOI)000709117800004 ()2-s2.0-85110379550 (Scopus ID)
Funder
Swedish Research Council, 2017-04550Swedish Research Council, 2019-05232XPRES - Initiative for excellence in production research
Note

QC 20211116

Available from: 2021-07-22 Created: 2021-07-22 Last updated: 2025-02-05Bibliographically approved
3. Online reinforcement learning for the shape morphing adaptive control of 4D printed shape memory polymer
Open this publication in new window or tab >>Online reinforcement learning for the shape morphing adaptive control of 4D printed shape memory polymer
2022 (English)In: Control Engineering Practice, ISSN 0967-0661, E-ISSN 1873-6939, Vol. 126, p. 105257-105257, article id 105257Article in journal (Refereed) Published
Abstract [en]

Combining 3D printing and smart materials, 4D printing technologies enable the printed actuators to furtherchange their shapes or other properties after prototyping. However, the shape morphing of 4D printed actuatorssuffers from poor controllability and low precision. One of the main challenges is that the 4D printed actuatorsare hard to be modeled and it is difficult to develop an appropriate controller for them. In this study, variouspopular reinforcement learning (RL) methods are applied to address the problem of online and adaptive model-free control of 4D printed shape memory polymer (SMP). Their training efficiencies are compared and anadaptive LQR controller based on Q learning is developed to realize efficient online learning. The RL controllerachieves precise and quick shape control within 2 − −3 learning episodes and is adaptive to the changingproperties of SMP. The RL controller performance is then compared with a model-based LQR controller andshows high control precision and excellent adaptability to the varying control plant.

Place, publisher, year, edition, pages
Elsevier, 2022
Keywords
Closed loop control; Reinforcement learning; Q-learning; 4D printing; Shape memory polymer
National Category
Control Engineering
Identifiers
urn:nbn:se:kth:diva-315616 (URN)10.1016/j.conengprac.2022.105257 (DOI)000828015800006 ()2-s2.0-85133428357 (Scopus ID)
Projects
Closed-Loop 4D Printing with High Precision
Funder
Swedish Research Council, 2017-04550Swedish Research Council, 2019-05232KTH Royal Institute of Technology, XPRESKTH Royal Institute of Technology, IRIS Area 2
Note

QC 20220728

Available from: 2022-07-14 Created: 2022-07-14 Last updated: 2022-10-07Bibliographically approved
4. Customized protective visors enabled by closed loop controlled 4D printing
Open this publication in new window or tab >>Customized protective visors enabled by closed loop controlled 4D printing
2022 (English)In: Scientific Reports, E-ISSN 2045-2322, Vol. 12, no 1, article id 7566Article in journal (Refereed) Published
Abstract [en]

The COVID-19 pandemic makes protective visors important for protecting people in close contacts. However, the production of visors cannot be increased greatly in a short time, especially at the beginning of the pandemic. The 3D printing community contributed largely in fabricating the visor frames using the rapid and adaptive manufacturing ability. While there are many open source designs of face visors for affordable 3D printers, all these designs fabricate mono-sized frames without considering diverse users’ dimensions. Here, a new method of visor post-processing technology enabled by closed loop controlled 4D printing is proposed. The new process can further deform the printed visor to any customized size for a more comfortable user experience. FEM analysis of the customized visor also shows consistent wearing experience in different circumstances compared with the old visor design. The fabrication precision and time cost of the method is studied experimentally. A case study regarding the reducing, reusing and recycling (3R) of customized visors in classrooms is proposed to enable the customized visors manufactured in a more sustainable way.

Place, publisher, year, edition, pages
Springer Nature, 2022
Keywords
4D Printing, Closed-loop Control, Sustainable Production, Protective Visor
National Category
Control Engineering Production Engineering, Human Work Science and Ergonomics
Research subject
Industrial Information and Control Systems
Identifiers
urn:nbn:se:kth:diva-312150 (URN)10.1038/s41598-022-11629-3 (DOI)000792845300039 ()35534667 (PubMedID)2-s2.0-85129460707 (Scopus ID)
Funder
Swedish Research Council, 2017-04550KTH Royal Institute of Technology, XPRESKTH Royal Institute of Technology, IRIS Area 2Swedish Research Council, 2019-05232
Note

QC 20220531

Available from: 2022-05-13 Created: 2022-05-13 Last updated: 2022-10-07Bibliographically approved
5. Synthesizing the optimal gait of a quadruped robot with soft actuators using deep reinforcement learning
Open this publication in new window or tab >>Synthesizing the optimal gait of a quadruped robot with soft actuators using deep reinforcement learning
Show others...
2022 (English)In: Robotics and Computer-Integrated Manufacturing, ISSN 0736-5845, E-ISSN 1879-2537, Vol. 78, p. 102382-102382, article id 102382Article in journal (Refereed) Published
Abstract [en]

Quadruped robots have the advantages of traversing complex terrains that are difficult for wheeled robots. Most of the reported quadruped robots are built by rigid parts. This paper proposes a new design of quadruped robots using soft actuators driven by tendons as the four legs. Compared to the rigid robots, the proposed soft quadruped robot has inherent safety, less weight and simpler mechanism for fabrication and control, but the corresponding challenge is that the accurate mathematical model applicable to model-based control design of the soft robot is difficult to derive by dynamics. To synthesize the optimal gait controller of the soft-legged robot, the paper makes the following contributions. First, the flexible components of the quadruped robot are modeled with different finite element and lumped parameter methods. The model accuracy and computation efficiency are analyzed. Second, soft actor–critic methods and curriculum learning are applied to learn the optimal gaits for different walking tasks. Third, The learned gaits are implemented in an in-house robot to transport hand tools. Preliminary results show that the robot can walk forward and correct the walking directions.

Place, publisher, year, edition, pages
Elsevier, 2022
Keywords
Quadruped robot, Soft actuators, Tendon-driven motion, Reinforcement learning, Robot gait, Motion control
National Category
Control Engineering Computer Sciences Robotics and automation
Research subject
Industrial Information and Control Systems; Computer Science; Machine Design; Production Engineering
Identifiers
urn:nbn:se:kth:diva-313485 (URN)10.1016/j.rcim.2022.102382 (DOI)000811874600001 ()2-s2.0-85131411362 (Scopus ID)
Funder
Swedish Research Council, 2017-04550Swedish Research Council, 2019-05232XPRES - Initiative for excellence in production researchKTH Royal Institute of Technology, IRIS Area 2
Note

QC 20220630

Available from: 2022-06-05 Created: 2022-06-05 Last updated: 2025-02-05Bibliographically approved
6. Omnidirectional walking of a quadruped robot enabled by compressible tendon-driven soft actuators
Open this publication in new window or tab >>Omnidirectional walking of a quadruped robot enabled by compressible tendon-driven soft actuators
Show others...
2022 (English)In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022), Kyoto, October 23–27, 2022, Institute of Electrical and Electronics Engineers (IEEE) , 2022, p. 11015-11022Conference paper, Published paper (Refereed)
Abstract [en]

Using soft actuators as legs, soft quadruped robots have shown great potential in traversing unstructured and complex terrains and environments. However, unlike rigid robots whose gaits can be generated using foot pattern design and kinematic model of the rigid legs, the gait generation of soft quadruped robots remains challenging due to the high DoFs of the soft actuators and the uncertain deformations during their contact with the ground. This study is based on a quadruped robot using four Compressible Tendon-driven Soft Actuators (CTSAs) as the legs, with the actuator's compression motion being utilized to improve the walking performance of the robot. For the gait design, an inverse kinematics model considering the compression of the CTSA is developed and validated in simulation. Based on this model, walking gaits realizing different motion speeds and directions are generated. Closed loop direction and speed controllers are developed for increasing the robustness and precision of the robot walking. Simulation and experimental results show that omnidirectional locomotion and complex walking tasks can be realized by tuning the gait parameters and the motions are resistant to external disturbances.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2022
Series
IEEE International Conference on Intelligent Robots and Systems, ISSN 2153-0858
National Category
Robotics and automation
Identifiers
urn:nbn:se:kth:diva-319486 (URN)10.1109/IROS47612.2022.9981314 (DOI)000909405303010 ()2-s2.0-85146334217 (Scopus ID)
Conference
2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022), Kyoto, October 23–27, 2022
Note

QC 20221004

Available from: 2022-09-29 Created: 2022-09-29 Last updated: 2025-02-09Bibliographically approved
7. Design and closed loop control of a 3D printed soft actuator
Open this publication in new window or tab >>Design and closed loop control of a 3D printed soft actuator
Show others...
2020 (English)In: 2020 16th IEEE International Conference on Automation Science and Engineering (CASE), IEEE, 2020, p. 842-848Conference paper, Published paper (Refereed)
Abstract [en]

Soft robots, made of soft materials such as di-electric elastomer or shape memory polymers, have receivedtremendous attentions due to its dexterousness, flexibility andsafety compared with rigid robots. However, wider applicationof soft robots is limited due to their complex fabrication processand poor controllability. Here, we introduce a closed loopcontrolled soft actuator that is fully 3D printed with flexiblematerial. The structure of the soft actuator is optimized withFinite Element Method (FEM) to acquire shortest fabricationtime and highest deformation for same stimulus input. A desk-top Fused Deposition Modeling (FDM) 3D printer is used forlow-cost fabrication of such actuators. A webcamera is used forthe image feedback which offers the real time shape monitoringof the soft actuator. An output feedback Proportional IntegralDerivative (PID) controller with lowpass filter is developed withpole placement design method based on a data-driven modelof the 3D printed soft actuator. The controller is implementedto regulate the input air pressure to ensure a fast-response, precise and robust shape changing for any work environments.

Place, publisher, year, edition, pages
IEEE, 2020
Keywords
3D print, soft actuator
National Category
Other Mechanical Engineering
Research subject
Machine Design
Identifiers
urn:nbn:se:kth:diva-287572 (URN)10.1109/CASE48305.2020.9216946 (DOI)000612200600120 ()2-s2.0-85094110350 (Scopus ID)
Conference
16th IEEE International Conference on Automation Science and Engineering (CASE, ELECTR NETWORK, AUG 20-21, 2020
Funder
Swedish Research Council, 2017-04550XPRES - Initiative for excellence in production research
Note

Part of proceedings ISBN

QC 20201216

Available from: 2020-12-16 Created: 2020-12-16 Last updated: 2023-03-13Bibliographically approved
8. Precise control of a 3D printed soft actuator with soft position sensors
Open this publication in new window or tab >>Precise control of a 3D printed soft actuator with soft position sensors
Show others...
(English)Manuscript (preprint) (Other academic)
National Category
Robotics and automation
Identifiers
urn:nbn:se:kth:diva-319488 (URN)
Note

Submitted to Mechatronics

QC 20221004

Available from: 2022-09-29 Created: 2022-09-29 Last updated: 2025-02-09Bibliographically approved

Open Access in DiVA

kappa(17652 kB)1152 downloads
File information
File name FULLTEXT01.pdfFile size 17652 kBChecksum SHA-512
bac19c4937fdb5502fe37fa74b5c46ee0568c19f0fbce0d378d992e96022d2509c20bcfaa19436899e180c1d434694491417af5991d5b75419f3208f729ba30d
Type fulltextMimetype application/pdf

Other links

http://Du som saknar dator/ datorvana kan kontakta service@itm.kth.se (English)

Authority records

Ji, Qinglei

Search in DiVA

By author/editor
Ji, Qinglei
By organisation
Sustainable Production Systems
Robotics and automationControl EngineeringProduction Engineering, Human Work Science and Ergonomics

Search outside of DiVA

GoogleGoogle Scholar
Total: 1152 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 1492 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf