ORCID ID
van den Bogert, Antonie J/0000-0002-3791-3749 Kirsch, Robert/0000-0003-2564-1800
Document Type
Article
Publication Date
10-1-2016
Publication Title
IEEE Transactions On Human-Machine Systems
Abstract
High-level spinal cord injury (SCI) in humans causes paralysis below the neck. Functional electrical stimulation (FES) technology applies electrical current to nerves and muscles to restore movement, and controllers for upper extremity FES neuroprostheses calculate stimulation patterns to produce desired arm movement. However, currently available FES controllers have yet to restore natural movements. Reinforcement learning (RL) is a reward-driven control technique; it can employ user-generated rewards, and human preferences can be used in training. To test this concept with FES, we conducted simulation experiments using computer-generated ``pseudohuman{''} rewards. Rewards with varying properties were used with an actor-critic RL controller for a planar two-degree-of-freedom biomechanical human arm model performing reaching movements. Results demonstrate that sparse, delayed pseudo-human rewards permit stable and effective RL controller learning. The frequency of reward is proportional to learning success, and human-scale sparse rewards permit greater learning than exclusively automated rewards. Diversity of training task sets did not affect learning. Longterm stability of trained controllers was observed. Using human-generated rewards to train RL controllers for upper-extremity FES systems may be useful. Our findings represent progress toward achieving human-machine teaming in control of upper-extremity FES systems for more natural arm movements based on human user preferences and RL algorithm learning capabilities.
Recommended Citation
Jagodnik, Kathleen M.; Thomas, Philip S.; van den Bogert, Antonie J.; Branicky, Michael S.; and Kirsch, Robert F., "Human-like Rewards To Train A Reinforcement Learning Controller For Planar Arm Movement" (2016). Mechanical Engineering Faculty Publications. 361.
https://engagedscholarship.csuohio.edu/enme_facpub/361
DOI
10.1109/THMS.2016.2558630
Version
Postprint
Publisher's Statement
© 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.
Volume
46
Issue
5
