Issue

Vol. 17 (2025)

A Rapid Adaptation Approach for Dynamic Air-Writing Recognition Using Wearable Wristbands with Self-Supervised Contrastive Learning

https://doi.org/10.1007/s40820-024-01545-8
Yunjian Guo
Department of Electronic Convergence Engineering, Kwangwoon University, Seoul 01897, South Korea
Kunpeng Li
Department of Electronic Convergence Engineering, Kwangwoon University, Seoul 01897, South Korea
Wei Yue
Radio Frequency Integrated Circuit (RFIC) Bio Centre, Kwangwoon University, Seoul 01897, South Korea
Nam‑Young Kim
Radio Frequency Integrated Circuit (RFIC) Bio Centre, Kwangwoon University, Seoul 01897, South Korea
Yang Li
School of Microelectronics, Shandong University, Jinan 250101, People’s Republic of China
Guozhen Shen
School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing 100081, People’s Republic of China
Jong‑Chul Lee
Department of Electronic Convergence Engineering, Kwangwoon University, Seoul 01897, South Korea

Corresponding Author: Jong‑Chul Lee

jclee@kw.ac.kr

Nano-Micro Letters, Vol. 17 (2025), Article Number: 41

Abstract

Wearable wristband systems leverage deep learning to revolutionize hand gesture recognition in daily activities. Unlike existing approaches that often focus on static gestures and require extensive labeled data, the proposed wearable wristband with self-supervised contrastive learning excels at dynamic motion tracking and adapts rapidly across multiple scenarios. It features a four-channel sensing array composed of an ionic hydrogel with hierarchical microcone structures and ultrathin flexible electrodes, resulting in high-sensitivity capacitance output. Through wireless transmission from a Wi-Fi module, the proposed algorithm learns latent features from the unlabeled signals of random wrist movements. Remarkably, only few-shot labeled data are sufficient for fine-tuning the model, enabling rapid adaptation to various tasks. The system achieves a high accuracy of 94.9% in different scenarios, including the prediction of eight-direction commands, and air-writing of all numbers and letters. The proposed method facilitates smooth transitions between multiple tasks without the need for modifying the structure or undergoing extensive task-specific training. Its utility has been further extended to enhance human–machine interaction over digital platforms, such as game controls, calculators, and three-language login systems, offering users a natural and intuitive way of communication.


Highlights:
1 Utilizing self-supervised learning, the proposed wearable wristband with a four-channel sensing array and wireless transmission module is developed for tracking air-writing and dynamic gestures.
2 The model can learn prior features from unlabeled signals of random wrist movements, significantly reducing the dependency on extensive labeled data for training.
3 The wristband system rapidly adapts to multiple scenarios after fine-tuning using few-shot data, enhancing user interaction through natural and intuitive communication.

Keywords

Wearable wristband Self-supervised contrastive learning Dynamic gesture Air-writing Human–machine interaction
Guo, Yunjian, Kunpeng Li, Wei Yue, Nam‑Young Kim, Yang Li, Guozhen Shen, and Jong‑Chul Lee. 2024. “A Rapid Adaptation Approach for Dynamic Air-Writing Recognition Using Wearable Wristbands With Self-Supervised Contrastive Learning”. Nano-Micro Letters 17 (October):41. https://doi.org/10.1007/s40820-024-01545-8.
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
References
  1. A. Moin, A. Zhou, A. Rahimi, A. Menon, S. Benatti et al., A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition. Nat. Electron. 4, 54 (2020). https://doi.org/10.1038/s41928-020-00510-8
  2. M. Wang, Z. Yan, T. Wang, P. Cai, S. Gao et al., Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors. Nat. Electron. 3, 563 (2020). https://doi.org/10.1038/s41928-020-0422-z
  3. Z. Sun, M. Zhu, X. Shan, C. Lee, Augmented tactile-perception and haptic-feedback rings as human-machine interfaces aiming for immersive interactions. Nat. Commun. 13, 5224 (2022). https://doi.org/10.1038/s41467-022-32745-8
  4. X. Liang, H. Li, W. Wang, Y. Liu, R. Ghannam et al., Fusion of wearable and contactless sensors for intelligent gesture recognition. Adv. Intell. Syst. 1, 1900088 (2019). https://doi.org/10.1002/aisy.201900088
  5. T. Watanabe, Md. Maniruzzaman, Md.A.M. Hasan, H.-S. Lee, S.-W. Jang et al., 2D camera-based air-writing recognition using hand pose estimation and hybrid deep learning model. Electronics 12, 995 (2023). https://doi.org/10.3390/electronics12040995
  6. S. Ahmed, W. Kim, J. Park, S.H. Cho, Radar-based air-writing gesture recognition using a novel multistream cnn approach. IEEE Internet Things J. 9, 23869 (2022). https://doi.org/10.1109/JIOT.2022.3189395
  7. Y. Guo, F. Yin, Y. Li, G. Shen, J. Lee, Incorporating wireless strategies to wearable devices enabled by a photocurable hydrogel for monitoring pressure information. Adv. Mater. 35, 2300855 (2023). https://doi.org/10.1002/adma.202300855
  8. W. Yue, E.-S. Kim, E. Ganbold, B.-H. Zhu, B. Oh et al., A miniature and reusable radiofrequency biosensor combining microfluidic and integrated passive technology for glucose detection. Sens. Actuators B Chem. 392, 134108 (2023). https://doi.org/10.1016/j.snb.2023.134108
  9. Y. Li, Q. Lin, T. Sun, M. Qin, W. Yue, S. Gao, A perceptual and interactive integration strategy toward telemedicine healthcare based on electroluminescent display and triboelectric sensing 3D stacked device. Adv. Funct. Mater. (2024). https://doi.org/10.1002/adfm.202402356
  10. H. Niu, H. Li, S. Gao, Y. Li, X. Wei et al., Perception-to-cognition tactile sensing based on artificial-intelligence-motivated human full-skin bionic electronic skin. Adv. Mater. 34, 2202622 (2022). https://doi.org/10.1002/adma.202202622
  11. Y. Liu, C. Yiu, Z. Song, Y. Huang, K. Yao et al., Electronic skin as wireless human-machine interfaces for robotic VR. Sci. Adv. 8, eabl6700 (2022). https://doi.org/10.1126/sciadv.abl6700
  12. J. Huang, R. Zhou, Z. Chen, Y. Wang, Z. Li et al., Highly stable and reliable capacitive strain sensor for wearable electronics based on anti-dry hydrogel electrode. Mater. Today Phys. 35, 101123 (2023). https://doi.org/10.1016/j.mtphys.2023.101123
  13. K.K. Kim, I. Ha, M. Kim, J. Choi, P. Won et al., A deep-learned skin sensor decoding the epicentral human motions. Nat. Commun. 11, 2149 (2020). https://doi.org/10.1038/s41467-020-16040-y
  14. Z. Zhou, K. Chen, X. Li, S. Zhang, Y. Wu et al., Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays. Nat. Electron. 3, 571 (2020). https://doi.org/10.1038/s41928-020-0428-6
  15. T. Wang, Y. Zhao, Q. Wang, A flexible iontronic capacitive sensing array for hand gesture recognition using deep convolutional neural networks. Soft Robot. 10, 443 (2023). https://doi.org/10.1089/soro.2021.0209
  16. P. Tan, X. Han, Y. Zou, X. Qu, J. Xue et al., Self-powered gesture recognition wristband enabled by machine learning for full keyboard and multicommand input. Adv. Mater. 34, 2200793 (2022). https://doi.org/10.1002/adma.202200793
  17. H. Fang, L. Wang, Z. Fu, L. Xu, W. Guo et al., Anatomically designed triboelectric wristbands with adaptive accelerated learning for human–machine interfaces. Adv. Sci. 10, 2205960 (2023). https://doi.org/10.1002/advs.202205960
  18. X. Le, Q. Shi, Z. Sun, J. Xie, C. Lee, Noncontact human–machine interface using complementary information fusion based on MEMS and triboelectric sensors. Adv. Sci. 9, 2201056 (2022). https://doi.org/10.1002/advs.202201056
  19. A. Tashakori, Z. Jiang, A. Servati, S. Soltanian, H. Narayana et al., Capturing complex hand movements and object interactions using machine learning-powered stretchable smart textile gloves. Nat. Mach. Intell. 6, 106 (2024). https://doi.org/10.1038/s42256-023-00780-9
  20. J. Park, D. Kang, H. Chae, S.K. Ghosh, C. Jeong et al., Frequency-selective acoustic and haptic smart skin for dual-mode dynamic/static human-machine interface. Sci. Adv. 8, eabj9220 (2022). https://doi.org/10.1126/sciadv.abj9220
  21. K.K. Kim, M. Kim, K. Pyun, J. Kim, J. Min et al., A substrate-less nanomesh receptor with meta-learning for rapid hand task recognition. Nat. Electron. 6, 64 (2023). https://doi.org/10.1038/s41928-022-00888-7
  22. Q. He, Z. Feng, X. Wang, Y. Wu, J. Yang, A smart pen based on triboelectric effects for handwriting pattern tracking and biometric identification. ACS Appl. Mater. Interfaces 14, 49295 (2022). https://doi.org/10.1021/acsami.2c13714
  23. H. Lee, S. Lee, J. Kim, H. Jung, K.J. Yoon et al., Stretchable array electromyography sensor with graph neural network for static and dynamic gestures recognition system. npj Flex. Electron. 7, 20 (2023). https://doi.org/10.1038/s41528-023-00246-3
  24. L. Cui, C. Hu, W. Wang, J. Zheng, Z. Zhu et al., An adhesive, stretchable, and freeze-resistant conductive hydrogel strain sensor for handwriting recognition and depth motion monitoring. J. Colloid Interface Sci. 677, 273 (2024). https://doi.org/10.1016/j.jcis.2024.07.214
  25. K.R. Pyun, K. Kwon, M.J. Yoo, K.K. Kim, D. Gong et al., Machine-learned wearable sensors for real-time hand motion recognition: toward practical applications in reality. Natl. Sci. Rev. 11, nwad298 (2024). https://doi.org/10.1093/nsr/nwad298
  26. X. Wei, H. Li, W. Yue, S. Gao, Z. Chen et al., A high-accuracy, real-time, intelligent material perception system with a machine-learning-motivated pressure-sensitive electronic skin. Matter 5, 1481 (2022). https://doi.org/10.1016/j.matt.2022.02.016
  27. Y. Li, Z. Qiu, H. Kan, Y. Yang, J. Liu et al., A human-computer interaction strategy for an FPGA platform boosted integrated “perception-memory” system based on electronic tattoos and memristors. Adv. Sci. (2024). https://doi.org/10.1002/advs.202402582
  28. H. Niu, F. Yin, E. Kim, W. Wang, D. Yoon et al., Advances in flexible sensors for intelligent perception system enhanced by artificial intelligence. InfoMat 5, e12412 (2023). https://doi.org/10.1002/inf2.12412
  29. L. Jin, Z. Li, Z. Liu, B. Richardson, Y. Zheng et al., Flexible unimodal strain sensors for human motion detection and differentiation. npj Flex. Electron. 6, 74 (2022). https://doi.org/10.1038/s41528-022-00205-4
  30. W. Yang, H. Kan, G. Shen, Y. Li, A network intrusion detection system with broadband WO3-x/WO3-x-Ag/WO3-x optoelectronic memristor. Adv. Funct. Mater. 34, 2312885 (2024). https://doi.org/10.1002/adfm.202312885
  31. T. Kim, Y. Shin, K. Kang, K. Kim, G. Kim et al., Ultrathin crystalline-silicon-based strain gauges with deep learning algorithms for silent speech interfaces. Nat. Commun. 13, 5815 (2022). https://doi.org/10.1038/s41467-022-33457-9
  32. H. Zhang, H. Li, Y. Li, Biomimetic electronic skin for robots aiming at superior dynamic-static perception and material cognition based on triboelectric-piezoresistive effects. Nano Lett. 24, 4002 (2024). https://doi.org/10.1021/acs.nanolett.4c00623
  33. F. Wen, Z. Zhang, T. He, C. Lee, AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove. Nat. Commun. 12, 5378 (2021). https://doi.org/10.1038/s41467-021-25637-w
  34. M.L. Hammock, A. Chortos, B.C.-K. Tee, J.B.-H. Tok, Z. Bao, 25th Anniversary : the evolution of electronic skin (e-skin): a brief history, design considerations, and recent progress. Adv. Mater. 25, 5997 (2013). https://doi.org/10.1002/adma.201302240
  35. T. Chen, S. Kornblith, M. Norouzi, G. Hinton, A simple framework for contrastive learning of visual representations, in Proceedings of the 37th International Conference on Machine Learning (PMLR) (2020), pp. 1597–1607
  36. J.-B. Grill, F. Strub, F. Altché, C. Tallec, P.H. Richemond et al., Bootstrap your own latent-a new approach to self-supervised learning. arXiv, 2006.07733. https://doi.org/10.48550/arXiv.2006.07733
  37. E. Eldele, M. Ragab, Z. Chen, M. Wu, C.-K. Kwoh et al., Self-supervised contrastive representation learning for semi-supervised time-series classification. IEEE Trans. Pattern Anal. Mach. Intell. 45, 15604 (2023). https://doi.org/10.1109/TPAMI.2023.3308189
  38. B.K. Iwana, S. Uchida, Time series data augmentation for neural networks by time warping with a discriminative teacher, in 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy (2021), pp. 3558–3565
  39. M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski et al., Unsupervised learning of visual features by contrasting cluster assignments, in Advances in Neural Information Processing Systems 33 (NeurIPS 2020) (2020), pp. 9912–9924
  40. J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: pre-training of deep bidirectional transformers for language understanding. arXiv, 1810.04805 (2018). https://doi.org/10.48550/arXiv.1810.04805
  41. Y. Chang, L. Wang, R. Li, Z. Zhang, Q. Wang et al., First decade of interfacial iontronic sensing: from droplet sensors to artificial skins. Adv. Mater. 33, 2003464 (2021). https://doi.org/10.1002/adma.202003464
  42. J. Shi, Y. Dai, Y. Cheng, S. Xie, G. Li et al., Embedment of sensing elements for robust, highly sensitive, and cross-talk-free iontronic skins for robotics applications. Sci. Adv. 9, eadf8831 (2023). https://doi.org/10.1126/sciadv.adf8831
  43. W. Cheng, J. Wang, Z. Ma, K. Yan, Y. Wang et al., Flexible pressure sensor with high sensitivity and low hysteresis based on a hierarchically microstructured electrode. IEEE Electron Device Lett. 39, 288 (2018). https://doi.org/10.1109/LED.2017.2784538
  44. B. Ji, Q. Zhou, M. Lei, S. Ding, Q. Song et al., Gradient architecture-enabled capacitive tactile sensor with high sensitivity and ultrabroad linearity range. Small 17, 2103312 (2021). https://doi.org/10.1002/smll.202103312
  45. Z. Wang, W. Yan, T. Oates, Time series classification from scratch with deep neural networks: a strong baseline, in 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA (2017), pp. 1578–1585. https://doi.org/10.1109/IJCNN.2017.7966039
  46. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones et al., Attention is all you need, in Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, United States (2017)
  47. Q. Wang, B. Li, T. Xiao, J. Zhu, C. Li et al., Learning deep transformer models for machine translation, in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy (2019), pp. 1810–1822. https://doi.org/10.18653/v1/P19-1176
  48. G.S. Dhillon, P. Chaudhari, A. Ravichandran, S. Soatto, A baseline for few-shot image classification. arXiv, 1909.02729 (2019). https://doi.org/10.48550/arXiv.1909.02729
  49. W.-Y. Chen, Y.-C. Liu, Z. Kira, Y.-C. F. Wang, J.-B. Huang, A closer look at few-shot classification. arXiv, 1904.04232 (2019). https://doi.org/10.48550/arXiv.1904.04232
  50. J. Soni, N. Prabakar, H. Upadhyay, Visualizing high-dimensional data using t-distributed stochastic neighbor embedding algorithm, in Principles of Data Science (Springer, Cham, 2020), pp. 189–206. https://doi.org/10.1007/978-3-030-43981-1_9
Read More
Author biographies are not available.
Statistic
Read Counter : 1945 Download : 1354

Most read articles by the same author(s)

1 2 > >>