Quantifying normal and parkinsonian gait features from home movies: Practical application of a deep learning–based 2D pose estimator

English version

Autoři: Kenichiro Sato ^aff001; Yu Nagashima ^aff001; Tatsuo Mano ^aff001; Atsushi Iwata ^aff001; Tatsushi Toda ^aff001
Působiště autorů: Department of Neurology, Graduate School of Medicine, University of Tokyo, Hongo, Bunkyo-ku, Tokyo, Japan ^aff001
Vyšlo v časopise: PLoS ONE 14(11)
Kategorie: Research Article
prolekare.web.journal.doi_sk: https://doi.org/10.1371/journal.pone.0223549

Souhrn

Objective

Gait movies recorded in daily clinical practice are usually not filmed with specific devices, which prevents neurologists benefitting from leveraging gait analysis technologies. Here we propose a novel unsupervised approach to quantifying gait features and to extract cadence from normal and parkinsonian gait movies recorded with a home video camera by applying OpenPose, a deep learning–based 2D-pose estimator that can obtain joint coordinates from pictures or videos recorded with a monocular camera.

Methods

Our proposed method consisted of two distinct phases: obtaining sequential gait features from movies by extracting body joint coordinates with OpenPose; and estimating cadence of periodic gait steps from the sequential gait features using the short-time pitch detection approach.

Results

The cadence estimation of gait in its coronal plane (frontally viewed gait) as is frequently filmed in the daily clinical setting was successfully conducted in normal gait movies using the short-time autocorrelation function (ST-ACF). In cases of parkinsonian gait with prominent freezing of gait and involuntary oscillations, using ACF-based statistical distance metrics, we quantified the periodicity of each gait sequence; this metric clearly corresponded with the subjects’ baseline disease statuses.

Conclusion

The proposed method allows us to analyze gait movies that have been underutilized to date in a completely data-driven manner, and might broaden the range of movies for which gait analyses can be conducted.

Klíčová slova:

Sequence analysis – Cameras – Data processing – Gait analysis – Walking – Feet – Parkinson disease – Control sequences

Zdroje

1. Alcock L, Galna B, Lord S, Rochester L. Characterisation of foot clearance during gait in people with early Parkinson׳s disease: Deficits associated with a dual task. J Biomech. 2016 Sep 6;49(13):2763–2769. doi: 10.1016/j.jbiomech.2016.06.007 27363617

2. Alcock L, Galna B, Perkins R, Lord S, Rochester L. Step length determines minimum toe clearance in older adults and people with Parkinson’s disease. J Biomech. 2018 Apr 11; 71: 30–36. doi: 10.1016/j.jbiomech.2017.12.002 29429622

3. Nutt JG, Bloem BR, Giladi N, Hallett M, Horak FB, Nieuwboer A. Freezing of gait: moving forward on a mysterious clinical phenomenon. Lancet Neurol. 2011 Aug;10(8):734–44. doi: 10.1016/S1474-4422(11)70143-0 21777828

4. Morris M, Iansek R, Matyas T, Summers J. Abnormalities in the stride length-cadence relation in parkinsonian gait. Mov Disord. 1998 Jan;13(1):61–9. doi: 10.1002/mds.870130115 9452328

5. Hausdorff JM. Gait dynamics in Parkinson’s disease: common and distinct behavior among stride length, gait variability, and fractal-like scaling. Chaos. 2009 Jun;19(2):026113. doi: 10.1063/1.3147408 19566273

6. Yang CC, Hsu YL. A review of accelerometry-based wearable motion detectors for physical activity monitoring. Sensors (Basel). 2010;10(8):7772–88.

7. Muro-de-la-Herran A, Garcia-Zapirain B, Mendez-Zorrilla A. Gait analysis methods: an overview of wearable and non-wearable systems, highlighting clinical applications. Sensors (Basel). 2014 Feb 19;14(2):3362–94.

8. Cao Z, Simon T, Wei S, Sheikh Y. Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. In CVPR, 2017. arXiv:1611.08050

9. Chainer version of Realtime Multi-Person Pose Estiamtion (https://github.com/DeNA/chainer_Realtime_Multi-Person_Pose_Estimation) (accessed by authors on February 2019)

10. Keenan DB, Wilhelm FH. Classification of locomotor activity by acceleration measurement: validation in Parkinson disease. Biomed Sci Instrum. 2005;41:329–34. 15850127

11. Yang CC, Hsu YL, Shih KS, Lu JM. Real-time gait cycle parameter recognition using a wearable accelerometry system. Sensors (Basel). 2011;11(8):7314–26.

12. Harris FJ. On the use of Windows for Harmonic Analysis with the Discrete Fourier Transform. Proceedings of the IEEE. 66 (1): 51–83. doi: 10.1109/PROC.1978.10837

13. Bennett TR, Wu J, Kehtarnavaz N, Jafari R. Inertial Measurement Unit-Based Wearable Computers for Assisted Living Applications: A signal processing perspective. IEEE Signal Processing Magazine. 2016, vol. 33, no.2, pp. 28–35. doi: 10.1109/MSP.2015.2499314

14. Yu S, Tan D, Tan T. A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition. 18th International Conference on Pattern Recognition (ICPR'06). 2006.

15. FFmpeg Developers. (2016). ffmpeg tool (Version be1d324) [Software]. (http://ffmpeg.org/)

16. Sofuwa O, Nieuwboer A, Desloovere K, Willems AM, Chavret F, Jonkers I. Quantitative gait analysis in Parkinson’s disease: comparison with a healthy control group. Arch Phys Med Rehabil. 2005 May;86(5):1007–13. doi: 10.1016/j.apmr.2004.08.012 15895349

17. Pistacchi M, Gioulis M, Sanson F, De Giovannini E, Filippi G, Rossetto F, et al. Gait analysis and clinical correlations in early Parkinson’s disease. Funct Neurol. 2017 Jan/Mar;32(1):28–34. doi: 10.11138/FNeur/2017.32.1.028 28380321

18. Kharb A, Saini V, Jain YK, Dhiman S. A review of gait cycle and its parameters. IJCEM International Journal of Computational Engineering & Management, Vol. 13, 2011.

19. Schreven S, Beek PJ, Smeets JB. Optimising filtering parameters for a 3D motion analysis system. J Electromyogr Kinesiol. 2015 Oct;25(5):808–14. doi: 10.1016/j.jelekin.2015.06.004 26159504

20. Giorgino T (2009). “Computing and Visualizing Dynamic Time Warping Alignments in R: The dtw Package”. Journal of Statistical Software, 31(7), 1–24. http://www.jstatsoft.org/v31/i07/.

21. FNN: Fast Nearest Neighbor Search Algorithms and Applications. (https://cran.r-project.org/web/packages/FNN/index.html)

22. Saini I, Singh D, Khosla A. QRS detection using K-Nearest Neighbor algorithm (KNN) and evaluation on standard ECG databases. J Adv Res. 2013 Jul;4(4):331–44. doi: 10.1016/j.jare.2012.05.007 25685438

23. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J, et al. (2011). pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics, 12, p. 77. doi: 10.1186/1471-2105-12-77 21414208

24. Sih BL, Hubbard M, Williams KR. Correcting out-of-plane errors in two-dimensional imaging using nonimage-related information. J Biomech. 2001 Feb;34(2):257–60. doi: 10.1016/s0021-9290(00)00185-8 11165291

25. Mannon K, Anderson T, Cheetham P, Cornwall MW, McPoil TG. A comparison of two motion analysis systems for the measurement of two-dimensional rearfoot motion during walking. Foot Ankle Int. 1997 Jul;18(7):427–31. doi: 10.1177/107110079701800710 9252813

26. Fu SW, Li PC, Lai YH, Yang CC, Hsieh LC, Tsao Y. Joint Dictionary Learning-Based Non-Negative Matrix Factorization for Voice Conversion to Improve Speech Intelligibility After Oral Surgery. IEEE Trans Biomed Eng. 2017 Nov;64(11):2584–2594. doi: 10.1109/TBME.2016.2644258 28026747