An Asymmetric Modeling for Action Assessment

Jibin Gao1,4, Wei-Shi Zheng1,2,5*, Jia-Hui Pan1, Chengying Gao1*, Yaowei Wang2, Wei Zeng3, and Jianhuang Lai1

1School of Data and Computer Science, Sun Yat-sen University, China
2Peng Cheng Laboratory, Shenzhen 518005, China
3School of Electronics Engineering and Computer Science, Peking University, China
4Pazhou Lab
5Key Laboratory of Machine Intelligence and Advanced Computing, MOE, China

Abstract

Action assessment is a task of assessing the performance of an action. It is widely applicable to many real-world scenarios such as medical treatment and sporting events. However, existing methods for action assessment are mostly limited to individual actions, especially lacking modeling of the asymmetric relations among agents (e.g., between persons and objects); and this limitation undermines their ability to assess actions containing asymmetrically interactive motion patterns, since there always exists subordination between agents in many interactive actions. In this work, we model the asymmetric interactions among agents for action assessment. In particular, we propose an asymmetric interaction module (AIM), to explicitly model asymmetric interactions between intelligent agents within an action, where we group these agents into a primary one (e.g., human) and secondary ones (e.g., objects). We perform experiments on JIGSAWS dataset containing surgical actions, and additionally collect a new dataset, TASD-2, for interactive sporting actions. The experimental results on two interactive action datasets show the effectiveness of our model, and our method achieves state-of-the-art performance. The extended experiment on AQA-7 dataset also demonstrates the generalization capability of our framework to conventional action assessment.

 

Download

Please download the dataset TASD-2 and code using the links below:
  • Please download the dataset TASD-2 using the links below: BaiDu Cloud Disk (code: h651), or Google Drive.

    • Codes for AIM model: AIM_model

       To be updated.

    Citation

    Please cite the following paper if you use the data in your research: