We introduce the Open Sequential Repetitive Action Counting(OSRAC)task,which aims to count all repetitions and locate transition boundaries of sequential actions from large-scale video data,without relying on predefin...We introduce the Open Sequential Repetitive Action Counting(OSRAC)task,which aims to count all repetitions and locate transition boundaries of sequential actions from large-scale video data,without relying on predefined action categories.Unlike the Repetitive Action Counting(RAC)task that focuses on a single-action assumption,OSRAC handles diverse and alternating repetitive action sequences in real-world scenarios,which is fundamentally more challenging.To this end,we propose UniCount,a universal system capable of counting multiple sequential repetitive actions from video data.Specifically,UniCount designs three primary modules:the Universal Repetitive Pattern Learner(URPL)to capture general repetitive patterns in alternating actions,Temporal Action Boundary Discriminator(TABD)to locate the action transition boundaries,and Dual Density Map Estimator(DDME)to achieve action counting and repetition segmentation.We also design a novel actionness loss to improve the detection of action transitions.To support this task,we conduct in-depth data analysis on existing RAC datasets and construct several OSRAC benchmarks(i.e.,MUCFRep,MRepCount,and MInfiniteRep)by developing a pipeline on data processing and mining.We further perform comprehensive experiments to evaluate the effectiveness of UniCount.On MInfiniteRep,UniCount substantially improves the Off-By-One Accuracy(OBOA)from 0.39 to 0.78 and decreases the Mean Absolute Error(MAE)from 0.29 to 0.14 compared to counterparts.UniCount also achieves superior performance in open-set data,showcasing its universality.展开更多
基金supported by the National Key Research and Development Program of China(No.2022YFF0604504)the National Natural Science Foundation of China(No.62172439)+2 种基金the Major Project of Natural Science Foundation of Hunan Province(No.2021JC0004)the National Natural Science Fund for Excellent Young Scholars of Hunan Province(No.2023JJ20076)the Central South University Innovation-Driven Research Programme(No.2023CXQD061).
文摘We introduce the Open Sequential Repetitive Action Counting(OSRAC)task,which aims to count all repetitions and locate transition boundaries of sequential actions from large-scale video data,without relying on predefined action categories.Unlike the Repetitive Action Counting(RAC)task that focuses on a single-action assumption,OSRAC handles diverse and alternating repetitive action sequences in real-world scenarios,which is fundamentally more challenging.To this end,we propose UniCount,a universal system capable of counting multiple sequential repetitive actions from video data.Specifically,UniCount designs three primary modules:the Universal Repetitive Pattern Learner(URPL)to capture general repetitive patterns in alternating actions,Temporal Action Boundary Discriminator(TABD)to locate the action transition boundaries,and Dual Density Map Estimator(DDME)to achieve action counting and repetition segmentation.We also design a novel actionness loss to improve the detection of action transitions.To support this task,we conduct in-depth data analysis on existing RAC datasets and construct several OSRAC benchmarks(i.e.,MUCFRep,MRepCount,and MInfiniteRep)by developing a pipeline on data processing and mining.We further perform comprehensive experiments to evaluate the effectiveness of UniCount.On MInfiniteRep,UniCount substantially improves the Off-By-One Accuracy(OBOA)from 0.39 to 0.78 and decreases the Mean Absolute Error(MAE)from 0.29 to 0.14 compared to counterparts.UniCount also achieves superior performance in open-set data,showcasing its universality.
