Comparison of object detection methods for abstract and concrete concepts

Abstract

In human communication, there are concrete concepts which are perceivable with one of the human senses (e.g. table or apple) and abstract concepts (e.g. freedom or love). The latter can not be perceived with the senses and relies more on subjective concepts. It is a challenging task to translate abstract concepts to automatic systems in computer-human interaction. This study examines characteristics of the visual representation of abstractness and concreteness by applying seven different object detection tools on images depicting abstract and concrete concepts. Further, the seven tools are compared with respect to model performance and their relation to abstractness and concreteness. In order to achieve this, image inference with a toolbox named MMDetection is performed on an image dataset with 500/500 of the most extreme abstract/concrete concepts. Further, the counts of detected objects and their distribution across abstractness, concreteness and models are analysed. Following, the model results are evaluated with the help of human annotators. The study finds that significantly more objects are detected for concreteness. Further, abstractness yields better results if only those cases are included that have a high prediction confidence. While RetinaNet performs well overall, Deformable DETR is particularly suitable for abstract data. On concrete data, both RetinaNet and Cascade R-CNN perform well. Overall, these findings are a step towards the characteristics of abstract and concrete concepts, suggesting that concrete concepts tend to occur with more objects per image than abstract ones. When considering the number of object counts per image as context, this argues that abstractness tends to occur with less context than concreteness and vice versa.