Development of a Data Science Solution for Clay Pigeon Shooting Scoring
The Customer is a European IT startup developing software solutions for shooting sports.
The Customer was developing image analysis software for clay pigeon shooting¹ that would automatically determine shooting results. However, partially developed software was ineffective, e.g., it failed to detect target hit/miss. The Customer needed to review software under development and deliver the solution that would meet the stated requirements.
- Clay pigeon shooting (clay target shooting) – a shooting sport that involves shooting a firearm at flying targets (clay pigeons). The sport has more than 20 different forms of regulated competition; the most popular of them are trap, skeet, and sporting.
To fulfill the software development project, the Customer resorted to ScienceSoft’s data science team, who started work with the audit of the existing solution.
The Customer’s analytical system was based on two ResNet-50 convolutional neural networks (CNNs) trained on 300x300 video frames cropped from 2440x2048 video images. The first CNN classified objects into two groups – a clay target and the background. The second CNN localized the target position. The CNNs had been trained on historical data (shooting data elicited from different periods, weather conditions, locations, etc.).
The system was meant to work as follows:
- The pre-installed camera translated the images of the region of interest (the so-called trap region) to a user application installed on the on-site PC located 50-200 meters away from the shooting range.
- The region of interest was divided into image crops, each of which was collected for the image processing and forecasting.
- Two CNNs classified objects and defined the clay target center coordinates.
- Trajectory trackers analyzed the clay target locations proposed by the CNNs and related frame predictions of the flying target trajectory.
Once a clay target throwing machine signaled to fire a shot, the system started reporting renders to the user application in near real time. And when the CNNs detected a clay target, users could see the flying trajectory and the result on the screen in real time – target hit, target miss or no shot.
As a result of the assessment, ScienceSoft’s team identified a number of errors, which didn’t allow efficient software operation: low frame rate – ~15 frames per second, excessiveness of the environment setup script, synchronization failures, inaccurate target localization, etc.
As the next project step, ScienceSoft’s data science team carried out several enhancements to rectify the identified errors.
Firstly, they enabled software to correctly define shooting results for one flying target:
- To distinct clay targets from the target fragments lying on the ground after previous launches, the team applied a background subtraction algorithm.
- To increase the accuracy and efficiency of the image processing, ScienceSoft’s team implemented EfficientNet CNNs instead of the ResNet-50 ones.
- To optimize computational resources by half, the two EfficientNet CNNs were merged into the unified one.
Then, ScienceSoft’s data science team enabled faultless simultaneous detection, classification, and tracking of multiple flying targets and target fragments regardless of their location. For that, the team amended the system with the temporal information. Combining temporal information with the spatial data, it became possible to define the speed and the direction of a clay target (and its parts) and distinguish it from other moving objects.
To detect and track multiple targets, the team implemented two CNNs:
- CenterTrack – as the main CNN for simultaneous target detection and tracking. The model did not require image cropping to tiles and was efficient even when trained on tiny datasets.
- SiamRPN – as a possible supplementary CNN, which was good at the distinction of clay targets from other moving objects and was fast, especially with the MobileNet backbone.
Based on the unique spatiotemporal pattern of the movement of clay target parts, the new CNNs were designed to function in the real-life outdoor environment and could be used for scoring in different types of shooting games, which involve more than one flying target.
The Customer obtained fully functioning software for automated results detection in clay pigeon shooting, which can be employed in major international competitions.
Technologies and Tools
C++ (Boost, libconfig, gtkmm), Python (pySerial, OpenCV, PyTorch, NumPy), СNNs (RestNet-50, EfficientNet, CenterTrack, SiamRPN).