Singapore, 4 March 2025 – Singapore’s national swim team will get a new and improved “coach” – powered by artificial intelligence and leveraging drone technology.
Developed by researchers from the Singapore Management University (SMU) and the Singapore University of Technology and Design (SUTD), the system helps coaches analyse their swimmers’ performance, including stroke duration, swimming velocity, and the symmetry of the swimmer’s moving body, in real-time1.
The team is currently working with Singapore Aquatics (SAQ) to produce real-time analytics of swimmers at the National Training Centre (NTC) to improve their performance.
This is how it works: Research Fellow Shane Kyi Hla Win from SUTD’s Engineering Product Development (EPD) pillar operates the drone, which comes with a high-resolution camera, to fly about eight metres above the swimmers as they swim. The video images of the swimmers are then downloaded and analysed by custom analytics and user interface (UI) software built by Dr Tran Ngoc Doan Thu, a post-doctoral student and recent SMU Computer Science PhD graduate who is part of a team led by SMU Professor of Computer Science Rajesh Balan and advised by Assistant Professor Kenny Choo from SUTD’s Information Systems Technology and Design pillar and Design and Artificial Intelligence programme.
The analytics software uses AI models and computer vision algorithms to recognise human poses and swimming events based on the swim coaches’ expertise for the swimmers’ videos. The custom UI allows the coaches to visualise the results and gain a deeper understanding of the athlete’s swimming technique, namely the symmetry which reflects overall body balance, stroke duration and swimming velocity in real-time, as well as how these performance factors differ between training and competition (Please refer to Annex). The results are then made available to the coaches, at poolside, on a tablet device using video analytics and the custom UI.
Said Prof Balan: “This exciting research can potentially elevate our sporting performance by making coaching more precise, efficient and cost-effective without huge investments in computing hardware. Real-time insights from video analytics are proving to be a valuable tool in helping coaches to fine-tune their training strategies with greater accuracy. To fully benefit from this technology, coaches need tailored video analytics that align with their coaching methods – allowing them to explore and test key performance hypotheses with confidence.”
Assistant Prof Choo said: “Technology has tremendous potential when we design for humans and AI to work collaboratively. By integrating AI with human expertise, we can unlock new ways to enhance performance, efficiency, and decision-making. In projects like these, where drones and AI-driven analytics work alongside coaches, we see how human-AI interaction can lead to more precise, actionable insights that ultimately improve outcomes.”
SMU Associate Professor of Strategy & Entrepreneurship (Education) Kenneth Goh, who is also the President of SAQ, observed that this project shows how an interdisciplinary approach to human and computer interaction can encourage innovation. He said: “To innovate and integrate digital solutions effectively, we need to overcome regulatory and practical hurdles, which often involve close collaboration and building trust with regulators and stakeholders. This project is a great opportunity to foster collaboration and elevate aquatics in Singapore as well as showcase an interdisciplinary approach—across academia and industry—to solve real-world problems.”
Mr Gary Tan, National Head Coach (Swimming) at SAQ, commented: “Innovation plays a key role in shaping the future of elite sports, and this research is an exciting step in exploring new ways to analyse our performance. While we are still evaluating its full potential, this collaboration with SMU and SUTD reinforces our commitment to continuous improvement.”
“As we advance into the next phase of NTC’s development, this partnership will help us refine training strategies and stay competitive on the world stage. By leveraging research-driven insights, we can fine-tune our athlete’s performance, where every marginal gain matters.”
On using drones, Associate Professor Foong Shaohui from SUTD’s EPD pillar said: “Drones are cost-effective and portable, offering a fraction of the expenses of specialised training facilities. They provide two main benefits. First, they give an unobstructed overhead view of swimmers, capturing the movement of all limbs from both sides during strokes. Second, they remove the need for multiple underwater or poolside cameras to calculate swimming parameters. In addition, this system is not restricted to any specific facility and can be deployed to any swimming pool rapidly with no set-up time, making it a valuable training aid for swimmers at every level.”
This analytics method is, however, not limited to elite swimmers or the sport of swimming alone. Associate Prof Goh said: “This technology shows great promise for diving, artistic swimming, water polo, and open water swimming. Its applications can extend beyond the NTC to clubs, making real-time performance analysis available for more swimmers.”
This project, supported by the Ministry of Education, Singapore, under its Academic Research Fund (AcRF) Tier 1 grant, and funded through the SMU-SUTD Internal Research Grant Call, represents a key milestone in the Joint Research Collaboration between SMU and SUTD, which began in 2022 – a testament to both universities' dedication to innovation and academic excellence.
SMU Vice Provost (Research) Professor Archan Misra shared: “I’m very pleased to observe how my colleague Rajesh has partnered his SUTD collaborator Shaohui to harness their individual expertise in video analytics and drone technology to create an innovative, readily deployable system for sports analytics research, in partnership with Singapore Aquatics. This is an excellent example of impactful outcomes from our joint programme, which was set up precisely to support such collaboration and harness the complementary knowhow of faculty members across both institutions.”
SUTD Associate Provost for Research Professor Chua Chee Kai added: “This accomplishment reflects the power of cross-institutional collaboration, where the pooling of resources and expertise from both universities has resulted in a tangible, impactful solution. It serves as proof of the transformative potential of strategic partnerships in advancing research and delivering real-world benefits.”
ANNEX
Factsheet: What the video analytics system shows
The AI-driven video analytics project is designed to identify critical factors influencing swimmer speed, optimise training and improve performance.
The computer vision algorithms and analytics system is developed by Dr Tran Ngoc Doan Thu, a Research Scientist and recent SMU Computer Science PhD graduate, who is part of a team led by SMU Professor of Computer Science Rajesh Balan, and advised by Assistant Professor Kenny Choo from SUTD’s Information Systems Technology and Design pillar and Design and Artificial Intelligence programme. Dr Tran has designed the system to function in real-time at the pool to detect and analyse key body joints, including the wrist, elbow, knee, ankle, hip, and shoulder. These joints are indicated by the dots and the square on the swimmer’s body (Figure 1).
Coaches are interested in these joints because they play a significant role in swimming mechanics and performance. For example, the distance between the wrist and head might be stretched further to improve the swimmer’s speed.
Over the past one year (2024), Prof Balan and his team have continually refined the project to enhance its accuracy and effectiveness.
For example, Dr Tran said the AI model initially calculated swimming velocity using red markers placed 10 metres apart on the lane dividers separating the swimmers (represented by the dots that appear on the lane dividers in the video). This method allowed speed measurements to be recorded each time the swimmer passed a marker, typically taking several seconds.
As the project progressed, however, she refined the approach to calculate the swimmer’s speed every 0.5 seconds instead, by projecting the swimmer's head position onto the lane dividers between the swimmers (see Figure 2 below).
This enhancement enables the system to detect when a swimmer's speed starts dropping, thus helping his coach to improve his performance.
See the action in this short video. (Photo and video credit: Singapore Management University)
1 The research, “Analysing Swimming Performance Using Drone Captured Aerial Videos,” was published in DroNet '24: Proceedings of the 10th Workshop on Micro Aerial Vehicle Networks, Systems, and Applications.