Around 60% of the training images that Team NimbRo's home robot used for object recognition were labeled automatically, according to the team's leader, Raphael Memmesheimer. The method won the best paper award at RoboCup 2026 and helped the team finish third in the RoboCup@Home competition. The paper is titled "Efficient Image Annotation via Semi-Supervised Object Segmentation with Label Propagation."
The two-day preparation problem
RoboCup@Home teams get two preparation days before the competition starts. In that time they assemble their robots and bring them into a model apartment they have not seen before. The robots know nothing about the space or the objects in it, and they must learn to recognise items such as a spoon or a specific drink.
The object set can reach 50 items the team has never encountered. Organisers typically buy local groceries from a shopping mall and spread them around the apartment, so the details of drinks and snacks differ from one venue to the next. Some items come from the public YCB Dataset, and the team can collect data on those beforehand and pre-train on large general object sets. Most of the rest cannot be anticipated.
Memmesheimer says the team looked at open-vocabulary approaches and found they struggled to tell apart objects within one category, such as drinks. Promptable detection and segmentation models worked with five to ten classes, he says, but did not scale to the roughly 50 classes the competition involves.
How the labeling method works
The approach has several stages. The team first trains one model on objects that robots saw in past editions of the competition. That model learns only to flag that something might be an object, at the segmentation level.
The team then collects reference images of the actual objects and labels them semi-automatically. The method combines the generic judgment that a region is an object with a reference showing what that object is. This simplifies labeling, and it replaces many hours of manual annotation for the roughly 60% of the data that is labeled automatically.
The team used the method at RoboCup 2025 in Brazil and at the RoboCup GermanOpen. For the 2026 World Cup in Korea, it extended the approach to generate more data from videos. Memmesheimer says the saved time lets the team spend more of the preparation days testing tasks rather than building the vision model.
Third place and a new laundry task
Team NimbRo finished third among 24 teams. Memmesheimer says the team had problems, as it does at every competition. The 2026 edition added a Doing Laundry task. The robot had to detect the washing machine door, find a grasp point on the handle, open the door, and carry a laundry basket from the machine to a folding table. It then had to pull out t-shirts and fold them. The robot cannot see inside the machine, so it must feel whether it has a t-shirt in its hand.
Memmesheimer says NimbRo's robot performed best on this task. It opened the washing machine and began folding, and he attributes that to the semi-automatic labeling.
The competition has four other main tasks. In the human-robot interaction task, the robot opens the door for party guests, greets them, guides them to a free seat and introduces them to others. A pick-and-place task this year required clearing a breakfast table, loading cutlery and a dishwasher tablet into the dishwasher, closing it and resetting the table. The general-purpose service task gives the robot a random natural-language command. In the restaurant task, the robot works in an unfamiliar venue where only the menu is known in advance, and it must find waving customers and take voice orders in a noisy room. Robots must also pass a safety and interaction inspection before they can compete.
The league was once split into two standard-platform leagues and an open platform league. These have been merged into one. Memmesheimer's team won the open platform league the year the event was held in Eindhoven.
What this means for the field
The paper addresses a narrow and practical bottleneck. A home robot that needs a new vision model within two days is limited by how fast it can get labeled data, and the paper targets that step directly. The reported result is that more than half of the labels come from the automated pipeline.
The comparison with promptable models is the most informative point. The team found they worked at five to ten classes and did not hold up at about 50 visually similar items. That suggests a tuned, reference-based pipeline still has an edge when a robot must separate near-identical packaged goods. This is the team's own assessment, and the source gives no head-to-head figures.
Several things remain unproven. The source reports no accuracy numbers, so the quality of the automatic labels is not quantified. The pipeline relies on a model trained on past competition objects and on reference images collected for each item, which fits a competition with a known setup. Whether it would hold up in homes with a changing and unbounded set of objects is not shown. The video-based extension used in Korea has also not been described in detail.
The result also shows how much of the competition lies outside perception. NimbRo led on laundry, yet finished third overall. Memmesheimer notes that robots must navigate, interact with people, manipulate objects and plan tasks, with only a few trials. Better labeling shortens one part of the work. The team's stated next steps are faster picking of varied objects and better two-arm manipulation, which indicates where it sees the remaining gaps in the RoboCup@Home challenge.