TIL-AI 2026: Robot War in Another World

Last updated on 05/07/2026

Tl;dr

Our team NeuralNetanyahwork (later censored to BenBotV3) participated in Today I Learned AI at BrainHack 2026, organised by AngelHack and DSTA. We placed 5th in the advanced track, and were one of two winning teams in the surprise challenge.

Couldn’t have done it without my teammates:

• Rui Zhuo (who also wrote most of this write up)
• Jet

Main Challenge

Backstory

In the fictional world of Clairos, a disaster known as the Cascade led to the collapse of most of civilisation, leaving megacorporations with automated war robots to fight for control of Haven, the last great city.

Competition Overview

The competition started with a one-month qualifier, where all 68 participating teams developed solutions for four tasks.

• Autonomous Exploration (AE)
• Automatic Speech Recognition (ASR)
• Computer Vision (CV)
• Natural Language Processing (NLP)

During qualifiers, each task was scored independently, and teams were ranked by their average score across all tasks. At the end of qualifiers, the top two teams were seeded directly into the finals, while the next 18 teams competed in the semi-finals for the four remaining finals spots. It’s in the semi-finals and finals that all four solutions get integrated together, and teams play against each other live at Marina Bay Sands.

Autonomous Exploration (AE)

Six teams play as megacorporation robots fighting over Haven, represented as a 16 by 16 maze with bases placed equidistant from the center. Each autonomous agent starts next to their assigned base. The core mechanics of the game are heavily inspired by Bomberman. At each turn, every agent can only see a small patch around itself and its base. With this limited information, the agent must navigate the fog of war and choose one of six actions: moving forwards, backwards, turning left, right, staying still, or dropping a bomb. The match runs for 200 turns, and agents are ranked by how many points they earn.

The main way to score points is to damage or destroy enemy agents and bases with bombs. Dropping a bomb consumes an agent’s resources, which must be collected from the environment. A bomb detonates three time steps after it’s dropped, damaging all agents and bases in its blast radius, and can destroy some walls marked as destructible. When an agent’s health drops below zero, it’s frozen for three time steps before it can act again. When a base’s health drops below zero, it’s permanently destroyed.

The secondary scoring method is visiting mission tiles. Mission tiles award a small number of points but also trigger an ASR, CV, or NLP mission. The overall accuracy across all missions acts as a multiplier on the agent’s score. For example, if an agent scores 400 in the environment and has an overall mission accuracy of 80%, its effective score is 320.

Solution

Like many other teams, we ended up writing our own algorithm instead of training a reinforcement learning (RL) model. To deal with the partial observability of the game, we track the state of the world and update it with information from the viewcones every step.

The first working agent followed a fixed priority chain. If a base is nearby, bomb it; otherwise collect a tile; otherwise explore. It was enough to validate the world model and pathfinding, but the priority chain kept snapping on edge cases. The agent would abandon a one-bomb base kill to chase a tile that happened to be slightly closer. Replacing the fixed chain with an expected-value scorer (EV = raw_value × urgency / (bfs_steps + 1)) fixed this cleanly. Every trade-off gets computed instead of hardcoded, so a dying enemy base one step away automatically beats a mission tile three steps away without any special-casing.

The real discovery came from staring at the immortality mechanic long enough. Enemies freeze for three ticks on death and respawn at the exact same cell. If you bomb a cell holding three or more live enemies, they respawn straight into your next bomb’s blast zone, which clusters them again for the bomb after that. The densest cell on the map self-perpetuates. We first tried coding this up directly as a dedicated respawn-camp goal, timing a bomb to land the instant a frozen enemy revives. An audit later found the timing never actually worked in terms of the leaderboard score.

A cluster-harvest override sits on top of the normal bomb-routing logic and unconditionally re-targets any bomb toward the densest reachable cluster of three or more live enemies within a short detour, whenever one exists. When no cluster is in range, it changes nothing, so the override is byte-identical to the underlying decision by construction. The respawn-lock isn’t explicitly coded anywhere. It falls out of the mechanic plus the threshold. Against random opponents it almost never fires. Against competitive opponents who cluster, it runs on its own.

At that point we tried to go neural. We collected trajectories from the rule agent and built a behavioral-cloning pipeline with an attention-based policy, a position-penalty module, and opponent memory. Imitation capped out around 61% of the rule agent’s own performance and wouldn’t close further regardless of architecture or training time. The failure wasn’t capacity. The reward is driven by where the immortal swarm will cluster several ticks in the future, and that information simply isn’t in the current viewcone. The network could imitate what the rule agent did in observed situations, but not the judgement calls that depend on future state it can’t see. PPO fine-tuned from the BC checkpoint still lost roughly two-to-one against the rule agent, and a handful of distillation and lookahead variants tried afterward landed somewhere between 6% and 20% of rule performance. The conclusion was the same every time. The remaining gap is information-theoretic. The cluster-harvest override already beats that wall because it harvests damage without needing to predict anything, almost everything else requires predicting future swarm positions and hits the same ceiling. A later, more exhaustive pass confirmed it from the strategy side too. More aggression regressed results, pure defense washed out, and the remaining economy and tile-routing decisions were already close to optimal, so 0.71 on the real leaderboard settled in as the actual rule ceiling, against frontier teams sitting around 0.77.

The rest of the work was composing independently validated levers on top of that base. An HP-weighted, horizon-aware bomb re-router that prefers weaker enemy bases. The cluster-harvest override. A fuel-aware tile-routing pass that, only when the team is bomb-starved, boosts the value of resource tiles so the agent refuels while it farms instead of stalling out unable to attack. And a persistent opponent-memory layer, since the competition container survives across tournament rounds even though the match manager gets rebuilt between them, that tracks our own death rate and how aggressively the opponent bombs, then widens the agent’s goal-sampling randomness whenever either is worse than its historical average, a hedge against the same opponent reading a pattern over repeated rounds. Each lever was validated on the leaderboard on its own before being stacked, with flag-off gates so that disabling any one of them reproduces the prior best byte-for-byte. That way every submission had a non-negative expected score delta by construction.

The agent that was used, nicknamed ffh3x, is that rule-based planner ( newer leaderboard 0.519) plus one more thin layer bolted on at the very end. A persistent world model reconstructs the map edge-by-edge from sequential viewcone observations, tracks enemy position, velocity, and freeze state, predicts enemy base locations from grid symmetry before they’re ever seen, and infers off-screen threats from unexplained HP drops. A planner scores eight goal types, flee, last-hit a low-HP base, hunt a base, hunt an agent, collect a tile, defend, explore, or fall back to random, by expected value every few steps, and a wall-aware BFS navigator turns the winning goal into a path.

The last addition was a short adversarial safety net. A 6-tick BFS search runs over the agent’s own future position and overrides the planner’s action only when that action is provably fatal, walking into a bomb that will detonate before an escape is possible, and a surviving alternative exists. Otherwise it defers completely. It’s wrapped in try/except so it can never crash the agent, and by construction it can only reduce deaths, so its floor is not too bad. A matched local A/B across 40 seeds showed a small, consistent win-rate edge over the plain rule agent for about 10 milliseconds of extra latency per move, though we were careful to note that no local proxy reliably predicts the real leaderboard, so this shipped as a bounded-downside bet rather than a proven gain. Fast, deterministic, and fully debuggable, which mattered a lot when every submission needed to be probably no worse than the last.

But carefully analysing the finals game led us to believing that more aggression should have been needed rather than tile farming so a higher weightage to find an opponent may have been the way.

Automatic Speech Recognition (ASR)

ASR missions involve transcribing a noisy voice recording. Given an audio file, the model must return a text transcript of whatever was said. The recordings are set in the fictional world and contain atypical terms specific to it. They may be in English, Malay, Tamil, or Chinese, and are scored on word error rate.

Solution

The dataset was relatively similar to the 2025 edition, but I didn’t save any of my 2025 work, confident it would be different enough to start fresh. But uh whatever…

The key structural change from 2025 was the number of languages, from 1 to 4. Optimistic about Whisper’s multilingual capabilities, I plugged in whisper-medium and fine-tuned it on the raw dataset without any data cleaning or analysis. That yielded a disappointing ~75% accuracy with slow inference. I added augmentation next (noise, speed, pitch), which is the standard suggestion and felt productive, but I was going through the motions without thinking about the fundamentals. Results didn’t move meaningfully.

After sitting with the per-language breakdown, Tamil was clearly the problem. The other languages were reasonable. Tamil WER was 0.45, dragging the overall down.

I switched to whisper-large-v3-turbo, as I thought a larger model would have better consistency across languages, which improved the overall accuracy but did not affect the poor performance on Tamil.

Then that’s where I added external data. Adding the FLEURS dataset pushed accuracy to 0.842 and carried us to 84.2% accuracy, 68.0% speed score in the qualifiers using vanilla faster-whisper defaults. Even then, Tamil remained our limiting factor throughout. Its WER floor sat around 0.45-0.63 regardless of inference tuning, which is a training-data problem rather than a model one. But I just gave up finding how to fix this problem albeit I should give polygot another try seeing other teams success on it.

The finals push came from two angles. Realising that CPU→GPU transfer cost was dominating per-request latency (which pointed toward batching, though batching turned out to be hardware-specific, helpful on an RTX 5080 but even better on the competition’s T4s), and discovering that chunking Tamil clips to 18 seconds instead of 30 reduced encoder calls by 14% with no accuracy cost. That combination, along with careful inference-parameter tuning (int8 quantisation, single-pass greedy decoding, language-detection tuned to one segment instead of five, and, counterintuitively, keeping timestamp tokens on, since disabling them cost 0.10 on the score by letting Chinese and Tamil transcripts drift), got us to 87.1% accuracy, 77.8% speed score before the finals.

Before finals I ran through several alternatives:

• Polyglot, which other teams reportedly did well with but which only got me to 65%, likely because the FLEURS-augmented training distribution clashed with its pretraining assumptions.
• Wav2Vec, a genuinely bad fit for the task.
• Sealion’s encoder, which I regretted trying almost immediately.

Computer Vision (CV)

CV missions involve object detection, returning a list of bounding boxes and predicted classes for land, air, and sea vehicles. Teams can also disrupt their enemies’ CV missions by adding adversarial noise to the images before the opponent receives them. Bounding box predictions are scored on mAP@.5:.05:.95.

Object Detection

For the qualifiers we trained a YOLO12x model with offline K=2 noise augmentation, scoring 0.475 on the blind set.

A problem we ran into early on was class imbalance. Aerial vehicles and uncommon-view classes were sparse enough that the model would happily ignore them until we applied repeat factor sampling (RFS).

We tried preprocessing the noisy images with CLAHE and unsharp masking, but it actually hurt accuracy as it shifted the input distribution away from what the model was trained on. What actually helped was simply running inference at a higher resolution than training, 960 rather than the 640 training resolution, which gave the detector enough resolution to pick out fine detail without the noise-amplification that showed up at 1280.

Before finals, we put together a WBF ensemble of YOLO26x and YOLO12x that showed real internal gains. YOLO26x was actually a bit weaker with a ~0.425 score, but a WBF[1,1] ensemble of the two pushed noisy-validation mAP50-95 from 0.7304 solo to 0.7452, a real gain of about 0.015, projecting to ~0.49 blind. The ensemble weighting barely mattered, [1,1], [1,2], and [2,1] all landed within 0.002 of each other. The two models share the same YOLO architecture but differ enough in size and training recipe that their failure modes decorrelate, which is exactly what WBF needs to add value instead of just averaging noise. An earlier YOLO26x plus RT-DETR ensemble had failed for the opposite reason. RT-DETR was undertrained on the noisy distribution and dragged the ensemble down instead of complementing it.

In the end, we trained YOLO11x and got 0.487 accuracy score and 0.942 speed score, an improvement over yolo12x. We decided to use this in the finals instead of the ensemble as it has less moving parts which means a lower chance of unexpected incidents during the actual day.

Disrupting the Opponent

The adversarial-noise side of CV was a brand-new addition for finals. Plain Gaussian noise turned out to be essentially useless (+0.006 effective disruption against a held-out detector). It’s unstructured, so a detector shrugs it off. A classifier-based sign-delta attack did better, but it was attacking the wrong target whereby iIt degrades classification confidence, not detection confidence, which isn’t quite what the scoring cares about. The move that actually worked was training a universal adversarial perturbation (UAP) directly against detector loss across an ensemble of YOLO models, 47 times more disruptive than Gaussian noise.

From there the problem became constraint engineering rather than attack design. The evaluator enforces hard caps on global image distortion and on distortion inside ground-truth boxes specifically (plus a perceptual-similarity floor inside those boxes). Violate any of them and the constraint gets rejected, meaning the opponent receives the clean image and the disruption is zero. Adding a local L2 cap (scaling down the perturbation in any small window where it got too concentrated, rather than scaling the whole image down) let us push the perturbation budget higher while keeping the pass rate high, recovering the same effective disruption as the unconstrained peak but with a much safer margin. We also tried an SSIM-based “repair” step that locally softened the perturbation wherever it looked like it might fail the similarity floor. That made things 2.2 times worse, because the repair heuristic couldn’t tell a flat ground-truth box from flat background, and ended up weakening the attack exactly where the actual vehicles were.

On top of the UAP itself, about half of the images also get a small random combo blended in, one or two photometric ops (gaussian noise, JPEG compression, gamma, contrast, saturation) picked and parameterised at random, mixed in at only 15% weight against the main perturbation direction. On its own this combo step is disruption-neutral, it doesn’t move the numbers. The point isn’t disruption, it’s per-image diversity. If an opponent trains a denoiser against our fixed perturbation fingerprint, a UAP that looks identical on every image is exactly what that denoiser learns to strip. Randomising a small slice of each image breaks that. Only additive photometric ops made the cut. Anything spatial, chromatic aberration, motion blur, distortion, smears edges and tanks inside-box SSIM even at low RMSE, and things like resolution round-trips, weather, glare, or mosaic effects are either too destructive at any usable mix ratio or create artefacts that blow the local L2 budget outright.

We should have also trained a dedicated attack against RT-DETR as a second target architecture, but we honestly forgot about it until it was too late to fit in so it was quite shit and some TEAM defeated that.

Natural Language Processing (NLP)

NLP missions involve retrieval-augmented generation, or at least they’re supposed to. Before each match, the agent is given a corpus of documents set in the world of Clairos, so that LLMs can’t answer from pre-existing knowledge. During the mission, the agent receives a question and must retrieve relevant documents and generate an answer. Questions come in five levels of difficulty, from direct quoting of documents to inference across multiple documents. The correct documents must be retrieved, otherwise the answer scores 0. A model trained by the organisers then judges whether the answer is equivalent to a reference answer. A correct answer scores 1, with partial credit of 0.4 for successful retrieval alone.

Solution

We started approaching the task with a regular RAG pipeline. We began with the smallest available Qwen3 embedding model and LLM, Qwen3-Embedding-0.6B and Qwen3-0.6B. The solution generated an embedding for every document in the corpus and indexed it with Faiss; when our container received a question, we’d embed it, retrieve the most similar documents, and feed them into the LLM prompt.

We quickly realised the solution took far too long to run. Still, it was hard to believe the RAG challenge could really be beaten without doing any actual RAG. So, we pressed on. We tried adding a cross-encoder reranker on top of the initial retrieval to squeeze more accuracy out of a smaller candidate set, tried trimming the LLM’s context and output length, and tried swapping in smaller/faster generation models.

Eventually we tried something we couldn’t quite believe would work. We looked at the runtimes other teams were posting and realised there was no way any model (embedding or LLM) could run that fast. So we scrapped the entire RAG pipeline, embeddings and all, and rebuilt it as pure keyword search. No sentence embedder, no cross-encoder, no LLM anywhere in the final pipeline, just BM25 and a pile of regex.

We knew we were on the right track the moment we matched the speed of the other teams. From there, all the accuracy came from engineering the retrieval and the answer text harder, since the grader’s own answer-similarity scorer turned out to be lenient enough that finding the right passage was most of the battle.

• Two BM25 indices instead of one. A full-document index handles which documents to return (the 0.4x retrieval-score component), while a separate sub-chunk index, built at both section level and paragraph level in the same index, supplies the actual answer text, restricted to chunks whose parent document made the top three. Letting BM25 pick the right granularity itself, rather than committing to one chunk size, mattered more than any single retrieval trick.
• Lexical tricks standing in for semantics. With no embedder anywhere, anything a bi-encoder would normally catch for free has to be faked with text transforms before indexing. Hyphenated IDs get a joined duplicate appended, so “77-015” also matches “77015”. Multi-word phrases get bigram tokens appended so they match as a unit instead of as scattered single words. A small hardcoded synonym table expands things like “company” into “corporation” and “firm” on the query side. None of it is glamorous, but together it recovers a lot of what an embedding model would have given us automatically.
• Local question-and-answer chunks. Question-and-answer pairs mined from our own local practice set get folded into the chunk index as tiny extra chunks per document. They’re exempt from the usual near-duplicate filtering, so they survive into the returned answer text even when they’d otherwise look like a substring of a larger chunk.
• Question-type-aware answer prepending. Before returning an answer, we check the question against a few shapes, “how many,” “when,” “who,” and if one matches, we scan the top chunks for the first sentence that both fits the expected answer type (a number, a date, a capitalised name) and shares at least two content words with the question, then prepend it. It’s a small nudge, but it means the actual number or date being asked for tends to sit at the front of the answer instead of buried in the middle of a chunk.
• A much bigger L4 net. Beyond the year-in-the-question check, we built out a long list of real-world entities (companies, politicians, cities, brands) verified absent from the fictional corpus, plus a fallback check. If every capitalised phrase in the question is absent from the corpus and the top BM25 chunk score is very low, we treat it as unanswerable too. Real-world questions dressed up to look like corpus questions were a real source of false positives early on, and this caught most of what the year regex alone missed.

A good chunk of our early iteration cycles were wasted chasing the wrong signal. Our local evaluator scored answers with a much shorter context window than the real grader used, which rewarded tricks (like stuffing extra text into the answer) that looked good locally but did nothing, or actively hurt, on the real leaderboard. Once we caught the discrepancy and started validating every change against the actual leaderboard instead of the local score, progress got a lot more honest. The final score was 0.893, entirely from BM25, regex, and hand-tuned heuristics, with the remaining gap mostly coming from a handful of adversarial “false premise” questions that read like normal questions but contradict the corpus. That’s genuine reading comprehension rather than retrieval, and no amount of keyword engineering was going to fix it.

Surprise Challenge

Challenge Specifications

One day before the finals, a surprise challenge was announced. All 20 semi-finalist teams had eight hours to prepare an agent to play a “Civilization”-style game. Twenty agents spawn onto a hexagonal grid, each with only one base, 500 gold, and a single goal, survival.

The game lasts 300 turns and shares the fog-of-war and simultaneous-turn structure of the main challenge. Every turn, agents can only observe tiles visible to their bases and units, and all agents move at once. Buildings can be constructed with gold. Bases can go up anywhere visible; everything else must be adjacent to existing buildings. Bases generate gold passively every turn; mines are a cheaper, higher-income alternative. Barracks produce infantry, scouts, and medics; factories produce tanks and artillery; airbases produce fighters and bombers, and all of them exist to attack other teams or defend your own base. A team is eliminated once all of its bases are destroyed.

Diplomacy runs alongside all of this through an in-game chat, with both global broadcasts and direct messages. For the first 200 turns, agents can propose and accept treaties with each other, which prevents allied teams from attacking one another and opens the door to real cooperation. In the final 100 turns, every treaty is automatically broken, and the game forces a fight to be the last team standing.

Our Approach

With only eight hours and effectively zero prior playbook for this exact game, the guiding principle was survival first, not aggression. Rather than chase early kills, the agent was built as a deterministic, defensive strategy that prioritises economy, expansion, and durability. It expands into safe, unclaimed territory, builds a stable income base, keeps a defensive presence around key buildings, and avoids overcommitment to risky attacks. Treaties get accepted and proposed whenever possible, because in a game that’s effectively a race to survive, quietly outlasting the turn limit alongside an ally is often more reliable than winning a war outright.

The implementation splits a simple agent interface from a concrete policy. A base agent defines the required decision interface, and a policy layer handles the full game logic (terrain awareness, construction, production, diplomacy, combat, and defensive counter-strikes), reading the published game rules and engine constants as the source of truth for all balance decisions. A second, optional layer handles diplomacy specifically. It’s an LLM-based advisor that never touches the core game loop and can only emit chat messages and posture hints. That separation matters in a setting where a single bad action is silently dropped and the cost of a brittle or overaggressive policy is high. It keeps the underlying gameplay fully deterministic while still letting the agent negotiate in something resembling natural language.

The main approach we threw away was a more aggressive, war-first policy. It looked attractive on paper, but it created unnecessary exposure and made the agent far more vulnerable to the game’s simultaneous-combat and fog-of-war dynamics. We also avoided letting an LLM drive the action policy directly. The deadline was tight and the environment unforgiving, so the system needed a stable deterministic fallback, and the safest design was to let a model assist with diplomacy rather than control the whole game. That combination (rule-based survival policy, bounded LLM diplomacy, and a containerised setup we could test locally before submission) was enough to make BenBotV3 one of two winning teams in the surprise challenge.

(Fable 5 with 4 prompts nailed this)