The alpha, twice a week
That was 4 min.
Get the best new moments in your inbox every Wednesday and Saturday. It's free.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
The Shawn Ryan Show · 4 min · Sep 3, 2026
Autonomy needs training data from varied terrain
Byron Boots explains why a vehicle trained on one terrain may struggle on another, and why training data should cover likely operating environments.
In this episode
Full analysis- 50:15DARPA breakthroughs need a company to reach warfighters8 min
- 1:38:24Autonomy keeps vehicles moving when communications fail4 min
- 1:46:10Autonomous breaching can keep dozens of troops out of danger3 min
- 58:39Overland built vehicles when software alone could not reach troops6 min
- 1:35:11Field exercises expanded the vehicles beyond resupply3 min
- 32:08Autonomy needs training data from varied terrain4 min
- 1:27:38Ground robots will augment human formations before replacing them6 min
- 2:05:01Ukraine adopts battlefield technology faster because the stakes are existential3 min
“So generally speaking, you want to collect data in a wide variety of environments that are inclusive of the types of places that you might want to be driving in the future.”
Takeaways
- Training on an oval track may not transfer to right turns.
- Include likely future terrain types in training data.
- New surfaces may require new data and further learning.
About this segment
Byron Boots explains why a vehicle trained on one terrain may struggle on another, and why training data should cover likely operating environments. New surfaces can still require fresh data.
Byron Boots is cofounder and CEO of Overland AI; professor at the University of Washington.
Shawn Ryan, Host · Host, The Shawn Ryan Show
From “Byron Boots - 5 People Could Control an Army of 500 Autonomous Vehicles | SRS #336”, Sep 3, 2026. Watch the full episode
Thanks. We'll look at and fix what's off.
Transcript
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
- 32:12
high-speed chase in the middle of i don't know manhattan so this is kind of like a fundamental um challenge in machine learning um like if you learn for example how to drive really fast on oval track where you're always going in one direction, right? You'll learn how to drive really fast while always turning left essentially, let's say. But then if you go on another track which has right turns, will it be able to generalize and be able to perform well on a track like that? And the short answer is not without some sort of work. So generally speaking, you want to collect data in a wide variety of environments that are inclusive of the types of places that you might want to be driving in the future.
- 33:04
So for example, you might have a complex dirt track with left and right turns and some wider turns and some sharper turns and things like this. If you train on a track like that, it's very easy to then race on like an oval track, right? Because you've seen all of the things that you're likely to see. So that becomes easy. But if you, for example, train on a dirt track and now you have to drive on asphalt, you may not be able to do that super well.
- 33:32
And so you need to collect new data as you move to that new type of environment or new type of problem, incorporate that into your learning algorithm, and then it will start to do better on that new type of track. And that matters even now when we think about where we want to drive our vehicles, because if you only train, for example, in the desert, you're not going to be able to necessarily drive well through a forest or vice versa.
- 34:01
And so you want to really train these systems, have them collect data and learn from as wide a set of environments as possible so that they're able, when they encounter a new environment, to still perform well, that there'll still be aspects of that environment that they've seen before. So I guess what my question is, will it get to the point where it runs the route for the very first time, like a brand new route for a very first time?
- 34:32
It could be off-road, it could be in the middle of a city, but we're talking left turns, right turns, heavy braking, fast acceleration, drifting, all of that stuff. Will it eventually learn what it needs to learn to be able to have a complicated new route, run it the first time, and it will run it perfectly? It will go as fast as the machine is capable of. Yeah, it's certainly possible. And that's what we're always striving for.
- 35:07
So when we're putting, you know, uncrew ground vehicles, autonomous vehicles in new environments, they're already performing really well because it's seen many aspects of that before. And you're trying to get it to perform, you know, optimally, right? Like that is the goal, like potentially faster than a human driver, even on environments that it's never seen before. every time our autonomous vehicles are out in the world they are basically you know we we've trained them so that they're we don't like pre-map environments or anything like that it's as if they're encountering that environment for the first time and they're constantly learning so the more environments that we encounter the more data we get like the better and better the system gets. Then the goal is always kind of moving towards that optimal movement.