This is the 4th in a series of articles I’m calling ‘Opening the Starsky Kimono,’ where I reveal the Starsky insights we previously considered top secret. You can read about Starsky’s approach to safety here, business model here, and thoughts on AI here.

When it comes to autonomous safety, simpler is almost always better.

That might seem strange.  The consequences of an autonomous car breaking are huge - death, injury, damage, and the likelihood that the entire company will go under as a result.  You might assume, then, that in order to make such a system safe you need to use every tool at your disposal and then chip away at those that don’t seem necessary.  Like how a sculptor might start with a slab of marble which they chisel down into a masterpiece.

While incredibly sophisticated, this early Zoox prototype would be magnitudinally harder to prove safe than a few sensors and drive by wire on a pre-existing car. SCREENSHOT FROM BLOOMBERG (YOUTUBE)

The more parts there are in a given system, the more possible combinations of slight failures that might result in a big one.  Too many sensors pointed at the vehicle in front of you and the autonomous agent might jump between which one to follow, creating strange and potentially unsafe driving behavior.  You could solve this by only following one or two of those sensors, which would negate the value of those sensors for that task while maintaining their ability to make other subsystems act strangely.

The more complex a system, the harder it is to understand.  The less understandable, the fewer the people who will be able to raise valid safety concerns.  That can split an engineering organization between those who understand safety and those who understand the system - meaning that many of the most critical safety concerns are never raised.  To empower everyone to make a system safe you need one that everyone can understand.

While a flying shuttle is faster than a needle and thread, it was also more likely to cause weavers to lose eyes when they were deflected and shot clear of the machine. SSPL VIA GETTY IMAGES

I think of it as Occam’s Safety Razor, from the principle that the simplest solution is most likely the right one.  The simpler a system is, the easier it is to make safe; the more complicated a system the harder it is to make safe.

To be clear, however, that isn’t to say that simple systems are always the safest, just that they’re the easiest to make safe.  A taut belt around a steering wheel and a brick on the accelerator do not make for a safe robotaxi - even if you would save a lot of money on roboticists.

That isn’t to say that a belt & brick unmanned car can’t be safe.  If you were on a closed track and pointed it at a concrete wall 1km in the opposite direction from you and your team you could be reasonably sure that it wouldn’t hurt anyone.

Your system could also be safe if, instead, you hacked the car’s drive-by-wire system and ordered it to go straight at 45mph towards that wall.  To make it safe, though, you’d have to do a lot more work.  Does your command to drive straight actually work, or is it offset by 15° and going to come back around at you?  Is any of the code telling the transmission to switch into reverse?  Those are all things you can check, but doing so is far harder than the safety checks for the Belt & Brick system.

All of this is counterintuitive - we’re taught to correlate complexity with sophistication,  Autonomy is hard so a better and safer system must be more complex.  That just isn’t the way engineering works, the safest bridge isn’t a futuristic suspension bridge but a causeway.  

And when it comes to autonomous vehicles the safest are those that are the most understandable.

This post was originally published in Forbes

A Zoox prototype robotaxi, the newest Amazon asset. DPA/PICTURE ALLIANCE VIA GETTY IMAGES

Last week e-commerce giant and connoisseur of applied robotics, Amazon, announced it was acquiring Zoox, the 1000 person robotaxi company that sought not just to deploy L4 vehicles but to fundamentally re-imagine the car.

When rumors of a deal were first surfaced a month ago in the Journal many, including myself, brushed it off as an effort to stir up acquisition interest. Dating back to its Kiva acquisition, Amazon’s robotics sweet spot seems to be things that already work at scale.

Zoox, on the other hand, has often been considered the sick man of autonomy. In the early days the team, then hundreds strong, had a rumored 10-person team re-imagining the horn. Their prototype omnidirectional vehicle has been rumored to be satirized in Comma.ai’s office with the slogan “this is why we don’t make cars.”

None of this is to say, of course, that Zoox hasn’t built impressive technology — I have universally heard great things about their ridealongs. Their problem instead has stemmed from a fallacious slippery slope. When Zoox was founded everyone seemed to think autonomy would be easy. Step one would therefore be to finish off L5, step two would logically then be an expansive robotaxis business, which would of course need vehicles designed to not have a human driver.

The first problem, of course, has been that L5 autonomy seems as far away today as it ever did and many teams are increasingly retrenching away from anything that takes the driver out of the vehicle. To make matters worse, there’s a very real question of whether or not there’s a business to be had in the robotaxi market.  Which makes a purpose designed robotaxias viable as a martian outerwear brand.

Nevertheless, as reported in The Information Amazon has paid around $1 billion for the 1000-person Zoox team. With a tight refocus of the autonomous product this could end up being one of Amazon’s best acquisitions.  

Abandon robotaxis, move to delivery.

When compared to the realities of the ridesharing market, the robotaxis start to pale. Before paying self-employment taxes, the average American Uber driver makes $11.77/hr. Assuming that eight hours are lost each day cleaning, maintaining and fueling the car, and that it’s out of service an additional 10% of the time; a L5 robotaxi would only make $63k/yr driving over 100,000 miles. That margin doesn’t include the $200,000 bill of materials that separate cars from robotaxis. With the likely tight ODD of an early robotaxi, it’s more likely that $20,000-$40,000 is a good expectation of per robotaxi income.

A UPS Van presumably full of Amazon Prime orders CORBIS VIA GETTY IMAGES

This isn’t even considering the fact that Amazon is in the movement of goods business, and not that of the movement of people. To move the needle most, Zoox should be immediately refocused toward helping cut into the $37.9 billion/year their new parent spends shipping.

Abandon San Francisco, move to the ‘burbs

I love living in San Francisco – great restaurants, bars and entertainment are just blocks away from my Hayes Valley apartment; which is precisely what makes it a challenging place for autonomy. Tight streets, frequent pedestrian encounters; it’s hard to imagine an environment more challenging for autonomous vehicles than dense cities like San Francisco or New York.

In total, only about 20% of America lives in the 25 most populous counties (many of which aren’t particularly dense). While that 20% is richer and presumably orders more from Amazon than residents of the least dense counties, the real target customer for an autonomous Amazon delivery fleet is the vast expanse of suburbs where prime delivery is quicker than remembering to buy something next weekend.

They might not have outward personality, but they're certainly easy to drive around. FAIRFAX MEDIA VIA GETTY IMAGES

Sprawling suburban hamlets of McMansions feature well-maintained roads and surprisingly few pedestrians. Besides the occasional jogger, suburbans mostly drive to go about their business which makes them far easier to predict and avoid. There is no good reason to keep Zoox focused on dense urban driving, they should instead entirely refocus their efforts on driving in the suburbs.

Set a Roadster Goal

Of all the things Elon Musk has done to earn acclaim (or derision), I’ve always found the TeslaTSLA Roadster to be the most laudable. By choosing to borrow the body of an existing production car, Musk was able to focus Tesla’s early team on just the challenge of building a luxury electric drive train within a relatively short period of time.

The Tesla Roadster combined the chassis of a Lotus and Tesla's first ever drive train. By only needing to focus on drive train development and production, Tesla was able to avoid the dustbin of Frontiertech GAMMA-RAPHO VIA GETTY IMAGES

Focusing on shipping has another great value: it forces teams to limit scope so that they can be successful as opposed to expanding it to be able to work on interesting what-ifs. Driving hard toward a safe, unmanned pilot meant that at Starsky we couldn’t really play “trolley-problem what if” for weeks at a time. We had to build and refine.

For Amazon-Zoox that goal probably looks like deploying in a suburb within one of the states with a robust supportive regulatory framework, like Florida or Texas. Given that Amazon already controls some endpoints via the Amazon Hub, their smart locker, they could choose to ship from a distribution center to their hubs with their safety driver playing the role of unloading the vehicle.

It would be realistic to give the teams three months for the dust to settle, three months to make a first (manned) delivery, and to be regularly stocking certain Amazon hubs by Q1 2021. The focus would then, of course, need to be getting the human out of the vehicle by the end of 2021.

None of this, of course, is for certain. Big companies are notoriously bad at shipping pre-launch technology startups and Zoox could be yet another example. But, if played correctly, Amazon might just be the company to ship autonomy.

This article was originally published in Forbes

(This is the third in a series of articles I’m calling ‘Opening the Starsky Kimono,’ where I reveal Starsky Robotics’ key insights we previously kept top-secret. The first covers the end of Starsky and limitation of AI and can be found here. The second (here) covers the business use case for AV trucks and the commercial irrelevance of true AI to that aim.)

A few months ago I wrote that people part of the challenge of deploying an autonomous truck was that people didn’t really value, or understand statistical safety.

How things have changed.

In the last two months everyone has developed a qualified opinion on statistical safety.  At least, that is, when it comes to public health in response to Covid-19. VCs who told me they thought the risk of unmanned trucks was too great are now tweeting that we should accept a higher death rate so as to re-open the American economy.

The statistical arguments that underpin proposed responses to Covid-19 aren’t that different from the models we used at Starsky to perform our public unmanned test.  The entire Covid-19 crisis, in fact, presents a surprising parallel to explain what safety engineering really is.

Safety is not the absence of risk, but the absence of unacceptable risk.  Just as every public health policy will lead to some number of fatal Covid-19 cases, any deployed AV will have a greater-than-zero fatality rate.  Making that system safe is a matter of understanding how, why, and when it will hurt people and ensuring that those reasons are acceptable.  

These social distance circles at a park mitigate the risk of a full-on reopening. Much like having backup drivers ready and able to jump into an emergency-stopped unmanned truck. TORONTO STAR VIA GETTY IMAGES

It is unacceptable to deploy a system that regularly hurts people while it’s working as expected in normal driving conditions.  On the other hand, it can be acceptable to deploy one that might hurt people while failing in rare ways in uncommon driving situations.  As long as you know the exact risks you’re taking.

Think of it this way - if you walk through a Covid-19 ward you won’t necessarily die.  To die you’d first need to be in contact with the virus, catch it, have a particularly bad case, and ultimately succumb to the illness.  If you need to walk through that ward, you can mitigate those circumstances by taking precautions while walking (6’ apart, masks, hand-washing), responding quickly to potential exposure (testing and going on early treatments), and quickly going on a full course of treatment.  While the chance of fatality is still greater than zero, it’s significantly lower.

This Mumbai doctor is taking multiple independent steps which each reduce his likelihood of exposure. AFP VIA GETTY IMAGES

For AVs you can also break the problem down.  A failed system or freak incident doesn’t necessitate a fatality  The freak incident might either happen when the AV isn’t nearby or the system failure might be when the AV isn’t near a person, the AV system’s onboard diagnostics then have the opportunity to catch the failure, and assuming that they do the system then has the opportunity to avoid incident. 

Through a decent amount of work you can figure out the statistical likelihood of each of those steps.  For some you can look at road safety data, your design team can conduct FMEAs to understand which failures pose the risk of harm and their causes, and you can do an incredible amount of real world testing — I’d estimate we drove on the same 8mi stretch of road 1500-2500 times for our unmanned run.  

One of the surprise MVPs of the entire Unmanned program was diagnostics.  At Starsky we were able to build a highly modular and measurable system.  Each node was only supposed to do very specific and measurable things.  The front normal camera, for example, was supposed to spit out an image every so many milliseconds.  If it failed for a few milliseconds we would log a failure, and if that failure continued we would go to a minimal risk condition.  The lane detection model similarly was supposed to spit out lane lines every few milliseconds and those lane lines should look fairly similar to the previous set (give or take a few radians).  If that failed for too long we would pull over.  In the two months before our unmanned-run every safety-driver disengagement was predicted by the diagnostic system.

That is to say, that if the safety driver hadn’t been in the vehicle we wouldn’t have crashed.  We would have come to a safe stop.

For some branches in the failure tree we didn’t like our odds.  For example - even if we successfully avoided the accident there was a measurable likelihood that someone would rear-end our truck while pulled over on the side of the road.  We could, however, mitigate that risk by having a safety driver in a follow-car who was rated at being able to get in and start the truck in under 60 seconds to decrease our exposure to that risk meaningfully.  

in May 2019 Starsky did an unmanned practice run on a closed highway to drill the team in procedure and truck recovery. Knowing that a driver could get a stopped truck moving in 60s mitigated the risk of a sitting duck accident. STARSKY ROBOTICS

Doing an unmanned run is a matter of certainty - we needed to be statistically confident that we wouldn’t need a safety driver for the test that didn’t have one.  To stretch the parallel - we needed to be incredibly sure that we knew that our precautions would make us unlikely to catch Covid-19 if we walked through that ward as the first step towards a broader re-opening.

Our simple high level metric was the number of consecutive zero-disengagement runs we had completed.  A zero-disengagement run is a run where the safety driver wasn’t needed from the beginning of the test to the very end.  

When we did our first zero-disengagement run, back in Aug’17, it was a matter of luck.  We had been trying for 3 days nonstop and everything finally worked as planned.  That would have been the first time we could have taken the person out of the truck, but we would have truly been rolling the dice.  As a metric, consecutive zero-disengagement runs are useful because if you haven’t needed a safety driver for 1000 consecutive tests it’s highly likely that that you won’t on your 1,001th test and could therefore take the safety driver out.

You can then do additional work to lower that number of consecutive tests. By doing an incredible amount of documentation to understand how the system worked and making sure that it was safe as intended, by building rigorous diagnostics which allowed the system to know if it was failing, controlling the conditions we drove in and countless smaller mitigations; we were able to model out that 80 zero-disengagement runs in a row would indicate a 1 in a million chance of fatality accident were we to take the driver out on the 81st test.

On June 11th, 2019, at Starsky Robotics we completed our 80th consecutive zero-disengagement run.  On June 15th we completed our 141st.  And on the 16th, we took the person out and completed the first ever unmanned public highway test.

The Starsky unmanned truck, and follow vehicles for rescue, head out to complete the worlds first ever unmanned public highway test. STARSKY ROBOTICS

Which is to say, we walked through the Covid-19 ward and didn’t get infected, let alone died.  For us to healthily live full-time in the Covid-19 ward there would have been a fair amount more work.  Throughout the second half of last year we were in the process of ruggedizing our system to support full-time unmanned operations, we would have needed to drive on the pre-selected routes thousands more times, and probably would have found a whole lot more diagnostics to write.

Just like wide scale re-opening of the economy without mass Covid-19 death, it was possible for our approach to lead to the deployment of unmanned vehicles.  And someday, someone will do it.

This post was originally published in Forbes

A Product Dive on how most ROI comes from Unmanned Remotely-Supervised Trucks

This is the second in a multipart series I’m calling “Opening the Starsky Kimono” where I share Starsky strategic insights that we previously considered top-secret.

True autonomy is worth almost nothing when it comes to trucks. For robot-driven unmanned trucks, or robotrucks, the magnitudinal improvement actually comes from solving the driver shortage the efficiencies fleets can focus on once they’re no longer preoccupied with the hiring and retention of drivers. Per dollar invested, making a robotruck truly drive itself has the least ROI of all of those improvements, which is fortunate because it’s also the least likely to happen.

At Starsky we modeled that our robotrucks could achieve 42% margins even if each had a dedicated remote person paying attention 100% of the time. Those margins would jump to 58% if that remote driver only needed to pay attention for the first and last miles.

True autonomy, on the other hand, would have added less than 2% to our bottom line. Which means that the technological achievement $70b has been invested in over the last decade is worth less to the trucking industry than automatic billing.

What!?

The US trucking industry is structured around the systemic shortage of long-haul truck drivers. The 3.5 weeks/mo an over-the-road (OTR) driver is expected to spend in a truck is so miserable that few will do it for even $60k/yr. On the other side, trucking is a highly fragmented commoditized industry, which puts no fleet in the position where they can raise prices sufficiently to afford to pay drivers more. The result is that the market typically has at least 50,000 too few drivers to meet demand.

This shortage defines everything in American trucking — it’s why our trucks have nice comfortable cabs (worse fuel efficiency but better driver retention), it’s why our railroads are so profitable (which is why Warren Buffet buys them), and why our supply chain has even been shaped to minimize how much time trucks need to drive in cities (drivers get paid by mile and hate driving in slow-speed cities).

On a fleet level, almost every trucking company has more demand than it has drivers. That means that the easiest way to increase profit is almost always to add more drivers. Platooning or automated dispatch would save fleets plenty, but almost every fleet would see better ROI if they instead invested in driver hiring and retention. As a result the industry invests in little outside of drivers.

That changes when you no longer need the driver in the truck. $60k/yr isn’t enough to entice over-the-road drivers but it is more than enough to recruit drivers who get to sleep at home. Without the pressure of the driver shortage fleets can tackle the problems the industry has traditionally neglected and change the traditional economics of trucking.

Trucking Economics

Trucking is a business with both high fixed and variable costs. For every dollar that comes in, the best run firms typically spend 75% of it evenly divided between fuel, equipment, and labor. They then spend another 17% or so of administrative overhead per truck before eking out an 8% profit margin.

The variability of truck drivers is part of why these costs are so hard to control. A driver may be willing to drive 25 days in May but only 20 in June so as to make it home for a family event. Still harder to manage, a different fleet may offer your driver a lucrative signing bonus which causes them to abandon your truck far afield from your office, which might effectively make your truck sit idle for a week and then cost money when you recover it.

That isn’t really a risk when it comes to robotrucks. While the AI system might have fits, it should be fairly predictable. A robotruck will never demand to go in an inefficient direction so as to be at their kid’s birthday. They’ll never quit to work for a competitor who pays more. They’ll never demand to replace a truck with uncomfortable seats.

The greater predictability of robotrucks allows for costs to be optimized like never before.

Fuel

Through a number of means, at Starsky we thought we could decrease our fuel spend by at least 20%, increasing base profit rate by 62%.

At peak Starsky spent over $150k/mo on fuel across our 50 truck fleet. Where big dollars are spent, savings can be found.

Increased fuel economy from smoother acceleration and deceleration, as well as the weight cut by eliminating the sleeper cab should save robotrucks 7.5–10% of their fuel consumption. 7.5% volume discounts are common for larger fleets from major truck stops. Some of the more sophisticated fleets use fuel futures to save 15–20%.

Much of the inefficiency in fuel purchasing, however, is human driven. Starsky’s $150k in monthly fuel spend was across 250 truck stops. Even if all those trucks were on the same route purchases would be spread across 10–20 truck stops, simply because it’s hard to schedule your drivers’ bathroom breaks.

But what if you could perfectly concentrate your fuel purchasing? $150k is 5–10% of monthly income for many truck stops. If a fleet only purchased fuel from one gas station per route, you should be able to purchase it for cost plus.

In total we modeled that we could save at least 20% of our fuel spend, bringing it down to 20% of gross. That would bring the overall profit margin of robotrucks up to 13%.

Equipment

By not needing a sleeper cab, self-insuring, and scale in a fragmented market we thought we could cut equipment costs by 40% which alone would increase profit 125%.

The 25% of gross, or $0.50/mi, that trucking companies spend on equipment is one of the harder costs to manage. The vast majority of it, or about $0.40/mi, goes into the purchase, maintenance, and depreciation of the tractor and trailer. The remaining $0.10/mi pays for insurance.

By the end, at Starsky we paid about $0.30/mi in fixed costs if our average truck drove 8200 miles/mo (a minimum that would have been considerably lower if we owned and didn’t lease our trucks). We then had a variable cost of about $0.10/mi to cover maintenance split 70:30 between the tractor and the trailer.

The most expensive part of this equation is the tractor. While a daycab (a truck without room for the driver to sleep) costs about $100k, a sleeper cab is typically $165k at list price. If you don’t need a driver to sleep in the truck, you don’t need a sleeper cab which allows you to immediately knock $65k off your fixed costs. At scale we projected that we could buy day cab tractors for ~$75k which would lower our cost per mile by about $0.12, or a savings of 16% of gross.

Since the Starsky robot primarily relied on cameras, our entire system cost only about $65k last June — without any scale our unmanned day cab would cost as much as a traditional truck. In time we expected our system to cost about $10k which would put our equipment cost at a thrifty $0.30/mi.

Insurance is another opportunity for cost optimization. An owner-operator truck driver might spend $12k/yr on truck insurance and a fleet with 10–50 trucks might spend $7000/truck/year. The big fleets, however, are able to see large savings by self-insuring.

Truck deductibles are typically over $10k. The most expensive risk to insure is all of the liability from the $10,001th dollar to the $500,000th, which is what fleets are spending $6500 of the $7000 premium on. If that money is instead escrowed aside you can instead buy insurance on just the relatively cheap risk above $500k. That would make your cash outlay for insurance a mere $500/truck/yr, or less than $0.01/mi.

Factoring all listed above, a safe autonomous trucking fleet should spend only 15% of gross on equipment instead of 25%. That would bring robotruck profit margin up to 23% when added to fuel savings.

Admin

By making the backoffice of trucking as automated as Uber we thought we could cut the admin costs of trucking by 2/3rds, which would increase profit relative to normal trucking by 140%

Due to it’s high stakes-high volume nature, the trucking industry has been largely reluctant to adopt new technology over the last 30 years and as a result most trucking fleets run on software and processes that would drive a tech entrepreneur crazy. It isn’t uncommon for a fleet to have 3 fulltime payroll people per 100 trucks, or a dispatcher per 15 trucks, or for dispatchers to spend hours a day telling shippers where their freight is…all tasks whose automation are table stakes for on-demand startups.

Trucking companies spend at least 17% of their revenue doing tasks that Uber spends zero incremental dollars on. We estimated that fully automating dispatch alone could save us upwards of $5000/truck/year. Even automatically ingesting paper Bills of Lading into our TMS could save us $500/truck/year.

In total we conservatively thought we could eliminate 2/3rds of overall trucking admin cost to bring it down to 5% of gross; bringing our profit margin up to 35%.

Labor

Almost all labor savings come from eliminating highway supervision and then from decoupling the driver from the truck.

The vision of autonomy has been that a computer program capable of driving everywhere with zero marginal cost can easily replace the $60k drivers earn per year, representing 25% of what the truck brings in.

The problem though, is that vision seems to require a number of asterisks; most notably that those systems won’t be able to drive everywhere. At least $70b has been invested in trying to make that vision reality, but so far no one has been able to deploy unmanned systems at scale. It seems unlikely a breakthrough will happen any time soon.

Just the improvements above would allow for a trucking company to more than triple its margins. Without counting on breakthroughs in AI, it is possible to decrease the labor cost of operating a robotruck.

Decoupling the driver from the truck is the easiest way to cut labor costs.

Decoupling the driver from the truck is the easiest way to cut labor costs. American truck drivers are paid only for the miles they haul freight — the hours spent waiting to be loaded or unloaded and taking mandatory breaks are all unpaid. As a result, the $200/day drivers typically earn is really only for the 7 hours/day they move freight and not for the 14 total hours they’re on-duty. This delta is big — it means that the trucking company feels like they pay drivers $28/hr while drivers feel like they only get $14/hr (or $8/hr if you consider the 24 hours/day drivers spend in a truck).

Drivers who aren’t physically in a truck don’t need to lose productivity when the truck stops moving. Rather than twiddling their thumbs at a distribution center the driver could simply switch to a different truck in the fleet which has been loaded and is ready to move. The driver actually cares about earning $200/day — if they earned that by driving for 11 hours (vs being on duty for 14) their hourly rate would increase. The fleet would see their $200 buy them 57% more hours of a truck moving which would drop their labor cost by 30%, bringing it to just 17.5% of gross.

As a result a robotruck which is remotely monitored 100% of the time by a teleoperator who is looking only at that truck while it moves would cut labor cost from 25% of gross to 16.5% and have profit margins of 42.5%. Not bad for an MVP.

95% of the hours an OTR truck moves is on the highway. If you were able to eliminate remote supervision for the bulk of those hours you would see per-truck labor cost drop to just 1.75% of gross and profit margin would soar to 58%.

The hardest 1% of the technical problem, automating the surface streets and interchanges, would end up being worth only about $600/truck/yr. Level 4 truck autonomy has less value than a daily coffee.

Level 4 truck autonomy has less value than a daily coffee.

So What?

In Y-Combinator they tell you to make something people want. By which they don’t mean to build something your engineers want, but that your customers want. The US trucking industry wants a steady supply of trucks, and don’t particularly care if they’re autonomous or not.

If the industry doesn’t care about higher level, or L4, autonomy; if it’s really hard to build; and if it barely improves the margins of a robotruck fleet, then why build it?

At Starsky we always kept an eye on the business that we were building, which allowed us to tightly narrow the definition of our product.

At Starsky we always kept an eye on the business that we were building, which allowed us to tightly narrow the definition of our product. The high margins of unmanned but supervised trucking allowed us to sidestep most of the hardest parts of autonomy. If our robotrucks only drove as much as regular trucks we would make about $270k/truck in revenue or $156k in margin; which means 10,000 robots would make us $2.7b/yr.

That figure isn’t important because of all the cool things you can do with billions in margin, but because of all the things you don’t have to do to get to that size. Regulations in France not amenable to unmanned rollout? Don’t need to deploy there. Autonomous driving in snow appears to be an open research question? Don’t drive where it snows. Highway driving difficult at night? Drive only during the day. Connectivity not good enough everywhere? Only drive on roads with good connectivity.

Once you stop focusing on the quagmire which is true AI, the solvable challenge of designing a system safe enough for an unmanned test becomes paramount.

If L4 autonomy is worth little more than L3, which is worth only 30% more than L2; it becomes pretty clear that the primary focus of a robotruck startup shouldn’t be autonomy. It should be to make the simplest possible system that can complete a trip without a person in it. Once you stop focusing on the quagmire which is true AI, the solvable challenge of designing a system safe enough for an unmanned test becomes paramount. Which is exactly what we were able to focus on at Starsky.

-Stefan

Bonus: Increasing the topline

Robotrucks should be able to easily garner a 50% premium

The above improvements assume two easily disprovable assumptions: that robottrucks will earn the same revenue per mile as regular trucks and that they’ll drive no more hours a day.

The first assumption is easy to disprove. We found that not only did brokers not expect a discount (there is a shortage after all) but that in time we would be able to charge a premium because of a higher quality of service.

Most OTR fleets have greater than 100% in annual turnover, with most drivers quitting with little to no notice. If you’re expecting 100 trucks to pick up yoru freight tomorrow, it wouldn’t be unusual for only 95 to show up. Larger fleets can actually charge a 25–50% premium to be held accountable — if you want to guarantee that they’ll have 100 trucks you need to pay more. Given that robotrucks won’t call in sick, it should be easier to earn that premium.

The second disprovable assumption is that robotrucks will drive no more than a traditional truck. This was modelled in both for ease of comparison (it affects fixed equipment cost in interesting ways) and because it gives the robotruck 17 hours/day to wait out conditions that are currently difficult for autonomy. It should be fairly easy for robotrucks to drive 16–20 hours a day which allows for faster delivery. Traditionally that’s only accomplished by having two drivers in a truck, which adds about 50% to trucking’s price per mile.

A good benchmark for trucking rates is $2/mi in revenue. With these two premiums robotrucks should be able to garner $3/mi fairly easily.

If cost decreased from $1.80/mi to $0.80/mi with all of the improvements listed above then robotrucks (unmanned but supervised 10% of the time) should get a 73% margin.

Which is pretty good, for a product in such demand that I’m still getting calls from brokers.

This post was originally published in Medium

This blog was originally published on Starsky's Medium Blog, a la Medium.  You can read the original article here.

In 2015, I got obsessed with the idea of driverless trucks and started Starsky Robotics. In 2016, we became the first street-legal vehicle to be paid to do real work without a person behind the wheel. In 2018, we became the first street-legal truck to do a fully unmanned run, albeit on a closed road. In 2019, our truck became the first fully-unmanned truck to drive on a live highway.

And in 2020, we’re shutting down.

I remain incredibly proud of the product, team, and organization we were able to build; one where PhDs and truck drivers worked side by side, where generational challenges were solved by people with more smarts than pedigree, and where we discovered how the future of logistics will work.

Like Shackleton on his expedition to Antarctica, we did things no one else ever has. Similarly, though, it didn’t turn out as planned.

So what happened?

Timing, more than anything else, is what I think is to blame for our unfortunate fate. Our approach, I still believe, was the right one but the space was too overwhelmed with the unmet promise of AI to focus on a practical solution. As those breakthroughs failed to appear, the downpour of investor interest became a drizzle. It also didn’t help that last year’s tech IPOs took a lot of energy out of the tech industry, and that trucking has been in a recession for 18 or so months.

The AV Space

There are too many problems with the AV industry to detail here: the professorial pace at which most teams work, the lack of tangible deployment milestones, the open secret that there isn’t a robotaxi business model, etc. The biggest, however, is that supervised machine learning doesn’t live up to the hype. It isn’t actual artificial intelligence akin to C-3PO, it’s a sophisticated pattern-matching tool.

Back in 2015, everyone thought their kids wouldn’t need to learn how to drive. Supervised machine learning (under the auspices of being “AI”) was advancing so quickly — in just a few years it had gone from mostly recognizing cats to more-or-less driving. It seemed that AI was following a Moore’s Law Curve:

Projecting that progress forward, all of humanity would certainly be economically uncompetitive in the near future. We would need basic income to cope, to connect with machines to stand a chance, etc.

Five years later and AV professionals are no longer promising Artificial General Intelligence after the next code commit. Instead, the consensus has become that we’re at least 10 years away from self-driving cars.

It’s widely understood that the hardest part of building AI is how it deals with situations that happen uncommonly, i.e. edge cases. In fact, the better your model, the harder it is to find robust data sets of novel edge cases. Additionally, the better your model, the more accurate the data you need to improve it. Rather than seeing exponential improvements in the quality of AI performance (a la Moore’s Law), we’re instead seeing exponential increases in the cost to improve AI systems — supervised ML seems to follow an S-Curve.

The S-Curve here is why Comma.ai, with 5–15 engineers, sees performance not wholly different than Tesla’s 100+ person autonomy team. Or why at Starsky we were able to become one of three companies to do on-public road unmanned tests (with only 30 engineers).

If L1 is the line of human equivalence, then leading AV companies merely have to prove safety to be able to deploy. I don’t think I know anyone serious who believes this, but it is a possibility. If L2 is the case, the bigger teams are somewhere from $1–25b away from solving this problem. When big AV investors say that autonomy is an industry just for big companies, this is the bet that they’re making. If, however, L3 is the line of human equivalence it’s unlikely any of the current technology will make that jump. Whenever someone says autonomy is 10 years away that’s almost certainly what their thought is. There aren’t many startups that can survive 10 years without shipping, which means that almost no current autonomous team will ever ship AI decision makers if this is the case.

There aren’t many startups that can survive 10 years without shipping

Why We Didn’t Survive

To someone unfamiliar with the dynamics of venture fundraising, all of the above might seem like a great case to invest in Starsky. We didn’t need “true AI” to be a good business (we thought it might only be worth ~$600/truck/yr) so we should have been able to raise despite the above becoming increasingly obvious. Unfortunately, when investors cool on a space, they generally cool on the entire space. We also saw that investors really didn’t like the business model of being the operator, and that our heavy investment into safety didn’t translate for investors.

Trucking Blues

If teleop solves half the challenge of autonomy, the other half is solved by being the operator. As the trucking company you can choose where you operate — allowing you to pick your battles. Your system only has to be safe on the routes and in the conditions you choose to drive in (on the easiest routes and pulling over and waiting in bad conditions).

The nature of the participants in the trucking industry also reinforces the decision to be an operator. Trucking companies aren’t great technology customers (you should see what they use), and no one knows how to buy safety-critical on-road robots. Even if Starsky perfected general autonomy and perfectly validated safety, it would take years to deploy sufficient systems to make the necessary profits.

While trucking companies don’t know how to buy safety critical robots, they do know how to buy trucking capacity. Every large trucking company does so — their brokerages buy capacity from smaller fleets and owner-operators, many of whom they keep at an arm’s length because they don’t know how much to trust their self-reported safety metrics. At Starsky we found 25+ brokers and trucking companies more than willing to dispatch freight to trucks they already suspected were unmanned. While this is a lower margin business than software’s traditional 90%, we expected to be able to get to a 50% margin in time.

It took me way too long to realize that VCs would rather a $1b business with a 90% margin than a $5b business with a 50% margin, even if capital requirements and growth were the same.

And growth would be the same. The biggest limiter of autonomous deployments isn’t sales, it’s safety.

No One Really Likes Safety, they like Features

In January of 2019, our Head of Safety, our Head of PR, and I gathered in a conference room for a strategy session. The issue: how could we make safety seem exciting enough to cover. A month earlier we had publicly released our VSSA, a highly technical document that detailed how we decided to approach safety. We had pitched it to a particularly smart reporter, but instead of covering it in detail they mostly wrote about teleoperation. We left the meeting in a fluster — we couldn’t figure out how to make safety engineering sexy enough to garner its own reporting.

And we never really figured out how.

Safety engineering is the process of highly documenting your product so that you know exactly the conditions under which it will fail and the severity of those failures, and then measuring the frequency of those conditions such that you know how likely it is that your product will hurt people versus how many people you’ve decided are acceptable to hurt.

Doing that is really, really hard. So hard, in fact, that it’s more or less the only thing we did from September of 2017 until our unmanned run in June of 2019. We documented our system, built a safety backup system, and then repeatedly tested our system to failure, fixed those failures, and repeated.

The problem is that all of that work is invisible. Investors expect founders to lie to them — so how are they to believe that the unmanned run we did actually only had a 1 in a million chance of fatality accident? If they don’t know how hard it is to do unmanned, how do they know someone else can’t do it next week?

Our competitors, on the other hand, invested their engineering efforts in building additional AI features. Decision makers which could sometimes decide to change lanes, or could drive on surface streets (assuming they had sufficient map data). Really neat, cutting- edge stuff.

Investors were impressed. It didn’t matter that that jump from “sometimes working” to statistically reliable was 10–1000x more work.

So, what’s next?

Around November 12 of 2019, our $20m Series B fell apart. We furloughed most of the team on the 15th (probably the worst day of my life), and then started work on selling the company and making sure the team didn’t go without shelter (or visa status, or healthcare for the new and expectant parents).

We were able to land many of the vulnerable jobs by the end of January and I’m in the process of selling the assets of the company (which includes a number of patents essential to operating unmanned vehicles). Like the captain of a sinking ship, I’ve gotten most of the crew on lifeboats and am now noticing the icy waters at my ankles while I start to think about what I do next.

From my vantage point, I think the most likely line of human equivalence is L3 which means that no one should be betting a business on safe AI decision makers. The current companies who are will continue to drain momentum over the next two years, followed by a few years with nearly no investment in the space, and (hopefully) another unmanned highway test for 5 years.

I’d love to be wrong. The aging workforce which will almost certainly start to limit economic growth in the next 5–10 years; the 4000 people who die every year in truck accidents seem a needless sacrifice. If we showed anything at Starsky, it’s that this is buildable if you religiously focus on getting the person out of the vehicle in limited-use cases. But it will need to be someone else to realize that vision.

Signing off,

Shipping by truck accounts for 70% of overland freight in the US

Image: Nigel Tadyanehondo/Unsplash

Disruptions to the economics of freight can help or hinder humanity – so as shipping technology advances, we must leverage it to build a more prosperous and more peaceful world.