I'll admit that I don't really know shit about self-driving cars or AI, but I keep thinking about this, so I might as well dump it here. It occurred to me that the safest way to drive (not the most efficient or most convenient) would be to assume that everything around the car is completely stationary. In other words, if it were driving 80 mph on a highway, and there's a car visible in front of it, assume the car is going to instantly stop as if it hit a wall and comfortably decelerate to prevent hitting it long before that becomes a risk. The closer it is to something that it's driving towards, the slower it has to get, but driving away from something, it can accelerate as fast as the driver is comfortable with, until it starts approaching something else. If it were parking then getting just shy of touching a wall would be ideal, but while driving it's best to at least keep enough distance to drive around the car in front of it. Perpendicular movement is tricky, since cars can easily pass each other in opposite lanes inches from each other without accidents being common, but just the same if it were driving alongside a wall and something walked out from a doorway in that wall, it could be immediately in front of the car without warning, so the safest behavior is simply to drive slower, so driving perpendicular to something is no different than driving towards it, especially when on a winding road where you never know what's around the corner. Obstacle avoidance could be based on the whatever direction it can safely move in the fastest. I think you could even apply the same logic to flying and higher speeds at higher altitudes, although with regular cars you'd need to slow or steer as it approaches potholes or ditches on the side of the road. Or maybe I'm just a retard with Dunning–Kruger effect and driving safely is really a lot more complicated than that.
Regardless, I'd love to see a simple simulation with a bunch of cars driving around following that simple logic, even with perfect omnidirectional vision not being realistic, I think the real-world hardware would amount to cameras on the bumpers and sides of the car, and the closer it gets to anything, the lowest value determines the max speed the car can go. There'd need to be a lot more added before it'd anything more than a glorified always-on parking assist.
Though I this kind of driving behavior would only really be safe if every other car on the road drove the same way, since humans are reckless and impatient assholes, but flashing your blinkers a lower speeds could be enough to help occasionally.