I cycle round London every day and accounting for the reckless behaviours of drivers is no easy task. But I'm not convinced that a driverless car need be any worse at doing so than a human. Someone slamming into the back of you or flying out of a side road without warning is a problem for human drivers and non-human drivers alike. The human proclivity to overreact in response to such situations, slam the brakes and throw the steering wheel to one side potentially ploughing into a bunch of pedestrians or causing a pile up of oncoming traffic on the other side of the road, might even be better dealt with by a machine.
An obvious answer to reckless drivers might be to make all cars driverless..... Individual vehicles operating as units in a wider single over-arching system. But that is probably some way off.