We are the teachers of a new form of intelligence now emerging on the planet, and we don’t quite know how long our intellectual progeny will continue listening to our wishes.
Some recent breakthroughs in training machines based on human preferences could well serve as early versions of governance systems for shaping the evolution of artificial intelligence. One of the reasons that human preferences are important to machine governance is that many of the tasks we’d like to hand off to machines aren’t well-suited training machines because they lack objective and clearly articulated rewards.
“Reward functions” provide feedback to a machine learning system that helps it know whether it’s headed in the right direction. When DeepMind trained its algorithms to play old-fashioned Atari video games, it used game scores as a reward function to measure and improve the algorithm’s strategies. Again, the problem is that the majority of the world’s problems lack that kind of objective reward function.
Take self-driving cars. While it’s true that aspects of collision avoidance can be turned into objective reward functions, they aren’t the only factor that goes into making a successful self-driving vehicle. For mainstream adoption to take off, we also need to consider the comfort of the ride, its convenience, how fun it is and countless other subjective variables. To turn these subjective measures into the kind of data needed for useful reward functions, we need good systems for collecting the subjective preferences of human riders.
Our future intelligent systems will depend upon subjective human feedback. We will need reliable, cost-effective mechanisms for capturing human preferences and turning that information into the reward functions that teach our machine learning systems.
In most cases, at least some of that capturing of human preferences will happen during product development, as companies train algorithms for their various new products and services. In most cases though, these initial instructions will prove insufficient. We will want our products and services to learn and adjust themselves as we use them over time. In the world where all products are tethered to an Internet of Things, we will come to see all products as services and we will expect those service to learn from their interactions with us.
We already expect this kind of learning today, thanks to the personalization that the Internet has made possible. I expect Netflix to customize my welcome screen, Google to uniquely tailor my search results and Amazon to personalize my shopping experience. In this sense, personalization acts as a stepping stone for building reward functions for machine governance based on human preferences.
We are going to need to think more deliberately about the governance of machines. When we think of a word like “governance” we tend to think of governments and maybe boards of directors.
A “governor” is also a mechanical system, however. It is used to constrain the speed of engines and other industrial machines. Now, as our automated machines do much more than just whir and spin, our concerns expand beyond simply throttling their operating speeds. As machines become increasingly intelligent, machine governance has to expand to something closer to the way we think of the governance of human systems.
Machine governance is critical to the emerging marriage of humanity and machines. Getting it right is of critical importance, and thus an emerging focus for my writing here on the Vital Edge.