I am in the camp that is concerned about super intelligence. First the machines will do a lot of jobs
for us and not be super intelligent. That should be positive if we manage it well. A few decades after
that though the intelligence is strong enough to be a concern. I agree with Elon Musk and some
others on this and don’t understand why some people are not concerned.The Future of Life Institute is a volunteer-run research and outreach organization that
has been set up to measure and mitigate existential risks facing humanity, including superin-
telligence. The institute is currently focusing on potential risks from the development of
human-level artificial intelligence. Its advisory board includes individuals such as Stephen
Hawking, Alan Alda, and Elon Musk, to name a few. This demonstrates that some of the lead-
ing minds across various industries are genuinely worried about the perils of superintelli-
gence and that they want to contribute their time and effort to make sure we think through
the risks appropriately.
As we get closer to designing machines that are capable of superintelligence, it is likely
that professionals with cybersecurity experience are going to be called upon to assist in
designing algorithms that can help curtail potentially intelligent machines by running simula-
tions of artificial intelligence in a controlled environment (a sandbox) that protects the safety of
human beings.
Computing devices capable of greater levels of intelligence will have access to IoT devices
that they can control. The intelligent code itself will be a “thing” on the network it is executed
on and may have the intelligence to branch out onto other networks. The unique risk with
superintelligence is that large-scale catastrophes may occur if scientists in the lab are not able
a superintelligent machine in ways they originally thought they could. It is easy to see how the
knowledge of how to create and unleash superintelligent computers could be attractive to ter-
rorists who want to wreak destruction and havoc.
The threat of superintelligence is at the top of the minds of many scientists and research-
ers in the area of computer science, and this is quite likely the greatest human-made existen-
tial threat to humankind.
Conclusion
From thingbots to drones to device backdoors, the attack surface presented by interconnected
devices is going to be attractive to threat agents and provide them with unparalleled opportu-
nities to compromise our privacy and physical safety.
Vulnerabilities such as Heartbleed that suddenly affect millions of computing devices
have already been known to impact IoT devices, such as the Nest Thermostat. It is quite likely
that many other IoT devices in the market are vulnerable to Heartbleed and will continue to
be vulnerable due to lack of security patches, either because the vendors have not released
them or because the patches were not applied or failed. There are also devices that do not
incorporate any mechanism for patches to be applied, and these devices will remain insecure
CHAPTER 8: SECURELY ENABLING OUR FUTURE—A CONVERSATION ON(^248) UPCOMING ATTACK VECTORS