Å·±¦ÓéÀÖ

"The best path toward the development of beneficial superintelligence is one in which AI developers and AI safety researchers are on the same side"

Technology Coupling
leading discovery in other fields

so it is good to postpone and yet it is not

"We find ourselves in a thicket of strategic complexity, surrounded by a dense mist of uncertainty."

By delaying the arrival of super intelligence, we will advance in other areas leading to overall increased progress and also advance in possible AI control. However, in the same time we face risks presented by those advancements. Thus, by pursuing super AI first we could in theory effectively face the risks presented by other advancement, while dealing only with the risk poised by super AI.

"It is not necessary for us to create a highly optimized design. Rather, our focus should be on creating a highly reliable design, one that can be trusted"

"offloading even more cognitive work onto the AI? Where is the limit to our possible laziness?"

If were to offload all the hard cognitive work about goals, values of AI to itself to find out how to reign itself (more or less what I understood)

"it would anyway be impossible—even for a superintelligence—to find out what humanity would actually want"

"Because the complexity is largely transparent to us, however, we often fail to appreciate that it is there"

"How could our programmer transfer this complexity into a utility function?"

"Asimov probably having formulated the laws in the first place precisely so that they would fail in interesting ways, providing fertile plot complications for his stories"

"The need to solve the control problem in advance—and to implement the solution successfully in the very first system to attain superintelligence"

From here on the book predominantly focuses on the control problem