I work on neurotechnology (defined broadly ) for 2.5 reasons.
The first is to give people tools to fix whatever about their brains they want to fix. More wellbeing, less depression, more intelligence, more wisdom, designer synesthesia â up to them. Iâm an enlightenment pluralist.
The second is answering the most important questions in ethics. What is consciousness? Is it what morality should be based on? What entities have it? What donât? Will the intelligences we build be greater moral patients than us? Can âAI alignmentâ possibly be operationalized? If these questions are answerable at all, it will only be with better neurotechnology.
The remaining half reason is existential risk. I suspect, though am not sure, that most of humanityâs greatest threats would disappear if most people could simply act the way they already want themselves to act. Neurotechnology is the path to achieving this.
Some interesting points I havenât seen elsewhere (including in the comments).
(DeepMind)
Paul Christiano on AI Risk.
(Holden Karnofsky)
Iâm going to lay out a set of possible utopias that span a spectrum from conservative (anchored to the status quo) to radical (not anchored).
I donât expect any single point on my spectrum to sound very satisfying, but I hope to help the reader find (and visualize) some point on this spectrum that they (the reader) would find to be a satisfying utopia. The idea is to give a feel for the tradeoff between these two ends of the spectrum (conservatism and radicalism), so that envisioning utopia feels like a matter of finding the right balance rather than like a sheer impossibility.
I recently realized how unstable my contemptuous feelings are. Imagine instead our posthuman descendants taking the form of Buddhas sitting on vast lotus thrones in a state of blissful tranquility. Their minds contain perfect awareness of everything that goes on in the Universe and the reasons why it happens, yet to each happening, from the fall of a sparrow to the self-immolation of a galaxy, they react only with acceptance and equanimity. Suffering and death long since having been optimized away, they have no moral obligation beyond sitting and reflecting on their own perfection, omnipotence, and omniscience â at which they feel boundless joy.
I am pretty okay with this future. This okayness surprises me, because the lotus-god future seems a lot like the wirehead future. All you do is replace the dingy room with a lotus throne, and change your metaphor for their no-doubt indescribably intense feelings from âdrug-addled pleasureâ to âcosmic blissâ. It seems more like a change in decoration than a change in substance. Should I worry that the valence of a future shifts from âheavily dystopianâ to âheavily utopianâ with a simple change in decoration?
Abstract:
Using backpropagation to compute gradients of objective functions for optimization has remained a mainstay of machine learning. Backpropagation, or reverse-mode differentiation, is a special case within the general family of automatic differentiation algorithms that also includes the forward mode. We present a method to compute gradients based solely on the directional derivative that one can compute exactly and efficiently via the forward mode. We call this formulation the forward gradient, an unbiased estimate of the gradient that can be evaluated in a single forward run of the function, entirely eliminating the need for backpropagation in gradient descent. We demonstrate forward gradient descent in a range of problems, showing substantial savings in computation and enabling training up to twice as fast in some cases.
(Yudkowsky)
So recently I was talking with various people about the question of why, for example, Steve Jobs could not find somebody else with UI taste 90% as good as his own, to take over Apple, even while being able to pay infinite money. A successful founder I was talking to was like, âYep, I sure would pay $100 million to hire somebody who could do 80% of what I can do, in fact, people have earned more than that for doing less.â
I wondered if OpenPhil was an exception to this rule, and people with more contact with OpenPhil seemed to think that OpenPhil did not have 80% of a Holden Karnofsky (besides Holden).
And of course, what sparked this whole thought process in me, was that Iâd staked all the effort I put into the Less Wrong sequences, into the belief that if Iâd managed to bring myself into existence, then there ought to be lots of young near-Eliezers in Earthâs personspace including some with more math talent or physical stamina not so unusually low, who could be started down the path to being Eliezer by being given a much larger dose of concentrated hints than I got, starting off the compounding cascade of skill formations that I saw as having been responsible for producing me, âon purpose instead of by accidentâ.
I see my gambit as having largely failed, just like the successful founder couldnât pay $100 million to find somebody 80% similar in capabilities to himself, and just like Steve Jobs could not find anyone to take over Apple for presumably much larger amounts of money and status and power. Nick Beckstead had some interesting stories about various ways that Steve Jobs had tried to locate successors (which I wasnât even aware of).
I see a plausible generalization as being a âSparse World Hypothesisâ: The shadow of an Earth with eight billion people, projected into some dimensions, is much sparser than plausible arguments might lead you to believe. Interesting people have few neighbors, even when their properties are collapsed and projected onto lower-dimensional tests of output production. The process of forming an interesting person passes through enough 0-1 critical thresholds that all have to be passed simultaneously in order to start a process of gaining compound interest in various skills, that they then cannot find other people who are 80% as good as what they /do/ (never mind being 80% similar to them as people).
There are three kinds of genies: Genies to whom you can safely say âI wish for you to do what I should wish forâ; genies for which /no/ wish is safe; and genies that arenât very powerful or intelligent .
The goal of the Open-Source Wish Project is to create perfectly-worded wishes, so that when the genie comes and grants us our wish we can get precisely what we want. The genie, of course, will attempt to interpret the wish in the most malicious way possible, using any loophole to turn our wish into a living hell. The Open-Source Wish Project hopes to use the collective wisdom of all humanity to create wishes with no loopholes whatsoever.
The essential /physical/ law underlying the Second Law of Thermodynamics is a theorem which can be proven within the standard model of physics: /In the development over time of any closed system, phase space volume is conserved./
The /mutual information/ of two variables is defined as the difference between the entropy of the joint system and the entropy of the independent systems:
I(X;Y) = H(X) + H(Y) - H(X,Y).
I digress here to remark that the symmetry of the expression for the mutual information shows that Y /must/tell us as much about Z, on average, as Z tells us about Y. I leave it as an exercise to the reader to reconcile this with anything they were taught in logic class about how, if all ravens are black, being allowed to reason Raven(x)->Black(x) doesnât mean youâre allowed to reason Black(x)->Raven(x). How different seem the symmetrical probability flows of the Bayesian, from the sharp lurches of logicâeven though the latter is just a degenerate case of the former.
If we use, to achieve our purposes, a mechanical agency with whose operation we cannot interfere effectively ⌠we had better be quite sure that the purpose put into the machine is the purpose which we really desire.â Norbert Wiener #quote
In combinatorial mathematics, the PrĂźfer sequence (also PrĂźfer code or PrĂźfer numbers) of a labeled tree is a unique sequence associated with the tree. The sequence for a tree on
nvertices has lengthn â 2, and can be generated by a simple iterative algorithm.
The McNamara fallacy (also known as the quantitative fallacy), named for Robert McNamara , the US Secretary of Defense from 1961 to 1968, involves making a decision based solely on quantitative observations (or metrics) and ignoring all others. The reason given is often that these other observations cannot be proven.
The first step is to measure whatever can be easily measured. This is OK as far as it goes. The second step is to disregard that which canât be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what canât be measured easily really isnât important. This is blindness. The fourth step is to say that what canât be easily measured really doesnât exist. This is suicide.
(Abram Demski)
Reductive utility:
Subjective utility:
So, utility is not a function:
In a logical induction (LI) framework, the central idea becomes /âupdate your subjective expectations in any way you like, so long as those expectations arenât (too easily) exploitable to Dutch-book.â/ ⌠This replaces the idea of âutility functionâ entirely â there isnât any need for a /function/ any more, just a logically-uncertain-variable (LUV, in the terminology from the LI paper).
(Dominic Cummings)
The actual motives and thinking of specific leaders was discouraged in favour of calculating balances of weapons. Bureaucracies focused on what can be counted and calculated rather than imponderable questions with no clear and comforting answers. The former was comforting. The latter seemed paralysing and/or nightmarish. Bureaucracies naturally gravitate toward the former unless very strong counter-pressures apply.
a Chaitin constant (Chaitin omega number) [1] or halting probability is a real number that, informally speaking, represents the probability that a randomly constructed program will halt
Sophisticated food packaging in an MRI scanner.