The Goals of Neurotechnology

I work on neurotechnology (defined broadly ) for 2.5 reasons.

The first is to give people tools to fix whatever about their brains they want to fix. More wellbeing, less depression, more intelligence, more wisdom, designer synesthesia — up to them. I’m an enlightenment pluralist.

The second is answering the most important questions in ethics. What is consciousness? Is it what morality should be based on? What entities have it? What don’t? Will the intelligences we build be greater moral patients than us? Can “AI alignment” possibly be operationalized? If these questions are answerable at all, it will only be with better neurotechnology.

The remaining half reason is existential risk. I suspect, though am not sure, that most of humanity’s greatest threats would disappear if most people could simply act the way they already want themselves to act. Neurotechnology is the path to achieving this.

Some thoughts on vegetarianism and veganism - EA Forum

Some interesting points I haven’t seen elsewhere (including in the comments).

Magnetic control of tokamak plasmas through deep reinforcement learning | Nature


What failure looks like - LessWrong

Paul Christiano on AI Risk.

Visualizing Utopia

(Holden Karnofsky)

I’m going to lay out a set of possible utopias that span a spectrum from conservative (anchored to the status quo) to radical (not anchored).

I don’t expect any single point on my spectrum to sound very satisfying, but I hope to help the reader find (and visualize) some point on this spectrum that they (the reader) would find to be a satisfying utopia. The idea is to give a feel for the tradeoff between these two ends of the spectrum (conservatism and radicalism), so that envisioning utopia feels like a matter of finding the right balance rather than like a sheer impossibility.

Wirehead Gods On Lotus Thrones | Slate Star Codex

I recently realized how unstable my contemptuous feelings are. Imagine instead our posthuman descendants taking the form of Buddhas sitting on vast lotus thrones in a state of blissful tranquility. Their minds contain perfect awareness of everything that goes on in the Universe and the reasons why it happens, yet to each happening, from the fall of a sparrow to the self-immolation of a galaxy, they react only with acceptance and equanimity. Suffering and death long since having been optimized away, they have no moral obligation beyond sitting and reflecting on their own perfection, omnipotence, and omniscience – at which they feel boundless joy.

I am pretty okay with this future. This okayness surprises me, because the lotus-god future seems a lot like the wirehead future. All you do is replace the dingy room with a lotus throne, and change your metaphor for their no-doubt indescribably intense feelings from “drug-addled pleasure” to “cosmic bliss”. It seems more like a change in decoration than a change in substance. Should I worry that the valence of a future shifts from “heavily dystopian” to “heavily utopian” with a simple change in decoration?

Longevity FAQ: A beginner’s guide to longevity research

Gradients without Backpropagation (Arxiv)


Using backpropagation to compute gradients of objective functions for optimization has remained a mainstay of machine learning. Backpropagation, or reverse-mode differentiation, is a special case within the general family of automatic differentiation algorithms that also includes the forward mode. We present a method to compute gradients based solely on the directional derivative that one can compute exactly and efficiently via the forward mode. We call this formulation the forward gradient, an unbiased estimate of the gradient that can be evaluated in a single forward run of the function, entirely eliminating the need for backpropagation in gradient descent. We demonstrate forward gradient descent in a range of problems, showing substantial savings in computation and enabling training up to twice as fast in some cases.

Conversation on technology forecasting and gradualism - LessWrong


So recently I was talking with various people about the question of why, for example, Steve Jobs could not find somebody else with UI taste 90% as good as his own, to take over Apple, even while being able to pay infinite money. A successful founder I was talking to was like, “Yep, I sure would pay $100 million to hire somebody who could do 80% of what I can do, in fact, people have earned more than that for doing less.”

I wondered if OpenPhil was an exception to this rule, and people with more contact with OpenPhil seemed to think that OpenPhil did not have 80% of a Holden Karnofsky (besides Holden).

And of course, what sparked this whole thought process in me, was that I’d staked all the effort I put into the Less Wrong sequences, into the belief that if I’d managed to bring myself into existence, then there ought to be lots of young near-Eliezers in Earth’s personspace including some with more math talent or physical stamina not so unusually low, who could be started down the path to being Eliezer by being given a much larger dose of concentrated hints than I got, starting off the compounding cascade of skill formations that I saw as having been responsible for producing me, “on purpose instead of by accident”.

I see my gambit as having largely failed, just like the successful founder couldn’t pay $100 million to find somebody 80% similar in capabilities to himself, and just like Steve Jobs could not find anyone to take over Apple for presumably much larger amounts of money and status and power. Nick Beckstead had some interesting stories about various ways that Steve Jobs had tried to locate successors (which I wasn’t even aware of).

I see a plausible generalization as being a “Sparse World Hypothesis”: The shadow of an Earth with eight billion people, projected into some dimensions, is much sparser than plausible arguments might lead you to believe. Interesting people have few neighbors, even when their properties are collapsed and projected onto lower-dimensional tests of output production. The process of forming an interesting person passes through enough 0-1 critical thresholds that all have to be passed simultaneously in order to start a process of gaining compound interest in various skills, that they then cannot find other people who are 80% as good as what they /do/ (never mind being 80% similar to them as people).

How to make cryogenic Multi-Layer Insulation (MLI) shields (at home)

Emoji to Scale

The Hidden Complexity of Wishes - LessWrong

There are three kinds of genies: Genies to whom you can safely say “I wish for you to do what I should wish for”; genies for which /no/ wish is safe; and genies that aren’t very powerful or intelligent .

Open-Source Wish Project

The goal of the Open-Source Wish Project is to create perfectly-worded wishes, so that when the genie comes and grants us our wish we can get precisely what we want. The genie, of course, will attempt to interpret the wish in the most malicious way possible, using any loophole to turn our wish into a living hell. The Open-Source Wish Project hopes to use the collective wisdom of all humanity to create wishes with no loopholes whatsoever.

The Second Law of Thermodynamics, and Engines of Cognition - LessWrong

The essential /physical/ law underlying the Second Law of Thermodynamics is a theorem which can be proven within the standard model of physics: /In the development over time of any closed system, phase space volume is conserved./

Mutual Information, and Density in Thingspace - LessWrong

The /mutual information/ of two variables is defined as the difference between the entropy of the joint system and the entropy of the independent systems: I(X;Y) = H(X) + H(Y) - H(X,Y).

I digress here to remark that the symmetry of the expression for the mutual information shows that Y /must/tell us as much about Z, on average, as Z tells us about Y. I leave it as an exercise to the reader to reconcile this with anything they were taught in logic class about how, if all ravens are black, being allowed to reason Raven(x)->Black(x) doesn’t mean you’re allowed to reason Black(x)->Raven(x). How different seem the symmetrical probability flows of the Bayesian, from the sharp lurches of logic—even though the latter is just a degenerate case of the former.

If we use, to achieve our purposes, a mechanical agency with whose operation we cannot interfere effectively … we had better be quite sure that the purpose put into the machine is the purpose which we really desire.” Norbert Wiener #quote

Prüfer sequence - Wikipedia

In combinatorial mathematics, the Prüfer sequence (also Prüfer code or Prüfer numbers) of a labeled tree is a unique sequence associated with the tree. The sequence for a tree on n vertices has length n − 2, and can be generated by a simple iterative algorithm.

Alignment research exercises - LessWrong

McNamara fallacy - Wikipedia

The McNamara fallacy (also known as the quantitative fallacy), named for Robert McNamara , the US Secretary of Defense from 1961 to 1968, involves making a decision based solely on quantitative observations (or metrics) and ignoring all others. The reason given is often that these other observations cannot be proven.

The first step is to measure whatever can be easily measured. This is OK as far as it goes. The second step is to disregard that which can’t be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can’t be measured easily really isn’t important. This is blindness. The fourth step is to say that what can’t be easily measured really doesn’t exist. This is suicide.

Daniel Yankelovich

An Orthodox Case Against Utility Functions - LessWrong

(Abram Demski)

Reductive utility:

  • Worlds are basically physical
  • Utility is a function of worlds
  • Utility is computable

Subjective utility:

  • Beliefs are /for things you can think about/.
  • Utility is a function of events
  • Updates are computable

So, utility is not a function:

In a logical induction (LI) framework, the central idea becomes /“update your subjective expectations in any way you like, so long as those expectations aren’t (too easily) exploitable to Dutch-book.”/ … This replaces the idea of “utility function” entirely — there isn’t any need for a /function/ any more, just a logically-uncertain-variable (LUV, in the terminology from the LI paper).

‘People, ideas, machines’ II: catastrophic thinking on nuclear weapons

(Dominic Cummings)

The actual motives and thinking of specific leaders was discouraged in favour of calculating balances of weapons. Bureaucracies focused on what can be counted and calculated rather than imponderable questions with no clear and comforting answers. The former was comforting. The latter seemed paralysing and/or nightmarish. Bureaucracies naturally gravitate toward the former unless very strong counter-pressures apply.

Chaitin’s constant - Wikipedia

a Chaitin constant (Chaitin omega number) [1] or halting probability is a real number that, informally speaking, represents the probability that a randomly constructed program will halt

Iconic Bottle Caps That Changed Packaging

Sophisticated food packaging in an MRI scanner.