Interactive exploration of the Kelly criterion
Conventional startup and business wisdom has become “there are plenty of good ideas, all that matters is execution.” We don’t buy it. Empirically, all our aspiring founder friends are desperate for startup ideas, with few managing to land on something worthwhile, and all our scientist friends are desperate for research ideas. Good ideas are essential, even ones that seem obvious in retrospect were non-obvious ex ante, and in fact good execution itself depends critically on good ideas.
academics are terrified of being scooped, why shouldn’t founders be? Even when professors will share a lot about what they’re working on, they’ll be more hesitant to tell you how they arrived at their ideas or why they decided to pursue them
Even if founders think ideas are precious, they have to share them if they want to raise funding, so the observation that founders share their ideas may in part just come from them needing $ and attention
Bitcoin - electronic money obvious, solution to double spend problem non-obvious
(Geoffrey Hinton)
This paper does not describe a working system. Instead, it presents a single idea about representation which allows advances made by several different groups to be combined into an imaginary system called GLOM. The advances include transformers, neural fields, contrastive representation learning, distillation and capsules. GLOM answers the question: How can a neural network with a fixed architecture parse an image into a part-whole hierarchy which has a different structure for each image? The idea is simply to use islands of identical vectors to represent the nodes in the parse tree. If GLOM can be made to work, it should significantly improve the interpretability of the representations produced by transformer-like systems when applied to vision or language
If we want to make neural networks that understand images in the same way as people do, we need to figure out how neural networks can represent part-whole hierarchies. This is difficult because a real neural network cannot dynamically allocate a group of neurons to represent a node in a parse tree. The inabil- ity of neural nets to dynamically allocate neurons was the motivation for a series of models that used “capsules”.
The GLOM architecture is composed of a large number of columns which all use exactly the same weights.
it is still unclear how to dynamically create a graph structure without the ability to allocate neurons on the fly
Capsules do not have the signature of really practical ideas like stochastic gradient descent or transformers which just want to work.
An even more profligate version is to dedicate several different levels of ubiquitous universal capsule to each location so that a location can belong to a scene, an object, a part and a sub-part simultaneously. This paper explores this profligate way of representing the part-whole hierarchy. It was inspired by a biological analogy, a mathematical analogy, and recent work on neural scene representations
I believe that our primary mode of reasoning is by using analogies which are made possible by the similarities between learned high-dimensional vectors
It’s amazing how much discussion this has triggered online.
TL;DR: The game takes ~6m to start in online mode. Turns out it’s parsing a 10mb JSON file and the parser is /really/ bad. Guy disassembles the game to fix it on his machine.
One follow-up: It Can Happen to You
As someone that has been programming for many years, this was a perfectly-timed reminder that there are /always/ pitfalls out there. The documentation for sscanf does not include a time complexity, so this is particularly tricky footgun , and I’m sure it’s not the only one lurking in the darkness.
Notice what the Rosser sentence accomplishes: it creates a symmetry between the cases that R(F) is provable and R(F) is disprovable, correcting the asymmetry between the two cases in Gödel’s original argument.
I simply observe Gödel’s Theorem as a trivial corollary of what I see as its conceptually-prior (even though historically-later) cousin: Turing’s Theorem on the unsolvability of the halting problem.
The halting theorem, Cantor’s theorem (the non-isomorphism of a set and its powerset), and Goedel’s incompleteness theorem are all instances of the Lawvere fixed point theorem
Speech synthesis software has reached the point where you can listen to The Notorious B.I.G. rap H.P. Lovecraft’s “Nemesis” , in case for some reason that is a thing you want to do:
Julia Galef interviews (Vitalik Buterin)
Curating isn’t undemocratic or “elitist,” a term that is now used so often that it’s become meaningless. It’s an act of generosity—you’re sharing what you love and what has inspired you. (The best streaming platforms, such as the Criterion Channel and MUBI and traditional outlets such as TCM, are based on curating—they’re actually curated.) Algorithms, by definition, are based on calculations that treat the viewer as a consumer and nothing else.
for the same body mass, a mammal typically uses ~10x the food and oxygen of a comparable reptile This is also why it takes so much food to raise a cow or sheep, but so little to raise the same mass of tilapia.
Interesting thread on how non-obvious AWS was. There was an Amazon Web Services in 2002, but it was just Amazon retail services. AWS had no competition for 7 years (really?). Paul Graham worked on a similar idea (to AWS and Replit) in 2000.
Original tweet deleted, but linked to Multimodal Neurons in Artificial Neural Networks. Related: