How to turn a collection of small building blocks into a versatile tool for solving regression problems.
The goal of this handbook is to demystify algorithmic differentiation, the tool that underlies modern machine learning. It begins with a calculus-101 style understanding and gradually extends this to build toy implementations of systems similar to PyTorch and TensorFlow. I have tried to clarify the relationships between every kind of differentiation I can think of – including forward and reverse, symbolic, numeric, tracing and source transformation. Where typical real-word ADs are mired in implementation details, these implementations are designed to be coherent enough that the real, fundamental differences – of which there are surprisingly few – become obvious.
Great article based on a close read of the vim manual.
With little evidence that failing to complete a prescribed antibiotic course contributes to antibiotic resistance, it’s time for policy makers, educators, and doctors to drop this message
Table Of Organic Compounds And Their Smells. Smell is…a lot more logically organized in a dimensional space than I thought. And in case you have the same question I do: “ethereal” = “smells like ether”.
a Navy scientist has filed a patent for a quantum superconducter antigravity drive capable of UFO-like feats of impossible aeronautics. When the Patent Office rejected it as outlandish, the Chief Technical Officer of naval aviation personally wrote the Patent Office saying it was totally possible and a matter of national security, after which the Patent Office relented and granted the patent. The patent thanks UFO researchers in the acknowledgements, includes a picture of a UFO recently sighted by Navy pilots, and does everything short of print in capital letters ‘THIS COMES FROM A UFO’. Scientists who were asked to comment say the proposed drive is “babble” and none of the supposed science checks out at all.
LW: autopsy of last year’s self-driving Uber crash.
Kurt Vonnegut’s ice-9 is science fiction, but the same process – a new crystalline form arising in a substance, spreading unstoppably, and destroying everything that relied on the old form – happened in real life to the AIDS drug ritonavir
This is a collection of Gwern’s most important writing that comprises
3 essays 5 tweets 1 obscure note on his site 1 sidenote 1 special edition of his newsletter 1 reddit comment 4 book reviews Plus,
2 reddit comments written in response to Gwern 2 PDFs on his site a few more important but not essential essays and book reviews
Covers the neural annealing metaphor, Friston’s free energy, psychedelics, etc. Most interesting part for me:
For a general overview of the math, see Wikipedia on simulated annealing, Metropolis-Hastings algorithm, Parallel tempering; for more on Carhart-Harris’s and Friston’s work, see Scott Alexander’s and Milan Griffes’ commentary.
Spoiler: it’s a Dart formatter. I love the skull footnotes of wrong turns. Great overview of why this kind of code is hard. But it left me wondering… did he consider using the standard algorithms?
The original article by Alexey Guzey is well worth reading, as is this follow-up.