How to turn a collection of small building blocks into a versatile tool for solving regression problems.
The goal of this handbook is to demystify algorithmic differentiation, the tool that underlies modern machine learning. It begins with a calculus-101 style understanding and gradually extends this to build toy implementations of systems similar to PyTorch and TensorFlow. I have tried to clarify the relationships between every kind of differentiation I can think of – including forward and reverse, symbolic, numeric, tracing and source transformation. Where typical real-word ADs are mired in implementation details, these implementations are designed to be coherent enough that the real, fundamental differences – of which there are surprisingly few – become obvious.
Great article based on a close read of the vim manual.
With little evidence that failing to complete a prescribed antibiotic course contributes to antibiotic resistance, it’s time for policy makers, educators, and doctors to drop this message
Table Of Organic Compounds And Their Smells. Smell is…a lot more logically organized in a dimensional space than I thought. And in case you have the same question I do: “ethereal” = “smells like ether”.
a Navy scientist has filed a patent for a quantum superconducter antigravity drive capable of UFO-like feats of impossible aeronautics. When the Patent Office rejected it as outlandish, the Chief Technical Officer of naval aviation personally wrote the Patent Office saying it was totally possible and a matter of national security, after which the Patent Office relented and granted the patent. The patent thanks UFO researchers in the acknowledgements, includes a picture of a UFO recently sighted by Navy pilots, and does everything short of print in capital letters ‘THIS COMES FROM A UFO’. Scientists who were asked to comment say the proposed drive is “babble” and none of the supposed science checks out at all.
LW: autopsy of last year’s self-driving Uber crash.
Kurt Vonnegut’s ice-9 is science fiction, but the same process – a new crystalline form arising in a substance, spreading unstoppably, and destroying everything that relied on the old form – happened in real life to the AIDS drug ritonavir
This is a collection of Gwern’s most important writing that comprises
3 essays 5 tweets 1 obscure note on his site 1 sidenote 1 special edition of his newsletter 1 reddit comment 4 book reviews Plus,
2 reddit comments written in response to Gwern 2 PDFs on his site a few more important but not essential essays and book reviews
Covers the neural annealing metaphor, Friston’s free energy, psychedelics, etc. Most interesting part for me:
For a general overview of the math, see Wikipedia on simulated annealing, Metropolis-Hastings algorithm, Parallel tempering; for more on Carhart-Harris’s and Friston’s work, see Scott Alexander’s and Milan Griffes’ commentary.
Spoiler: it’s a Dart formatter. I love the skull footnotes of wrong turns. Great overview of why this kind of code is hard. But it left me wondering… did he consider using the standard algorithms?
The original article by Alexey Guzey is well worth reading, as is this follow-up.
In 1953, John Wheeler accidentally left a file describing how to build the Hydrogen Bomb on a train: https://t.co/Q5W5IrOa36— michael_nielsen (@michael_nielsen) December 3, 2019
I'm try to imagine how, exactly, you'd report this to the authorities. pic.twitter.com/nqNyLwc2U8
if someone ever tries to tell you something "because godel" direct 'em to this paper. not only are most "popular" uses of "godel incompleteness" dead wrong, but it turns out, a good 2/3 of the "proper" uses of it make it out to be too strong as well. https://t.co/ZzI3qunOyA— sclv (@sclv) December 3, 2019
Even the most advanced languages squash everything into a single grand structured category, like they want to be topos theorists or something. Languages should live in many categories connected by adjunctions— julesh (@_julesh_) November 30, 2019
Extra: The thing that will eventually replace monads is adjunctions https://t.co/aWFNvR8YBX
This feels like a real breakthrough:https://t.co/eGG6cqCJS4— Ian Osband (@IanOsband) November 21, 2019
Take the same basic algorithm as AlphaZero, but now *learning* its own simulator.
Beautiful, elegant approach to model-based RL.
... AND ALSO STATE OF THE ART RESULTS!
Well done to the team at @DeepMindAI #MuZero
Belfast in 1973— createstreets (@createstreets) November 22, 2019
Subsets of a space are predicates i.e. “yes/no questions about points of the space.” So-called open sets are “the questions you’re allowed to ask”. A continuous function is a consistent rule for turning an allowed yes/no question about its “output” into one about its “input”. https://t.co/sOTfgSOjf4— davidad 🎇 (@davidad) November 20, 2019
1/ The ACA stipulates insurance companies cannot keep profits from reducing costs. The 'medical loss ratio' is capped at 80% (meaning insurers cannot earn margins higher than 20%). As a result, insurers cannot grow profits by reducing costs....— Inquisitive Investor (@QuisitiveInvest) November 4, 2019
Terry Tao & co. have just rewritten their paper about the eigenvector-eigenvalue identity that I covered in Quanta. They now review ~2 dozen independent discoveries of the ID that have come to light since our article took off and analyze the sociology: https://t.co/nYSMzAzvg4 https://t.co/c7bqQY4xLg— Natalie Wolchover (@nattyover) December 4, 2019
Steve Ballmer and Bill Gates once made an Austin Powers parody video pic.twitter.com/jahpuRg2nR— Jon Erlichman (@JonErlichman) December 2, 2019
Looking at different generations' size vs. share of US wealth over time pic.twitter.com/b7tZbvG0Rb— Noah Veltman (@veltman) December 2, 2019
Now you know: Proportion ever having sex with someone other than spouse, while married: By gender and birth cohort. pic.twitter.com/UHAsiF9SpS— Philip 💥 Cohen (@familyunequal) November 30, 2019