Roaring Bitmaps

  • “A better compressed bitset. Roaring bitmaps are compressed bitmaps. They can be hundreds of times faster.”
  • Used by a ton of impressive projects (Google Procella, Apache Lucene, etc)

Scientists rename human genes to stop MS Excel from misreading them as dates

TikTok and the Sorting Hat

Excellent post on how TikTok became what it is.

The Sorting Hat is perhaps the most curious plot device from the Harry Potter universe. Is it a metaphor for genetic determinism? Did Draco have any hope of not being a Slytherin? By sorting Draco into that house, did it shape his destiny? Is the hat a metaphor for the U.S. college admissions system, with all its known biases? Is Harry Potter, sorted into Gryffindor, a legacy admit?

Instagram is some strange hybrid mix of social and interest graph, and now it’s also a jumble of formats, with a Stories feed relegated to a top bar in the app while the more stagnant and less active original feed continues to run vertically as the default. Messaging is pushed to a separate pane and also served by a separate app. Longer form videos bounce you to Instagram TV, which is just an app for videos that exceed some time limit, I guess? And soon, perhaps commerce will be jammed in somehow? Meanwhile, they have a Discover tab, or whatever it is called, which seems like it could be the default tab if they wanted to take a more interest-based approach like TikTok. But they seem to have punted on making any hard decisions for so long now that the app is just a Frankenstein of feeds and formats and functions spread across a somewhat confused constellation of apps.

Chrisopher Alexander: A Primer (video)

See tweet section for more Chrisopher Alexander

My tier list of interesting YouTube channels

By Tristan Hume

Gelman Against Parsimony (lesswrong)

Robert Sapolsky Human Behavioral Biology

Fun series of lectures I’ve been watching when I have some free time and feel like thinking a bit but not too hard. Companion website

Adversarial Examples that Fool both Computer Vision and Time-Limited Humans

- adversarial examples often transfer between models
- leverage three key ideas:
    1. "we use the recent black box adversarial example construction techniques that create adversarial examples for a target model without access to the model’s architecture or parameters"
    2. "we adapt machine learning models to mimic the initial visual processing of humans, making it more likely that adversarial examples will transfer from the model to a human observer"
    3. "we evaluate classification decisions of human observers in a time-limited setting, so that even subtle effects on human perception are detectable"
- "We find that adversarial examples that transfer across computer vision models do successfully influence the perception of human observers, thus uncovering a new class of illusions that are shared between computer vision models and the human brain"
- Adversarial examples that transfer between models also affect human response time and accuracy

A Close Call: How a Near Failure Propelled Me to Succeed

  • Terry Tao story about his poor study habits through his second year of grad school, almost failed general exams, and learned better habits as a result.
  • Write-up of his generals

Common Crawl

“We build and maintain an open repository of web crawl data that can be accessed and analyzed by anyone.”

T5: Text-To-Text Transfer Transformer

Images of Panamax Ships in the Panama Canal Locks

Apparently neopanamax and post-neopanamax are ship size standards.

Tea preparation ISO standard

Abstract: The method consists in extracting of soluble substances in dried tea leaf, containing in a porcelain or earthenware pot, by means of freshly boiling water, pouring of the liquor into a white porcelain or earthenware bowl, examination of the organoleptic properties of the infused leaf, and of the liquor with or without milk or both.

NeuroSAT: Learning a SAT Solver from Single-Bit Supervision

  • reddit (video — Daniel Selsam presentation)
  • HN
  • training data
    • define a distribution SR(n): problems come in pairs: differ by negating one literal in one clause, one SAT and one UNSAT. n variables
    • prevent superficial analysis differentiating the two
  • network architecture
    • SAT problems as character strings
      • SAT problems are permutation invariant
      • also negation invariant
      • don’t want model to have to learn these invariants
    • build a graph representing the problem
      • maintain vector embedding at each node
      • pass messages along the edges of the graph
    • iterative
  • 85% accuracy on SR(40)
  • can read off a satisfying assignment from the literal votes
  • two-clustering recovers satisfying assignment
  • solves more and more problems over time (almost 100% by 1000 rounds)
  • never becomes confident in unsat (when training on SR(40))
  • author thinks it’s synthesized a local search algorithm
  • “NeuroSAT (trained on SR(U(10, 40))) can find satisfying assignments but is not helpful in constructing proofs of unsatisfiability. When it runs on an unsatisfiable problem, it keeps searching for a satisfying assignment indefinitely and non-systematically”

Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention)

An Untrollable Mathematician Illustrated

  • Abram Demski, Sam Eisenstat, Scott Garrabrant
  • Naively updating a probability distribution can lead to “trollability” — the ability to push beliefs arbitrarily high or low by feeding it true / false observations.
  • The distinction is about updating on A -> B vs observe[A -> B]
  • “Logical induction (which is untrollable but not exactly a Bayesian probability distribution) is still the gold standard for logical uncertainty

Relational inductive biases, deep learning, and graph networks

  • “Just as biology uses nature and nurture cooperatively, we reject the false choice between “hand-engineering” and “end-to-end” learning, and instead advocate for an approach which benefits from their complementary strengths.”
  • combinatorial generalization: constructing new inferences, predictions, and behaviors from known building blocks
  • “important critiques have highlighted key challenges it faces in complex language and scene understanding, reasoning about structured data, transferring learning beyond the training conditions, and learning from small amounts of experience. These challenges demand combinatorial generalization, and so it is perhaps not surprising that an approach which eschews compositionality and explicit structure struggles to meet them”
  • https://github.com/deepmind/graph_nets

What does this prove? Some of the most gorgeous visual “shrink” proofs ever invented

  • (2d) square grids don’t contain any regular polygons (except for squares)
    • this is easy to show by a shrink proof
  • 3d (and higher dimensions) cubical grid has only equilateral triangles, squares, and regular hexagons

11 Notes on McKinsey

Tweet section