Beren

Epistemic status: Obviously speculative but mechanism design is fun. Last year I wrote a post on reward hacking as we were then beginning to see concerning signs of scaling RLVR causing models to exhibit substantial reward hacking behaviours. Unfortunatel

Epistemic status: A lot of this is vibes and generally trying to express tacit knowledge from ML experience and training models. However there are a huge number of details here and the field of ML changes incredibly rapidly. This is an oversimplified, but

Author's note: This is quite a long post. It is highly speculative, obviously, but I think it offers an interesting perspective on, and proposal for, outer alignment. The specific idea here is to move away from demanding a final lightcone-scale conception

so much depends upon a von-neumann probe somewhere in the intergalactic void aimed at eternity

Epistemic Status: Obviously speculative and maybe obvious. The success of RL in LLMs has been puzzling me for a while. People have developed various information-theoretic style arguments by which they argue that RL is extremely informationally inefficient

Epistemic Status: A short note which posits a question without necessarily arriving at a definitive answer. When thinking about whether we are likely to end up in a monotheistic or polytheistic AI future, we often end up trying to identify and categorize

Epistemic status: Interesting as a mechanism and trend, but obviously highly variable over short timescales and ultimately speculative. On a plane recently I ended up noodling back around to my original concepts of amortised vs direct optimization and als

Epistemic Status: I really only did a fairly surface level skim of Baudrillard's actual work mostly cribbed from secondary sources, so I could very well be misrepresenting core things about what he actually said. On the other hand, Baudrillard really just

Author's note: More mystical than usual. A vibe rather than a point. The earliest prophets, Good and Vinge, predicted the existence of such a singularity. They sketched the first field equations and were the first to see the truth that time must end, not

Epistemic note: Probably already known or obvious, but this treatment seems clearer to me than the standard treatment in e.g. the FDT paper. The other day I went down a deep lesswrong rabbit-hole and ended up going through all the decision theory literatu