Connorjdavis
connorjdavis.com
Actively hiring
Follow
Overview
News
Technologies
Salaries
Products
People
Growth
Financials
Overview
To inspire others as others have inspired me. Click to read Connor's Substack, by Connor Davis, a Substack publication. Launched 8 months ago.
News
Intuitions for Transformer Circuits
In a previous post on language modeling, I implemented a GPT-style transformer. Lately I've been learning mechanistic interpretability to go deeper and understand why the transformer works on a mathematical level.This post is a brain dump of what I've lea
Read more
Report
Language Modeling, Part 7: BPE Tokenization
Welcome to Part 7 of this series on language modeling. In this post, we will implement tokenization. In the previous post, we trained a one-layer transformer for maximizing the likelihood of the next token, conditioned on the previous tokens over the Tiny
Read more
Report
Language Modeling, Part 6: Transformers
In Part 6 of this series on language modeling, we upgrade our model architecture from the LSTM to the Transformer. You can see Part 5 of this series here. Our latest LSTM gave us a sample story like this:Once upon a time there were two fearful of many toy
Read more
Report
Language Modeling, Part 5: Reverse Engineering LSTM Cells
In Part 5 of this series on language modeling, we linger a bit on the LSTM to peek under the hood in order to better understand the network's internals. If you're just joining, you can read Part 4 here. Our latest 1-layer LSTM we trained from Part 4 on th
Read more
Report
Language Modeling, Part 4: LSTMs
Welcome to Part 4 of a series on language modeling. In Part 3, we got familiar using vanilla RNNs for the next-character prediction task. We saw the main advantage of adding recurrence to the network is to keep track of long-term dependencies in the input
Read more
Report
Pro access
Upgrade to see all 14 mentions
Upgrade to a paid plan to read every media mention of this company - funding news, awards, product launches and press releases from all the outlets writing about it.
Every media mention and press release
Funding news, awards and product launches
Fresh coverage from every outlet writing about the company
Upgrade now
Cancel anytime. Secure checkout. Instant activation.