I am a final year linguistics PhD student at the Universitat Pompeu Fabra in Barcelona, Spain, supervised by Marco Baroni (previous: MIT ‘20, MEng ‘21 in CSAIL Infolab). I am primarily interested in how brains and machines process language, and have spent most of my PhD understanding the relationship between feature abstraction and the dimensionality of representations during language comprehension. I have also spent some time developing algorithms to understand how and whether LLMs can be controlled.
Quick Links
News
- (January ‘26) Paper accepted at ICLR with Apple! We propose a PAC bound on reachable sets of any black-box language model.
- (December ‘25) Visiting Rycolab at ETHz til March!
- (May ‘25) Started an internship at Apple MLR in Cambridge, UK
- (May ‘25) Presented a poster about Emergence of a High-Dimensional Abstraction Phase in Language Transformers at ICLR 2025
- (May ‘25) Paper accepted at ACL (main) as an oral presentation! Geometric feature complexity of LM representations encode the degree of linguistic compositionality.
New paper! 🌟How does LM representational geometry encode compositional complexity?
— Emily Cheng (@sparse_emcheng) October 6, 2024
A: it depends on how we define compositionality!
We distinguish compositionality of form vs. meaning, and show LMs encode form complexity linearly and meaning complexity nonlinearly...
1/9 pic.twitter.com/us30NCDjlH - (October ‘24) New preprint! Principles of semantic and functional efficiency in noun morphology.
- (September ‘24) New preprint! Intrinsic dimensionality of language model representations predicts LM-brain alignment very well.
We know that middle layers from LLMs best predict brain responses to natural language. But why? In our new short paper (https://t.co/s5fBD9nl14), we show that this prediction performance closely aligns with a peak in the ✨intrinsic dimensionality✨of the LLM's layers. (1/6) pic.twitter.com/kIQarXcVaJ
— Richard Antonello (@NeuroRJ) September 13, 2024 - (August ‘24) I TA’ed this year’s Brains, Minds, and Machines summer course.
- (Summer ‘24) I’m visiting MIT CSAIL this summer to work on geometric complexity in linguistic processing in the 🧠
- (June ‘24) I am visiting Area Science Park in Trieste to present about geometry and abstraction in Transformers
- (May ‘24) New preprint about Linearly Controlled Language Generation with Guarantees, on Arxiv
- (November ‘23) I gave a talk about information-theoretic and geometric compression in LMs at the IDSIA in Lugano, Switzerland
- (November ‘23) I presented a poster about cognitive pressures shaping noun morphology in Romance languages at X-PPL in Zurich
- (October ‘23) Paper accepted! about bridging information-theoretic and geometric compression in LMs (EMNLP 2023)
- (August ‘23) Attended the Brains, Minds, and Machines summer course in Woods Hole, MA
- (May ‘23) Paper accepted! about compositionality and imitation in emergent communication (ACL Findings 2023)
- (October ‘22) Started PhD at COLT in Barcelona