I am a linguistics PhD student at the Universitat Pompeu Fabra in Barcelona, Spain, supervised by Marco Baroni (previous: MIT ‘20, MEng ‘21 in CSAIL Infolab). I’m currently interested in how artificial and biological brains represent linguistic complexity, and what that entails for linguistic productivity. I’ve also dabbled in emergent communication in humans and machines.
Quick Links
News
- (October ‘24) New preprint! Principles of semantic and functional efficiency in noun morphology.
- (October ‘24) New preprint! Geometric feature complexity of LM representations encode the degree of linguistic compositionality.
New paper! 🌟How does LM representational geometry encode compositional complexity?
— Emily Cheng (@sparse_emcheng) October 6, 2024
A: it depends on how we define compositionality!
We distinguish compositionality of form vs. meaning, and show LMs encode form complexity linearly and meaning complexity nonlinearly...
1/9 pic.twitter.com/us30NCDjlH - (September ‘24) New preprint! Intrinsic dimensionality of language model representations predicts LM-brain alignment very well.
We know that middle layers from LLMs best predict brain responses to natural language. But why? In our new short paper (https://t.co/s5fBD9nl14), we show that this prediction performance closely aligns with a peak in the ✨intrinsic dimensionality✨of the LLM's layers. (1/6) pic.twitter.com/kIQarXcVaJ
— Richard Antonello (@NeuroRJ) September 13, 2024 - (August ‘24) I TA’ed this year’s Brains, Minds, and Machines summer course.
- (Summer ‘24) I’m visiting MIT CSAIL this summer to work on geometric complexity in linguistic processing in the 🧠
- (June ‘24) I am visiting Area Science Park in Trieste to present about geometry and abstraction in Transformers
- (May ‘24) New preprint about Emergence of a High-Dimensional Abstraction Phase in Language Transformers, and another about Linearly Controlled Language Generation with Guarantees, on Arxiv
- (November ‘23) I gave a talk about information-theoretic and geometric compression in LMs at the IDSIA in Lugano, Switzerland
- (November ‘23) I presented a poster about cognitive pressures shaping noun morphology in Romance languages at X-PPL in Zurich
- (October ‘23) Paper accepted! about bridging information-theoretic and geometric compression in LMs (EMNLP 2023)
- (August ‘23) Attended the Brains, Minds, and Machines summer course in Woods Hole, MA
- (May ‘23) Paper accepted! about compositionality and imitation in emergent communication (ACL Findings 2023)
- (October ‘22) Started PhD at COLT in Barcelona