I am a linguistics PhD student at the Universitat Pompeu Fabra in Barcelona, Spain, supervised by Marco Baroni (previous: MIT ‘20, MEng ‘21 in CSAIL Infolab). I’m currently interested in how artificial and biological brains represent linguistic complexity, and what that entails for linguistic productivity.
Quick Links
News
- (October ‘24) New preprint! Principles of semantic and functional efficiency in noun morphology.
- (October ‘24) New preprint! Geometric feature complexity of LM representations encode the degree of linguistic compositionality.
New paper! 🌟How does LM representational geometry encode compositional complexity?
— Emily Cheng (@sparse_emcheng) October 6, 2024
A: it depends on how we define compositionality!
We distinguish compositionality of form vs. meaning, and show LMs encode form complexity linearly and meaning complexity nonlinearly...
1/9 pic.twitter.com/us30NCDjlH - (September ‘24) New preprint! Intrinsic dimensionality of language model representations predicts LM-brain alignment very well.
We know that middle layers from LLMs best predict brain responses to natural language. But why? In our new short paper (https://t.co/s5fBD9nl14), we show that this prediction performance closely aligns with a peak in the ✨intrinsic dimensionality✨of the LLM's layers. (1/6) pic.twitter.com/kIQarXcVaJ
— Richard Antonello (@NeuroRJ) September 13, 2024 - (August ‘24) I TA’ed this year’s Brains, Minds, and Machines summer course.
- (Summer ‘24) I’m visiting MIT CSAIL this summer to work on geometric complexity in linguistic processing in the 🧠
- (June ‘24) I am visiting Area Science Park in Trieste to present about geometry and abstraction in Transformers
- (May ‘24) New preprint about Emergence of a High-Dimensional Abstraction Phase in Language Transformers, and another about Linearly Controlled Language Generation with Guarantees, on Arxiv
- (November ‘23) I gave a talk about information-theoretic and geometric compression in LMs at the IDSIA in Lugano, Switzerland
- (November ‘23) I presented a poster about cognitive pressures shaping noun morphology in Romance languages at X-PPL in Zurich
- (October ‘23) Paper accepted! about bridging information-theoretic and geometric compression in LMs (EMNLP 2023)
- (August ‘23) Attended the Brains, Minds, and Machines summer course in Woods Hole, MA
- (May ‘23) Paper accepted! about compositionality and imitation in emergent communication (ACL Findings 2023)
- (October ‘22) Started PhD at COLT in Barcelona