graduate In-Context Learning in LLMs for Simple Functions Conducted a deeper examination of in-context learning for simple functions using small LLMs. Builds off of experiments by Garg et al. Learning Mixtures of Linear Dynamical Systems Designed, implemented and reproduced experiments for unsupervised clustering of trajectories drawn from linear dynamical systems. Comparing Mamba, Attention and their Hybrids Studying length generalization in Mamba, attention-based transformers, and hybrids of both for associative recall and related tasks. Updating Autobidding Models in Near-Real Time Designed and implemented a new algorithm for making hourly updates to models used for autobidding in ad auctions. Worked with multitask Gaussian processes. RLHF for Very Small LLMs Compared various RLHF methods for summarization using GPT-2 small. Implemented trainers from scratch in JAX. undergraduate py_knots Knot algorithms in Python, 2022. topological data analysis Analysing UNGA votes, 1945-2015. superficial Algorithms for surfaces in Scala, 2020. idris Interactive proofs in Idris, 2019.
