Summary
The impact of (large) language models (LM) has been felt across a wide range of scientific domains. It is no exaggeration to state LMs have revolutionized several applications, including both commercial and scientific discovery. Among all the factors that contributed to the success of LMs, it is empirically clear that data is one of the most important. However, this relationship remains poorly understood and motivates foundational questions, given its importance to the success of LMs. Therefore, the future progress of LMs depends critically on the data, how they are obtained, selected, and ut
