< Back to previous page

Project

Recasting Transformer-based Language Models as a Tool for Analysis in both Linguistics and Other Fields with Text-driven Methodologies

Recent machine learning methods based on neural transformer architectures have greatly improved the state of the art for a wide range of natural language processing applications, such as machine translation and general natural language understanding. There is evidence that the self-supervised language modeling objective on which such neural methods are trained results in models that implicitly encode various kinds of linguistic knowledge, ranging from part-of-speech information to syntactic structure to co-reference information. The main research aim is to dissect, adapt, and exploit transformer architectures in order to investigate to which extent such implict linguistic representations can used as a means for exploring and analyzing linguistic phenomena. Additionally, in the context of the CLARIAH-VL work package SIC 5 (Digital Text Analysis Dashboard and Pipeline), the real-life utility of our insights will be tested through concrete implementation in a series of applications within the field of digital humanities.

Date:24 Oct 2021 →  12 Jun 2023
Keywords:NLP, Transformers, Deep Learning, Explainability, Linguistic resources
Disciplines:Computational linguistics, Machine learning and decision making, Natural language processing
Project type:PhD project