Clustering social sciences and humanities publications

Subtitle:can word and document embeddings improve cluster quality?

We study how different document representation techniques affect the outcomes of clustering based on textual information. Our dataset consists of titles and abstracts for 15,907 publications from the social sciences and humanities. We compare established document representation techniques such as TF-IDF and Latent Semantic Indexing with word and document embedding techniques (Word2Vec and Doc2Vec) in terms of the quality of the clustering outcomes. Quality is assessed by calculating silhouette scores and the average textual coherence of the clusters. The results show that word and document embeddings are promising feature engineering techniques in the context of clustering social sciences and humanities publications. The average of Word2Vec embeddings works best for identifying textually coherent clusters.

Publication year:2021

Keywords:P1 Proceeding

Accessibility:Closed