Library Header Image
LSE Research Online LSE Library Services

Comparing contextual embeddings for semantic textual similarity in Portuguese

Andrade Junior, José E., Cardoso-Silva, Jonathan and Bezerra, Leonardo C.T. (2021) Comparing contextual embeddings for semantic textual similarity in Portuguese. In: Britto, André and Valdivia Delgado, Karina, (eds.) Intelligent Systems - 10th Brazilian Conference, BRACIS 2021, Proceedings, Part 2. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Science and Business Media Deutschland GmbH, pp. 389-404. ISBN 9783030916985

[img] Text (BRACIS_STS_2021_NLP_Portuguese) - Accepted Version
Repository staff only until 28 November 2023.

Download (445kB) | Request a copy

Identification Number: 10.1007/978-3-030-91699-2_27


Semantic textual similarity (STS) measures how semantically similar two sentences are. In the context of the Portuguese language, STS literature is still incipient but includes important initiatives like the ASSIN and ASSIN 2 shared tasks. The state-of-the-art for those datasets is a contextual embedding produced by a Portuguese pre-trained and fine-tuned BERT model. In this work, we investigate the application of Sentence-BERT (SBERT) contextual embeddings to these datasets. Compared to BERT, SBERT is a more computationally efficient approach, enabling its application to scalable unsupervised learning problems. Given the absence of SBERT models pre-trained in Portuguese and the computational cost for such training, we adopt multilingual models and also fine-tune them for Portuguese. Results showed that SBERT embeddings were competitive especially after fine-tuning, numerically surpassing the results of BERT on ASSIN 2 and the results observed during the shared tasks for all datasets considered.

Item Type: Book Section
Additional Information: Publisher Copyright: © 2021, Springer Nature Switzerland AG.
Divisions: LSE
Date Deposited: 22 Feb 2022 10:30
Last Modified: 22 Feb 2022 10:33

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics