Comparing contextual embeddings for semantic textual similarity in Portuguese

Andrade Junior, José E., Cardoso-Silva, Jonathan and Bezerra, Leonardo C.T. (2021) Comparing contextual embeddings for semantic textual similarity in Portuguese. In: Britto, André and Valdivia Delgado, Karina, (eds.) Intelligent Systems - 10th Brazilian Conference, BRACIS 2021, Proceedings, Part 2. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Science and Business Media Deutschland GmbH, pp. 389-404. ISBN 9783030916985

Text (BRACIS_STS_2021_NLP_Portuguese) - Accepted Version Download (445kB)

• Author

Abstract

Semantic textual similarity (STS) measures how semantically similar two sentences are. In the context of the Portuguese language, STS literature is still incipient but includes important initiatives like the ASSIN and ASSIN 2 shared tasks. The state-of-the-art for those datasets is a contextual embedding produced by a Portuguese pre-trained and fine-tuned BERT model. In this work, we investigate the application of Sentence-BERT (SBERT) contextual embeddings to these datasets. Compared to BERT, SBERT is a more computationally efficient approach, enabling its application to scalable unsupervised learning problems. Given the absence of SBERT models pre-trained in Portuguese and the computational cost for such training, we adopt multilingual models and also fine-tune them for Portuguese. Results showed that SBERT embeddings were competitive especially after fine-tuning, numerically surpassing the results of BERT on ASSIN 2 and the results observed during the shared tasks for all datasets considered.

Item Type:	Book Section
Additional Information:	Publisher Copyright: © 2021, Springer Nature Switzerland AG.
Divisions:	LSE Methodology
Date Deposited:	22 Feb 2022 10:30
Last Modified:	19 Mar 2024 02:21
URI:	http://eprints.lse.ac.uk/id/eprint/113795

Actions (login required)

View Item