A Use Case of Patent Classification Using Deep Learning with Transfer Learning

Roberto Henriques · Adria Ferreira · Mauro Castelli

9 citations0/0 files storedJournal of Data and Information Science

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Intellectual Property and Patents Topic Modeling Text and Document Classification Technologies

Abstract

Abstract Purpose Patent classification is one of the areas in Intellectual Property Analytics (IPA), and a growing use case since the number of patent applications has been increasing worldwide. We propose using machine learning algorithms to classify Portuguese patents and evaluate the performance of transfer learning methodologies to solve this task. Design/methodology/approach We applied three different approaches in this paper. First, we used a dataset available by INPI to explore traditional machine learning algorithms and ensemble methods. After preprocessing data by applying TF-IDF, FastText and Doc2Vec, the models were evaluated by cross-validation in 5 folds. In a second approach, we used two different Neural Networks architectures, a Convolutional Neural Network (CNN) and a bi-directional Long Short-Term Memory (BiLSTM). Finally, we used pre-trained BERT, DistilBERT, and ULMFiT models in the third approach. Findings BERTTimbau, a BERT architecture model pre-trained on a large Portuguese corpus, presented the best results for the task, even though with a performance of only 4% superior to a LinearSVC model using TF-IDF feature engineering. Research limitations The dataset was highly imbalanced, as usual in patent applications, so the classes with the lowest samples were expected to present the worst performance. That result happened in some cases, especially in classes with less than 60 training samples. Practical implications Patent classification is challenging because of the hierarchical classification system, the context overlap, and the underrepresentation of the classes. However, the final model presented an acceptable performance given the size of the dataset and the task complexity. This model can support the decision and improve the time by proposing a category in the second level of ICP, which is one of the critical phases of the grant patent process. Originality/value To our knowledge, the proposed models were never implemented for Portuguese patent classification.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 9 citations exist upstream.

References

2018The state-of-the-art on Intellectual Property Analytics (IPA): A literature review on artificial intelligence, machine learning and deep learning methods for analysing intellectual property (IP) data 2011A patent quality analysis for innovative technology and product development 2003Using the patent co-citation approach to establish a new patent classification system 2018DeepPatent: patent classification with convolutional neural networks and word embedding 2021A survey on deep learning for patent analysis 2016A patent quality analysis and classification system using self-organizing maps with support vector machine 2012Ontology-based neural network for patent knowledge management in design collaboration

Cited by

2024Machine learning-based method to cluster a converging technology system: The case of printed electronics

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.