Ph.D. Student: Jose María Moyano
Advisors: Eva Lucrecia Gibaja, Krzysztof J. Cios, Sebastián Ventura
Defended on: September 2020
Keywords: multi-label learning, classification, ensemble learning
Digital version: PDF


In recent years, the multi-label classification gained attention of the scientific community given its ability to solve real-world problems where each instance of the dataset may be associated with several class labels simultaneously. For example, in medical problems each patient may have several diseases at the same time, and in multimedia categorization, each item might be related with different tags or topics.Thus, given the nature of these problems, dealing with them as traditional classification where just one class label is assigned to each instance, would lead to a loss of information. The fact of having more than one label associated with each instance leads to new classification challenges that should be addressed, such as modeling compound dependencies among the labels, imbalance of the label space, and high dimensionality of the output space.

A large number of methods for multi-label classification was proposed in the literature, including several ensemble-based methods. Ensemble learning is a technique which is based on combining the outputs of many diverse base models, in order to outperform each separate model. In multi-label classification, ensemble methods are those that combine the predictions of several multi-label classifiers, and they were shown to outperform simpler multi-label classifiers. Therefore, given its performance, we focus our research on the study of ensemble-based methods for multi-label classification.

Since most of the ensemble methods for multi-label classification are based on creating diverse members by randomly selecting instances, input features, or labels, our main objective is to propose novel ensemble methods for multi-label classification where the characteristics of the data are taken into account.

The main contributions of this Ph.D. thesis are summarized in the following points:

  • A thorough review of the state-of-the-art ensembles of multi-label classifiers (EMLCs) has been performed. This study analyzes the EMLCs in the literature, proposing a novel taxonomy to categorize them, and also an experimental study is carried out, in order to find the method or family of methods that perform better depending on the characteristics of the data, as well as provide some guidelines for selecting the best method according to the characteristics of a given problem.
  • Two evolutionary approaches to build EMLCs while considering the characteristics of the data are proposed. The first one (EME) evolves the entire ensemble as an individual, while the second (EAGLET) evolves separate members of the ensemble as individuals, then creating the ensemble by combining accurate and diverse individuals from the population. Both methods have been applied to real-world benchmarks and they have been demonstrated to perform significantly better than state-of-the-art EMLCs, also emphasizing the lower computational complexity of EAGLET.
  • A tool for analyzing and preprocessing multi-label datasets have been developed, including both a graphic interface and a Java API.
  • Two preliminary approaches to build EMLCs have been also proposed, one of them based on cooperative coevolutionary algorithms, and other based on grammar-guided genetic programming. These approaches serve as basis for establishing some lines of future work.
  • The code of all the developed tools and algorithms are publicly available in GitHub repositories under the GNU Public License (GLPv3).


The development of this thesis has been supported by:

  • Spanish Ministry of Science and Competitiveness, project TIN-2014-55252-P.
  • Spanish Ministry of Science and Competitiveness, project TIN-2017-83445-P.
  • Spanish Ministry of Education, Culture and Sports under the FPU program (FPU15/02948).


  1. J.M. Moyano, E. Gibaja and S. Ventura. MLDA: A tool for analyzing multi-label datasets. Knowledge-Based Systems, vol. 121, pp. 1-3. 2017. DOI: 10.1016/j.knosys.2017.01.018.
  2. J.M. Moyano, E. Gibaja, K.J. Cios and S. Ventura. Review of ensembles of multi-label classifiers: Models, experimental study and prospects. Information Fusion, vol. 44, pp. 33-45. 2018. DOI: 10.1016/j.inffus.2017.12.001.
  3. J.M. Moyano, E. Gibaja, K.J. Cios and S. Ventura. An evolutionary approach to build ensembles of multi-label classifiers. Information Fusion, vol. 50, pp. 168-180. 2019. DOI: 10.1016/j.inffus.2018.11.013.
  4. J.M. Moyano, E. Gibaja, K.J. Cios and S. Ventura. Combining multi-label classifiers based on projections of the output space using Evolutionary algorithms. Knowledge-Based Systems, vol. 196, pp. 105770. 2020. DOI: 10.1016/j.knosys.2020.105770.
  1. J.M. Moyano, E. Gibaja, K.J. Cios and S. Ventura. Generating ensembles of multi-label classifiers using cooperative coevolutionary algorithms. In European Conference on Artificial Intelligence (ECAI 2020), 1379-1386. 2020. 2020. DOI: 10.3233/FAIA200242.
  2. J.M. Moyano, E. Gibaja, K.J. Cios and S. Ventura. Tree-shaped ensemble of multi-label classifiers using grammar-guided genetic programming. In IEEE Congress on Evolutionary Computation (IEEE-CEC 2020), 1-8. 2020. 2020. DOI: 10.1109/CEC48606.2020.9185661.
  1. J.M. Moyano, E. Gibaja, A. Cano, J.M. Luna and S. Ventura. Algoritmo evolutivo para optimizar ensembles de clasificadores multi-etiqueta. In X Congreso Español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados (MAEB 2015), pp. 219-225. 2015.
  2. J.M. Moyano, E. Gibaja, A. Cano, J.M. Luna and S. Ventura. Diseño automático de multi-clasificadores basados en proyecciones de etiquetas. In XVI Conferencia de la Asociación Española para la Inteligencia Artificial (CAEPIA 2015), pp. 355–365. 2015.
  3. J.M. Moyano, E. Gibaja and S. Ventura. Una herramienta para analizar conjuntos de datos multi-etiqueta . In XVII Conferencia de la Asociación Española para la Inteligencia Artificial (CAEPIA 2016), pp. 857–866. 2016.


At the ages of fifty four fifty can advise you that I'onal ended up lucky not to have wanted the product sooner, nevertheless loosing your partner of 25yrs 2010 became a curve which modified me personally for a long time. Now there came out a place exactly where click to read fifty had visit to have my tastes fulfilled only to find out this plumbing related desired just a little poke. So I questioned my own Computer system doc intended for a little something with tiny facet is affecting. He or she provided the particular recommended you read Cialis regular 5mg. 1st working day fifty had 5mg without any help to discover more help in the event that t discovered virtually any difference considering the woman never was planning determine, which'azines our system and also l'meters being dedicated to the idea. Regardless, these materials Operates, along with is useful. And click here then up coming night time with your ex m took 10mg at 8pm, and the rest is heritage. Through 13:double zero fifty manufactured my own move, but it appeared to be the most element to live in place until finally the sunlight came up upward , 100%Pleased :)When i't thirty-two as well as gone pretty much 1,5 years without intercourse. I had created pop over to this website plenty of anxiousness related penile erection challenges. And hop over to here then We found this specific great which woman My spouse and i started courting, along with first 2 times us all sex didn'to determine which properly, and i also appeared to be worried about just what exactly your woman considered this matter. We obtained braveness to visit and request cialis approved from the health practitioner. When i had taken 10mg product and it labored perfectly. I could truthfully continue on having sex many times a day without the problems. Merely bad thing is a smallish frustration. For me this can be truly a wonder drug.