Project webpage for the project entitled “Addressing Class Imbalance in Visual Recognition Problems by Measuring Class Imbalance and Using Epistemic Uncertainty” with acronym BALANCE (DENGE), supported by TUBITAK 1001 (no 120E494).

Figure 1: Many problems in practice exhibit strong imbalance towards certain classes or categories, which can severely impact learning performance. (a) Foreground-background imbalance problem in object detection. (b) Imbalance problem among object categories. (Figure source: [Oksuz et al., 2021](https://arxiv.org/abs/1909.00169)).

What is DENGE?

DENGE is a project supported by TUBITAK as a two-year project (2021-2023) for developing better methods for addressing imbalance problems. The project is being conducted at METU Image Lab, Dept. of Computer Eng., METU. It builds on the team’s experience on and extends the results of the team’s previous project entitled “Object Detection in Videos with Deep Neural Networks” (funded by TUBITAK with no 117E054).
Project No 120E494
Project Type ARDEB-1001
Budget ~ TRY 340K (~ $ 45K)
Dates 1 March 2021 - 1 March 2023
Coordinators Emre Akbas [METU Image Lab, Dept. of Computer Eng., METU] and Sinan Kalkan [METU Image Lab, Dept. of Computer Eng., METU]
Researchers Kemal Oksuz [METU Image Lab, Dept. of Computer Eng., METU]
Students Zeynep Sonat Baltaci, Baris Can Cam, Gunes Cepic, Sinem Donmez, Cagri Eser, Ece Gokcay, Baran Gulmez, Ertugrul Gungor, Fehmi Kahraman, Berkin Kerim Konar, Selim Kuzucu, Alpay Ozkan, Kivanc Tezoren, Feyza Yavuz

Summary

Class imbalance (CI) is an important problem affecting the generalization performance of machine learning methods. CI can basically be defined as the problem that some classes contain fewer samples than others in the dataset. According to this definition and the general opinion in the literature, this unbalanced distribution between classes causes machine learning methods to perform better on classes containing a large number of samples, while being insufficient for classes containing a small number of samples.

Many methods have been developed against CI problems in recent years. These methods generally suggest (i) correcting bias by over-sampling or sub-sampling, or (ii) attaching more importance to “hard” samples. These methods are known to alleviate the CI problem and improve performance.

The proposed project will focus on improving existing methods in the following ways:

  • Observation 1: Having relatively fewer samples for one class does not mean that the classification performance of that class will be poor.

  • Observation 2: Sample hardness may not be an indicator of imbalance, although it has shown promising results.

Publications

The list of publications (from DENGE and its precursor project):

  • F. Yavuz, B. C. Cam, A. H. Dogan, K. Oksuz, E. Akbas, S. Kalkan, “Bucketed Ranking-based Losses for Efficient Training of Object Detectors”, 18th European Conference on Computer Vision (ECCV), 2024.
  • B. Cetinkaya, S. Kalkan, E. Akbas, “RankED: Addressing Imbalance and Uncertainty in Edge Detection Using Ranking-based Losses”, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. Arxiv, Code
  • Z. S. Baltaci, K. Oksuz, S. Kuzucu, K. Tezoren, B. K. Konar, A. Ozkan, E. Akbas, S. Kalkan, “Class Uncertainty: A Measure to Mitigate Class Imbalance”, under review, 2023. Arxiv
  • F. Kahraman, K. Oksuz, S. Kalkan, E. Akbas, “Correlation Loss: Enforcing Correlation between Classification and Localization”, Thirty-Seventh AAAI Conference on Artificial Intelligence, 2023. Arxiv, Code
  • K. Oksuz, B. C. Cam, S. Kalkan, E. Akbas, “One Metric to Measure them All: Localisation Recall Precision (LRP) for Evaluating Visual Detection Tasks”, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 44(12):9446-9463, 2022. Arxiv, Code.
  • K. Oksuz, B. C. Cam, E. Akbas, S. Kalkan, “Rank & Sort Loss for Object Detection and Instance Segmentation”, International Conference on Computer Vision (ICCV), oral presentation, 2021. Arxiv, Code.
  • K. Oksuz, B. C. Cam, S. Kalkan, E. Akbas, “Imbalance Problems in Object Detection: A Review”, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 43(10):3388-3415, 2021. Arxiv, Publisher, Repo.
  • K. Oksuz, B. C. Cam, E. Akbas, S. Kalkan, “A Ranking-based, Balanced Loss Function Unifying Classification and Localisation in Object Detection”, Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS), spotlight paper, 2020. Arxiv, NeurIPS, Code.
  • K. Oksuz, B. C. Cam, E. Akbas, S. Kalkan, “Generating Positive Bounding Boxes for Balanced Training of Object Detectors”, IEEE Winter Conference on Applications of Computer Vision (WACV), 2020. Arxiv, Publisher, Code.

Contact

Please use the following email address for course related issues: metu-denge [@] googlegroups [dot] com