Final Work Product for mlr3torchAUM on CRAN

Project Goals

Below is the project goal, referenced from the official GSoC project idea page for mlr3torchAUM on CRAN, under the R Project for Statistical Computing.

Real classification data is usually imbalanced — a small minority of positive cases against a large majority of negatives:

• 3% autism diagnoses in health-survey data
• a small minority of object pixels in image segmentation
• a small minority of rare words in language modeling

ROC curves are the standard way to evaluate such problems, and torch::nn_aum_loss() (added to torch in 2025) makes it possible to optimize the ROC curve directly with gradient descent.

mlr3torchAUM already provided that AUM loss plus some custom samplers. The GSoC goal was to:

• implement standard and state-of-the-art baselines for imbalanced classification inside the mlr3torch framework
• cover all-pairs / pairwise AUC-style losses, e.g. arXiv:2302.11062 and LibAUC
• cover over- and under-sampling, including SMOTE
• add a re-weighted logistic loss for cost-sensitive learning
• add tests, documentation, and vignettes, and submit the package to CRAN

This project will make it easier to compare different approaches for imbalanced classification in R.

Summary of Contributions

For each of the implementation PRs below, I added a test script and a man page alongside the code, to keep the package CRAN-compatible and maintainable.

Merged Pull Requests

• PR #14 — added a regression test that reproduces the ROCAUM device-mismatch bug and fixed it, unblocking CRAN checks.
• PR #56 — implemented the all-pairs squared hinge loss from arXiv:2302.11062 in R torch, using cumulative sums instead of a naive O(n²) pairwise loop (loss implementation, package registration, docs, unit tests, and a vignette).
• PR #63 — ported LibAUC's four pairwise AUC surrogate losses (squared, squared-hinge, logistic, barrier-hinge) to R torch (loss implementation, registration, docs, and unit tests).
• PR #77 — added edge-case tests for SMOTE, such as a minority class with a single sample and an already-balanced dataset.
• PR #104 — implemented random over-sampling as a training-time batch sampler (implementation, docs, tests).
• PR #105 — moved the shared quota/validation logic out of the individual sampler classes into Sampler.R, so over-sampling and under-sampling reuse the same code instead of duplicating it.
• PR #85 — ADASYN oversampling, adapted from the reference Python implementation (implementation, docs, tests).
• PR #110 — random under-sampling as a training-time batch sampler (implementation, docs, tests).
• PR #109 — fixed a couple of CI-breaking issues (a stale .Rd file and a test with a missing dependency) so the automated checks passed cleanly again.

Open or Not Yet Merged Work

• PR #8 — refactored duplicated logic across three batch-sampler classes into a shared base class.
• PR #106 — a re-weighted logistic loss that turns a user-supplied cost matrix into a pos_weight for nn_bce_with_logits_loss, following Elkan's (2001) formula (implementation, registration, docs, tests).

A couple of things I did outside of PRs: I started the coordination-roadmap discussion between the two contributors in issue #35, and shared a benchmark comparing the pairwise-AUC squared-hinge loss with log-linear version on EMNIST and FashionMNIST in issue #87 — the two matched in learning trajectory.

Challenges and What I Learned

• While implementing the log-linear squared hinge loss, I ran into precision issues from mixing single- and double-precision floats, and had to look into different sorting approaches and how ties get broken during sorting.
• Writing test scripts taught me how tests actually keep code maintainable — they are what catches an unintended regression later, not just a checkbox to tick before opening a PR.
• I picked up a much better practical understanding of Git and GitHub: branches, the different ways to merge a PR (merge, squash, rebase), and the stacked-PR workflow for splitting a feature into reviewable steps.
• Beyond the technical side, I learned about discipline, communication, and teamwork — especially while coordinating scope and reviewing PRs with another contributor working on the same project.

Current State and Remaining Work

Today, a user can load mlr3torchAUM and use new losses directly through the standard t_loss() dictionary:

• t_loss("sq_hinge_loglinear") — the all-pairs squared hinge loss.
• t_loss("pairwise_auc_surrogate") — the pairwise AUC loss, with its four surrogate variants.

RandomOverSampler, RandomUnderSampler, and ADASYN are also merged and usable directly as R6 classes for training-time over- and under-sampling. Every implementation above ships with its own test file and man page (see Summary of Contributions), and CI on main — including R CMD check, test coverage — is currently green.

mlr3torchAUM also gained multiclass losses and LibAUC optimizer support from a co-contributor working on the project at the same time — this page only describes my part of that combined state.

Remaining Work

One more piece is implemented and tested but still open as a pull request — the re-weighted logistic loss (PR #106 above) — and should get merged soon.

Beyond that, a few ideas are future scope rather than something I got to during GSoC: multiclass support for the re-weighted logistic loss, and additional SMOTE variants such as KMeansSMOTE and BorderlineSMOTE.

Thanks to My Mentor

I want to thank my mentor, Toby Hocking, for giving me the chance to work on this project and for being patient and clear with feedback all the way from my very first PR on mlr3torchAUM through to the end of GSoC. This summer taught me a lot about writing code that other people can actually rely on, and I'm grateful for the opportunity.