Problem formulation outweighs loss-function choice in deep-learning cervical vertebral maturation staging: A multi-seed study on an imbalanced public benchmark
| dc.authorid | 0000-0001-9840-4211 | |
| dc.contributor.author | Tokatlı, Nazlı | |
| dc.date.accessioned | 2026-10-12T12:09:52Z | |
| dc.date.available | 2026-10-12T12:09:52Z | |
| dc.date.issued | 2026 | |
| dc.department | Fakülteler, Mühendislik ve Doğa Bilimleri Fakültesi, Bilgisayar Mühendisliği Bölümü | |
| dc.description.abstract | (1) Background: Cervical vertebral maturation (CVM) staging on lateral cephalograms informs the timing of orthodontic treatment, and automated deep-learning approaches are widely studied. A recurring but under-examined obstacle is that public CVM datasets are severely imbalanced across the six stages. Using a public dataset and a rigorous eight-seed protocol, we quantify how strongly imbalance constrains performance, whether common imbalance-handling and ordinal-aware loss functions overcome it, and whether a clin ically motivated three-group reformulation is more tractable than the native six-stage task. (2) Methods: Using the public Aariz dataset (1000 lateral cephalograms with expert CVM-stage labels; 700/150/150 train/validation/test, one radiograph per patient), we trained an ImageNet-pretrained ResNet-18 under four strategies: unweighted cross-entropy (baseline), class-weighted cross-entropy with balanced sampling, focal loss, and a rank consistent ordinal (CORAL) head. Each configuration was trained with eight random seeds. Both tasks were evaluated over this multi-seed protocol, and sensitivity analyses examined the ordinal model’s sampling scheme and the focal-loss focusing parameter. Performance was assessed by accuracy, macro-F1, mean absolute error in stages (MAE), and quadratic weighted kappa (QWK), and compared across strategies with the Kruskal–Wallis test; sec ondary pairwise contrasts were Holm-corrected, and Wilson binomial confidence intervals (accuracy), seed-level confidence intervals (macro-F1), and analytically derived major ity and frequency-weighted random classifiers were added as trivial reference baselines. Both the native six-stage task and a three-group scheme (pre-peak = CS1-3, peak = CS4, post-peak = CS5-6) were evaluated. (3) Results: On the six-stage task the extreme imbal ance (CS1 n = 18 vs. CS5 n = 311 in training) produced poor minority-stage recognition (macro-F1 ≈ 0.22–0.25), with the rarest stages essentially unclassified even after imbalance handling. Reframing into three clinically meaningful groups roughly doubled macro-F1 (to ≈0.47–0.51) across all strategies. Critically, in the three-group setting no loss function significantly outperformed the others: across eight seeds the Kruskal–Wallis test was non significant for every metric (macro-F1 p = 0.09, QWK p = 0.05, MAE p = 0.07, accuracy p = 0.41), and the unweighted baseline matched the best specialized loss on macro-F1 (0.507 ± 0.045 vs. ordinal 0.505 ± 0.044). In secondary pairwise comparisons the ordinal loss nominally exceeded the weighted and focal losses (uncorrected p = 0.04 and p = 0.01), but after Holm correction only the comparison with focal loss remained significant(adjusted p =0.03), and no pairwise difference survived a conservative six-comparison Bonferroni bound; focal loss was the least stable. All strategies clearly exceeded trivial baselines on macro-F1 (majority classifier 0.259; frequency-weighted random ≈0.333), although the majority classifier’s accuracy (0.633) exceeded that of every trained model. (4) Conclusions: On this imbalanced public dataset, the dominant factor associated with CVM classification performance was the problem formulation rather than the loss function: a clinically grounded three-group scheme was far more tractable than six-stage classification, whereas specialized imbalance and ordinal losses did not reliably outperform a simple baseline. These findings, established under a fixed backbone, training budget, and a single public dataset, caution against assuming that loss-level fixes resolve CVM imbalance, and high light data scale and problem formulation as the primary levers. Multi-dataset external validation is a necessary next step. | |
| dc.identifier.citation | Tokatlı, N. (2026). Problem formulation outweighs loss-function choice in deep-learning cervical vertebral maturation staging: A multi-seed study on an imbalanced public benchmark. Diagnostics, 16(19), pp. 1-13. https://doi.org/10.3390/diagnostics16193183 | |
| dc.identifier.doi | 10.3390/diagnostics16193183 | |
| dc.identifier.endpage | 13 | |
| dc.identifier.issn | 2075-4418 | |
| dc.identifier.issue | 19 | |
| dc.identifier.scopusquality | Q1 | |
| dc.identifier.startpage | 1 | |
| dc.identifier.uri | https://doi.org/10.3390/diagnostics16193183 | |
| dc.identifier.uri | https://hdl.handle.net/20.500.13055/1665 | |
| dc.identifier.volume | 16 | |
| dc.identifier.wosquality | Q1 | |
| dc.indekslendigikaynak | Web of Science | |
| dc.indekslendigikaynak | Scopus | |
| dc.indekslendigikaynak | PubMed | |
| dc.indekslendigikaynak.other | SCI-E - Science Citation Index Expanded | |
| dc.institutionauthor | Tokatlı, Nazlı | |
| dc.institutionauthorid | 0000-0001-9840-4211 | |
| dc.language.iso | en | |
| dc.publisher | MDPI Publishing | |
| dc.relation.ispartof | Diagnostics | |
| dc.relation.publicationcategory | Makale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanı | |
| dc.rights | info:eu-repo/semantics/openAccess | |
| dc.subject | Cervicalvertebral Maturation | |
| dc.subject | Deeplearning | |
| dc.subject | Class Imbalance | |
| dc.subject | Ordinal Classification | |
| dc.subject | Orthodontics | |
| dc.subject | Cephalometry | |
| dc.title | Problem formulation outweighs loss-function choice in deep-learning cervical vertebral maturation staging: A multi-seed study on an imbalanced public benchmark | |
| dc.type | Article | |
| dspace.entity.type | Publication |
