Identity-Preserving Sketch-to-Photo Synthesis for Forensic Face Retrieval Using Discriminator Regularization

Abstract

Background and Objectives: Forensic investigations often begin with a hand-drawn sketch, and the practical question is not whether it can be turned into a plausible face but whether that face retrieves the right person from a gallery of photographs. CycleGAN has become the standard route because it trains without paired supervision. Evaluation of such models, however, leans on reconstruction measures such as SSIM and PSNR, which reward smooth, photograph-like output; whether a face scoring well on them also carries the identity information a recognizer depends on is rarely tested. A second gap concerns the discriminator, where regularization is treated as a matter of optimization stability and its effect on identity-discriminative structure is left unexamined. Our objective was to determine whether regularizing the CycleGAN discriminator with all-block dropout, on top of spectral normalization, improves identity-preserving retrieval, whether generator depth or dropout rate contributes independently, and whether pixel-level fidelity can serve as a proxy for retrieval when a model is selected for forensic use.

Methodology: Experiments used the CUHK student subset of CUFS, 188 sketch–photo identity pairs at 128 × 128 resolution, partitioned under fixed seeds into five identity-disjoint splits of 131 training, 20 validation, and 37 test identities. Six configurations were trained on every split, giving thirty runs: two spectral-normalization-only baselines with ResNet-6 and ResNet-9 generators, and four configurations adding all-block discriminator dropout at rates of 0.3 and 0.5 for each depth. Both baselines are previously published spectral-normalized architectures, retrained here so that all configurations share identical partitions, checkpoints, and criteria. Training ran 200 epochs with the standard CycleGAN objective, with cycle and identity loss weights held fixed so that discriminator regularization was the only variable. Evaluation ran in two stages that never share data: checkpoints saved every twenty epochs from epoch 100 were scored on the validation gallery, the best validation Top-1 was selected with ties resolved by a rule fixed in advance, and that checkpoint was evaluated once on the held-out test identities. Synthesized photographs were embedded with a fixed InsightFace ArcFace recognizer, ranked by cosine similarity, and scored as Top-1, Top-5, and Top-10 accuracy alongside SSIM and PSNR. Retrieval is reported under two denominators: a conditional figure over identities whose query and gallery counterpart were both embedded, and an end-to-end figure over all 37 test identities counting a query as incorrect whenever any pipeline stage fails. Paired Wilcoxon signed-rank tests across the five splits are descriptive, since five paired observations floor the attainable p-value at 0.0625 and the test sets overlap.

Main Results: Every configuration carrying all-block dropout outperformed both baselines on conditional Top-1, on all five splits without exception; end-to-end, the same holds in thirty-nine of the forty configuration–split comparisons, with one tie and no reversal. Holding depth fixed, dropout raised mean conditional Top-1 by 14.6 percentage points at ResNet-9, from 80.14% to 94.74%, and by 8.0 points at ResNet-6, from 83.17% to 91.16%, both at the minimum attainable p-value of 0.0625; end-to-end the gaps are 13.0 and 6.5 points. Neither depth nor rate produced a comparable effect: the baselines differed by roughly three points, and no depth or rate comparison held a consistent direction across splits, pointing to the presence of regularization rather than its strength or generator size as the operative factor. The synthesis measures inverted this ordering. The ResNet-9 baseline achieved the highest SSIM, 0.6930, and the highest PSNR, 18.23 dB, while returning the lowest Top-1 of the six. Across all thirty method–split combinations, mean SSIM and Top-1 were negatively correlated (Pearson r = −0.55, p = 0.002), and at the level of individual images the two were effectively unrelated. Training curves are consistent with discriminator overfitting in the unregularized baselines, which the dropout runs avoid.

Conclusions: Crossing generator depth with discriminator dropout, all-block dropout in the discriminator consistently improves identity-preserving retrieval in sketch-to-photo synthesis, and the benefit comes from the presence of the regularization rather than a particular rate or depth. The more consequential finding is that pixel fidelity did not predict retrieval and can point the opposite way: the configuration reconstructing most faithfully retrieves worst, so a model chosen on SSIM or PSNR would be among the weakest search tools here. For forensic systems, where the deliverable is a correct ranked identity rather than a faithful portrait, synthesis models should be selected and stopped on retrieval measured against a held-out gallery. Results come from a single small, clean dataset with five splits, and confirmation on larger, noisier forensic-like data remains necessary.