Evaluating ESM3 Representations and Whole-Sequence Generative Models for HIV Broadly Neutralizing Antibody Modeling

HIV-1 broadly neutralizing antibodies (bnAbs) neutralize diverse viral variants by binding conserved envelope glycoprotein regions [1][2]. Unlike ordinary antibodies, bnAbs mature over years through somatic hypermutation, producing clonal lineages that share an unmutated common ancestor (UCA) but diverge in sequence and function (Figure 1).

Predicting neutralization breadth from variable heavy-chain ($V_H$) sequence alone is difficult because functional readouts depend on heterogeneous virus panels and inconsistent laboratory assays.

Protein language models learn contextual sequence representations from large unlabeled corpora. ESM3 reasons jointly over sequence, structure, and function [3][4]. We ask whether general-protein ESM3 embeddings support out-of-lineage bnAb modeling from heavy-chain sequence alone, and whether unsupervised adaptation on the Observed Antibody Space (OAS) [5] improves transfer.

Because antibody datasets contain somatically related variants, naive random splits can place members of the same clonal lineage in both training and validation folds. We implement metadata-derived lineage grouping with group-resampled uncertainty analysis on CATNAP [6]. Because the study compares multiple heads and pooling schemes on the same folds, cross-validation scores represent model-selection estimates rather than fully locked confirmatory performance.

We organize the study around three analyses: (1) a metadata-derived grouping protocol with group-resampled uncertainty quantifying sensitivity to cohort composition for CATNAP classification and regression; (2) matched evaluation of sequence features, frozen ESM3, and OAS-adapted ESM3 (LoRA) under identical downstream folds and heads; and (3) evaluation of a whole-sequence categorical VAE on out-of-fold reconstruction and a known-intermediate lineage interpolation test.

2.1. HIV bnAb measurement and development

CATNAP standardizes antibody-by-virus neutralization panels and metadata [6], curating IC50 measurements across antibody-virus pairs. Longitudinal lineage studies show breadth can change substantially as somatic variants accumulate, including within the CAP256/VRC26 lineage [7][8]. These observations motivate lineage-held-out evaluation: high sequence similarity does not imply identical breadth, and lineage membership is a strong shortcut under naive random splits.

2.2. Machine learning for HIV neutralization

Most computational work predicts virus resistance to a fixed antibody panel rather than scoring antibody breadth from sequence [9]. Multi-task learning across antibodies mitigates subtype bias [22], and BRAVE couples ESM2 embeddings of antibody and virus sequences to predict resistance across 33 bnAbs [10]. We evaluate the converse: whether ESM3 embeddings of heavy-chain sequence support antibody-centric breadth estimation.

2.3. Protein and antibody language models

ESM2 demonstrated evolutionary-scale representation learning [11]; ESM3 extends multimodal reasoning over sequence, structure, and function [3]. Antibody-specific models including AntiBERTy, AbLang, and IgBERT train on immunoglobulin repertoires [12][13][14][23]. Surveys find general protein language models can match domain-specific models when size and layer selection are controlled [15].

2.4. Adaptation and latent interpolation

We adapt ESM3 with LoRA [16] on OAS via masked language modeling [19], without supervised bnAb labels during adaptation. We also compare VAE latent interpolation [17] with straight-line interpolation in raw ESM3 embedding space along maturation trajectories.

3.1. CATNAP cohort and labels

We compile a dataset from CATNAP, which aggregates antibody sequences and quantitative viral neutralizations. The half-maximal inhibitory concentration ($\text{IC}_{50}$) measures potency against a specific viral isolate: lower values indicate higher antibody potency. We evaluate each antibody–virus pair as neutralized when $\text{IC}_{50} \leq 50\,\mu\mathrm{g/mL}$. Neutralization breadth $b_i$ for antibody $i$ is the fraction of tested measurements below that threshold:

We restrict analysis to antibodies tested against at least 20 virus strains. For binary classification, we assign broad ($1$) when $b_i \geq 0.50$ and narrow/weak ($0$) when $b_i \leq 0.20$, excluding the intermediate band to establish a high-contrast classification task rather than an arbitrary one-dimensional boundary. This is an analysis choice, not a biological or clinical cutoff: among 450 sequence-complete antibodies tested against at least 20 viruses, it excludes 150 middle-band antibodies and retains 300 labeled examples. We require complete complementarity-determining regions (CDRs), which form much of the antigen-binding surface (Figure 2).

The final labeled set contains 300 unique $V_H$ sequences (202 broad, 98 narrow), with lengths 116–146 amino acids. For continuous-breadth regression, we predict $b_i$ directly for these 300 antibodies.

3.2. Lineage grouping and split

To prevent leakage of closely related sequences, we partition antibodies using metadata-derived groups assigned sequentially: CATNAP clonal-lineage metadata (216 antibodies), shared-donor patient metadata (70), single-linkage clustering at $\geq 80\%$ Levenshtein identity (2), and singletons (12). This hierarchical assignment first produces 113 conservative group IDs; we then merge connected components linked by residual pairs at $\geq 80\%$ identity, producing 109 final groups, after which no cross-group near-duplicate pairs remain at this threshold. Across 862 in-group pairs, median sequence identity is $0.771$ (IQR $0.683$–$0.870$); across 43,988 out-group pairs, median identity is $0.424$ (IQR $0.377$–$0.504$) (Figure 3).

Both tasks use a single 5-fold StratifiedGroupKFold split so all antibodies in a metadata-derived group remain in train or validation while stratifying the broad-vs-narrow ratio per fold.

3.3. Metrics, selection, and uncertainty

Classification metrics are AUROC and AUPRC (average precision). AUROC measures the probability that a random positive ranks above a random negative; AUPRC evaluates precision–recall trade-offs under class imbalance. Regression metrics are Pearson $r$, Spearman $\rho$, and $R^2$.

Configuration selection uses mean validation-fold score (mean AUROC or mean Spearman $\rho$). Primary reporting uses pooled out-of-fold (OOF) predictions from all folds. We estimate 95% intervals with 10,000 bootstrap draws resampling complete metadata-derived groups. Paired deltas resample groups and compute metric differences on aligned OOF predictions. These intervals quantify sensitivity to the observed group composition but do not correct for configuration-selection bias.

3.4. OAS adaptation corpus

Domain adaptation uses 219,837 globally deduplicated human heavy-chain sequences from 11 OAS data units (108,675 clonotypes); the largest unit contributes 20,000 sequences (9.1%). Systematically 5$'$-truncated data units are excluded. A clonotype-aware split yields 198,130 training and 21,707 validation sequences. OAS supplies adaptation data only—we neither label OAS sequences as HIV bnAbs nor include them in downstream test folds [5]. For whole-sequence VAE pretraining we additionally protect every sequence that could later enter an evaluation: exact-match and $\geq 80\%$ normalized Levenshtein-identity protection retains 169,876 of the 219,837 OAS candidates, and a protected CATNAP adaptation set retains 143 of 568 candidate rows. These are training sources only; neither contributes evaluation rows.

3.5. Evaluation cohorts

Classification, regression, and whole-sequence VAE OOF reconstruction use a common evaluation cohort of the same 300 antibody IDs, heavy-chain sequences, labels, and 109 metadata-derived groups. Classification and regression use matched folds; the VAE uses an independent five-fold grouped partition of this cohort while preserving the same group-level separation between training and evaluation data.

The known-intermediate lineage analysis is a separate evaluation. It includes six VRC34 states drawn from the evaluation cohort and lineage-trajectory data, yielding four ordered triples. All six states are excluded from the shared OAS and CATNAP data used for VAE pretraining and adaptation.

4.1. Handcrafted baseline

The baseline represents each $V_H$ sequence by seven handcrafted features computed over the whole variable domain. These are sequence length, the fractions of charged (D, E, K, R, H), hydrophobic (A, I, L, M, F, W, Y, V), and aromatic (F, W, Y, H) residues, cysteine count, cysteine fraction, and the number of $N$-linked glycosylation motifs matching N-X-[S/T] where X is not proline. We count these motifs in the antibody variable domain itself, not in the viral glycan shield described in Section 1. The label is the binary breadth class. These simple features test whether ESM3 adds signal beyond amino-acid composition.

We evaluate four baselines: length-only logistic regression; logistic regression using all seven features; random forest using all seven features [25]; and XGBoost using all seven features [18]. XGBoost is retained as the prespecified paired comparator for uncertainty analyses. We recompute features inside each training fold and standardize them using statistics fit on the training split only.

4.2. Frozen ESM3 and downstream heads

Residue embeddings from frozen esm3-sm-open-v1 (1.4B parameters) [4] yield $L \times 1536$ matrices per sequence (excluding BOS/EOS tokens). Pooling schemes are (1) mean pooling (1536-d) and (2) concatenated mean, max, and standard deviation (4608-d). Scaling and PCA fit strictly inside each training fold. All downstream heads use scikit-learn [26].

Classification candidates are L1/L2 logistic regression (SAGA solver [27] for L1), random forest, XGBoost, and a PCA-50 multilayer perceptron, plus a separately trained additive attention-pooling classifier. Regression candidates are Ridge, random forest, and XGBoost, with a length-only Ridge floor. Appendix Table A1 lists hyperparameters.

4.3. OAS LoRA adaptation

LoRA [16] freezes base weights and injects rank-16 adapters ($\alpha=32$, dropout 0.05), training 18,874,368 parameters (1.33%). Adaptation uses masked language modeling [19] on 15% corrupted tokens for one epoch with AdamW [20] (effective batch 32). Clonotype-held-out validation reaches loss $0.5025$ (perplexity $1.653$). Adapters merge into the base model before embedding extraction.

4.4. Whole-sequence VAE

To model each antibody as one coherent unit, we train a categorical VAE [17] directly on complete heavy-chain sequences. We left-align each sequence in 170 slots and pad the remainder; each slot is one-hot encoded over 20 amino acids plus a PAD token, producing a 3570-dimensional input. A multilayer encoder (widths 2048, 1024, 256) maps the sequence to a 32-dimensional Gaussian latent variable, and a mirrored decoder produces $170\times21$ categorical logits. The objective combines categorical reconstruction cross-entropy with KL regularization under a linear $\beta$ warm-up; we bound posterior log-variance to $[-10, 6]$ and clip gradients to prevent non-finite training.

Training follows a protected staged protocol: the model pretrains on the protected OAS background, adapts once on protected non-evaluation CATNAP sequences, and independently fine-tunes from the shared adapted checkpoint inside each canonical outer training fold. Inner group-held validation selects the epoch count; refitting then uses all eligible outer-training rows. Each canonical antibody receives exactly one OOF reconstruction and latent vector. We report residue reconstruction accuracy, categorical cross-entropy, and Levenshtein edit distance.

4.5. Lineage interpolation and cross-representation controls

For the known-intermediate analysis we evaluate four ordered VRC34 triples $(A,B,C)$ in which $B$ is a known inferred intermediate. We encode each sequence to its posterior mean and form the midpoint $z_{\mathrm{mid}}=\tfrac{1}{2}(\mu_A+\mu_C)$. The latent test asks whether $z_{\mathrm{mid}}$ is closer to $\mu_B$ than either endpoint; the sequence test compares the decoded midpoint with $B$, the decoded endpoints, and the raw endpoint sequences. We run this test after OAS pretraining, CATNAP adaptation, and the fold that holds out the complete VRC34 group.

To compare the VAE against the foundation model on identical triples, we add frozen-ESM3 and LoRA-ESM3 embedding-interpolation controls: for each triple we globally align the per-residue embeddings of $A$ and $C$, form the embedding midpoint, and measure its distance to $B$. Because raw distances live on incomparable scales across the VAE latent, frozen ESM3, and LoRA ESM3, we report all arms after per-dimension standardization (unit variance), making midpoint errors directly comparable. The frozen- and LoRA-ESM3 embedding midpoints are decoded with the frozen ESM3 sequence head (the matched readout for both models); the VAE midpoint is decoded by its own head.

We also evaluate the whole-sequence VAE's 32-dimensional latent as a discriminative feature vector. To stay leakage-safe, each canonical antibody is encoded by the fold checkpoint that held its lineage out (its OOF posterior mean); we then apply the same classification and regression heads and grouped cross-validation as the ESM3 representations. The PCA-50 multilayer perceptron head is omitted for this arm because its component count exceeds the 32 latent dimensions.

5.1. Classification

Table 1 separates the configuration-selection score (mean over folds) from the pooled OOF estimate and group-bootstrap interval. Among the four baselines (Table 2), random forest has the largest mean-fold AUROC ($0.671$), while the prespecified XGBoost comparator has the largest pooled OOF AUROC ($0.651$) and AUPRC ($0.771$). Against that XGBoost comparator, frozen ESM3 raises the pooled AUROC point estimate from $0.651$ to $0.773$ and pooled AUPRC from $0.771$ to $0.869$. On aligned predictions, the paired frozen-minus-baseline difference is $+0.122$ AUROC ($[-0.008, 0.240]$) and $+0.099$ AUPRC ($[-0.003, 0.189]$). Both intervals include zero; they describe sensitivity to held-out lineage composition and do not remove model-selection optimism.

LoRA's selected classifier reaches pooled AUROC $0.729$, below frozen by $-0.044$ (paired interval $[-0.108, 0.019]$). The corresponding AUPRC difference is $-0.051$ ($[-0.090, -0.010]$). This single adaptation run does not improve classification and reduces selected-model AUPRC on the observed cohort. Pooled OOF ROC and precision–recall curves appear in Figure 4.

The learned attention pool reaches mean-fold AUROC $0.769 \pm 0.117$, below the best fixed-pooling frozen model. Adding learned attention pooling did not improve mean-fold AUROC on this cohort.

5.2. Continuous breadth regression

Regression uses the same 300 threshold-selected antibodies. Frozen ESM3 reaches pooled Spearman $\rho = 0.368$ ($[0.157, 0.574]$), Pearson $r = 0.363$ ($[0.163, 0.564]$), and $R^2 = 0.107$ ($[-0.122, 0.307]$). The length-only pooled Spearman diagnostic baseline is $0.007$ (Table 3). The selected representation shows a positive pooled ranking association, while its absolute breadth predictions remain weak and sensitive to group composition.

Selecting LoRA configurations by mean-fold Spearman chooses random forest with mean pooling (pooled $\rho = 0.305$, $[0.052, 0.549]$). The paired LoRA-minus-frozen Spearman difference is $-0.062$ ($[-0.201, 0.082]$).

5.3. Whole-sequence VAE reconstruction

Across the same canonical 300-antibody cohort used by classification and regression, the whole-sequence VAE attains mean OOF residue reconstruction accuracy $0.642$, categorical cross-entropy $1.348$, and sequence edit distance $43.35$. Reconstruction varies across the five grouped folds, with mean residue accuracy ranging from $0.563$ to $0.688$. Broad/positive antibodies are harder to reconstruct than narrow/weak antibodies (mean residue accuracy $0.597$ versus $0.735$). The model learns substantial sequence regularity but does not provide high-fidelity OOF reconstruction.

5.4. Known-intermediate lineage recovery

Table 4 reports the exact four-triple VRC34 midpoint test. At every stage, the latent midpoint is closer to the known intermediate than both latent endpoints in two of four triples — partial latent-geometry evidence, not consistent recovery across the trajectory. Decoded sequence recovery is unsuccessful under the stronger control: no decoded midpoint is closer to the known intermediate than the nearer raw flanking sequence. The OAS-pretrained model has the lowest decoded-midpoint mean edit distance ($24.75$); CATNAP adaptation and fold fine-tuning do not improve it.

5.5. Standardized cross-representation midpoint comparison

To place the antibody-trained VAE next to the foundation model on identical triples, we add frozen-ESM3 and LoRA-ESM3 embedding-interpolation controls and report every arm after per-dimension standardization, so the three representations share a common unit-variance scale (raw midpoint distances differ by orders of magnitude — VAE latent $\approx 0.02$, frozen ESM3 $\approx 11$, LoRA ESM3 $\approx 14$). Table 5 reports the CATNAP-adapted stage over the four VRC34 triples. The sequence VAE and frozen ESM3 each place the midpoint nearer the known intermediate than both endpoints in two of four triples; the LoRA-ESM3 midpoint beats its own endpoints in all four, although its standardized midpoint distance is not the smallest. With four triples from a single lineage and no confidence intervals, this is descriptive only.

5.6. Decoded-midpoint sequence recovery

The standardized comparison above is in representation space. To compare in sequence space, we decode each midpoint to amino acids and measure edit distance to the true intermediate $B$ (Table 6). Decoding the ESM3 embedding midpoints recovers the intermediate markedly better than the VAE (mean edit distance: LoRA $12.25$, frozen $18.75$, versus VAE $29.25$), and LoRA matches the raw-endpoint baseline on the first triple. None beats the raw-endpoint baseline on average ($5.00$), so midpoint interpolation still does not surpass simply taking the nearer flanking sequence; but the ESM3 decoders land far closer to the intermediate than the VAE.

5.7. Sequence-VAE latent as a discriminative representation

We also test the whole-sequence VAE's 32-dimensional latent as a discriminative feature. To remain leakage-safe, each canonical antibody is encoded by the fold checkpoint that held its lineage out (its OOF posterior mean), and the same heads and grouped cross-validation as the ESM3 representations are applied. As Table 7 and Figure 7 show, the latent's best classifier reaches mean-fold AUROC $0.682$ and its best regressor pooled Spearman $0.114$ — both well below frozen ESM3 ($0.779$ / $0.368$) and LoRA ESM3 ($0.746$ / $0.305$). On this low-dimensional latent, linear heads outperform tree ensembles (random forest $0.608$, XGBoost $0.606$ AUROC), reversing the pattern on the high-dimensional ESM3 embeddings. The 32-dimensional reconstruction bottleneck therefore retains the least breadth-discriminative signal, though it remains above chance.

6.1. Exploratory model selection

We compare heads and pooling schemes on the same five grouped folds used for reporting. Selecting the best mean-fold configuration makes reported cross-validation performance optimistic due to hyperparameter and model-selection leak. Group-bootstrap intervals condition on the selected predictions and do not account for that selection process. Confirmatory evaluation requires nested grouped cross-validation or a fully locked external lineage holdout.

6.2. Cohort and target construction

The cohort is small (300 sequences across 109 metadata-derived groups), limiting the statistical power of the architectural comparisons. Only $V_H$ chains enter the models, neglecting light-chain ($V_L$) contributions and heavy–light pairing, which are biochemically critical for folding, stability, and neutralization breadth of many HIV-1 bnAbs. CATNAP narrow/weak antibodies are not equivalent to arbitrary non-bnAbs, and breadth is treated as an unweighted fraction over viral panels that vary in size and strain composition, without accounting for laboratory-specific or panel-level effects.

6.3. Handcrafted baseline scope

The handcrafted features average amino-acid composition over the whole $V_H$ domain. Most of that domain is conserved framework, so signal localized in the CDRs is diluted. The feature set also omits two sequence properties commonly associated with neutralization breadth: CDRH3 loop length, and somatic hypermutation relative to the germline V gene. Somatic hypermutation needs an external germline reference that we do not use. Of the seven features, only sequence length and charged fraction separate the classes individually, and the rest contribute little on their own. The charged fraction sums acidic and basic residues together and counts histidine as charged, so it does not measure net charge or isoelectric point. The aromatic set also counts histidine, unlike the usual F, W, Y definition. Because our $V_H$-completeness filter requires the two conserved framework cysteines, cysteine count is nearly constant here, so cysteine fraction tracks inverse sequence length rather than cysteine content. This baseline is a simple composition reference, not a strong biologically informed comparator. A richer feature set could narrow the reported gap.

6.4. Grouping assumptions

Metadata-derived clonal lineage is the strongest available grouping key, but patient-level fallback groups may merge unrelated antibodies from the same donor, while singletons may mask unrecorded genetic relationships. The $\geq 80\%$ sequence-identity clustering heuristic supports separation in aggregate but cannot biologically prove complete developmental independence or rule out convergent evolution.

6.5. Adaptation and ESM3 constraints

LoRA domain adaptation was evaluated on a single dataset with one adapter configuration and one training run. Unsupervised adaptation on OAS may align representations toward general repertoire characteristics at the expense of somatic hypermutation features required for HIV-1 gp120/gp41 binding. ESM3's 1.4B-parameter scale introduces substantial computational overhead compared to simpler sequence models, and non-commercial licensing constraints limit downstream translation.

6.6. Interpolation and VAE constraints

The whole-sequence VAE preserves antibody-level sequence context, but its left-aligned slots do not use biological IMGT/ANARCI numbering, so insertions and deletions can shift downstream positions. OOF reconstruction remains moderate, especially for broad antibodies. The known-intermediate analysis contains only four ordered triples from one inferred VRC34 trajectory, so it is a case study rather than a population estimate. Raw flanking sequences are naturally close to their known intermediate, which creates a strong and appropriate control. Finally, the model uses one random seed, and CATNAP adaptation reaches its epoch cap while validation loss is still improving.

6.7. Reproducibility boundary

Reported calculations can be reconstructed from saved OOF predictions and evaluation artifacts without retraining ESM3, the LoRA adapter, or the whole-sequence VAE. These artifacts support verification of internal consistency and reported calculations, but not bitwise reproduction of the GPU training runs, which used external clusters (Google Colab). The group bootstrap resamples the observed 109 metadata-derived groups and does not represent a new external cohort.

Under metadata-derived grouped evaluation, frozen ESM3 yields higher classification point estimates than the baseline XGBoost comparator and a positive pooled association with continuous breadth. However, paired classification intervals include zero and pooled $R^2$ remains low, so the evaluated $V_H$-only models do not provide accurate absolute breadth prediction on this cohort. The single OAS LoRA adaptation run does not improve downstream classification or regression performance.

The evaluated task differs from virus-centered resistance studies that score Env sequences against fixed antibodies [9][22][10] and from an antibody-centered structure-based breadth estimator for CD4-binding-site antibodies [29]. Here the model receives the variable heavy-chain amino-acid sequence as its sole input, without light-chain sequence, viral-envelope sequence, epitope labels, assay-panel metadata, or modeled structure. Because these studies use different inputs, targets, and split protocols, their AUROCs are not directly comparable. Against the prespecified baseline XGBoost comparator, frozen ESM3 increases pooled AUROC from $0.651$ to $0.773$, but the paired $+0.122$ difference has group-bootstrap interval $[-0.008, 0.240]$.

The whole-sequence VAE reconstructs $64.2\%$ of held-out residues on average. Its latent midpoint recovers the known VRC34 intermediate in two of four lineage triples, but decoded midpoints do not outperform the nearer raw endpoint. This supports partial latent organization, not successful sequence-level intermediate recovery.

Overall, the strongest contribution is methodological: metadata-derived grouping, matched evaluation, pooled OOF reporting, and group-resampled intervals materially change how these experiments should be interpreted. Future work should evaluate paired heavy/light-chain inputs and structural context, use larger external cohorts, repeat training across seeds, and replace left alignment with biologically numbered sequence positions.