From Clicks to Intent: Cross-Platform Session Embeddings with LLM-Distilled Taxonomy for Financial Services Recommendations

Source: arXiv:2606.26277 · Published 2026-06-24 · By Dianjing Fan, Yao Li, Kyaw Hpone Myint, Dwipam Katariya, Alexandre G. R. Day, Pranab Mohanty et al.

TL;DR

This paper addresses the challenge of bridging pre-login web clickstream behaviors and post-login personalization in financial services recommender systems. Pre-login users on financial websites typically explore products anonymously, while post-login users engage in account servicing. Because cross-channel identity linkage is difficult, valuable pre-login intent signals are often discarded and underused for personalization. The authors propose a scalable dual-output system that converts raw clickstreams into a dense session embedding and interpretable intent labels distilled from a large language model (LLM)-generated taxonomy. This approach leverages a self-supervised Transformer for multi-modal session encoding and an iterative, clustering-enhanced LLM taxonomy generation pipeline, followed by knowledge distillation to a lightweight classifier for scalability.

Empirically, the session embeddings outperform baseline features on two downstream tasks: mobile homepage tile ranking and user conversion prediction. The best embedding variant reduces Log Loss by 13.38% and improves macro Recall@1 by 1.88% relative to the production baseline. The distilled intent classifier achieves similar interpretability with only a 7% relative drop in micro F1, while delivering ultra-low latency inference over millions of sessions daily. The results demonstrate the value of jointly modeling rich behavioral patterns as dense embeddings and extracting human-readable intent labels at scale, specifically addressing the unique characteristics of financial service user interactions.

Key findings

• Bidirectional Transformer with mean pooling on clickstream sequences improves macro Recall@1 by 1.88% and reduces Log Loss by 13.38% on mobile homepage tile ranking vs baseline features (Table 3).
• Session embedding outperforms LLM-generated intent labels by 4.3% on micro F1 for user conversion prediction (Table 5).
• Distilled MLP classifier reproduces 93% of LLM teacher label performance, with only 7% relative micro F1 drop, enabling scalable intent labeling at ultra-low latency.
• Hand-crafted site retargeting features degrade micro F1 by 35.8% relative to LLM intent labels, confirming insufficiency of coarse features.
• Clustering-based stratified sampling (K=50) improves LLM taxonomy generation diversity as measured by silhouette (0.535), DBI (1.478), and CHI (19250).
• LLM-generated taxonomy labels achieve 97.0% strict agreement and 99.5% lenient agreement with human annotation over representative sessions.
• Transformer embedding encoding 100K sessions takes 60 seconds, with distilled intent MLP inference under 0.01 seconds—over 3,000× faster than direct LLM inference.

Threat model

The paper implicitly assumes a benign production environment where users engage in pre-login web sessions anonymously, and cross-device linkage via first-party cookies and login credentials reliably matches pre-login behavior to authenticated users. The adversary is not explicitly modeled; the system does not consider adversarial behaviors such as click fraud, spoofing, or privacy attacks. The focus is on improving recommender personalization by leveraging latent intent rather than security or adversarial robustness.

Methodology — deep read

The paper proposes a two-stage pipeline to derive intent representations from pre-login web clickstreams in a financial services context.

1. Threat Model & Assumptions: The adversary model is implicit, focusing on typical production scenarios with anonymized pre-login sessions linked post-login via first-party cookies and login credentials. Adversarial resistance or privacy limitations are not discussed. The system assumes cross-device and cross-channel entity resolution can connect anonymous browsing to authenticated users.

2. Data: Internal clickstream data from a first-party financial services website is used. Approximately 10M sessions are generated daily; the study filters to about 100K sessions/day that include both pre-login and post-login events. Each session consists of chronologically ordered events capturing page info, event types, dwell times, cookies, device metadata, and extracted semantic content via page HTML embedding. Data is split temporally across four months for training, validation, and testing to avoid leakage.

3. Architecture/Algorithm: The session embedding model uses a self-supervised Transformer encoder that integrates heterogeneous features per event: learned embeddings for categorical event attributes, pretrained sentence transformer vectors for page content, and positional embeddings. Event features are concatenated and fused via an MLP, then input to a Transformer encoder with either causal or bidirectional self-attention. The Transformer is trained with masked language modeling objectives predicting next or masked event titles/URLs, enabling it to learn contextual session embeddings without labeled data. The final session vector is produced by mean pooling over event outputs.

For interpretability, a three-phase LLM-driven pipeline generates an intent taxonomy: clustering session embeddings (K-Means, K=50) provides a stratified sample; iterative minibatch LLM prompting generates, updates, and reviews the taxonomy over several epochs; assigned intent labels from the LLM teacher are distilled into a lightweight MLP classifier that inputs session embeddings and predicts multi-label intent probabilities using soft binary cross-entropy with teacher confidence as soft targets.

4. Training Regime: The Transformer is trained self-supervised on large clickstream sequences over multiple epochs; details on batch size, hyperparameters, and hardware are not explicitly stated. The distilled MLP is supervised on a labeled set of sessions with LLM intent labels; early stopping and class weighting are applied.

5. Evaluation Protocol: Downstream tasks include mobile homepage tile ranking (multi-label classification, macro Recall@K and Log Loss), and user conversion prediction (multi-label classification, micro precision, recall, F1). Baselines include site retargeting features (hand-crafted counts) and LLM-generated intent labels. Ablations compare different Transformer variants and embedding pooling schemes. Human annotation validates the LLM taxonomy label quality.

6. Reproducibility: The dataset is internal and proprietary to a financial institution. No code or pretrained models are released. Pipeline details are given, but full hyperparameters and random seeds are not disclosed.

Example end-to-end: A pre-login web session is parsed into a sequence of event feature vectors combining categorical embeddings and semantic page vectors. These are fed into the Transformer encoder to produce a 64-dimensional session embedding. The clustering step stratifies samples to the LLM, which iteratively creates an intent taxonomy. The LLM labels sampled sessions, training a distilled MLP to predict those labels from embeddings. At inference, the Transformer encodes new sessions rapidly, and the MLP predicts interpretable intent labels efficiently for millions of daily sessions.

Technical innovations

• Combined self-supervised Transformer encoding of multi-modal clickstream data with LLM-generated intent taxonomy to produce dual-output session representations.
• Clustering-enhanced stratified sampling for LLM taxonomy generation ensuring representative coverage of diverse user behaviors.
• Iterative generate-update-review LLM taxonomy refinement for fixed-size, human-interpretable intent categories tailored to financial services.
• Knowledge distillation of LLM intent labels into a lightweight MLP classifier operating directly on dense session embeddings for scalable, low-latency inference.
• Use of pretrained sentence transformers to incorporate semantic webpage content embeddings alongside traditional categorical clickstream features.

Datasets

• Internal financial services web clickstream data — ~100K daily sessions filtered to contain both pre- and post-login activity — proprietary internal dataset

Baselines vs proposed

• Baseline (CTR + Site Retargeting Features): macro Recall@1 = baseline; Log Loss = baseline
• Causal Transformer + Last Hidden State: Recall@1 +0.56%, Log Loss +1.37%
• Causal Transformer + Mean Pooling: Recall@1 +1.50%, Log Loss -9.69%
• Bidirectional Transformer + Mean Pooling: Recall@1 +1.88%, Log Loss -13.38% vs baseline (Table 3)
• Site Retargeting Features only (hand-crafted): -35.8% micro F1 vs LLM intent teacher baseline
• Distilled MLP intent labels: -7.0% relative micro F1 vs LLM teacher intent labels
• Session embedding: +4.3% micro F1 vs LLM teacher intent labels (Table 5)

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.26277.

Fig 1: System architecture overview. Left (Section 3.2): multi-modal clickstream events are fused and encoded into session

Limitations

• Evaluations are limited to internal proprietary financial services clickstream data from a single institution, limiting generalizability.
• No adversarial robustness or privacy impact assessments were conducted.
• The effectiveness of cross-device identity linkage methods is assumed but not explored or evaluated in detail.
• Lack of full transparency on model training hyperparameters, random seeds, and reproducibility resources (code, weights) restricts outsider verification.
• The iterative taxonomy generation requires domain expert human-in-the-loop review, adding manual overhead and subjectivity.
• Downstream evaluation tasks focus on offline metrics without reported online A/B testing or direct user impact measurements.

Open questions / follow-ons

• How would the system generalize to other domains beyond financial services with different pre-login/post-login behavior patterns?
• Can the taxonomy refinement be fully automated to adapt in real time without manual human review?
• What is the impact of imperfect or missing cross-device linkage on embedding and intent quality?
• Could adversarial or privacy-preserving variants of the embedding and distillation pipeline improve trustworthiness?

Why it matters for bot defense

Bot-defense and CAPTCHA practitioners can take interest in this work because it demonstrates effective use of anonymous pre-login web behavioral signals transformed into robust intent embeddings and interpretable labels via large language models. This cross-platform session embedding approach highlights how rich clickstream data can be leveraged prior to authentication to better understand user intent, which may aid in distinguishing humans from bots based on diverse, temporally structured session features. Additionally, the knowledge distillation of expensive LLM inference into lightweight classifiers facilitating scalable real-time inference is directly applicable to system design constraints common in bot mitigation.

However, the focus here is primarily on financial recommender personalization and not adversarial detection or bot classification. Practitioners designing bot defenses might explore adapting similar self-supervised multi-modal embedding techniques combined with distilled interpretable intent representations to detect anomalous or automated browsing patterns early in sessions. The methodology for building scalable, explainable user intent taxonomies could also assist in segmenting suspicious traffic or understanding nuanced bot behaviors across multi-channel user touchpoints.

Cite

@article{arxiv2606_26277,
  title={ From Clicks to Intent: Cross-Platform Session Embeddings with LLM-Distilled Taxonomy for Financial Services Recommendations },
  author={ Dianjing Fan and Yao Li and Kyaw Hpone Myint and Dwipam Katariya and Alexandre G. R. Day and Pranab Mohanty and Giri Iyengar },
  journal={arXiv preprint arXiv:2606.26277},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.26277}
}

Read the full paper

• arXiv:2606.26277 (abstract)
• arXiv:2606.26277 (PDF)