[Submitted on 14 Oct 2026 (v1), last revised 6 Nov 2026 (this version, v4)]
Abstract:Surrogate models approximate the solutions of partial differential equations given a certain geometry and boundary conditions. Traditional transformer-based approaches employ a single mixing mechanism to the whole solution. although the solution fields exhibit variations across multiple spatial scales. We propose a mixture of transformer experts that combines a routing mechanism with multiscale supervision. Specifically, the individual experts are routed through a controller that utilizes learnable Laplacian matrices to compute weights that are subsequently used to define a weighted sum of the experts for the output feature vectors at each point of the domain. Subsequently, a decoder receives the weighted sum of the experts output and predicts the solution fields. During training, the reference fields are decomposed into components at different spatial scales by repeated smoothing with a mesh Laplacian. We evaluate our approach on the official AirFRANS dataset using remeshed solution domains leading to unstructured triangular meshes of $4.9$k vertices. The experimental outcomes reveal lower mean relative $L_2$ error and interior mean squared errors among the compared models of this study.
Submission history
From: Stavros Nousias PhD [view email]
[v1] Wed, 14 Oct 2026 17:57:24 UTC (3,262 KB)
[v2] Fri, 16 Oct 2026 10:48:30 UTC (3,262 KB)
[v3] Sat, 31 Oct 2026 07:13:42 UTC (3,262 KB)
[v4] Fri, 6 Nov 2026 11:19:01 UTC (3,282 KB)
