Publications by Type: Journal Article

Ziyan Zhu, Marios Mattheakis, Weiwei Pan, and Efthimios Kaxiras. Submitted. “HubbardNet: Efficient Predictions of the Bose-Hubbard Model Spectrum with Deep Neural Networks”.Abstract

We present a deep neural network (DNN)-based model (HubbardNet) to variationally find the ground state and excited state wavefunctions of the one-dimensional and two-dimensional Bose- Hubbard model. Using this model for a square lattice with M sites, we obtain the energy spectrum as an analytical function of the on-site Coulomb repulsion, U, and the total number of particles, N, from a single training. This approach bypasses the need to solve a new hamiltonian for each different set of values (U, N ). We show that the DNN-parametrized solutions are in excellent agreement with results from the exact diagonalization of the hamiltonian, and it outperforms the exact diagonalization solution in terms of computational scaling. These advantages suggest that our model is promising for efficient and accurate computation of exact phase diagrams of many-body lattice hamiltonians.

Ariel Norambuena, Marios Mattheakis, Francisco J. Gonzalez, and Raul Coto. Submitted. “Physics-informed neural networks for quantum control”. Publisher's VersionAbstract

Quantum control is a ubiquitous research field that has enabled physicists to delve into the dynamics and features of quantum systems. In addition to steering the system, quantum control has delivered powerful applications for various atomic, optical, mechanical, and solid-state systems. In recent years, traditional control techniques based on optimization processes have been translated into efficient artificial intelligence algorithms. Here, we introduce a computational method for optimal quantum control problems via physics-informed neural networks (PINNs). We apply our methodology to open quantum systems by efficiently solving the state-to-state transfer problem with high probabilities, short-time evolution, and minimizing the power of the control. Furthermore, we illustrate the flexibility of PINNs to solve the same problem under changes in parameters and initial conditions, showing advantages in comparison with standard control techniques.

Trevor David Rhone, Romakanta Bhattarai, Haralambos Gavras, Bethany Lusch, Misha Salim, Marios Mattheakis, Daniel T. Larson, Yoshiharu Krockenberger, and Efthimios Kaxiras. Forthcoming. “Artificial intelligence guided studies of van der Waals magnets.” Advanced Theory and Simulations.
Marios Mattheakis, Hayden Joy, and Pavlos Protopapas. 2/2023. “Unsupervised Reservoir Computing for Solving Ordinary Differential Equations.” International Journal on Artificial Intelligence Tools, 32, 1. Publisher's VersionAbstract

There is a wave of interest in using physics-informed neural networks for solving differen-
tial equations. Most of the existing methods are based on feed-forward networks, while
recurrent neural networks solvers have not been extensively explored. We introduce a
reservoir computing (RC) architecture, an echo-state recurrent neural network capable
of discovering approximate solutions that satisfy ordinary differential equations (ODEs).
We suggest an approach to calculate time derivatives of recurrent neural network outputs
without using back-propagation. The internal weights of an RC are fixed, while only a lin-
ear output layer is trained, yielding efficient training. However, RC performance strongly
depends on finding the optimal hyper-parameters, which is a computationally expensive
process. We use Bayesian optimization to discover optimal sets in a high-dimensional
hyper-parameter space efficiently and numerically show that one set is robust and can
be transferred to solve an ODE for different initial conditions and time ranges. A closed-
form formula for the optimal output weights is derived to solve first-order linear equa-
tions in a one-shot backpropagation-free learning process. We extend the RC approach
by solving nonlinear systems of ODEs using a hybrid optimization method consisting of
gradient descent and Bayesian optimization. Evaluation of linear and nonlinear systems
of equations demonstrates the efficiency of the RC ODE solver.

Marios Mattheakis, David Sondak, Akshunna S. Dogra, and Pavlos Protopapas. 6/30/2022. “Hamiltonian neural networks for solving equations of motion.” Phys. Rev. E, 105, Pp. 065305. Publisher's VersionAbstract

There has been a wave of interest in applying machine learning to study dynamical systems. We present a Hamiltonian neural network that solves the differential equations that govern dynamical systems. This is an equation-driven machine learning method where the optimization process of the network depends solely on the predicted functions without using any ground truth data. The model learns solutions that satisfy, up to an arbitrarily small error, Hamilton’s equations and, therefore, conserve the Hamiltonian invariants. The choice of an appropriate activation function drastically improves the predictability of the network. Moreover, an error analysis is derived and states that the numerical errors depend on the overall network performance. The Hamiltonian network is then employed to solve the equations for the nonlinear oscillator and the chaotic H ́enon-Heiles dynamical system. In both systems, a symplectic Euler integrator requires two orders more evaluation points than the Hamiltonian network in order to achieve the same order of the numerical error in the predicted phase space trajectories.


Mattia Angelia, Georgios Neofotistos, Marios Mattheakis, and Efthimios Kaxiras. 1/2022. “Modeling the effect of the vaccination campaign on the Covid-19 pandemic.” Chaos, Solitons and Fractals, 154, Pp. 111621. Publisher's VersionAbstract

Population-wide vaccination is critical for containing the SARS-CoV-2 (Covid-19) pandemic when combined with restrictive and prevention measures. In this study we introduce SAIVR, a mathematical model able to forecast the Covid-19 epidemic evolution during the vaccination campaign. SAIVR extends the widely used Susceptible-Infectious-Removed (SIR) model by considering the Asymptomatic (A) and Vaccinated (V) compartments. The model contains sev- eral parameters and initial conditions that are estimated by employing a semi-supervised machine learning procedure. After training an unsupervised neural network to solve the SAIVR differ- ential equations, a supervised framework then estimates the optimal conditions and parameters that best fit recent infectious curves of 27 countries. Instructed by these results, we performed an extensive study on the temporal evolution of the pandemic under varying values of roll-out daily rates, vaccine efficacy, and a broad range of societal vaccine hesitancy/denial levels. The concept of herd immunity is questioned by studying future scenarios which involve different vaccination efforts and more infectious Covid-19 variants.

Hayden Joy, Marios Mattheakis, and Pavlos Protopapas. 2022. “RcTorch: a PyTorch Reservoir Computing Package with Automated Hyper-Parameter Optimization.” arXiv paper. Publisher's VersionAbstract

Reservoir computer (RC) are among the fastest to train of all neural networks, especially when they are compared to other recurrent neural networks. RC has this advantage while still handling sequential data exceptionally well. However, RC adoption has lagged other neural network models because of the model’s sensitivity to its hyper-parameters (HPs). A modern unified software package that automatically tunes these parameters is missing from the literature. Manually tuning these numbers is very difficult, and the cost of traditional grid search methods grows exponentially with the number of HPs considered, discouraging the use of the RC and limiting the complexity of the RC models which can be devised. We address these problems by introducing RcTorch, a PyTorch based RC neural network package with automated HP tuning. Herein, we demonstrate the utility of RcTorchby using it to predict the complex dynamics of a driven pendulum being acted upon by varying forces. This work includes coding examples. Example Python Jupyter notebooks can be found on our GitHub repository and documentation can be found at

Shaan Desai, Marios Mattheakis, David Sondak, Pavlos Protopapas, and Stephen Roberts. 9/2021. “Port-Hamiltonian Neural Networks for Learning Explicit Time-Dependent Dynamical Systems.” Phys Rev. E, 104, Pp. 034312. Publisher's VersionAbstract
Accurately learning the temporal behavior of dynamical systems requires models with well-chosen learning biases. Recent innovations embed the Hamiltonian and Lagrangian formalisms into neural networks and demonstrate a significant improvement over other approaches in predicting trajectories of physical systems. These methods generally tackle autonomous systems that depend implicitly on time or systems for which a control signal is known apriori. Despite this success, many real world dynamical systems are non-autonomous, driven by time-dependent forces and experience energy dissipation. In this study, we address the challenge of learning from such non-autonomous systems by embedding the port-Hamiltonian formalism into neural networks, a versatile framework that can capture energy dissipation and time-dependent control forces. We show that the proposed \emph{port-Hamiltonian neural network} can efficiently learn the dynamics of nonlinear physical systems of practical interest and accurately recover the underlying stationary Hamiltonian, time-dependent force, and dissipative coefficient. A promising outcome of our network is its ability to learn and predict chaotic systems such as the Duffing equation, for which the trajectories are typically hard to learn.
Shaan Desai, Marios Mattheakis, and Stephen Roberts. 9/2021. “Variational Integrator Graph Networks for Learning Energy Conserving Dynamical Systems.” Phys. Rev. E, 104, Pp. 035310. Publisher's VersionAbstract
Recent advances show that neural networks embedded with physics-informed priors significantly outperform vanilla neural networks in learning and predicting the long term dynamics of complex physical systems from noisy data. Despite this success, there has only been a limited study on how to optimally combine physics priors to improve predictive performance. To tackle this problem we unpack and generalize recent innovations into individual inductive bias segments. As such, we are able to systematically investigate all possible combinations of inductive biases of which existing methods are a natural subset. Using this framework we introduce Variational Integrator Graph Networks - a novel method that unifies the strengths of existing approaches by combining an energy constraint, high-order symplectic variational integrators, and graph neural networks. We demonstrate, across an extensive ablation, that the proposed unifying framework outperforms existing methods, for data-efficient learning and in predictive accuracy, across both single and many-body problems studied in recent literature. We empirically show that the improvements arise because high order variational integrators combined with a potential energy constraint induce coupled learning of generalized position and momentum updates which can be formalized via the Partitioned Runge-Kutta method.
Yue Luo, Rebecca Engelke, Marios Mattheakis, Michele Tamagnone, Stephen Carr, Kenji Watanabe, Takashi Taniguchi, Efthimios Kaxiras, Philip Kim, and William L. Wilson. 8/2020. “In-situ nanoscale imaging of moiré superlattices in twisted van der Waals heterostructures.” Nature Communication, 11, 4209, Pp. 1-7. Publisher's VersionAbstract
Direct visualization of nanometer-scale properties of moiré superlattices in van der Waals
heterostructure devices is a critically needed diagnostic tool for study of the electronic and optical phenomena induced by the periodic variation of atomic structure in these complex systems. Conventional imaging methods are destructive and insensitive to the buried device geometries, preventing practical inspection. Here we report a versatile scanning probe  microscopy employing infrared light for imaging moiré superlattices of twisted bilayers graphene encapsulated by hexagonal boron nitride. We map the pattern using the scattering dynamics of phonon polaritons launched in hexagonal boron nitride capping layers via its interaction with the buried moiré superlattices. We explore the origin of the double-line features imaged and show the mechanism of the underlying effective phase change of the phonon polariton reflectance at domain walls. The nano-imaging tool developed provides a non-destructive analytical approach to elucidate the complex physics of moiré engineered heterostructures.
G. A. Tritsaris, S. Carr, Z. Zhu, Y. Xie, S. Torrisi, J. Tang, M.Mattheakis, D. Larson, and E. Kaxiras. 6/2020. “Electronic structure calculations of twisted multi-layer graphene superlattices.” 2D Materials, 7, Pp. 035028. Publisher's VersionAbstract
Quantum confinement endows two-dimensional (2D) layered materials with exceptional physics and novel properties compared to their bulk counterparts. Although certain two- and few-layer configurations of graphene have been realized and studied, a systematic investigation of the properties of arbitrarily layered graphene assemblies is still lacking. We introduce theoretical concepts and methods for the processing of materials information, and as a case study, apply them to investigate the electronic structure of multi-layer graphene-based assemblies in a high-throughput fashion. We provide a critical discussion of patterns and trends in tight binding band structures and we identify specific layered assemblies using low-dispersion electronic bands as indicators of potentially interesting physics like strongly correlated behavior. A combination of data-driven models for visualization and prediction is used to intelligently explore the materials space. This work more generally aims to increase confidence in the combined use of physics-based and data-driven modeling for the systematic refinement of knowledge about 2D layered materials, with implications for the development of novel quantum devices.
Georgios A. Tritsaris, Yiqi Xie, Alexander M. Rush, Stephen Carr, Marios Mattheakis, and Efthimios Kaxiras. 6/2020. “LAN -- A materials notation for 2D layered assemblies.” J. Chem. Inf. Model. . Publisher's VersionAbstract
Two-dimensional (2D) layered materials offer intriguing possibilities for novel physics and applications. Before any attempt at exploring the materials space in a systematic fashion, or combining insights from theory, computation and experiment, a formal description of information about an assembly of arbitrary composition is required. Here, we introduce a domain-generic notation that is used to describe the space of 2D layered materials from monolayers to twisted assemblies of arbitrary composition, existent or not-yet-fabricated. The notation corresponds to a theoretical materials concept of stepwise assembly of layered structures using a sequence of rotation, vertical stacking, and other operations on individual 2D layers. Its scope is demonstrated with a number of example structures using common single-layer materials as building blocks. This work overall aims to contribute to the systematic codification, capture and transfer of materials knowledge in the area of 2D layered materials.
Feiyu Chen, David Sondak, Pavlos Protopapas, Marios Mattheakis, Shuheng Liu, Devansh Agarwal, and Marco Di Giovanni. 2/2020. “NeuroDiffEq: A Python package for solving differential equations with neural networks.” Journal of Open Source Software, 5, 46. Publisher's Version 2020_joss_neurodiffeq.pdf
G. Barmparis, G. Neofotistos, M.Mattheakis, J. Hitzanidi, G. P. Tsironis, and E. Kaxiras. 2/2020. “Robust prediction of complex spatiotemporal states through machine learning with sparse sensing.” Physics Letters A, 384, Pp. 126300. Publisher's VersionAbstract
Complex spatiotemporal states arise frequently in material as well as biological systems consisting of multiple interacting units. A specific, but rather ubiquitous and interesting example is that of “chimeras”, existing in the edge between order and chaos. We use Machine Learning methods involving “observers” to predict the evolution of a system of coupled lasers, comprising turbulent chimera states and of a less chaotic biological one, of modular neuronal networks containing states that are synchronized across the networks. We demonstrated the necessity of using “observers” to improve the performance of Feed-Forward Networks in such complex systems. The robustness of the forecasting capabilities of the “Observer Feed-Forward Networks” versus the distribution of the observers, including equidistant and random, and the motion of them, including stationary and moving was also investigated. We conclude that the method has broader applicability in dynamical system context when partial dynamical information about the system is available.
M. Maier, M.Mattheakis, E. Kaxiras, M. Luskin, and D. Margetis. 10/2019. “Homogenization of plasmonic crystals: Seeking the epsilon-near-zero behavior.” Proceedings of the Royal Society A, 475, 2230. Publisher's VersionAbstract
By using an asymptotic analysis and numerical simulations, we derive and investigate a system of homogenized Maxwell's equations for conducting material sheets that are periodically arranged and embedded in a heterogeneous and anisotropic dielectric host.  This structure is motivated by the need to design plasmonic crystals that enable the propagation of electromagnetic waves with no phase delay (epsilon-near-zero effect). Our microscopic model incorporates the surface conductivity of the two-dimensional (2D) material of each sheet and a corresponding line charge density through a line conductivity along possible edges of the sheets. Our analysis generalizes averaging principles inherent in previous Bloch-wave approaches. We investigate physical implications of our findings. In particular, we emphasize the role of the vector-valued corrector field, which expresses microscopic modes of surface waves on the 2D material. By using a Drude model for the surface conductivity of the sheet, we construct a Lorentzian function that describes the effective dielectric permittivity tensor of the plasmonic crystal as a function of frequency.
M.Mattheakis, P. Protopapas, D. Sondak, M. Di Giovanni, and E. Kaxiras. 4/2019. “Physical Symmetries Embedded in Neural Networks.” arXiv paper, 1904.08991. Publisher's VersionAbstract
Neural networks are a central technique in machine learning. Recent years have seen a wave of interest in applying neural networks to physical systems for which the governing dynamics are known and expressed through differential equations. Two fundamental challenges facing the development of neural networks in physics applications is their lack of interpretability and their physics-agnostic design. The focus of the present work is to embed physical constraints into the structure of the neural network to address the second fundamental challenge. By constraining tunable parameters (such as weights and biases) and adding special layers to the network, the desired constraints are guaranteed to be  satisfied without the need for explicit regularization terms. This is demonstrated on  supervised and unsupervised networks for two basic symmetries: even/odd symmetry of a function and energy conservation. In the supervised case, the network with embedded constraints is shown to perform well on regression problems while simultaneously obeying the desired constraints whereas a traditional network fits the data but violates the underlying  constraints. Finally, a new unsupervised neural network is proposed that guarantees energy conservation through an embedded symplectic structure. The symplectic neural network is used to solve a system of energy-conserving differential equations and out-performs an  unsupervised, non-symplectic neural network.
G. N. Neofotistos, M.Mattheakis, G. Barmparis, J. Hitzanidi, G. P. Tsironis, and E. Kaxiras. 3/1/2019. “Machine learning with observers predicts complex spatiotemporal behavior.” Front. Phys. - Quantum Computing , 7, 24, Pp. 1-9. Publisher's VersionAbstract
Chimeras and branching are two archetypical complex phenomena that appear in many physical systems; because of their different intrinsic dynamics, they delineate opposite non-trivial limits in the complexity of wave motion and present severe challenges in predicting chaotic and singular behavior in extended physical systems. We report on the long-term forecasting capability of Long Short-Term Memory (LSTM) and reservoir computing (RC) recurrent neural networks, when they are applied to the spatiotemporal evolution of turbulent chimeras in simulated arrays of coupled superconducting quantum interference devices (SQUIDs) or lasers, and branching in the electronic flow of two-dimensional graphene with random potential. We propose a new method in which we assign one LSTM network to each system node except for “observer” nodes which provide continual “ground truth” measurements as input; we refer to this method as “Observer LSTM” (OLSTM). We
demonstrate that even a small number of observers greatly improves the data-driven (model-free) long-term forecasting capability of the LSTM networks and provide the framework for a consistent comparison between the RC and LSTM methods. We find that RC requires smaller training datasets than OLSTMs, but the latter require fewer observers. Both methods are benchmarked against Feed-Forward neural networks (FNNs), also trained to make predictions with observers (OFNNs).
Najm Hassan, Marios Mattheakis, and Ming Ding. 11/2018. “Sensorless Node Architecture for Events Detection in Self-Powered Nanosensor Networks.” Nano Communication Networks, 19, Pp. 1-9. Publisher's VersionAbstract
Due to size, computational and power limitations an integrated nanosensor device needs to be redesigned with a limited number of components. A sensorless event detection node can overcome these limitations where such node can be powered using energy harvested from various events. The harvested energy could also be a significant factor for events detection without using any sensors. This study presents a detailed description of a sensorless event detection node which consists of two components — an energy harvester and a pulse generator. We discuss the state of the art configurations for these two components. However, due to the low complexity of the nanoscale device, the pulse generator should be kept simple. We, therefore, theoretically investigate different approaches for the pulse generator to generate Surface Plasmon Polaritons (SPPs) which reasonably resemble femtoseconds long pulses in graphene. Based on our analysis, we find that SPPs can be excited using a near-field excitation method for the THz band which is simple and can produce Electromagnetic (EM) radiation with a wide range of high wavenumber. Hence, the coupling condition can be easily satisfied and consequently, the SPP wave can be excited. However, such method excites SPPs locally, which requires improvement in practice. Thus we numerically investigate how operating frequency, the doping amount of graphene and the properties of the evanescent source affect the plasmon resonance of SPPs. We also studied different evanescent sources such as electric dipole, and hexapole, and find that the former provides better SPP resonance. We also observe that through fine-tuning of the chemical potential, frequency and source phase angle, higher amplitude SPPs can be excited on graphene surface in the THz band. The proposed model can be a good candidate for a low-complexity realization of a THz pulse generator in self-powered sensorless events detection node.
Marios Mattheakis, G. P. Tsironis, and Efthimios Kaxiras. 6/2018. “Emergence and dynamical properties of stochastic branching in the electronic flows of disordered Dirac solids.” EPL, 122, Pp. 27003. Publisher's VersionAbstract
Graphene as well as more generally Dirac solids constitute two dimensional materials where the electronic flow is ultra relativistic. When a Dirac solid is deposited on a different substrate surface with roughness, a local random potential develops through an inhomogeneous charge impurity distribution. This external potential affects profoundly the charge flow and induces a chaotic pattern of current branches that develops through focusing and defocusing effects produced by the randomness of the surface. An additional bias voltage may be used to tune the branching pattern of the charge carrier currents. We employ analytical and numerical techniquesin order to investigate the onset and the statistical properties of carrier branches in Dirac solids. We find a specific scaling-type relationship that connects the physical scale for the occurrenceof branches with the characteristic medium properties, such as disorder and bias field. We usenumerics to test and verify the theoretical prediction as well as a perturbative approach that gives a clear indication of the regime of validity of the approach. This work is relevant to deviceapplications and may be tested experimentally.
Sharmila N. Shirodkar, Marios Mattheakis, Paul Cazeaux, Prineha Narang, Marin Soljačić, and Efthimios Kaxiras. 5/2018. “Quantum plasmons with optical-range frequencies in doped few-layer graphene.” Phys. Rev. B, 97, Pp. 195435. Publisher's VersionAbstract
Although plasmon modes exist in doped graphene, the limited range of doping achieved by gating restricts the plasmon frequencies to a range that does not include the visible and infrared. Here we show, through the use of first-principles calculations, that the high levels of doping achieved by lithium intercalation in bilayer and trilayer graphene shift the plasmon frequencies into the visible range. To obtain physically meaningful results, we introduce a correction of the effect of plasmon interaction across the vacuum separating periodic images of the doped graphene layers, consisting of transparent boundary conditions in the direction perpendicular to the layers; this represents a significant improvement over the exact Coulomb cutoff technique employed in earlier works. The resulting plasmon modes are due to local field effects and the nonlocal response of the material to external electromagnetic fields, requiring a fully quantum mechanical treatment. We describe the features of these quantum plasmons, including the dispersion relation, losses, and field localization. Our findings point to a strategy for fine-tuning the plasmon frequencies in graphene and other two-dimensional materials.