transcribe

Optimal Basis Elements: The POD Expansion

Nathan Kutz · 30m · transcribed Aug 2026
More from Nathan Kutz Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Section Insights

# 0:00

Introduction to Reduced Order Models

What is the concept of reduced order models and how are they constructed?

Reduced order models aim to simplify complex systems, typically represented by partial differential equations (PDEs), into lower-dimensional representations. This involves constructing an optimal basis for the solution space, allowing for efficient simulations.

  • Reduced order models simplify high-dimensional PDEs into lower-dimensional proxies.
  • The goal is to perform computations at a reduced size R instead of the original size N.
  • Optimal basis construction is key for effective modeling.
# 6:08

Constructing Optimal Basis Elements

How are the basis elements for reduced order models determined?

The basis elements are derived from simulation data, utilizing techniques like Singular Value Decomposition (SVD) to identify an optimal subspace that minimizes error in approximating the data.

  • Basis elements are constructed from simulation data rather than predefined functions.
  • SVD helps in finding an optimal subspace for the reduced order model.
  • The resulting basis is guaranteed to be optimal in an L2 sense.
# 12:16

Example of Harmonic Oscillator

What is the example used to illustrate the construction of a basis for reduced order models?

The harmonic oscillator is used as an example, where the goal is to find an optimal embedding space through simulation data, potentially without prior knowledge of the system's analytical solutions.

  • The harmonic oscillator serves as a practical example for constructing reduced order models.
  • Simulation data can reveal optimal basis sets even without prior knowledge of the system's characteristics.
  • The approach emphasizes data-driven modeling techniques.
# 18:24

Simulation and Data Generation

How is data generated for the harmonic oscillator simulation?

Data is generated by solving the PDE of the harmonic oscillator, applying Fourier transforms, and visualizing the results through various plots to analyze the system's behavior.

  • Fourier transforms are used to analyze the harmonic oscillator's potential and dynamics.
  • Visualization techniques help in understanding the simulation results.
  • The process involves generating data that will inform the construction of the reduced order model.
# 24:32

Efficiency of Reduced Order Models

What are the benefits of using reduced order models in simulations?

Reduced order models significantly decrease computational complexity, allowing for simulations with fewer modes while maintaining accuracy, resulting in substantial savings in computational resources.

  • Reduced order models can achieve significant reductions in computational requirements.
  • Using only a few dominant modes can replace a much larger number of modes without loss of accuracy.
  • This efficiency is crucial for practical applications in complex systems.

Transcript

0:06 we're gonna move on now in this lecture to talk about how to build or how to think about constructing optimal basis elements for building reduced order models this is out of chapter eleven eleven point two out of the book data-driven science and engineering by Brundtland cuts you can go to date of book u-dub comm has all the code i'm going to work with today plus all a bunch of the notes on this so you can find more details there so so we were

0:39 talking about this concept of the reduced order model which is to take a typically what we're looking at as partial differential equation whether it's a time dependent system or it can be in fact a time independent PD as well but the idea is to think about how would you take the computational engine which is typically discretizing some PDE into a large number of discrete points and then solving each of those as an Eau de right which is very high dimensional and

1:09 how do you construct a proxy model which is much lower dimensional much easier to simulate so that you can do instead of computations at size n you can do computations at size R where R is the rank of the system we want to be working with so new sort of modeling is all about trying to find that optimal basis in which I can represent my solution and build models in that's but subspace okay

1:40 so how we're gonna do this well first want to come back to our generic method I want just a reminder here which is we were thinking about systems of this form u T equals L u plus some known in your terms okay can be parama tries by some parameter let's say beta so I have this PDE and what I really want to do with this PDE is to think about solving this numerically using some modal expansion

2:05 given by here so I want to do some kind of at least for the time dependent problem some galerkin projection on to some set of modes 5k where somehow I want to these modes to somehow some nice structure about the solution either and to solve this rapidly what we talked about last time is five K could be for instance even for emos where you could use the FFT to compute this so that is a very fast algorithm so even

2:33 though you're committed now to working with a of K in and differential equations they have K the question is can I still though compute that rapidly and to move into that space the a of k space rapidly and you can do that with the fast Fourier transform but it still brings up a broader question which is what what would be the optimal basis the optimal expansion to be using what elements should I use there 5k I can

3:02 certainly make them up ahead of time Fourier modes chebyshev polynomials special functions there's lots of different options the real question comes is what in fact would be optimal to use as a basis as functions now what we mean my optimal is in some sense a set of modes that are so efficient representing the solution of this that instead of using n of them I can use R of them to get very high

3:33 accuracy where R is much less than n then my proxy model and that basis set would be much smaller much simpler to simulate so here's how we're gonna do it we're gonna do it through what's called the proper orthogonal decomposition let's come back here here's my governing PDE and let's then say in fact that I do a very expensive calculation I I solve this with some basis set maybe for a maybe I do finite differences but I do

4:05 some high resolution very expensive computation its size in okay and I generate data from that so what I'm gonna do with that data is I'm going to make a matrix X here and of course I'm going to use the singular value decomposition to find the optimal low rank subspace in which the dynamics of this PDE is evolving so the way you would do this is you would take snapshots of you here I've devoted from you one you to all you whether you

4:33 of em so this is the solution you at time T one time T to all the way to time T of M so I take full state measurements line them up in a snapshot matric like this call this X and of course what I'm gonna do is a singular value decomposition of this to find the guaranteed in an l2 sense best low rank embedding of the data generated from this PDE so that's the idea and this is

5:05 called a proper orthogonal decomposition or POV method which is to say take the snapshots perform an SVD so here's what you want to do now of course we don't want the full SVD that's an expensive computation and in fact we're not gonna use most of what comes out of that what we're looking for is the low rank subspace and so let's call this X tilde u tilde so this is a reconstructing using in fact the low rank components to

5:34 rank our reconstruction of u Sigma and V so that's going to give me it's going to approximate my original data matrix X by a new data matrix X tilde but in fact in an l2 sense is guaranteed to be the optimal reconstruction if I choose to truncate at our modes and here's how I'm going to compute it so the SVD which is in fact the first chapter of the book that's all we talk about is what is the best way to build

6:03 some of these low ranks in subspaces and this is one of the best ways out they're very efficient algorithms to do it and that's that's what you're gonna get and it has some nice guarantees and properties so what's the goal the goal then is if I find this subspace the columns of U tilde there are columns of U tilde are gonna make up my side in other words I'm thinking of these u Tilda's as the basis elements in which i

6:35 want to expand my PDE previously i said let's just go ahead and expand in fact let me just go back to this slide here here you go this is the expansion we talked about this this is like a spectral expansion for solving the PD es 5k was typically chosen ahead of time some special functions for emos chebychev whatever you might want it to be but now what we're doing is we're actually using simulation data so this

7:03 is a data-driven architecture in some way we're using the simulation data to construct a set of Phi of K that are optimal where now I don't want to sum from 1 to n I want to sum from 1 to R so there's our rank subspace I'm gonna work with and I found that subspace through the SVD and I know it's optimal at least in an l2 sense and here they are then so the columns of U tilde are Phi 1 Phi 2

7:32 to 5 are they are my basis elements that I'm gonna use to solve this PDE it is an optimal subspace guaranteed in an l2 sense you cannot construct another basis element set of basis elements of rank R that are better at próxima ting the data from which you drew this subspace couple comments I just said something very

8:02 important there which is you can't find a better basis for the from which you from the data that you use to construct it the interesting thing that you're going to do now though is you're going to start doing simulations based upon this basis and you're gonna walk away potentially from the data that you just used to create this subspace right so in some sense you're gonna start doing some extrapolation with these reduced order models and extrapolation is much more

8:32 difficult than interpolation and the subspace will eventually break down that's one of the problems with taking data building a subspace you want to when you want to walk out of that subspace you don't have the modes to do it ok so what you hope to do though what you hope though is that your evolution dynamics is well captured by the subspace and the dynamic stays in the Subspace so no there's invariant in that subspace okay and you

9:01 know line on many systems that's that's that's largely true at least it's true for quite some time to the future and then eventually you have to switch back over to the high fidelity model generate a new basis set again so that's the concept we are going to take high-fidelity simulations to build a best embedding subspace for us to project the dynamics into and get out our evolution equations so here's a slide that you see that's very standard

9:34 in reduced order modeling community which is to say I have the PDE and I'm gonna do the Gherkin projection the following way I'm going to take my you field I'm gonna project it onto those modes the are rank subspace and all I want to do now is find some equations of dynamical equations for the a of T in other words how does each mode evolve in time and I know that these this has to satisfy as best as possible the

10:00 governing equations so I take this u field and I basically plug it in to the governing equations the governing PDE and then I take an inner product with that PD with each one of those modes size the net result is my low rank dynamic rused order model which is given by here so you have da DT is equal to see this here so now side transpose else I aft so this is the linear operator project it into

10:32 this subspace now the nice thing about this linear operator size transpose L sy is that you can compute it once upfront once you've computed it you're done a itself is our dimensional not n dimensional as where we started with right the solution U is n dimensional very high dimensional and the idea is to trade to a which is our dimensional where R is much less than n so the linear portion is nice because you can

11:02 do at one time cost computed up front and now this is an r by r matrix and done with it the non-linear term is much more difficult because as the solution is updated you have to go from your a space back to your use space to compute the nonlinearities and then project back down this is extremely expensive okay so if it weren't for the fact that there's some other tricks we're going to use in

11:33 fact this the cost of doing this would essentially make render this route a sort of model you know wouldn't be worth doing it might as well just stay in doing the full PD simulations but in fact we're gonna do is we're gonna use interpolation methods here and this is going to be the subject it's chapter 12 is how do we kill them over overcome this difficulty of computing these inner products in efficient and fast ways and

11:58 we do this by exploiting the low rank structure and using some greedy algorithms for doing interpolation with that and we're gonna get accurate results there so there's the two pieces of reduced order modeling it is not just about finding a low dimensional subspace it is also about computing a quickly and efficiently these non-linear products if you do not have one or the other you don't really have a real store model so those are the two pieces we have to

12:28 have chapter 11 is gonna be mostly talking about building out these subspaces in the structure of that chapter 12 is going to talk a lot about how do I interpret to make that model actually those computations fast okay so this is our reduced order model this is you know driving this from the governing equations using that very simple to do it just plug in take inner products that's the whole game this is what you get very standard representation and

13:00 that is your Gallican projective model the simplest thing you could do okay so what I want to do is actually kind of go through an example we're gonna have in this lecture what I want to do is talk about building that basis or finding that basis and so the month' example i'm gonna give i'm not gonna build a reduced order model in this lecture but the next one but what we're going to do here because I want to consider the

13:27 harmonic oscillator so this is a Schrodinger operator and the front end here and with a potential which is x squared so this actually has is this is one of the special functions that gives solutions to this where they call gauss hermite polynomials so what I'm going to do here is we're going to do a simulation of this system and as we do the simulation of the system what we want to look at is from the simulation

13:51 can I find an optimal embedding space suppose I didn't even know about Gauss to meet polynomials I just have this PDE I do a simulation what would I find as a good representation or a good basis set to be using to represent solutions to this system and what's interesting about what we're going to find is we find things that look very similar to our Gauss permeat polynomials so that's a it's encouraging sign that some of these

14:18 more data-driven methods which are making no biased assumptions about the kind of sturm-liouville problem that's sitting underneath there give you things that we've known for a long long time from mathematical physics okay so let's get to it the next piece here then is to walk through a little bit of code we'll walk through the simulation of that harmonic oscillator and show you some of the results of that okay so here we are so

14:54 this is the code what we're going to do here is I'm going to just walk you through a pity PDE build model and we're gonna walk through the pieces of it so that we can simulate the harmonic oscillator generate the data do an SVD on the simulation snapshots to see what does the low rank subspace look like so here we go so the first set of codes lines of code here first we start off with a domain size L is gonna be 30 so

15:23 I'm gonna think about going from negative 15 to 15 the total domain size will be 30 so that's right here and I chop it up into 512 points so I'm going to take a domain size of 30 chop it into 512 and so my domain will be a linear space going from negative L over 2 to L over 2 n plus 1 points and I'm gonna throw away I'm gonna keep just the first one through and I'm gonna use

15:47 periodic boundary conditions so the last point is gonna be the same as the first that's why I throw away the last point and just keep X from 1 to n n plus first point is the same as the first one so I'm going to use Fourier methods or spectral decomposition x' using fast Fourier transform to solve this so because I'm using a fast Fourier transform I divide some wave numbers K here which is 2 pi L so the Fourier

16:10 transform thinks you're working on a domain which is 2 pi but of course I've decided to work on a domain which is size 30 so I have to do a rescaling from 2 pi to L so 2 PI over L and then here my wave numbers for the Fourier transform here's my potential x squared this is the harmonic potential remembers the Schrodinger equation with x squared potential so the potential is x squared right there I'm going to solve it from

16:37 time 0 to time 20 save the results every point two units so those give me my snapshots coming out of the system now I'm gonna do two simulations here here's one block here's the second one the first one I'm going to take a Gaussian initial condition centered at x equals 1 so I have this potential well it's sitting in so the way the harmonic oscillator you think about it is it's it's modeling quantum mechanics and you

17:08 think about a potential x squared as being this potential which is where the particle would live and I'm gonna start with a Gaussian the bottom of the well and get one unit over in the bottom from the well so this one here the first simulation I Center this Gaussian at X equal 1 the second simulation i centered at x equals 0 so this is centered dead center in the well so since I'm using the FFT I'm going to take the data I'm

17:36 going to take you transform into the Fourier domain through the FFT and then run it through OD for 5 so now it's gonna step that thing forward so basically what I've done here is actually a choleric and expansion using Fourier modes okay so it but I'm using 512 for you modes and the question really is is could I do better than using for you modes for the kind of solutions that I'm gonna be looking at so I run this thing

18:05 ot for five call right hand side run it come back since I'm in the for a domain I come back right here and I undo the Fourier domain so I can look at it back into my normal XT domain okay this is my solution and this is the second solution you saw you salt to what the right hand sides look like for that is very simple right here it is so what I've got here is the right hand side which describes

18:37 this PD system which is the harmonic oscillator so what I first come in here they're bringing in UT I undo the Fourier transform I can compute here's the second derivative and here is the FFT of the potential times you remember the potential is coming in here is x squared so I have to get back to you so I can multiply by x squared then take the Fourier transform of that so that's it it's a pretty simple model it's a

19:06 fairly standard will I can there'll be other resources if you look at the bottom of the wed link there's another link to some other course work I've done on numerical methods and so this is just comes right out of of solving PDEs and i have a book for that and so that's that's directly out of that here this is just using it to generate data for us okay once we've done that I can plot these in the figures so subplots I do a

19:38 surface plot and I do some fancy visualizations of this I will come back to see what they look like but I do a surface plot but also look a topographical plot and you'll see what these things look like again this code you can play around with individuals you can play around with as you go okay but what I really want to do is take that data and let's pay attention here to you saw and you saw - those are the two

20:02 simulations I out of this thing you you sim here is I took the asymmetric start where I started one you know a takes one I just took it and I said well I started X and minus one in other words I have these two simulations that are now because it started one on one side one on the other I get symmetric I symmetrized the data okay that's this you three here but really I want to focus in on the two

20:28 simulations the SVD if you saw the SVD of you solve two and when I'm going to plot is I'm going to start plotting the SVD Diagnostics in particular we're gonna start plotting here the diagonal term of the SVD s right to look at the singular value decay and then down here look at the modes start looking at what are the modes it's pulling out remember the modes that I pull out of here and the u columns of u are telling me what

21:01 the subspace or the basis elements should look like that are in fact optimal for those simulations okay so I'm just gonna run this and show some of these results so the run time here it's 512 modes and again it's not even that big of a system running out 20 units of time this thing will basically run but this is exactly the point I'm currently running this in a for a basis with 512 for emotes what if instead I

21:35 could run this and solve this with much with a significant fewer number of modes so I'm going to show you here we blow this picture up first all right so this is what the simulations look like here it is so this is the evolution dynamics wherein it's starting in the center here's the evolution dynamics when it's starting off center so notice that if you start off center in this well it starts to oscillate in the well okay and

22:06 in fact back there this line is the potential x squared that's sitting around for us so so this this thing ran the code again is available directly from the website so you can just run this play around with it's gonna create some data what I want to do is show this data back into the slides because it's a little bit nicer to look at when we're back in the slides so let me come back to here

22:33 that's the code and let me just show you some of the results so here are some of the results of this again this is just the simulation where I do the simulation where it's centered in the domain and offset by one so these are these two simulation cases now what you're looking at here and the two bottom plots is the Sigma matrix in the SVD right which is telling me the variance contained within each of the modes so what's interesting

23:05 that you see remember I did a simulation with 512 modes and yet in these basis elements the first mode want to do the simulation the first mode has whatever about 70% of the variance second mode has about 20% and then down here five eight percent and drops off in fact you know by five modes four or five modes it captures this entire dynamics here I used 512 and this is telling me that I

23:36 could have used if I use the right modes five modes six modes something like this if I start off center notice what happens I get a different modal distribution so these modes here are particular to that data these modes here are particular to this data and but notice here it's a little bit the first mode doesn't have as much variance but the tail is a bit fatter and broader okay which makes sense you have an

24:07 asymmetric data set over here and this is symmetric so it's a little bit simpler it's it's lower rank that's one way to think about this data versus that data over there so look what are these mode what are these modes look like remember this is this is the entire game of reduced order Mon languages I'm gonna do some high fidelity simulation in this case I mean this is these are very simple model right now but my high fidelity

24:32 simulation was with 512 4e modes to give me some very accurate solution that I have here but then once I get it and I take the SVD I find that in fact I have very few modes that really matter and I could use those modes instead of the 512 so imagine using in this case using the 5 or 6 modes so I could do a solve a 5 or 6 dimensional OTE system versus of 512 right so this is significant savings

25:03 this is two orders of magnitude reduction in this very simple model of a simulation engine so let me show you the modes look like and here they are so let me walk through this because this is very interesting so first of all at the top is not a simulation at the top are the gauss hermite polynomials that come out of Sturm móvil theory in fact the interesting thing about these Sturm level calculations is this is well known

25:32 quite the harmonic oscillators one of the canonical most famous examples that they typically give in a quantum mechanics course so every undergrad who has you know intro to quantum mechanics does the harmonic oscillator knows that these solutions exist there the Sturm Louisville solutions and they're in some sense the ideal representation of the dynamics to the right they're the eigenfunctions of that system and so that's the way quantum town mechanics thinks often about this is that you know represent your dynamics

26:04 of the quantum system in terms of the Tigan functions and there you go that's what they look like so that is in fact analytic theoretical constructions that we've known for nearly a hundred years maybe more I don't exactly know when the guests are meet were around but it's it's one of the canonical examples of a Sturm local problem now let's talk about what's next what's next is these are the dominant first five modes for the data

26:35 when you start centered right so this is just this thing breathing like this it's centered and notice mode one here mode one there now what's interesting about these modes none of them are asymmetric and that's because I gave it a symmetric initial condition so this is actually pointing out one of the failings that can happen in a reduced order modeling or an SVD based method which is it can only give you things that's seen in the data if it

27:05 never saw asymmetric dynamics none of these modes will have asymmetry either okay so it's a limitation so that this is about interpolation and extrapolation I made a big deal of that in sections chapters 4 & 5 when you take these snapshots of the data the modes you're gonna get out the feature space if you can build is only particular to that data the minute you start to walk

27:36 outside of that data you cannot extrapolate out in other words you can't extrapolate to an asymmetric mode from the modes that you have here because it's only sin seen symmetric dynamics regardless the structure of these modes here in fact these modes basically represent the first five symmetric modes that you get from theoretical considerations of the Gaussian hermit polynomials and you can convince yourself if you stare at these

28:08 long enough that that's true the second panel here of simulations so this one here this one is now the SVD where I've started the simulation off center slightly and notice that here you do get asymmetric modes I want you to notice so for instance mode two is slightly asymmetric there's there's here's a peak here here's bigger peak there notice the blue one the cyan one is also asymmetric

28:39 too in fact it looks somewhat like mode two up here and so this starts to actually pick up many of the canonical shapes that you would get if you had a symmetry or in other words it starts some variants of these match the asymmetric modes you would have from Sturm Louisville theory or the gauss hermite polynomials so this is very encouraging in the sense that one of the way to think about it is if I had several simulations for a

29:06 different you know symmetric and asymmetric I could start building modal sets that would expand span all of those and because if I start off-center I would get the asymmetric modes I would in fact potentially form a nice basis maybe even if enough simulation something that looks almost complete but also much lower dimensional right so the idea here is to trade out this 512 for e mode expansion to a basis set which is

29:39 much more natural to the system in fact optimal to the system these don't look like 4 e modes at all the only reason I use Fourier decomposition to do the simulations because I've n log n speed and spectral accuracy and this is of one of the standard solution techniques or numerical PDEs but notice that when I do this I can now move to a basis set that is optimal for that data itself and that is the idea behind the reduced order

30:08 modeling framework so I'll follow up in the next lecture with actually building a full-on ROM and then not only finding some space but then doing the direction projected system that comes out of that more can be found here data book u-dub comm you can find the notes here this is the PDF code this code that I used here it's all there so knock yourself out code try some simulations

Summary

The lecture discusses constructing optimal basis elements for reduced order models (ROMs) using proper orthogonal decomposition (POD) from high-dimensional partial differential equations (PDEs). The goal is to create a low-dimensional proxy model that retains essential dynamics while reducing computational complexity, allowing for efficient simulations.

- Reduced order modeling aims to simplify high-dimensional PDEs by finding an optimal low-dimensional subspace for simulations.
- Proper orthogonal decomposition (POD) is employed to extract the most significant modes from simulation data using singular value decomposition (SVD).
- The optimal basis elements are derived from high-fidelity simulations, allowing for accurate representation of the solution with fewer modes.
- The lecture emphasizes the importance of capturing the dynamics within the subspace and the challenges of extrapolating beyond the data used to construct the model.
- The harmonic oscillator is used as an example to illustrate the process, showing how simulation data can yield basis functions similar to known solutions (e.g., Gauss-Hermite polynomials).
- The effectiveness of the reduced order model depends on the ability to compute nonlinear terms efficiently, which will be addressed in subsequent discussions.
- The results indicate that a small number of modes can capture the majority of the variance in the data, demonstrating significant computational savings.

Questions Answered

What is the concept of reduced order models and how are they constructed?

Reduced order models aim to simplify complex systems, typically represented by partial differential equations (PDEs), into lower-dimensional representations. This involves constructing an optimal basis for the solution space, allowing for efficient simulations.

How are the basis elements for reduced order models determined?

The basis elements are derived from simulation data, utilizing techniques like Singular Value Decomposition (SVD) to identify an optimal subspace that minimizes error in approximating the data.

What is the example used to illustrate the construction of a basis for reduced order models?

The harmonic oscillator is used as an example, where the goal is to find an optimal embedding space through simulation data, potentially without prior knowledge of the system's analytical solutions.

How is data generated for the harmonic oscillator simulation?

Data is generated by solving the PDE of the harmonic oscillator, applying Fourier transforms, and visualizing the results through various plots to analyze the system's behavior.

What are the benefits of using reduced order models in simulations?

Reduced order models significantly decrease computational complexity, allowing for simulations with fewer modes while maintaining accuracy, resulting in substantial savings in computational resources.

© transcribe · For agents Built with care and craft by Gokul Rajaram