Round about vid 7 he spells it out how Kalman is making new estimates by balancing a prediction from physical law vs the new measurement inputs. The weight given to each of those being adjusted by the kalman gain. Not only that but the amount of "trust" you have in either of those is being adjusted dynamically.
So it seems my hunch above is about right.
My brain started to freeze over with the introduction of 2d and 3d kalman and all that matrix maths.
It did revive memories of things I have not thought about for decades, variance, covariance, standard deviation, heck even just matrix maths.
Aside: How do you continuously calculate a "running" standard deviation of a signal using hardly any memory, no square roots and only one multiply per sample point and of course integer maths? That is a neat little trick I saw an 8085 based control system using back 1980 or so (it had previously been implemented as just a bunch of TTL chips).
Hmm, algorithm construction is more like a servo than a filter. Difference is the feedback is internal as per IIR filters.
I got to number 5 and went huh? what about calculating Emea? He's giving an example but hasn't finished explaining ... just seems to magically fill it with the value of 4.
I think his first example using only one measurement is intentionally over simplified just to offer a flavour of what kalman is all about.
I was also wondering where he got that Emea = 4 from.
Of course one has to assume there is an error in measurement other wise there is no point in doing all of this.
Presumably Emeas is an estimate of standard deviation or variance of the input signal. An estimate of how noisy it is.
If one were doing this for real then one would have evaluated the sensor and characterized it's accuracy, resolution, noise etc. And that would set the Emeas. In this example that Emea is pretty arbitrary, does not matter much what it is.
Good point about the likeness to a servo. In this case the thing being controlled is error in estimate, Eest.
Problem comes when you want it to track a time varying signal. Kalman stubbornly tries to give the average of the whole thing !
Like this (even with "error in measurment" parameter set to 0.0001):
I'm just monkeying around trying to make sense of what I see in the first few kalman vids we were linked to above. When it comes to Kalman I feel like I have just started school, I can add but multiplying is a mystery. And why are we doing this anyway?
Actually I was just sitting here wondering what possible use this thing might be. So it would be great if you could say something about what you have in mind.
Actually I was just sitting here wondering what possible use this thing might be. So it would be great if you could say something about what you have in mind.
Sure! The overall project is an internal combustion engine ignition and fuel injection controller. The narrow view is that I'm working through different approaches to obtaining a better representation of mass air flow via the manifold pressure sensor reading. Sedate multicylinder engines are fairly easy to obtain stable, meaningful values from but one or two cylinder engines, especially performance oriented ones are a different beast, Ducati is one I'm working with right now. My existing code works very well for a generator based application but that's easy; steady load, unchanging RPM, etc. The Duc is a whole different beast. I'm trying different things - angle gating, time gating, postanalysis of detailed logs to divine a predictable pattern, etc. The factory uses an alpha-n algorithm; I'm trying to use speed density. Sometimes serendipity is in your corner....
Problem comes when you want it to track a time varying signal. Kalman stubbornly tries to give the average of the whole thing !
Like this (even with "error in measurment" parameter set to 0.0001):
Heater, that Kalman filter isn't designed to handle time varying signals. I'm pretty sure you need two Kalman filters: one to track the current position, and one to track the current velocity. If you integrate the velocity into your position with respect to time, it should track any signal whose velocity doesn't change quickly perfectly.
When you introduce more variables than just position and velocity (such as acceleration, more dimensions, and rotation), Kalman filters become a mess. That's what all the matrix stuff is about: instead of having a ton of separate Kalman filters and having to deal with all of their interrelations separately, you just feed your Kalman filter a state vector, which is a column vector where each element represents a different variable, through a single (vector capable) Kalman filter, and then multiply your state vector by a matrix describing how things influence each other. For example, if you just want to track position and velocity, you'd do x[t+1] = x[t] + v[t]*dt; v[t+1] = v[t], which can be represented by the following matrix:
There might be more to it than this, but my impression was that a multiple variable Kalman filter is just a single variable Kalman filter, except you feed a whole vector through it at once and then simulate your model for one timestep to account for the fact that the system probably changed in the manner consistent with your model. If you have a linear model, this is just a matrix multiplication. But AFAICT all this matrix stuff is just shorthand (isn't that all it ever is?) for feeding each variable through its own Kalman filter and then simulating your model for one timestep.
If you have some level of control over your system (e.g. motors), you should feed them into your model as well, to anticipate the fact that your system should react to the motors.
EDIT: There's actually more to it than this. The estimation errors are also integrated into each other. I'm not really sure how this works. I guess you would just feed your error vector through the model matrix separately to get a new error vector. I should probably watch those videos.
...that Kalman filter isn't designed to handle time varying signals.
Yep. That is the conclusion that has been slowly dawning on me. It really does want to give you the average of a steady signal. plus noise, over time. Which it does remarkably well.
The clue for me is in the line:
let currentEstimate = previousEstimate + kalmanGain * (measurement - previousEstimate)
That really wants to keep the next estimate the same as the last one, especially as the gain goes down with time. Unless there is some other input to disturb things.
The rest of you descriptions sums things up, as I am beginning to see, nicely. The matrices are indeed a shorthand for dealing with a lot of equations at the same time.
Problem is I have not done any serious matrix munging since the late 1970's early 80's. Solving systems of linear equations, matrix optics and such. I have forgotten even the basics.
Then there is the question of how those various matrices are arrived at. As you say somehow the activity on one measurement is being used to inform the error in estimate of the others. Somehow?
I too will be watch those videos through a few times!
One of my previous coworkers recommended "Introduction to Random Signals and Applied Kalman Filtering with Matlab Exercises" but you can see it's not something you're going to read in a week or two. And you might want to look for a used copy or international edition.
[In that group all of the software engineers had advanced degrees in EE since most CompSci majors aren't exposed to much signal processing. One of these engineers who is a nice guy told me that all of this is trivial. I told him not to undervalue all the years that he spent studying it, and that it's not so trivial to us muggles.]
Ewww, MATLAB, proprietary, lock-in, yet another language to learn, grumble grumble.
We don't need no MATLAB, we code up our own matrix manipulations. Did it as a teenager in the late 70's, in BASIC, must be possible to cope with it again.
"Trival", trival you say. That's why it took a Hungarian genius and Standford maths professor to come up with the idea and develop the theory.
The only I guy I ever met who might say this is trivial had a first class honors degree in control systems engineering from Oxford. (Last heard of developing the algorithms for the active suspension system on Williams F1 cars, until it was banned as being too effective and an unfair advantage:) )
Aside: Turns out Kalman died in 2016.
Other aside: After watching up to vid number 20 in the Kalman series we still have not "closed the loop" and arrived at a full, working, multi-dimensional Kalman filter! Trival indeed.
Still all those matrices that are now jumping in my dreams are starting to look very beautiful!
Octave is a good alternative to MATLAB - and it's free. For the sort of matrix operations and visualizations that 90% of MATLAB users employ, Octave works just the same.
Official Dartmouth BASIC back in 1968 did have some matrix operations - things like MAT A = B + C where A, B, and C were arrays - and MAT PRINT to display them. Later dialects of BASIC common on 8-bit home computers dropped the matrix parts though.
I was wondering about those MATLAB clones. I guess I could make life easier for myself here by getting into such things. Those young undergrads today make these things "trival" by using such tools. I'm old school, we can do it with regular languages. Besides I'm not much into learning yet another fíng language just now. And MATLAB solutions ain't going to run on a Prop or whatever, places where you actually need such a filter.
I don't know what the BASIC was that we used in technical college in '74, on some ICL computer far away and connected by 300 baud modem, it had no MAT. As we were studying numerical analysis and such at the time we had to slog through writing all the matrix manipulation routines for ourselves. It was part of the introduction to programming.
Comments
So it seems my hunch above is about right.
My brain started to freeze over with the introduction of 2d and 3d kalman and all that matrix maths.
It did revive memories of things I have not thought about for decades, variance, covariance, standard deviation, heck even just matrix maths.
Aside: How do you continuously calculate a "running" standard deviation of a signal using hardly any memory, no square roots and only one multiply per sample point and of course integer maths? That is a neat little trick I saw an 8085 based control system using back 1980 or so (it had previously been implemented as just a bunch of TTL chips).
I got to number 5 and went huh? what about calculating Emea? He's giving an example but hasn't finished explaining ... just seems to magically fill it with the value of 4.
I was also wondering where he got that Emea = 4 from.
Of course one has to assume there is an error in measurement other wise there is no point in doing all of this.
Presumably Emeas is an estimate of standard deviation or variance of the input signal. An estimate of how noisy it is.
If one were doing this for real then one would have evaluated the sensor and characterized it's accuracy, resolution, noise etc. And that would set the Emeas. In this example that Emea is pretty arbitrary, does not matter much what it is.
Good point about the likeness to a servo. In this case the thing being controlled is error in estimate, Eest.
let exponentialAverage = 0; const alpha = 1 / 8; let boxCarAverage = 0; let boxCarBuffer = new Array(16).fill(0); const boxCarLength = 16; let n = 0; let trueValue = 100; // Simulate an input signal with noise. function getSample() { //trueValue = 100 + Math.sin(n/32) * 50; return (trueValue + ((Math.random() - 0.5) * 2) * 10); } let kalmanGain = 0; let errorInEstimate = 2; let errorInMeasurement = 4; let previousEstimate = 0; function kalmanFilter(measurement) { // Calculate the kalman gain kalmanGain = errorInEstimate / (errorInEstimate + errorInMeasurement); // Calculate current estimate let currentEstimate = previousEstimate + kalmanGain * (measurement - previousEstimate); previousEstimate = currentEstimate; // Calculate current error in estimate errorInEstimate = (1 - kalmanGain) * errorInEstimate; return(currentEstimate); } function exponentialFilter (sample) { // Calculate exponential average. exponentialAverage = (sample * alpha) + (exponentialAverage * (1 - alpha)); return (exponentialAverage); } function boxCarFilter(sample) { // Calculate "true" boxcar average boxCarBuffer[n % boxCarLength] = sample; let boxCarSum = 0; for (let i = 0; i < boxCarLength; i += 1) { boxCarSum = boxCarSum + boxCarBuffer[i]; } boxCarAverage = boxCarSum / boxCarLength; return(boxCarAverage); } while (1) { // Read a sample. let sample = getSample(); // Apply different filters and print results console.log(n, sample, exponentialFilter(sample), boxCarFilter(sample), kalmanFilter(sample)); n += 1; if (n === 400) { break; } }The kalman produces by far the best estimate of the average value of a constant value signal plus noise:That makes them stable but slow. The kalman gets to a usable result far quicker and is smoother.
Like this (even with "error in measurment" parameter set to 0.0001):
-Mike R...
Gosh, thank you for saying so.
I'm just monkeying around trying to make sense of what I see in the first few kalman vids we were linked to above. When it comes to Kalman I feel like I have just started school, I can add but multiplying is a mystery. And why are we doing this anyway?
Actually I was just sitting here wondering what possible use this thing might be. So it would be great if you could say something about what you have in mind.
Heater, that Kalman filter isn't designed to handle time varying signals. I'm pretty sure you need two Kalman filters: one to track the current position, and one to track the current velocity. If you integrate the velocity into your position with respect to time, it should track any signal whose velocity doesn't change quickly perfectly.
When you introduce more variables than just position and velocity (such as acceleration, more dimensions, and rotation), Kalman filters become a mess. That's what all the matrix stuff is about: instead of having a ton of separate Kalman filters and having to deal with all of their interrelations separately, you just feed your Kalman filter a state vector, which is a column vector where each element represents a different variable, through a single (vector capable) Kalman filter, and then multiply your state vector by a matrix describing how things influence each other. For example, if you just want to track position and velocity, you'd do x[t+1] = x[t] + v[t]*dt; v[t+1] = v[t], which can be represented by the following matrix:
[x_{t+1}] = [ x_t ] * [1 Δt] [v_{t+1}] = [ v_t ] [0 1 ]There might be more to it than this, but my impression was that a multiple variable Kalman filter is just a single variable Kalman filter, except you feed a whole vector through it at once and then simulate your model for one timestep to account for the fact that the system probably changed in the manner consistent with your model. If you have a linear model, this is just a matrix multiplication. But AFAICT all this matrix stuff is just shorthand (isn't that all it ever is?) for feeding each variable through its own Kalman filter and then simulating your model for one timestep.
If you have some level of control over your system (e.g. motors), you should feed them into your model as well, to anticipate the fact that your system should react to the motors.
EDIT: There's actually more to it than this. The estimation errors are also integrated into each other. I'm not really sure how this works. I guess you would just feed your error vector through the model matrix separately to get a new error vector. I should probably watch those videos.
The clue for me is in the line: That really wants to keep the next estimate the same as the last one, especially as the gain goes down with time. Unless there is some other input to disturb things.
The rest of you descriptions sums things up, as I am beginning to see, nicely. The matrices are indeed a shorthand for dealing with a lot of equations at the same time.
Problem is I have not done any serious matrix munging since the late 1970's early 80's. Solving systems of linear equations, matrix optics and such. I have forgotten even the basics.
Then there is the question of how those various matrices are arrived at. As you say somehow the activity on one measurement is being used to inform the error in estimate of the others. Somehow?
I too will be watch those videos through a few times!
[In that group all of the software engineers had advanced degrees in EE since most CompSci majors aren't exposed to much signal processing. One of these engineers who is a nice guy told me that all of this is trivial. I told him not to undervalue all the years that he spent studying it, and that it's not so trivial to us muggles.]
We don't need no MATLAB, we code up our own matrix manipulations. Did it as a teenager in the late 70's, in BASIC, must be possible to cope with it again.
"Trival", trival you say. That's why it took a Hungarian genius and Standford maths professor to come up with the idea and develop the theory.
The only I guy I ever met who might say this is trivial had a first class honors degree in control systems engineering from Oxford. (Last heard of developing the algorithms for the active suspension system on Williams F1 cars, until it was banned as being too effective and an unfair advantage:) )
Aside: Turns out Kalman died in 2016.
Other aside: After watching up to vid number 20 in the Kalman series we still have not "closed the loop" and arrived at a full, working, multi-dimensional Kalman filter! Trival indeed.
Still all those matrices that are now jumping in my dreams are starting to look very beautiful!
Official Dartmouth BASIC back in 1968 did have some matrix operations - things like MAT A = B + C where A, B, and C were arrays - and MAT PRINT to display them. Later dialects of BASIC common on 8-bit home computers dropped the matrix parts though.
I don't know what the BASIC was that we used in technical college in '74, on some ICL computer far away and connected by 300 baud modem, it had no MAT. As we were studying numerical analysis and such at the time we had to slog through writing all the matrix manipulation routines for ourselves. It was part of the introduction to programming.