Smoothing Scatterplots

I found an implementation for well known algorithms in the book Exploratory Data Analyis with Matlab. I thought I would like to show you, how to do this in Euler Math Toolbox.

First you have to download the data from the book site. The format turns out to be useful Matlab only. But you can export the data from Matlab into raw files in the current directory with

>> load environmental
>> dlmwrite('oz.dat',Ozone)
>> dlmwrite('temp.dat',Temperature)

After, that you can import the data in EMT. Generate a notebook in the directory and simply run.

>oz=readmatrix("oz.dat")';
>temp=readmatrix("temp.dat")';
>plot2d(temp,oz,>points,style=".."):

We want to run the „loess“ algorithm on this. According to the book, this algorithm is named after the geological sediment Löss, which looks like a plot of the smoothing algorithm. It may also simply be a shortcut for „local regression“.

The algorithm takes a number of data neighbors of a given point x, and performs a polynomial fit on these neighboring data points only. To emphasize the point x, it uses a weighted polynomial fit to get a polynomial p, which of course depends on x. The weight in the book is

\(w(t) = \left(1-t^3\right)^3\)

The parameter t is the absolute distance to x scaled to [0,1], so that the neighbor with maximal distance gets t=1. Then it takes p(x) as the smoothing value. The implementation in EMT is

>function map loess (x;xd,yd,a=0.5,n=1) ...
$  {vx,i}=sort(abs(xd-x));
$  m=cols(xd)*a; xd=xd[i][1:m]; yd=yd[i][1:m];
$  d=abs(xd-x);
$  w=(1-(d/max(d))^3)^3;
$  return polyval(polyfit(xd,yd,n,w),x);
$endfunction

After that, we can add the plot of the smoothing function.

>plot2d("loess(x,temp,oz,0.5,1)",>add):

loess-002

In case, you want to try yourself, here are the data:

temp = ..
 [67,  72,  74,  62,  65,  59,  61,  69,  66,  68,  58,  64,  66,  57,
 68,  62,  59,  73,  61,  61,  67,  81,  79,  76,  82,  90,  87,  82,
 77,  72,  65,  73,  76,  84,  85,  81,  83,  83,  88,  92,  92,  89,
 73,  81,  80,  81,  82,  84,  87,  85,  74,  86,  85,  82,  86,  88,
 86,  83,  81,  81,  81,  82,  89,  90,  90,  86,  82,  80,  77,  79,
 76,  78,  78,  77,  72,  79,  81,  86,  97,  94,  96,  94,  91,  92,
 93,  93,  87,  84,  80,  78,  75,  73,  81,  76,  77,  71,  71,  78,
 67,  76,  68,  82,  64,  71,  81,  69,  63,  70,  75,  76,  68]
oz = ..
 [41,  36,  12,  18,  23,  19,  8,  16,  11,  14,  18,  14,  34,  6,
 30,  11,  1,  11,  4,  32,  23,  45,  115,  37,  29,  71,  39,  23,
 21,  37,  20,  12,  13,  135,  49,  32,  64,  40,  77,  97,  97,  85,
 10,  27,  7,  48,  35,  61,  79,  63,  16,  80,  108,  20,  52,  82,
 50,  64,  59,  39,  9,  16,  122,  89,  110,  44,  28,  65,  22,  59,
 23,  31,  44,  21,  9,  45,  168,  73,  76,  118,  84,  85,  96,  78,
 73,  91,  47,  32,  20,  23,  21,  24,  44,  21,  28,  9,  13,  46,
 18,  13,  24,  16,  13,  23,  36,  7,  14,  30,  14,  18,  20]

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.

Diese Website verwendet Akismet, um Spam zu reduzieren. Erfahre mehr darüber, wie deine Kommentardaten verarbeitet werden.