[LAU] [Sort of OT] Ringing in filters

[Mark Knecht]

Hi Fons

On Tue, May 11, 2010 at 4:19 PM, Fons Adriaensen <fons at kokkinizita.net> wrote:
> On Tue, May 11, 2010 at 03:36:08PM -0700, Mark Knecht wrote:
>
>> Actually though, as a test of an idea I had and just because I've got
>> the data here, I'm using stock market data, decomposing the data into
>> bands of energy if you will, and then summing the bands back together
>> to see how close I come to the original data. It works fairly well
>> most of the time, but stock data can have big directional moves at
>> times which appears as a big transient event. When those events
>> complete then the slower filter bands ring and cause large
>> displacements in the output.
>
> So you want a set of filters with the following properties:
>
> 1. each filter has a frequency response with a shape that allows
> an intuitive interpretation as a 'bandpass', i.e. a range of
> frequencies, and
>

Yes, a specific (small) range of frequencies and some ability to see
if the energy meets some minimum level of interest.

> 2. the sum of the filter outputs equals the original input.
>

This is not really a requirement, but if they did then I'd have a more
visual demonstration that the process was working.

> This is a *hard* problem. Basically none of the classical filter
> types have these properties. There are filter sets that can do
> this but they are quite esoteric.
>
> 'Ringing' is not really a problem here, each of the individual
> filters may ring, as long as this is cancelled by the others
> when you add the outputs.

Not sure I totally agree with this as it assumes that in the end I
listen to the reconstituted sum of all the filters. On the other hand,
if I listen to only a few then without the other 'negative'
contributions I end up hearing the ringing in the few I'm listening
to.

>
> An FFT will provide a set of filters having property (2), but
> each filter has a sin(x)/x shape, which is not really a bandpass.
> You can 'improve' the shape by windowing, but that destroys (2).
> And anyway using an FFT like this is a 'block' operation - how
> a sample is treated depends on its position within the block.
> Windowing requires overlapping blocks, and they won't add up
> to the original input.
>

I think also that an FFT assumes some sort of longer-term steady state
repetitive nature which isn't in the stock data. I could take a
representative sample - the last 1000 bars, and then treat that as if
it was a repeating cycle and extract frequencies. I'm not sure what
that will do for me and the sample points are likely to introduce lots
of new artifacts. Still, worth thinking about.

<SNIP>
>
> Now if the purpose of the filtering is to extract 'features' of
> the data, you really don't need property (2). And given the
> nature of stock market data, I suspect that classic audio filters
> are not the right way to extract such features. I'd have a look
> at wavelet filters for example.
>
> Ciao,
>

Will do. Thanks.

Cheers,
Mark