Relaxing Matrix Profile

[NimaSarajpoor]

I think it might be a good idea to come up with a method to relax the Matrix Profile. (I have been thinking about this idea for a while. I haven't got any conclusion yet. I am still thinking about it and I thought it might be a good idea to share it.

By the term "relax", I mean to consider close neighbors and not just closest data point. For instance, let's say we have seven data points where each data point is a Time Series of length two as: (x, y)

Let's say the distance between the two blue data points is 0.01 and the between any two red ones is 0.011. The matrix profile detect the two blue data points as the motif, but in my opinion, the 5 red ones are motifs and those two data points can be considered as outliers. Although I might have exaggerated here, similar cases might appear in some applications.

I appreciate any input regarding this idea.

[seanlaw]

I think @mexxexx has done a lot of wonderful recent work on motif discovery beyond the top-1 motif (coming out in our v1.9.0 release). You may be able to learn more about it in motifs.py and @mexxexx is working on a Motif Discovery Tutorial for it. @ninimama would you mind taking a look and also digging through the PR process for context? I'd be happy to help answer any questions.

[NimaSarajpoor]

@seanlaw
Thanks for letting me know. I will take a look at it after I take care of Time Series Chain.

[JaKasb]

The generalized solution is a k-NN search.
However the Matrix Profile is the result of 1-NN search.
The extension from 1-NN to k-NN search is currently a research topic.
The SCAMP repo contains an experimental method for k-NN.

I came up with a different solution, and my paper for k-NN matrix profile is currently under peer-review.

Extending from 1-NN to k-NN is somewhat straightforward.
The hard part is the interpretation of k-NN output for sliding window subsequence search.

[NimaSarajpoor]

@JaKasb
Interesting... I think,for instance, if you consider k>1, the number of CHAINs can exponentially increase. If I remember correctly, I tried chain for k=2 on my data a few months ago and I got a lot of chains (which makes sense) but how to analyze them was a headache for me at that time so I just left the idea.

Hope to see your paper published soon.