Finding Periods or Periodicity in Time Series Data

[ghost]

Hi guys,
I think this is a superb, amazing work. but I don't still fully get it.
For example, if you check the Taxi notebook example I, I do not understand why there are no low data points suggesting a 24 hours periodicity (or 1 day, with an m=48).

[seanlaw]

@BotasB Thank you for the question. Can you elaborate on what you mean by "no low data points"?

When I look at the plots, many of the values in the matrix profile are substantially less than one or close to zero. This implies that many of the subsequences have a very close/non-trivial match. This may be indicative of periodicity but is not a sufficient condition. Having said that, I'm not sure that the work by the original authors offer any hints as it relates to determining the periodicity. The goal is to discover motifs and, possibly, top-K matches (yet to be implemented).

I'm not a time series expert but you may find autocorrelation to be a better way to discover the periodicity or, better yet, straight visualization.

[ghost]

@BotasB Thank you for the question. Can you elaborate on what you mean by "no low data points"?

When I look at the plots, many of the values in the matrix profile are substantially less than one or close to zero. This implies that many of the subsequences have a very close/non-trivial match. This may be indicative of periodicity but is not a sufficient condition. Having said that, I'm not sure that the work by the original authors offer any hints as it relates to determining the periodicity. The goal is to discover motifs and, possibly, top-K matches (yet to be implemented).

I'm not a time series expert but you may find autocorrelation to be a better way to discover the periodicity or, better yet, straight visualization.

Hi. Thanks for the reply. What I meant is that in the notebook they said:

It happens to be that the 2 lowest data points are exactly 7 days apart, suggesting that in this dataset, there might be a periodicity of seven days, in addition to the more obvious periodicity of one day

I think that there should be quite a few low points exactly 1 day apart. In fact, if I take a look at the first 100 values in the Matrix, I think there are maybe just one or two of those with 1 day part. All the others seems to be 1 week apart. So, I presume the 1 week periodicity is much more evident that the 1 day one one. (I haven't done any other checked anyway, just thinking out loud).
I will read the papers more carefully this weekend.

[seanlaw]

Ahh, I think that there is a slight misunderstanding in terms of the interpretation of the matrix profile.

I think that there should be quite a few low points exactly 1 day apart. In fact, if I take a look at the first 100 values in the Matrix, I think there are maybe just one or two of those with 1 day part. All the others seems to be 1 week apart. So, I presume the 1 week periodicity is much more evident that the 1 day one one. (I haven't done any other checked anyway, just thinking out loud).
I will read the papers more carefully this weekend.

So, based solely on the matrix profile alone, one can only examine the distance between the global minima as those are the only two points that correspond to a "motif" in time series parlance. All other matrix profile values only tell you the distance to its closest neighbor but the matrix profile doesn't tell you where along the time series its closest neighbor is. That information (the where) is stored in the second column of the matrix profile which is the matrix profile index. I recommend reading the matrix profile papers (I, II, VII) found in the references and focus on the applications and discussions.

Let me know if that helps and we can close this issue.

[seanlaw]

@BotasB Feel free to reopen this issue if you have more questions