DETAILED DESCRIPTION OF EMBODIMENT

[0033] Selecting a Time Series Data Set

[0034] In this embodiment, a plurality of time series data sets is analyzed in order to identify one or more data sets for further analysis. Referring now to FIG. 1 , which is a flowchart of the major analysis steps, in step 100 each of the data sets is analyzed to determine if the set is strongly trending. For each set, the last M ₁ data elements are extracted from the time series, and a line is fit through the points using multiple regression. In this example, M ₁ is selected to be about 30 data points, or approximately 1.5 months of data. The normalized slopes and correlation coefficients are recorded. Upon such analysis of each of the data sets, a data set is selected for further analysis. When identifying financial instruments to trade, a time series that has a large increase or decrease in normalized slope with a correspondingly large absolute value of the correlation coefficient would be a likely candidate for subsequent processing. Time series data sets that have a small slope would generally be avoided.

[0035] Referring now to FIG. 2 , which is a more detailed flowchart of the data set selection steps, the initial search is performed through the following steps for each time series data set i:

[0036] In step 110 , each time series data set, y _1,j , j=1, N ₁ , where N ₁ is the number of elements in time series, is loaded into the processor. Note that a mathematical transform of a time series is also a time series (i.e. y _i,j ⁿ =ln(y _i,j ), j=1,N _i is also considered to be a time series).

[0037] In step 120 , an initial window size M ₁ , is read by the processor, and the last M ₁ data points from the time series, y _i,j =y _{i,N 1 −M 1 +j} , j=1,M ₁ , are selected.

[0038] In step 130 , the series of the last M ₁ data points is normalized by dividing each data point by y _i,1 , so that the first data value is 1.0. The normalized time series is designated as y′, where y′ _i,1 =1.0. The normalization is performed in order to permit sorting of the data sets on a consistent basis, such as the slope of a line fit through the last M ₁ data points. Alternate methods of normalization may be used such as: a) dividing the time series its mean or median; b) ranking the values in the time series and using the ranks instead of the time series values themselves in the regression; or c) calculate the Fourier transform of the time series, zero the zero-th frequency contribution, and back-transform the series back to the time domain.

[0039] In step 140 , a line, y′ _i,j =a′ _i x _i,j +b′ _i , is fit through the M ₁ data points (x _i,j ,y′ _i,j ) j=1,M ₁ , x _i,j =j using linear regression. The slope of the line, a′ _i , is stored on the processor so that it may be used as a time series data set selection criterion. Alternate methods of data fitting may be used such as higher order polynomial regression or nonlinear regression. In this embodiment, the average of the first derivatives would be stored on the processor as a′ _i .

[0040] In step 150 , if there are remaining time series data sets, steps 110 - 140 are repeated. After each data set has been analyzed in steps 110 - 140 , the results are reviewed in step 160 . In this embodiment, the time series data sets are sorted according to the normalized slopes; a time series data set is selected for further processing and loaded in the processor in step 170 .

[0041] Alternate methods of data set selection may be used such as analyzing both the trend and positive or negative fundamental information. For example, a stock that has both an increasing trend, and a low price to earnings ratio may be a more desirable candidate than an overpriced stock whose trend is just as strong. Similarly overpriced stocks may become good candidates for selling short once they start trending down.

[0042] Generating the Trend Determination Parameters

[0043] Referring again to FIG. 1 , once a time series data set has been selected and loaded in the processor, the trend determination parameters are selected at step 200 . Inherent in this step is a processing of a portion of the time series to derive the parameters. The Initial Window Size (m ₁ ) is the size of the first window that is the minimum number of data points needed to define a trend. The Deviation Limit (L _s ) is the number of deviations away from the trend that the next day's closing price or additional data value must be to signal a new trend. The deviations are defined as a measure of the spread of a distribution. Common statistical measures of spread of a distribution are variance, standard deviation, inter-quartile range, and range between the minimum and maximum values. In this invention, L _s refers to any and all measures of spread of a sample of values.

[0044] Each set of trend determination parameters in combination with the particular data fitting procedure, such as polynomial regression, produces a unique set of trends for a time series data set. At this step of the method, many different sets of trend determination parameters preferably are used to produce corresponding sets of trends. These multiple sets of trends are then evaluated to determine which of the sets are better at matching the current data from the time series data set.

[0045] The trend determination parameters can be selected in several ways. In this embodiment, a subset of the time series, the first M ₂ data points of the time series (where N _i is the number of data points in the entire time series data set under evaluation) are selected for defining the trend determination parameters. However, the time series is not limited to this embodiment, and trend determination parameters can be derived from any time series or one or more simulations of time series. In this embodiment, the parameter selection is manual so as to allow the selection of more than one set of parameters to generate multiple optimum trends with different trend lengths for a given time series as will be discussed in FIG. 6 .

[0046] Referring now to FIG. 3 , which is a detailed flowchart of the procedure for deriving the trend determination parameters. A set of trend determination parameters is generated in step 200 by looping in increments between minimum and maximum values for each of the trend determination parameters. In this embodiment, two trend determination parameters are used—the Initial Window Size (m ₁ ) which is evaluated between a starting value M1MIN and an ending value M1MAX in an increment M1INC; and Deviation Limit (L _s ) which is evaluated between a starting value LSMIN and an ending value LSMAX in an increment LSINC. Typical values for these variables are:

[0047] M1MIN=3

[0048] M1MAX=70

[0049] M1INC=1

[0050] LSMIN=1

[0051] LSMAX=20

[0052] LSINC=0.5

[0053] The trend determination minimum, maximum, and increment parameters are determined through experience. If either of the selected trend determination parameter values is equal to M1MAX or LSMAX for a selected trend set, then this would indicate the need to increase the LSMAX or M1MAX values. The M1INC and LSINC are selected with a tradeoff between accuracy and time. M1INC cannot be smaller than 1. The smaller the values for the increments, M1INC and LSINC, the longer it takes to process all of the trends. However, with smaller increments, there is an increased likelihood of obtaining trend parameters that produce more accurate trends. The lower bound Initial Window Size, M1MIN, is limited by the order of the polynomial that one is using for the trend. For a linear trend (a line) one needs at least two points to fit a line. For second-order polynomials, one needs three points to fit a curve to the data points.

[0054] In step 200 , these values are generated in a nested loop resulting in approximately 2,500 sets of trend parameters. Referring again to FIG. 1 , it is usually desirable to process these sets in step 400 as the values are generated. For each parameter set, the following values are calculated and exported:

[0055] the number of trends in the subset of the time series (N _Trends );

[0056] the RMS Error between the input data values and trend values;

[0057] the average length of the trends;

[0058] the average trend length/m ₁ , where m ₁ is the minimum number of data points needed to define a trend;

[0059] the average return (%) of the trends;

[0060] the cumulative return (%) of the trends;

[0061] the fraction of correct predictions, where the initial trend was increasing and the ending price was higher than the initial price, or the initial trend was decreasing and the ending price was lower than the initial price at the beginning of the trend;

[0062] the fraction of incorrect predictions, where the initial trend was increasing and the last price was lower than the initial price, or the initial trend was decreasing and the last price was higher than the initial price;

[0063] the RMS Error/(Average length of the trends/m ₁ ); and

[0064] and the RMS error *L _s

[0065] the efficiency of the trends (%), where efficiency is defined as the average return of the trends (%)/average length of the trends;

[0066] compounded return of the trends (%)

[0067] Each of these values is stored in the computer with its corresponding set of trend determination parameters (m ₁ and L _s ).

[0068] Selecting a Set of Useful Trend Determination Parameters

[0069] Referring again to FIG. 1 , at step 300 the values for each of the sets of trend determination parameters are evaluated in order to select one or more useful set of trend determination parameters for a particular time series data set. This step is based on the unexpected observation that trends, based on a given set of determination parameters, that fit one portion of the time series well also tend consistently to fit subsequent portions of the time series as well.

[0070] In a semi-automated method of this embodiment, the RMS error is crossplotted against the average trend length and a subset of the more effective parameter sets are identified from that plot. Referring now to FIG. 6 , which is a plot of the product of the RMS error and the deviation limit, L _s , versus the average trend length, the lower line 705 represents a fit of a curve through the trend determination parameter sets that have long trends and low error. The points along or near this curve represent the optimum combination of low error and longer trends for the time series data set. The points that are closest to that line, particularly those points corresponding to longer average trends such as points 710 , 720 , and 730 , are considered to be the most desirable for subsequent analysis of the time series data. In this embodiment, several values that have the maximum “Average Trend Length” for a minimum “RMS error * L _s ” are selected. More than one set of trend parameters is selected so that multiple trends can be generated. Generally two to three sets are selected for subsequent analysis, though other users may have different uses for the tool and may select more or less depending upon their purposes. Each trend picks up different features, and the trend produced by one set of trend parameters may be more useful during one period of time versus the trend produced by another set of trend parameters.

[0071] In other embodiments, other types of error measures such as other λ _p norms, where p is a real number, may be used. For example, the λ ₁ norm is very commonly used for robust estimation in problems that have large outliers (Tarantola, Albert, Inverse Problem Theory: Methods for Data Fitting and Model Parameter Estimation Elsevier, N.Y., 1998.). There are numerous other measures of error that can be employed based on probability distributions and are not limited to λ _p norms. These include those based on a Lorentzian distribution, Andrew's sinc, and Tukey's biweight (Press, William H., Brian P. Flannery, Saul A. Teukolsky, and William T. Vetterling, Numberical Recipes in C: The Art of Scientific Computing , Cambridge University Press, 1989.). In some embodiments, this semi-automated step of visual examination of the results can be replaced by an automation of the process by establishing one or more objective functions. In the case of this type of automation, there are numerous, well-known optimization techniques that may be employed to find good parameter sets without the exhaustive generation of parameter sets described in the section above.

[0072] Processing the Time Series with Trend Determination Parameter Sets

[0073] Referring again to FIG. 1 , when one or more trend determination parameter sets are selected, they are applied to process the time series and to generate the trends at step 400 . The result of this step is a set of trends for each trend determination parameter set as each new data point is considered for the time series.

[0074] FIG. 4A is a detailed flowchart of the processing of a time series at step 400 for determining whether or not a new data point represents a continuation of a trend. For each selected trend determination parameter set (m ₁ , L _s ), the latest trend is updated with each new sequential value of data for the time series data set. In this embodiment, a trend is described by a best-fit polynomial curve through all of the data within the trend and some number of data values from the previous trend. As a consequence, the current trend is dynamic and will be updated as each value of new data arrives. Once the trend changes, the previous trends are no longer dynamic.

[0075] A new trend is defined to have begun when one of the following occurs:

[0076] The derivative of the trend changes from positive to negative or vice versa; or

[0077] The number of deviations that the difference between the last closing price or sequential data element and the predicted trend value meets or exceeds L _s , the deviation limit, and the deviation is in the opposite direction to that of the trend. If the difference meets or exceeds L _s , but the deviation is in the same direction as that of the trend, then the trend is maintained. The reason for this is to maintain longer trends. There is less confidence in shorter trends. In some applications, such as financial management, the determination that new time series data is consistent with a longer upward or downward trend is more important than whether the magnitude of that trend has changed. In other applications, it is more important that the data be fitted better by a new trend, so the trend wouldn't be maintained.

[0078] Referring to FIGS. 4A and 4B , the processing is accomplished through the following steps:

[0079] In step 410 , load the time series, y _i,j , j=1, N ₁ , where N ₁ is the number of elements in time series i.

[0080] Initialize the trend determination parameters in step 420 . The trend determination parameters are m ₁ , the first window size; L _s , the deviation limit; and m ₂ , the second window size. The first window size is the minimum number of data points needed to define a trend. In this embodiment, the deviation limit is the number of deviations away from the trend that the next day's closing price or additional data value must be to signal a new trend. The deviations are defined as a measure of the spread of a distribution. Common statistical measures of spread of a distribution are variance, standard deviation, inter-quartile range, and range between the minimum and maximum values. In this patent application, L _s refers to any measure of spread of a sample of values. In this embodiment, the second window size is equal to m ₁ for the first iteration, and is equal to n+m ₁ for subsequent iterations where n is the length of a trend. This is the number of data values at the end of the trend's time series that are used to calculate a test measure of spread.

[0081] Set j, the time series index, to m ₁ +1 in step 430 .

[0082] Set n, the number of points in the current trend to 0 in step 440 .

[0083] For each trend, check to see if the current value, y _1,j , is in the trend in steps 450 - 530 .

[0084] For the First Trend

[0085] In step 450 , select_m ₁ values from the beginning of the segment of interest in the time series.

[0086] In step 460 , calculate a best-fit curve f _j (x), through the m ₂ data points. (x _1,k ,y _1,k ), x _1,k =[k−(j−m ₂ )+1], k=(j−m ₂ ), j−1. The subscript j in f _j (x) refers to the function estimate at index j, when evaluating whether y _1,j is in the trend or not. In the preferred implementation, least squares regression is used to determine a best-fit polynomial through the data points. However, the curve in this patent application is not restricted to a polynomial function nor to a polynomial function of any particular order. The curve can be another mathematical function, such as an exponential or logarithmic function. It could also be the estimate derived from a neural network or Kalman filter. Least squares regression (λ ₂ norm) is the preferred method used to calculate the curve, f _j (x), but this is not restricted to a λ ₂ norm. The invention is patent application includes other methods that can be used to determine the fit of a curve to a set of points. This includes other λ _p norms, where p is a real number.

[0087] In step 470 , evaluate the derivative 1 $\frac{f_j\left(x\right)}{x}{|}_{x_{i,j-i}}$

[0088] at the point x _i,j−1 =m ₂ and obtain an estimated value of the curve, f _j (x _i,j ) at index j. Note that the derivative is also calculated at index j, where x _i,j =m ₂ +1.

[0089] In step 480 , calculate the correlation coefficient (R) between the pairs of values, (y _i,k ,f _j (x _i,k )), k=(j−m ₂ ), j−1. The correlation coefficient is not critical to the process, but it is useful diagnostic information that is output for further analysis.

[0090] Calculate the residuals, d _k , between the time series values, y _1,j , and the trend values, f _j (x _1,j ) in step 490 . Although it is unnecessary in most cases, it is sometimes desirable to normalize the residuals, such as by dividing by the trend values. The purpose of this normalization is to ensure homoscedasticity of the deviations. If the series is trending up or down within a single trend, then the non-normalized deviations at the beginning of the trend will have a different variance than the non-normalized deviations at the end of the trend. Alternatively, other methods of normalization may be employed, such as dividing by a moving average of the trend. 2 $\begin{array}{cc}{d}_k=\left[\frac{\left(y_{i,k}-f_j\left(x_{i,k}\right)\right)}{f_j\left(x_{i,k}\right)}\right],k=\left(j-m_2\right),j-1& \left(1\right)\end{array}$

[0091] Calculate the standard deviation, s _j−1 , of the residuals, d _k in step 500 . The standard deviation is used in this embodiment; however, it is only one of many measures of the spread of a distribution. The invention includes other measures of spread, such as the variance, range, inter-quartile range, root mean square, differences between any quantiles of a distribution, or a measure of spread from a neural network.

[0092] Calculate the deviation between the predicted trend value and the actual value, y _i,j at x _i,j in step 510 . The invention includes either normalizing by dividing by the trend value, f _j (x _i,j ), or not normalizing. Other methods of normalization may be employed, such as dividing by a moving average of the trend. 3 $\begin{array}{cc}{d}_j=\left[\frac{\left(y_{i,j}-f_j\left(x_{i,j}\right)\right)}{f_j\left(x_{i,j}\right)}\right]& \left(2\right)\end{array}$

[0093] Normalize the deviation, d _j in step 520 . 4 $\begin{array}{cc}{d}_j^{\prime }=\frac{d_j}{s_{j-1}}& \left(3\right)\end{array}$

[0094] Test to determine whether the trend has changed at step 530 . In this embodiment, the trend changes if the sign of the first derivative, d _j , has changed from the previous iteration to the current iteration (equation 4): 5 $\begin{array}{cc}\mathrm{If}\left\{\mathrm{sign}\left[\frac{f_j\left(x\right)}{x}{}_{x_{i,j-1}}\right]\ne \mathrm{sign}\left[\frac{f_{j-1}\left(x\right)}{x}{}_{x_{i,j-2}}\right]\right\},\mathrm{then}\mathrm{true}& \left(4\right)\end{array}$

[0095] The trend also changes if the normalized deviation, d′ _j , exceeds L _s in the opposite direction to that which the trend is going. In an alternate embodiment, the trend may be determined to have changed if only the absolute value of the normalized deviation exceeds the deviation limit. 6 $\begin{array}{cc}\mathrm{If}\left\{\begin{array}{c}\left[\mathrm{abs}\left(d_j^{\prime }\right)>L_s\right]\mathrm{AND}\\ \left\{\left[{(\frac{f_j\left(x\right)}{x}}_{x_{i,j}}>0\right)\mathrm{AND}\left(d_j^{\prime }<0\right)\right]\mathrm{OR}\\ \left[{(\frac{f_j\left(x\right)}{x}}_{x_{i,j}}<0\right)\mathrm{AND}\left(d_j^{\prime }>0\right)]\}\end{array}\right\},\mathrm{then}\mathrm{true}& \left(5\right)\end{array}$

[0096] If the trend does not change at step 530 , then increment n to n+1 at step 550 .

[0097] If the trend does change at step 530 , then set n=0 at step 540 .

[0098] Increment j at step 560 .

[0099] At step 570 , if j<N _i , then repeat steps 450 - 560 for the next data point in the data set. If the last point of the time series is still within the first trend, then go to step 600 and calculate trend attributes.

[0100] For Subsequent Trends

[0101] Select m ₂ values at step 450 . m ₁ of the values are from the end of the previous trend, and n values are from the current trend. Calculate the best-fit curve, f _j (x), through the m ₂ data points at step 460 .

[0102] In step 470 , evaluate the derivative 7 $\frac{f_j\left(x\right)}{x}{|}_{x_{i,j-i}}$

[0103] at the point x _i,j−1 =m ₂ and obtain an estimated value of the curve, f _j (x _i,j ) at index j. The derivative may also be calculated at index j, where x _i,j =m ₂ +1.

[0104] In step 480 , calculate the correlation coefficient (R) between the pairs of values, └y _i,k , f _j (x _i,k )┘, k=(j−m ₂ ), j−1.

[0105] Calculate the residuals, d _k , between the time series values, y _i,j , and the trend values, f _j (x _i,j ) in step 490 . Optionally, the residuals may be normalized as described above.

[0106] Calculate the standard deviation, s _j−1 , of the residuals, d _k in step 500 .

[0107] Calculate the deviation between the predicted trend value at step 510 , as described previously.

[0108] Normalize the deviation at step 520 , as described previously. Test to determine whether the trend has changed. 530 , as described previously. If the trend changes then set n=0 at step 540 . If the trend doesn't change, then increment n at step 550 . Increment j at step 560 .

[0109] At step 570 , if j<N _i , then repeat steps 450 - 560 for the next data point in the data set. If the last point of the time series has been reached (j≧N _i ), then go to step 600 and calculate trend attributes.

[0110] The procedure preferably outputs the following arrays for each trend parameter set. There are N _i elements in each array. They can either be analyzed by themselves or output to a post-processing procedure to calculate statistics on the trends as shown in FIG. 5 .

[0111] Date or index array

[0112] Data array (y _i,j ), j=1, N _i .

[0113] Data value change=ln(y _i,j /y _i,j−1 ), j=1,N _i

[0114] Trend codes array

[0115] d′, the array of normalized deviations from the trend.

[0116] Array of minimum closing prices allowed based on trend

[0117] Array of maximum closing prices allowed based on trend

[0118] Array of the first derivatives of the dynamic trend

[0119] Array of the first derivatives of the normalized dynamic trend. The data values at the beginning of the trends are normalized to one, so that the initial trend data value is one for all trends. The derivative of the trend fit to the normalized data values can be compared to the derivatives of the other trends for filtering purposes. Alternate methods of normalization may be used as described previously. This can include: a) dividing the time series values in the trend by its mean or median; b) ranking the values in the trend and using the ranks instead of the time series values themselves in the regression; or c) calculate the Fourier transform of the time series values in the trend, zero the zero-th frequency contribution, and back-transform the series back to the time domain.

[0120] Array of the first derivatives of the final trend.

[0121] Array of correlation coefficients (R) of the final trend line through the data values for each index i.

[0122] The array of final trend line values once a trend has been finalized.

[0123] An array of the differences between the data values and the trend values.

[0124] FIG. 6 represents trends fit to a time series. The data that were used to select the trend parameters are the closing prices of Dell Computer Corporation from Jan. 1, 1994 through Jan. 1, 1999. The parameters were then applied to the closing prices from Jan. 1, 1994 through Jan. 23, 2001.

[0125] Determining Attributes of Each Trend

[0126] Referring again to FIG. 1 , after determining whether a new point represents the continuation of a current trend, a number of attributes of each of the trends are calculated or updated in step 600 . For each of the set of trend determination parameters, each trend is characterized.

[0127] Referring now to FIG. 5 , which is a post-processing chart for the trends of a time series, the trends are counted in step 610 , and statistics for each of the trends are calculated in steps 610 - 710 . The statistics include:

[0128] Determining the beginning date or index for each trend at step 620 ;

[0129] Determining the ending date or index for each trend at step 630 ;

[0130] Determining the beginning data value in each trend at step 640 ;

[0131] Determining the ending data value in each trend at step 650 ;

[0132] Determining the trend length at step 660 ;

[0133] Determining the average trend correlation at step 670 ;

[0134] Setting a flag at step 680 to indicate the direction of the trend, where up=+1, flat=0, and down=−1;

[0135] Determining the fraction of changes in data values that were increasing relative to the previous data value for each trend at step 690 ;

[0136] Determining the fraction of changes in data values that were decreasing relative to the previous data value for each trend at step 700 ; and

[0137] Determining the average slopes or first derivatives of each trend at step 710 .

[0138] In this embodiment, a series of arrays is output, each containing N _Trends elements, where N _Trends is the number of trends derived from the time series.

[0139] Implications of the End of a Trend

[0140] One of the uses of this invention is to provide an unbiased method for assessing whether the fluctuations in the prices of a financial instrument are normal fluctuations around a trend, or whether they are indicative of a change in the trend itself. In the first case, negative deviations from the trend provide buying opportunities if the trend is positive. In the second case, where the tool would provide an indication that the trend itself is changing; a change in the trend would provides an opportunity to reassess the potential direction of prices for a financial instrument. Other tools, such as the fundamental analysis of a stock's potential could be assessed to determine whether a position in a stock should be changed. For example, assume that a stock is in an uptrend. A downward change in the trend would provide a signal that the trend has changed and that the position needs to be re-evaluated. In another example, if the stock has a positive fundamental position, but the stock is in a downtrend, this tool would be used to assess when the downtrend has ended and signal an opportunity to purchase the stock at an even cheaper price.

DETAILED DESCRIPTION OF ALTERNATE EMBODIMENT

Non-Constant Interval Between the Samples in a Vector

[0141] This alternate embodiment relates to steps 200 through 400 . It is an alternative method for processing time series datasets with vectors of values that have non-constant spacing between the values. Instead of a time series being represented by y _i,j , j=1, N _i , a vector is now represented by (x _i,j , y _i,j ), j=1, N _i where x _i,j represents some measure of the location or time for each sampled data value y _i,j .

[0142] For example, (x _i,j , y _i,j ), j=1, N _i can represent the times and prices for individual trades that arrive at a non-constant interval of time.

[0143] Selecting a Vector Data Set

[0144] In this embodiment, as in the first embodiment, a plurality of vector data sets is analyzed in step 100 to identify one or more data sets for further analysis. The difference is that the values in the vector are no longer at a constant spacing. Therefore, the last M ₁ data elements are no longer applicable, because the last M ₁ data elements would represent different amounts of time, distance, or other variable that is used to represent the location x _i,j of each of the data values y _i,j . Therefore, instead of using a constant number of last points, it is desirable to use a constant measure of x _i,j called ΔX, and a minimum number of points that are necessary to fit the initial curves, M _Min . The number of points that are in the time span of ΔX and are greater than M _Min will be referred to as M _i . A curve will be fit through the data elements (x _i,j ,y _i,j )=(x _{i,N i −M 1+j} ,Y _{i,N i −M 1 +j} ), j=1, M _i using multiple regression.

[0145] As in the first embodiment, the normalized derivatives and correlation coefficients are recorded. Upon such analysis of each of the data sets, a data set is selected for further analysis. When identifying financial instruments to trade, a vector that has a large increase or decrease in normalized derivative with a correspondingly large absolute value of the correlation coefficient would be a likely candidate for subsequent processing. Vector data sets that have a small slope would generally be avoided.

[0146] Referring now to FIG. 2 , which is a more detailed flowchart of the data set selection steps, the initial search is performed through the following steps for each time series data set i:

[0147] In step 110 , the each vector data set, (x _i,j ,y″ _i,j ), j=1, N _i , where N _i is the number of elements in vector, is loaded into the processor. Note that a mathematical transform of a vector is also a vector. (i.e. (x _i,j ,y″ _i,j )=(x _i,j ,ln(y _i,j )), j=1,N _i is also considered to be a vector).

[0148] In step 120 , an initial window size ΔX, is read by the processor, and the last M _i data elements, where M _i ≧M _Min , are selected from the vector data set. (x _i,j ,y _i,j )=(x _{i,N i −M 1 +j} ,y _{i,N i −M 1 +j} ), j=1, M _i .

[0149] In step 130 , the series of the last M _i data elements is normalized by dividing each data element by y _i,1 , so that the first data value is 1.0. The normalized vector is designated as y′, where y′ _i,1 =1. The normalization is performed in order to permit sorting of the data sets on a consistent basis, such as the slope of a line fit through the last M _i data elements. Alternate methods of normalization may be used such as: a) dividing the vector by its mean or median or b) ranking the values in the vector and using the ranks instead of the vector values themselves in the regression.

[0150] In step 140 , a line, y′ _i,j =a′ _i,j +b′ _i , is fit through the M _i data points elements (x _i,j ,y′ _i,j ) j=1, M _i , using linear regression. The slope of the line, a′ _i , is stored on the processor so that it may be used as a time series data set selection criterion. Alternate methods of data fitting may be used such as higher order polynomial regression or nonlinear regression. In this case, the average of the first derivatives would be stored on the processor as a′ _i .

[0151] In step 150 , if there are remaining vector data sets, steps 110 - 140 are repeated.

[0152] After each data set has been analyzed in steps 110 - 140 , the results are reviewed in step 160 as in the first embodiment.

[0153] Generating the Trend Determination Parameters

[0154] Referring again to FIG. 1 , once a time series data set has been selected and loaded in the processor, the trend determination parameters are selected at step 200 . Inherent in this step is a processing of a portion of the time series to derive the parameters. The parameters include_the Initial Window Size (m ₁ ) and Deviation Limit (L _s ) as discussed in the first embodiment, and an additional parameter, Initial Window Length (ΔX ₁ ), which is the size of the first window. This parameter is the minimum length of data needed to define a trend. It must have more than m ₁ number of elements. In this embodiment, m ₁ acts only as the minimum number of elements that must be in ΔX ₁ . If this method is being applied to tick data for example, a value of m ₁ =30 over a time interval, ΔX ₁ =30 minutes would represent reasonable values.

[0155] Each set of trend determination parameters in combination with the particular data fitting procedure, such as polynomial regression, produces a unique set of trends for a vector data set. At this step of the method, many different sets of trend determination parameters are used to produce corresponding sets of trends. These multiple sets of trends are then evaluated to determine which of the sets are better at matching the current data from the vector data set.

[0156] The trend determination parameters can be selected in several ways. In this embodiment, a subset of the vector, the first M ₂ data points, representing some length ΔX ₂ of the vector, where N _i is the number of data elements in the entire vector data set under evaluation, are selected for defining the trend determination parameters. However, the vector is not limited to the preferred implementation. The parameters can be derived from any vector including other financial vectors as well as one or more simulations of the vector. In this embodiment, the parameter selection is manual as in the first embodiment.

[0157] Referring now to FIG. 3 , which is a detailed flowchart of the procedure for deriving the trend determination parameters. A set of trend determination parameters is generated in step 200 by looping in increments between minimum and maximum values for each of the trend determination parameters. The process is the same as in the first embodiment, except that M1MIN, M1MAX, and M1INC refer to the Initial Window Length (ΔX ₁ ).

[0158] Selecting a Set of Useful Trend Determination Parameters

[0159] Referring again to FIG. 1 , at step 300 the values for each of the sets of trend determination parameters are evaluated in order to select one or more useful sets of trend determination parameters for a particular vector data set. This process is the same as in the first embodiment.

[0160] Processing the Vector Data Set with Trend Determination Parameter Sets

[0161] Referring again to FIG. 1 , when one or more trend determination parameter sets are selected, they are applied to process the vector data set and to generate the trends at step 400 . The result of this step is a refreshed set of trends for each trend determination parameter set as each new data point is considered for the time series.

[0162] FIG. 4A is a detailed flowchart for determining whether a new data point represents a continuation of a trend or not. For each selected trend determination parameter set (ΔX ₁ , L _s ), the latest trend is updated with each new sequential value of data for the vector data set. As in the first embodiment, a trend is described by a best-fit polynomial curve through all of the data within the trend and some length of values from the previous trend. As a consequence, the current trend is dynamic and will be updated as each value of new data arrives. Once the trend changes, the previous trends are no longer dynamic.

[0163] A new trend is defined to have begun as described in the first embodiment.

[0164] Referring now to FIG. 4 A, which is a detailed flowchart of the processing of a time series at step 400 to determine the trends for a given trend determination parameter set (ΔX ₁ , L _s ). The processing is accomplished through the following steps:

[0165] In step 410 , load the vector, (x _i,j ,y _i,j ), j=1, N _i , where N _i is the number of elements in vector i.

[0166] Initialize the trend determination parameters in step 420 . The parameters are ΔX ₁ , the Initial Window Length; L _s , the deviation limit; and ΔX ₂ , the second window size. In this preferred embodiment, ΔX ₂ is equal to ΔX ₁ for the first iteration and n+ΔX ₁ for subsequent iterations, where n is the number of points in a trend. This is the length of data values at the end of the trend's vector data set that are used to calculate a test measure of spread.

[0167] Set j to Δm+1 in step 430 .

[0168] Set n, the number of points in the current trend to 0 in step 440 .

[0169] For each trend, check to see if the current value, (x _i,j ,y _i,j ) is in the trend in steps 450 - 570 .

[0170] For the First Trend

[0171] In step 450 , select_ΔX ₂ length of values from the beginning of the segment of interest in the vector data set. Because the number of data points represented by ΔX ₂ will probably be variable, represent the number of data points represented by ΔX ₂ be Δm.

[0172] In step 460 , calculate a best-fit curve, f _j (x), through the Δm data points. (x _i,k , y _i,k ), x _i,k =[k−(j−Δm)+1], k=(j−Δm), j−1. The subscript j in f _j (x) refers to the function estimate at index j, when evaluating whether (x _i,j ,y _i,j ) is in the trend or not. 8 $\begin{array}{cc}{d}_k=\left[\frac{\left(y_{i,k}-f_j\left(x_{i,k}\right)\right)}{f_j\left(x_{i,k}\right)}\right],k=\left(j-\Delta m\right),j-1& \left(6\right)\end{array}$

[0173] Calculate the standard deviation, s _j−1 , of the residuals, d _k in step 500 . This is similar to what is described in the first embodiment, except that there is a difference in the calculation of the statistics. The normal equation of the standard deviation is: 9 $\begin{array}{cc}{s}_{j-1}=\sqrt{\frac{\sum _{k=1}^{\Delta m}{\left(d_k-\stackrel{\_}{d}\right)}^2}{\Delta m}}& \left(7\right)\end{array}$

[0174] where {overscore (d)} is the average of the d _k values. However, this assumes a constant spacing between the sample values represented by d _k . Because the values in the vector are not sampled on a constant spacing, the values must be weighted to account for this non-constant spacing. 10 $\begin{array}{cc}{s}_{j-1}=\sqrt{\sum _{k=1}^{\Delta m}{w_k\left(d_k-{\stackrel{\_}{d}}^{\prime }\right)}^2}\mathrm{where}{\stackrel{\_}{d}}^{\prime }\mathrm{is}\mathrm{the}\mathrm{weighted}\mathrm{average}\mathrm{of}\mathrm{the}d_k\mathrm{values}.& \left(8\right)\\ {\stackrel{\_}{d}}^{\prime }=\sum _{k=1}^{\Delta m}\left(w_kd_k\right)& \left(9\right)\\ w_k=\frac{\frac{1}{2}\left(x_{i,k}-x_{i,k-1}\right)+\frac{1}{2}\left(x_{i,k+1}-x_{i,k}\right)}{\left(x_{i,\Delta m}-x_{i,1}\right)},k=2,\left(\Delta m-1\right)& \left(10\right)\\ w_1=\frac{\frac{1}{2}\left(x_{i,2}-x_{i,1}\right)}{\left(x_{i,\Delta m}-x_{i,1}\right)}& \left(11\right)\\ w_{\Delta m}=\frac{\frac{1}{2}\left(x_{i,\Delta m}-x_{i,\Delta m-1}\right)}{\left(x_{i,\Delta m}-x_{i,1}\right)}\mathrm{and}\sum _{k=1}^{\Delta m}w_k=1.& \left(12\right)\end{array}$

[0175] The weighted population standard deviation is used in this embodiment; however, this is only one of many measures of the spread of a distribution. This patent includes, but is not limited to, other measures of spread, such as the sample standard deviation, the population or sample variance, range, inter-quartile range, differences between any quantiles of a distribution, or a measure of spread from a neural network.

[0176] Steps 510 through 570 are the same as in the first embodiment with the modifications to steps 450 through 500 as described previously.

[0177] For Subsequent Trends

[0178] Select Δm+1 values at step 450 . ΔX ₁ length of the values are from the end of the previous trend, and n values are from the current trend.

[0179] Calculate the best-fit curve, f _j (x), through the Δm data points at step 460 .

[0180] In step 470 , evaluate the derivative 11 $\frac{f_j\left(x\right)}{x}{|}_{x_{i,j-i}}$

[0181] at the point x _i,j−1 , and obtain an estimated value of the curve, f _j (x _i,j ) at index j.

[0182] In step 480 , calculate the correlation coefficient (R) between the pairs of values, └y _i,k ,f _j (x _i,k )┘, k=(j−Δm), j−1.

[0183] Calculate the residuals, d _k , between the time series values, y _i,j , and the trend values, f _j (x _i,j ) in step 490 . This is the same as in the first embodiment, except that m ₂ has been replaced by Δm.

[0184] Calculate the standard deviation, s _j−1 , of the residuals, d _k in step 500 using equation 3. The weighted population standard deviation is used in this embodiment, however, this is only one of many measures of the spread of a distribution. The invention includes other measures of spread, such as the sample standard deviation, the population or sample variance, range, inter-quartile range, differences between any quantiles of a distribution, or a measure of spread from a neural network.

[0185] Steps 510 through 570 are the same as in the first embodiment with the modifications to steps 450 through 500 as described previously.

[0186] The procedure preferably outputs the same arrays for each trend parameter set as described in the first embodiment. The only difference is that instead of exporting the data array, (y _i,j ), j=1, N _i , the routine will export the data pairs arrays, (x _i,j ,y _i,j ), j−1, N _i .

[0187] Determining Attributes of Each Trend

[0188] As in the first embodiment, a number of attributes of each of the trends are calculated or updated in step 600 .

[0189] Referring now to FIG. 5 , which is a post processing chart for the trends of a time series, steps 610 through 710 are the same as in the first embodiment, except for:

[0190] Determining the beginning time or location for each trend at step 620 ;

[0191] Determining the ending time or location for each trend at step 630 ;

[0192] Determining the average trend correlation at step 670 ; This will be a weighted average as described above in step 500 .

[0193] Determining the average slopes or first derivatives of each trend at step 710 . This will be a weighted average as described above in step 500 .

[0194] 9

DETAILED DESCRIPTION OF ALTERNATE EMBODIMENT

Time Series Data Elements with Multiple New Data Values

[0195] This embodiment refers to step 200 through 400 in FIG. 1 . It is alternative method to describe the ability to consider adding multiple values to the trend at once versus only adding a single value at a time.

[0196] The difference is that instead of considering data elements to be individual points: (x _i,j ,y _i,j ), j=1, N _i , a data element is an aggregate of points within some neighborhood. For example, let (x′ _i,j ,y′ _i,j ), j=1, N _a refer to aggregates of points, where N _a is the number of aggregate points under