Data Type

Time Sequence Data

The candidate types of features should be the same as the type of target feature. For example, temperature at time1, temperature at time2, and temperature of time3, the target feature should be temperature at next time point instead of other types of data.

Many rows (including multiple features) mapping on sigle target

In this case, the data in each feature should be aggregated by sum,average, max, min for numerical data; partition by function for categorical or nominal data (eg: SELECT name, hair_colour, COUNT(hair_colour) OVER (PARTITION BY name) FROM MyTable GROUP BY name, hair_colour; ).

process

Classification and Regression

To choose algorithms which have best performance in classification or regression.

Variables Selection

Forward
Backward
Stepwise 

Feature Transformations

Now list several common techniques for feature transformations. Usually, it is helpful to combine some of these transformations (e.g., centering + scaling). In the following, we denote by $f = (f_1,...,f_m) \in R^m$ the value of the feature $f$ over the $m$ training examples. Also, we denote by $\bar{f} = \frac{1}{m} \sum_{i=1}^{m}f_i$ the empirical mean of the feature over all examples.

Centering

This transformation makes the feature have zero mean, by setting $f_i \gets f_i - \bar{f}$.

Unit Range

This transformation makes the range of each feature be $[0,1]$. Formally, let $f_{max} = max_i f_i$ and $f_{min} = min_i f_i$. Then, we set $f_i \gets \frac{f_i}{f_{min}}$.

Standardization

This transformation makes all features have a zero mean and unit variance. Formally, let $\nu = \frac{1}{m}\sum_{i=1}^{m}(f_i - \bar{f})^2$ be the empirical variance of the feature. Then, we set $f_i \gets \frac{f_i-\bar{f}}{\nu}$.

Clipping

This transformation clips high or low values of the feature. For example, $f_i \gets sign(f_i), max\{b,|f_i|\}$, where $b$ is a user-speciffed parameter.

Sigmoidal Transformation

As its name indicates, this transformation applies a sigmoid function on the feature. For example, $f_i \gets \frac{1}{1+exp^{bf_i}}$, where $b$ is a user-speciffed parameter. This transformation can be thought of as a “soft” version of clipping: It has a small effect on values close to zero and behaves similarly to clipping on values far away from zero.

References

• tex refer to https://en.wikibooks.org/wiki/LaTeX/Mathematics.
• “Feature Transformations” Refer to the book of “Understanding Machine Learning:From Theory to Algorithms”.