What is Feature Scaling

Feature scaling is a preprocessing technique used in machine learning to standardize or normalize the range of independent variables or features of data. It's an important step, especially for algorithms that are sensitive to the scale of input features. Feature scaling ensures that all features contribute equally to the model's learning process and avoids problems caused by differences in feature magnitudes. The two common methods for feature scaling are:

1. Standardization (Z-score normalization):

   • 
     
   • In standardization, each feature is scaled to have a mean of 0 and a standard deviation of 1.
   • The formula for standardization is: z = (x - μ) / σ, where z is the standardized value, x is the original value, μ is the mean of the feature, and σ is the standard deviation of the feature.
   • Standardization makes the data look like it roughly follows a standard normal distribution (mean = 0, standard deviation = 1).
   • 
     

2. Normalization (Min-Max scaling):

   • 
     
   • In normalization, each feature is scaled to a specific range, typically between 0 and 1.
   • The formula for normalization is: x' = (x - min) / (max - min), where x' is the normalized value, x is the original value, min is the minimum value in the feature, and max is the maximum value in the feature.
   • Normalization is useful when the algorithm you're using assumes that the data is on a similar scale, or when you want to preserve the original data distribution.

Here are some reasons why feature scaling is important:

• Many machine learning algorithms, such as support vector machines, k-nearest neighbors, and principal component analysis (PCA), are sensitive to feature scales. Features with larger scales can dominate those with smaller scales in these algorithms.

• Gradient-based optimization algorithms, like those used in neural networks, converge faster when features are on similar scales. It helps prevent the optimization process from being skewed towards certain features.

• Distance-based metrics in clustering and nearest neighbor methods are influenced by feature scales. Standardized data ensures that distances are computed correctly.

• Interpretability of model coefficients can be affected by feature scales, especially in linear regression.

Feature scaling is typically applied after data preprocessing and before feeding the data into a machine learning algorithm. The choice between standardization and normalization depends on the specific requirements of your model and the characteristics of your data.

Code :

from sklearn.preprocessing import StandardScaler # Sample data (replace this with your dataset) data = [[1.0, 2.0, 3.0], [4.0, 5.0, 6.0], [7.0, 8.0, 9.0]] # Create a StandardScaler object scaler = StandardScaler() # Fit the scaler to your data and transform it scaled_data = scaler.fit_transform(data) # The scaled_data variable now contains your standardized data print("Standardized Data:") print(scaled_data)

Output: 

Standardized Data: [[-1.22474487 -1.22474487 -1.22474487] [ 0. 0. 0. ] [ 1.22474487 1.22474487 1.22474487]]

In the standardized data:

• Each column (feature) has a mean of approximately 0.
• Each column has a standard deviation of approximately 1.

Code:

import numpy as np

# Sample dataset

data = np.array([[2, 5, 10],

                 [4, 8, 12],

                 [1, 3, 6]])

# Min-Max scaling

min_val = np.min(data, axis=0)

max_val = np.max(data, axis=0)

normalized_data = (data - min_val) / (max_val - min_val)

print("Normalized Data:")

print(normalized_data)

Output: 

Normalized Data:

[[0.5 0.5 0.5]

 [1.  1.  1. ]

 [0.  0.  0. ]]