Models learn from the columns you give them. Often, better features improve a model more than switching to a fancier algorithm. Feature engineering is the work of turning raw data into inputs that expose the patterns.
Encode categories
Most models need numbers.
- One-hot encoding: one 0/1 column per category. Safe for categories with no natural order (city, payment method).
- Ordinal encoding: numbers that reflect a real order (Low < Medium < High).
from sklearn.preprocessing import OneHotEncoder
enc = OneHotEncoder(handle_unknown="ignore")handle_unknown="ignore" prevents errors when a new category appears later. Group rare categories into "Other" first to avoid hundreds of near-empty columns.
Scale numbers
Distance-based and penalised models (KNN, SVM, K-Means, regularized regression) are sensitive to scale. A salary column in lakhs would dominate an age column.
StandardScaler: mean 0, standard deviation 1.MinMaxScaler: range 0 to 1.
Tree-based models do not need scaling.
Extract features from dates
A raw date is rarely useful. Extract what matters:
df["order_month"] = df["order_date"].dt.month
df["order_weekday"] = df["order_date"].dt.dayofweek
df["is_weekend"] = df["order_weekday"] >= 5
df["days_since_signup"] = (df["order_date"] - df["signup_date"]).dt.daysRatios and combinations
Domain knowledge creates the best features:
- Loan amount ÷ income (affordability).
- Late payments ÷ total payments (reliability).
- Revenue ÷ visits (value per visit).
- Price ÷ average price of the category (relative price).
Binning
Turning a continuous number into bands (age groups, order-size bands) can help simple models capture non-linear effects and makes results easier to explain, at some cost in detail.
Missing values as information
Sometimes "missing" is meaningful: no employer listed might itself predict loan risk. Impute the value and add a flag column:
from sklearn.impute import SimpleImputer
imp = SimpleImputer(strategy="median", add_indicator=True)Aggregated history
For customer-level predictions, summarise past behaviour: number of orders in the last 90 days, average rating given, days since last order. Make sure the summaries use only data available before the prediction date.
Text basics
For short text such as support tickets, convert words to counts or TF-IDF weights:
from sklearn.feature_extraction.text import TfidfVectorizer
vec = TfidfVectorizer(min_df=2, ngram_range=(1, 2))Put it all in a pipeline
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
prep = ColumnTransformer([
("num", StandardScaler(), ["income", "loan_amount", "tenure_months"]),
("cat", OneHotEncoder(handle_unknown="ignore"), ["city", "employment_type"]),
])
model = Pipeline([("prep", prep), ("clf", LogisticRegression(max_iter=1000))])A pipeline fits preprocessing only on training data, which prevents leakage, and applies exactly the same steps to new data.
Beware of leakage
Never create features using information from after the outcome, and never compute scaling or encoding statistics on the full dataset before splitting. See train-test split and cross-validation.
Practise
The KNN gym programme set shows why scaling matters, and the support ticket set turns text into features with counts and TF-IDF.
Practise what you just read
Reading helps; writing the code yourself is what sticks. Start here: