What is feature engineering and why does it matter?

Feature engineering is creating the input variables a model learns from. It routinely matters more than the choice of algorithm — a good feature set with a simple model usually beats a poor feature set with a sophisticated one.

Common techniques:

• Categorical encoding — one-hot for low cardinality, target or ordinal encoding for high cardinality. Target encoding must be computed within cross-validation folds or it leaks.
• Scaling — standardisation or normalisation, essential for distance-based and gradient-descent models, irrelevant for tree-based ones.
• Date and time decomposition — day of week, month, hour, is-holiday, days-since-last-event. Raw timestamps are almost useless; their components are highly predictive.
• Aggregations — count, mean, and recency of a customer's past behaviour. These are usually the most powerful features in business problems.
• Binning and interactions — grouping continuous values, or products and ratios of existing features.
• Text and domain-specific transforms — TF-IDF, embeddings, or ratios that a domain expert would recognise.

Handling missing values is part of this: understand why a value is missing before imputing. Missingness is often itself informative, and a binary "was missing" flag frequently helps.

Note: The critical discipline is that every feature must be computable at prediction time with data available then. A feature built from information that only exists after the outcome is leakage, and it is the most common reason a model performs brilliantly in testing and fails in production.