Reweighting, Rescaling, Smoothing, and Bagging: The Data Techniques Behind Every Model

These five terms show up constantly in ML pipelines — in papers, in library docs, in other people's code — usually with no explanation, as if everyone already knows them. They don't require deep math to understand. Each one solves a specific, concrete problem with your data. Here's each one plainly, with code.

1. Reweighting — making some examples count more than others

By default, every training example counts equally. Reweighting says: no, actually, weigh this one more (or less) than that one. Two common flavors:

Word reweighting — not every word in a sentence is equally informative. This is exactly what TF-IDF does (covered in the embeddings post earlier in this series): common words like "the" get down-weighted, rare and distinctive words get up-weighted.

from sklearn.feature_extraction.text import TfidfVectorizer
 
vectorizer = TfidfVectorizer()
tfidf = vectorizer.fit_transform(["the cat sat on the mat", "the dog ran fast"])
# "the" gets a low weight (appears everywhere), "cat" and "dog" get higher weight

Sample (class) reweighting — when one class in your dataset is rare (fraud detection, rare disease diagnosis), the model can get lazy and just predict the majority class every time, since that's "right" most of the time. Reweighting forces it to pay more attention to the minority class by literally multiplying its loss contribution:

from sklearn.utils.class_weight import compute_class_weight
import numpy as np
 
y = np.array([0, 0, 0, 0, 0, 0, 0, 0, 1, 1])  # 8 majority, 2 minority
weights = compute_class_weight(class_weight="balanced", classes=np.array([0, 1]), y=y)
# weights ≈ [0.625, 2.5] — the rare class (1) counts 4x more during training

The idea in one sentence: reweighting doesn't change your data, it changes how much each piece of data is allowed to influence the model.

2. Time scaling — putting temporal features on a comparable footing

Time-based features come in wildly different units — "seconds since last login" might be 30, "days since account created" might be 900. Fed in raw, a model can end up treating "days since signup" as automatically more important just because the numbers are bigger, which has nothing to do with actual importance. Time scaling normalizes these onto a comparable range.

import pandas as pd
from sklearn.preprocessing import MinMaxScaler
 
df = pd.DataFrame({
    "seconds_since_login": [30, 3600, 120],
    "days_since_signup": [900, 5, 200],
})
 
scaler = MinMaxScaler()
df_scaled = pd.DataFrame(scaler.fit_transform(df), columns=df.columns)
# both columns now range 0 to 1, on equal footing

A related, very common time-scaling move for time series specifically: converting an absolute timestamp into a cyclical feature, so a model understands "23:00 and 01:00 are close together," which raw hour numbers (23 vs. 1) don't express on their own.

import numpy as np
 
hour = 23
hour_sin = np.sin(2 * np.pi * hour / 24)
hour_cos = np.cos(2 * np.pi * hour / 24)
# now hour 23 and hour 1 end up close in (sin, cos) space, correctly

3. Data rescaling — the general version of the same idea

Rescaling is the broader term: transforming any numeric feature onto a standard range or distribution, not just time. Two standard approaches:

• Min-max scaling — squeeze everything into a fixed range, usually 0 to 1.
• Standardization (z-score scaling) — center each feature around a mean of 0 with a standard deviation of 1.

from sklearn.preprocessing import MinMaxScaler, StandardScaler
import numpy as np
 
X = np.array([[10], [200], [3000]])
 
minmax = MinMaxScaler().fit_transform(X)
# [[0.], [0.0636], [1.]]
 
standard = StandardScaler().fit_transform(X)
# mean 0, std 1 — outliers show up as large positive/negative values

Why this matters: algorithms that measure distance (k-nearest neighbors, k-means clustering) or that use gradient descent (neural networks) are sensitive to feature scale. A feature ranging 0–1,000,000 will dominate one ranging 0–1 purely because of its size, regardless of which one is actually more predictive. Rescaling removes that distortion.

4. Smoothing — refusing to let the model be overconfident about rare or sparse data

Smoothing is what you reach for when raw counts or raw probabilities would be too extreme, usually because you don't have enough data to trust them fully. The classic example: a word that never appeared in your training data gets a probability of exactly zero under a naive frequency count — which is too strong a claim, since "never seen" isn't the same as "impossible."

Laplace (additive) smoothing fixes this by pretending you saw every possible word at least once:

def laplace_smoothed_prob(word_count, total_words, vocab_size, alpha=1):
    return (word_count + alpha) / (total_words + alpha * vocab_size)
 
# a word that appeared 0 times still gets a small, non-zero probability
print(laplace_smoothed_prob(word_count=0, total_words=1000, vocab_size=5000, alpha=1))

Exponential smoothing solves a different but related problem — noisy time-series data — by blending each new value with the smoothed history, so a single spiky outlier doesn't yank your trend line around:

def exponential_smoothing(values, alpha=0.3):
    smoothed = [values[0]]
    for v in values[1:]:
        smoothed.append(alpha * v + (1 - alpha) * smoothed[-1])
    return smoothed
 
exponential_smoothing([10, 50, 12, 11, 60, 13])
# spikes get pulled toward the trend instead of whipsawing the series

The theme across every kind of smoothing: raw counts and raw signals are often too jagged or too confident to trust directly, and smoothing is the deliberate act of softening that.

5. Bagging — reducing variance by training on many random slices of your data

Bagging (short for Bootstrap AGGregatING) trains many copies of a model, each on a different random resample of your training data (sampled with replacement — so some rows appear multiple times in one resample, others not at all), then averages their predictions.

from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier
 
model = BaggingClassifier(
    estimator=DecisionTreeClassifier(),
    n_estimators=50,      # train 50 trees, each on a different bootstrap sample
    max_samples=0.8,       # each tree sees 80% of the data, resampled
)
model.fit(X_train, y_train)

Why this helps: a single decision tree is prone to overfitting — it can latch onto quirks of the exact training rows it saw. Train 50 trees on 50 slightly different resamples of that data and average their votes, and the individual quirks tend to cancel out, while the genuine underlying pattern (which shows up consistently across most resamples) survives. This is literally the mechanism behind Random Forests — a random forest is bagged decision trees, plus one extra trick (randomizing which features each tree is even allowed to look at).

How these connect

None of these five techniques are isolated tricks — they're all answers to the same underlying question: how do I stop my model from being misled by the specific shape of the data I happened to collect? Reweighting corrects for which examples are over- or under-represented. Rescaling (including time scaling) corrects for features being on incomparable numeric scales. Smoothing corrects for overconfidence from sparse or noisy signals. Bagging corrects for a model's sensitivity to the exact training rows it happened to see. Different problem, same instinct: don't let a quirk of the raw data become a false signal the model learns as if it were real.

Reference links

• scikit-learn preprocessing (rescaling, smoothing utilities): scikit-learn.org/stable/modules/preprocessing.html
• scikit-learn class weighting: scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html
• scikit-learn Bagging: scikit-learn.org/stable/modules/ensemble.html#bagging
• Laplace smoothing (Wikipedia, solid plain-language overview): en.wikipedia.org/wiki/Additive_smoothing

Ties back to the embeddings post earlier in this series — TF-IDF reweighting shows up there too, from the other direction.