How to Choose a Machine Learning Algorithm for Tabular Data

People often ask “which algorithm should I use?” as if there were a lookup table. There is not, and the scikit-learn team say as much on their own estimator flowchart, which is offered as a rough guide rather than a rule. What there is instead is a short sequence of questions that narrows the field quickly. This is the sequence I use for tabular data, which covers most problems in finance and reporting: rows of trades, holdings, customers or transactions with a few dozen columns each.

1. What exactly are you predicting?

Before any algorithm, pin down the target. A number (next month’s fund flows) is regression. A category (will this client churn, is this email phishing) is classification. No target at all (group these securities by behaviour) is unsupervised. It sounds obvious, but a surprising number of projects stall because the target was never defined precisely. “Detect bad data” is not a target. “Flag price records that a reviewer later corrected” is.

The definition also tells you what labels you have. If nobody has ever recorded which records were corrected, you have an unsupervised problem whether you like it or not.

2. How will the output be used?

This question eliminates more candidates than any technical one. Ask who acts on the prediction and what they need from it.

• If a person must explain each decision, for example to a client, an auditor or a model validation team, start with models whose reasoning can be read directly: logistic regression, linear regression or a shallow decision tree.
• If the output is a ranking for a review queue, you need well-ordered scores more than readable coefficients, and gradient-boosted trees usually do this well.
• If the prediction runs inside a nightly batch, training and scoring speed rarely matter. If it runs on every keystroke, they do.

3. How much data, and what shape?

A few hundred rows will not support a flexible model; a regularised linear model or a small tree is safer. With tens of thousands of rows and a mix of numeric and categorical columns, tree ensembles are a strong default. A 2022 benchmark across 45 tabular datasets found that tree-based models remained state-of-the-art on medium-sized data of around 10,000 samples, ahead of deep learning (Grinsztajn et al., 2022). Neural networks earn their place when the inputs are text, images or long sequences, which is a different problem from a spreadsheet.

Also check the class balance. If the positive class is 1% of rows, the algorithm matters less than how you evaluate it, which I cover in reading a confusion matrix when the positives are rare.

4. Start with a baseline you cannot argue with

Before training anything clever, measure what “dumb” gets you. scikit-learn ships a DummyClassifier that always predicts the most frequent class or predicts at random. Then train one simple, interpretable model. Every more complex model has to beat both by enough to justify the extra cost of explaining and maintaining it.

5. Compare a short list, fairly

Three or four candidates are enough. Compare them with the same cross-validation folds and the same metric, chosen for the business problem rather than out of habit. A sketch:

from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, HistGradientBoostingClassifier

candidates = {
    "baseline": DummyClassifier(strategy="most_frequent"),
    "logistic": make_pipeline(StandardScaler(),
                              LogisticRegression(max_iter=1000, class_weight="balanced")),
    "forest":   RandomForestClassifier(n_estimators=300, class_weight="balanced"),
    "boosting": HistGradientBoostingClassifier(),
}

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
for name, model in candidates.items():
    scores = cross_val_score(model, X, y, cv=cv, scoring="average_precision")
    print(f"{name:9s} {scores.mean():.3f} ± {scores.std():.3f}")

Stratified folds keep the class ratio stable in each split, and putting the scaler inside a pipeline stops information from the validation fold leaking into training. The scikit-learn guide to cross-validation covers both points. If your rows are ordered in time, which is true of almost all market data, swap the shuffled folds for TimeSeriesSplit so you never train on the future.

Look at the spread as well as the mean. If two models are within one standard deviation of each other, pick the simpler one.

A worked shortlist

To make this concrete, here is how the questions might play out for an illustrative problem: flagging holdings records likely to need manual correction before month-end reporting.

That leaves a baseline, logistic regression and gradient boosting, compared on later months. If boosting wins clearly, ship it with per-record explanations. If it wins narrowly, ship the logistic model, because a reviewer who understands why a record was flagged will trust the queue.

The part that matters more than the algorithm

In my experience the choice between two reasonable algorithms changes results far less than the quality of the features and labels. A clean, well-defined target and two or three features that capture how errors actually happen will do more than any amount of model shopping. Choose a sensible model quickly, then spend the time on the data.