Code Assistants: Five Default ML Values to Check

AI-generated snippets run without error, but many decisions are made on your behalf by the libraries. Five standard parameters in scikit-learn, often left implicit, have direct effects on the variance of ensembles, the strength of regularization, the validity of cross-validations, the quality of partitions, and the persistence of columns.

Clustering: automatic n_init offers only a single attempt

With init="k-means++", n_init="auto" in KMeans corresponds to a single complete run. The algorithm starts with initial centers, assigns each point to the nearest center, and updates until convergence, but poor initialization can lead to a local minimum from which it cannot escape. Setting n_init=10 launches ten runs and retains the one with minimal inertia, while a single run does not guarantee the best solution. Automatic tuning applies a rule related to initialization without examining the results or restarting if the solution is not optimal. Increasing max_iter does not replace these restarts; it merely adds iterations to a given attempt. Setting random_state=42 makes the initialization reproducible but does not test other starting points. To obtain multiple attempts, n_init must be explicitly set to a value like 10, accepting the additional computational cost.

Preprocessing: columns can disappear with SimpleImputer

SimpleImputer, configured by default with a mean strategy, removes a feature that contains no observed values during fitting, as no mean can be calculated. In a training set with 20 columns, where one is entirely missing, the downstream model actually learns from 19 columns. If subsequent data fills in this column, the trained imputer continues to eliminate it, and this new information does not reach the model. The pipeline maintains the same dimension between training and prediction, avoiding shape errors, but the decision comes from keep_empty_features=False.

Evaluation: cv=5 does not specify the construction of folds

Setting cv=5 in cross_val_score only indicates the number of folds. In regression, scikit-learn applies KFold with shuffle=False, producing consecutive blocks according to the order of the rows. A dataframe sorted by date, client, or region transfers this order into the evaluation design. For example, with 100 weeks, the first fold evaluates weeks 1 to 20 after training on weeks 21 to 100, which leads to predicting the past from the future, while actual forecasting only uses past data to predict the future. Activating shuffle=True would mix past and future. The separator must reflect what "unseen" means: mixed KFold for independent observations, a temporal scheme for forecasting, or grouped splitting to test on new clients. In classification, StratifiedKFold is used by default without mixing and does not initially account for time or groups. By choosing cv=5, the evaluation frequency has been set but its structure left implicit.

Regularization: LogisticRegression imposes L2 with C=1.0

LogisticRegression by default fits an L2 penalized model with C=1.0, where smaller values of C increase regularization and larger values loosen the constraint. The numerical size of a coefficient depends on the units of the variable: an income expressed in euros may produce 0.0001, while in thousands of euros, it requires 0.1 for the same contribution, with 50,000 × 0.0001 and 50 × 0.1 each equating to 5. The squared L2 penalty makes these unit changes critical for the optimization trade-off if C is left unchanged. A starting point is to standardize continuous variables in a Pipeline fitted fold by fold, then tune C with appropriate validation.

Ensembles: RandomForestRegressor does not sample variables by default

RandomForestClassifier uses max_features="sqrt" while RandomForestRegressor sets max_features=1.0, making all variables available at each split. The bootstrap of rows remains, but the randomness on features is neutralized. With a dominant predictor, multiple trees from different bootstrap samples may still select the same initial splits and become correlated, limiting variance reduction through aggregation. Forcing a subsampling of variables leads to alternative splits; adding trees does not eliminate the variance related to correlation. Setting max_features=0.33 restores this randomness by exposing about one-third of the variables at each split. The key question becomes: if feature randomness is sought, why leave it inactive?

Assistants: running code masks undisclosed choices

An assistant can generate a model implementation that runs in just a few lines. Libraries fill in omitted parameters with their default values, transforming an incomplete request into an operational pipeline. Language models, trained to predict the next token, reproduce frequent implementation patterns where arguments are left at their standard value. Code that works does not mean these trade-offs have been reviewed. Production feedback has shown that these oversights have caused debugging difficulties. The recommended practice is to systematically examine the impact of each accepted default value without discussion.