Software Engineer: AI Expert

How do you decide which evaluation metric to optimize for a classification model when the classes are imbalanced?

Accuracy is misleading on imbalanced data because a model can score well by always predicting the majority class. I look at precision, recall, and F1 for the minority class specifically, and pick the metric that matches the real cost of false positives versus false negatives in the deployment context, since those costs are rarely symmetric in practice.

What is your process for debugging a deep learning model that trains fine but performs poorly at inference time?

I start by checking for a train/inference mismatch: different preprocessing, batch normalization behaving differently in eval mode, or data leakage during training that inflated validation scores. Comparing the exact input pipeline used at training time against the one used at inference usually surfaces the discrepancy before I need to touch the model architecture itself.

How do you approach feature selection when working with high-dimensional tabular data?

I start with correlation and mutual information analysis to drop obviously redundant features, then use a tree-based model's feature importance as a first pass filter. From there, recursive feature elimination with cross-validation tells me where the marginal value of additional features drops off, which keeps the final feature set both smaller and more robust to overfitting.

Explain how you would detect and address data drift in a production machine learning pipeline.

I monitor the statistical distribution of input features and model outputs over time, using something like population stability index or KL divergence against a reference window. When drift crosses a threshold, I retrain on recent data rather than the full historical set, and I keep an alert in place so drift is caught before it shows up as a quality regression.