Problem
An electronics store wants to recommend products each shopper will rate highly, including new shoppers with no history.
Data: public Amazon electronics ratings (about 7.8 million, per the dataset description), filtered to users with 50+ ratings and products with 5+: 65,290 ratings, 1,540 users, 5,689 products. Ratings skew high (mean 4.29, median 5). Program certificate (external site).
Result / Impact
A layered recommender, not one model:
- Tuned SVD as the main personalized model: best rating accuracy (RMSE 0.8808 vs. 0.9526 for the best KNN), F1@10 of 0.866, and always a real personal estimate.
- Item-item KNN for “customers who liked this also liked…”, only where it has real neighbors; SVD fills the rest.
- Popularity for new shoppers, with a minimum-ratings rule.
The finding that changed the design. When a KNN model can’t find enough neighbors, it doesn’t fail: it returns the global average (4.29) and flags the prediction “impossible”. The tuned user-user model did this for both my example pairs; the tuned item-item model’s top five for one shopper were all this fallback. Since 4.29 clears the 3.5 relevance threshold, these guesses count as “relevant”: fine in the metrics, random to the shopper.
What I did
Built popularity, user-user and item-item KNN, and SVD, each tuned by 3-fold grid search on RMSE, and compared them on an 80/20 split (top-10 metrics; relevant = true rating 3.5+). The grid searches used the full filtered data, so the folds overlap the test split and tuned scores are somewhat optimistic.
| Model | RMSE | P@10 | R@10 | F1@10 |
|---|---|---|---|---|
| User-user KNN, baseline | 1.0012 | 0.855 | 0.858 | 0.856 |
| User-user KNN, tuned (k = 50, cosine) | 0.9526 | 0.847 | 0.894 | 0.870 |
| Item-item KNN, baseline | 0.9950 | 0.838 | 0.845 | 0.841 |
| Item-item KNN, tuned (k = 30, MSD) | 0.9576 | 0.839 | 0.880 | 0.859 |
| SVD, baseline | 0.8882 | 0.853 | 0.880 | 0.866 |
| SVD, tuned | 0.8808 | 0.854 | 0.878 | 0.866 |
Tools: Python, Pandas, Surprise (KNNBasic, SVD, GridSearchCV), Matplotlib, Seaborn, Google Colab.
Decisions
Lead with SVD, though user-user KNN had the top F1@10. Context: F1@10 ranked user-user first (0.870 vs. 0.866); RMSE ranked SVD first. Options: top-F1 model; SVD; models by role. Decision: models by role, SVD primary. Consequences: 0.004 lower F1 for real personal estimates everywhere.
Drop “impossible” KNN predictions. Context: they look relevant in the evaluation. Options: keep them; drop and backfill from SVD. Decision: drop and backfill. Consequences: fewer item-item suggestions on sparse products; no generic guesses shown as personal.
What I’d do next
- Add a “predict the average” baseline every model must beat; evaluate on a time-based split; measure coverage and novelty; A/B test the design.
Design, not experience
How I’d run this in production on Databricks (design, not production experience)
Ratings into Delta; scheduled retraining (Spark’s ALS at full scale) logged in MLflow, promoted only if better; precomputed top-N lists; alerts when the fallback share rises.