NUTRISNAP / FOOD RECOGNITION
From a meal photo
to food detection and nutrition logging.
I built NutriSnap?s ML pipeline and inference API,
from data preparation to lightweight serving.
A team project connecting a fine-tuned food detector to a nutrition-tracking application.
01 / Project & ownership
From meal photo to nutrition record.
NutriSnap connects food detection to nutrition logging. My responsibility was the ML component: preparing the data, training the detector, examining its errors, and serving predictions to the application.
Photograph a meal
The user uploads a food photo. The current model works best with one dish at a time.
Detect & return
The inference API returns food names, confidence scores, and bounding boxes in original-image coordinates.
Confirm & log
The backend looks up nutrition for the confirmed food. Portions are entered manually in the app.
Previously integrated with the app. A refreshed walkthrough and hosted model demo are planned.
02 / Data & model evaluation
Good predictions start
with questions about the data.
AIHub JSON annotations needed conversion to normalized YOLO labels. The conversion process also exposed mismatches that could silently remove data or distort the evaluation scope.
38 training classes.
87 validation classes.
One folder-name typo
hid an entire class.
Missing images and
invalid annotations.
Use training classes as the source of truth.
Align label and image folder names.
Log and skip unusable entries.
Fine-tuning a pretrained YOLOv8n
100 epochs ? 640 ? 640 input ? RTX 3050 ? approximately 22 hours. Metrics below describe the AIHub validation split, not real-world table-photo performance.
A duplicate or near-duplicate audit across splits is not documented here.
How did the model perform on validation data?
11,679 AIHub validation images.
03 / Inference & serving
A dedicated inference API.
A 93% smaller serving image.
The service answers a focused question: which foods are in this image, and where? Nutrition lookup, account data, and meal storage belong to the application backend.
Decode to RGB
Class-wise NMS → box rescaling
Low-confidence review flag
Kept the model, removed the training stack.
A lighter service with ONNX Runtime.
Reimplemented preprocessing and postprocessing with NumPy and OpenCV, separating inference dependencies from training.
How much smaller was the Docker image?
What changed in the serving environment?
Repository measurements; benchmark conditions and reproducible outputs will accompany the verification notes.
Labels show the measured before/after values and approximate reductions.
A smaller runtime should not mean
a different prediction.
The first preprocessing implementation used PIL resizing and produced confidence drift of up to 0.25. Matching Ultralytics with OpenCV INTER_LINEAR brought the outputs into alignment.
Repository-reported comparison against Ultralytics running the same ONNX model. Confidence difference reported as 0.0. This compares serving implementations, not independent model accuracy.
04 / Failure analysis & limitations
Strong validation scores.
A different story on a full table.
Training photos were centered on a single plate, which could contain multiple foods. Full-table photos with several dishes were harder; testing individual dish crops produced better detections.
Schematic illustrations of the reported crop experiment; these are not actual model-output images.
A diagnostic example, not an aggregate benchmark. Current guidance: photograph one plate at a time.
Technical notes & verification
Reproduction and benchmark notes are being prepared. Nutrition and portion accuracy are outside the detector evaluation.