Our team followed the professor’s instructions to compare CLIP ViT-B/16 and ViT-B/32 on CIFAR-10 classification for an assistive-glasses use case. We implemented the work together; I also managed the timeline, communication, and coordination. This was a coursework evaluation, not a deployed wearable.
My roleCollaborative implementation and team coordination
StatusCoursework evaluation complete · Results are team-reported
01
Comparing two representations
We organized 50,000 labeled images across ten classes into 25,000 training images, 10,000 test images, and 15,000 reserved for later training experiments.
We extracted and normalized 512-dimensional pretrained CLIP embeddings, then trained linear classifiers. Initial overall accuracy was 95.20% for patch-16 and 94.01% for patch-32. Our team recommended patch-16 for this experiment.
Results and charts come from our April 21, 2026 project report and have not been independently rerun.
Embeddings, patches, and training settings
An embedding is a numeric representation of an image. These models divide the model input into 16 × 16 or 32 × 32 pixel patches before generating a 512-dimensional representation. The team trained linear classifiers using one-hot class labels, mean squared error (MSE) loss, and stochastic gradient descent (SGD). The report lists 3,000 epochs and a learning rate of 5.5. An epoch is one pass through the training set; the learning rate controls the size of parameter updates.
We used confusion matrices to investigate weaker classes. Initially, cat accuracy was 88% for patch-16, while deer accuracy was 89% for patch-32.
Adding 1,500 cat examples did not change the reported cat result. Adding 1,500 deer examples improved that class result by about one percentage point.
How to read the confusion matrices
Rows represent the true class and columns represent the predicted class. Values along the diagonal represent correct classifications; off-diagonal values show which classes were confused. The displayed proportions help compare results within each true class.
Initial patch-16 confusion matrix from the team report.Initial patch-32 confusion matrix from the team report.
03
Examining an individual error
A deer classified as a horse gave us a concrete example to inspect alongside the aggregate results. Individual examples help make errors visible, but do not establish their cause.
CIFAR-10 dataset example: true class deer, predicted horse. Reproduced from the team report, Figure 9; not personal photography.
04
What more training data changed
We expanded training to 40,000 images while retaining the 10,000-image test set. The original reserve was used for these training experiments; it was not an untouched final evaluation set.
Patch-16 remained at 95.2%, and patch-32 showed only a small reported change. More data provided limited improvement in this setup; that does not establish that the models cannot improve.
Different losses, optimizers, classifier complexity, and preprocessing remain proposed future experiments.
Training-size versus accuracy comparison. Team report, Figure 13: limited reported improvement from 25,000 to 40,000 training images.