Evaluate your own model the way we evaluated the stand-in.

What to do

1. Prepare new images. For each of your classes put at least ~20 photos into test/<class>/ that are not in data/ (another day, another angle, another background — the more "from real life", the more honest). The class folder names must match data/.
2. Train and save. If you have not yet: python transfer.py --data data --epochs 5 --lr 0.001 --finetune.
3. Evaluate twice:
   • python evaluate.py --model model.pt --data data
   • python evaluate.py --model model.pt --data data --test-dir test
4. Try two photos that are in neither folder (e.g. just taken with a phone): python predict.py photo.jpg.

What to hand in

One text file or answer containing:

• both accuracies with the image count and the error: "X % of N" and ±… (use your margin_of_error from the drills);
• the confusion matrix for the new images;
• the most confused pair (use your most_confused);
• three mistakes, opened with your own eyes: give the file name and say in one sentence why the model may have gone wrong (bad light? two objects in the frame? genuinely similar? is the picture actually in the wrong folder?);
• one class whose mistake costs you more, and whether precision or recall matters more for it (one sentence);
• the results of the two phone photos and how much confidence the model showed;
• one honest sentence: whether the validation and new-image numbers differ, and whether that difference is larger than the error.

Two rules

• Look at test/ once. If after the result you change something and train again — write that in the report (how many times you looked at test/). It is no shame; the shame would be not saying.
• If your numbers are worse than ours (93.5 %), that is not a failure. Our four classes are small grey photos of clothes, very clear-cut. Your classes may be harder, and your number matters less here than knowing where it goes wrong.

Tip. In the next lesson you put all of this into one script that does every step on its own — and you will see that one "good habit" can fail to help on your data, or even hurt. The decisions will still be yours to make.