Evaluation & Deployment

Code

A small HTTP endpoint: serve.py

predict.py works as long as you sit at the same computer, in a terminal. For the model to be used, another person or another program must be able to send it something. The smallest form for that is an HTTP endpoint: you send an image, you get JSON back.

Your job

Create serve.py (on the right). It is complete and short, because it invents nothing new: the hard part is already done.

model, classes, size = load_model(args.model)          # evaluate.py
...
ranked = predict_image(model, classes, size, image)    # predict.py

The model is loaded once, when the server starts, not on every request. The HTTP part uses only the standard library (http.server) — no framework, so you can see how little happens here.

Start it in one terminal window:

python serve.py --port 8123

POST an image to http://127.0.0.1:8123/

And from another, send an image (Windows 10+ has curl; in PowerShell type curl.exe, because there curl is an alias of another tool):

curl --data-binary @test/sneaker/0003.png http://127.0.0.1:8123/

{"prediction": "sneaker", "probabilities": {"sneaker": 0.9999969005584717, "coat": 1.9309829895064468e-06, "pullover": 8.68784070462425e-07, "tshirt": 3.0835616371405195e-07}}

Now something that is not an image:

curl -i --data-binary "hello" http://127.0.0.1:8123/

HTTP/1.0 400 body must be an image file

The server does not crash — it returns 400 and carries on. Each answer took ~0.06–0.08 s (on our CPU).

What is NOT here

This is a demonstration, not production. An honest list of what is missing:

http.server handles requests one at a time — two people at once stand in a queue;
there is no authentication, no limit on file size, no HTTPS;
every answer is just a class and probabilities; no "I am not sure" warning;
the model understands nothing it was not taught: serve.py accepts any picture and says one of your four classes.

That last line is the most important. If you send a photo of a cat to this model (which knows only coat, pullover, sneaker, tshirt), it will still say one of the four: softmax always spreads 100 % over the classes, whatever the model sees. A model cannot say "I don't know" unless you teach it to (e.g. a fifth class, "other").

What you have now

model.pt → load_model → predict_image → serve.py. The same road all your data walked, only one image at a time.

serve.py CREATE

Create this file in ./

"""serve.py — a tiny HTTP endpoint: POST an image, get JSON back."""

import argparse
import io
import json
from http.server import BaseHTTPRequestHandler, HTTPServer

from PIL import Image

from evaluate import load_model
from predict import predict_image


def make_handler(model, classes: list[str], size: int):
    class Handler(BaseHTTPRequestHandler):
        def do_POST(self) -> None:
            body = self.rfile.read(int(self.headers.get("Content-Length", 0)))
            try:
                image = Image.open(io.BytesIO(body))
            except Exception:
                self.send_error(400, "body must be an image file")
                return
            ranked = predict_image(model, classes, size, image)
            payload = json.dumps(
                {"prediction": ranked[0][0], "probabilities": dict(ranked)}
            ).encode()
            self.send_response(200)
            self.send_header("Content-Type", "application/json")
            self.end_headers()
            self.wfile.write(payload)

    return Handler


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--model", default="model.pt")
    parser.add_argument("--port", type=int, default=8000)
    args = parser.parse_args()

    model, classes, size = load_model(args.model)
    server = HTTPServer(("127.0.0.1", args.port), make_handler(model, classes, size))
    print(f"POST an image to http://127.0.0.1:{args.port}/")
    server.serve_forever()


if __name__ == "__main__":
    main()