Message Tab

Projects Involved: GCS

Concepts: Protobuf in React Typescript

Abstract:

Using the image tab you made earlier, you now should add a dropdown for the possible objects and a submit button, which sends the information back to the OBC to verify that the object is matched successfully via the /message route (which you will be implementing in the next step). Essentially, this feature allows you to match the image on screen with one of the predefined objects from a list (this step), which allows you to see if the match is correct in the OBC (next step). This task mainly introduces you to how to use and interact with protos in React Typescript.

Protocal Buffers, or protobufs, or protos, are basically a tool that allows us to define and serialize structured data so different systems and languages can talk to each other. This basically means that we do not need to create separate data structures across OBC and GCS. Instead, we can just rely on its neat conversions to different languages, which also includes handy helper methods like setters, getters, and converting to different types. We already defined the protos for you in this project, which you can view in OBC or GCS's protos/onboarding.proto. Note: make sure you run make protos to compile protos in BOTH OBC and GCS first!

Goals:

Learn to use protos in React Typescript

Files Involved:

OBC

protos/onboarding.proto

GCS

src/pages/ImagePage.tsx
protos/onboarding.proto

Base Steps:

1. Import the Tsx Protobuf files

Add import { ODLCObjects, DetectedObject } from "../protos/onboarding.pb"; to the top of the file. This allows you to use built-in methods compiled by protos to work with what you need.

2. Edit the Protobuf

Define a message in onboarding.proto that holds the detected object of type ODLCObjects.

Create a dropdown to choose the objects. Create a submit button for this object, which should hit the /message route.

Tips:

1: Tips for getting the list of objects in the dropdown

You may find this method to be useful when getting a list of objects from protos.

Object.entries(ODLCObjects);

2: Tips for creating a message and payload for the submit button

These two boilerplates might be useful.

const message = DetectedObject.create({ object: selectedObject });
const payload = DetectedObject.toJSON(message);

3. If you are completely stuck...

Here are chunks of code that you may paste into ImagePage.tsx.

const [selectedObject, setSelectedObject] = useState<number>(
    ODLCObjects.Undefined
);
const [isSubmitting, setIsSubmitting] = useState<boolean>(false);
const [submitError, setSubmitError] = useState<string | null>(null);
const [submitResponse, setSubmitResponse] = useState<string | null>(null);

const enumEntries = Object.entries(ODLCObjects).filter(
    ([, value]) => typeof value === "number"
) as [string, number][];
const enumOptions = enumEntries
    .filter(([, value]) => value !== -1)
    .sort((a, b) => a[1] - b[1]);

const handleSubmit = async () => {
    setIsSubmitting(true);
    setSubmitError(null);
    setSubmitResponse(null);

    try {
        const message = DetectedObject.create({ object: selectedObject });
        const payload = DetectedObject.toJSON(message);

        const resp = await fetch("/api/v1/obc/message", {
            method: "POST",
            headers: { "Content-Type": "application/json" },
            body: JSON.stringify(payload),
        });
        // Prefer JSON, but gracefully fall back to text if not JSON
        const contentType = resp.headers.get("content-type") || "";
        const data = contentType.includes("application/json")
            ? await resp.json()
            : await resp.text();
        if (!resp.ok) {
            throw new Error(
                (typeof data === "object" &&
                    data &&
                    (data.error || data.message)) ||
                    `HTTP error! Status: ${resp.status}`
            );
        }
        setSubmitResponse(
            typeof data === "string" ? data : JSON.stringify(data, null, 2)
        );
    } catch (err: any) {
        setSubmitError(err?.message || "Failed to submit message");
    } finally {
        setIsSubmitting(false);
    }
};

<section
    style={{
        marginTop: "16px",
        padding: "12px",
        border: "1px solid #ccc",
        borderRadius: 6,
        backgroundColor: "#f9f9f9",
    }}
>
    <h3 style={{ marginTop: 0 }}>Submit Detected Object</h3>
    <div
        style={{
            display: "flex",
            gap: 10,
            alignItems: "center",
            flexWrap: "wrap",
        }}
    >
        <label htmlFor="detected-object">Object</label>
        <select
            id="detected-object"
            value={selectedObject}
            onChange={(e) => setSelectedObject(Number(e.target.value))}
        >
            {enumOptions.map(([name, value]) => (
                <option key={value} value={value}>
                    {name}
                </option>
            ))}
        </select>
        <button onClick={handleSubmit} disabled={isSubmitting}>
            {isSubmitting ? "Submitting..." : "Submit"}
        </button>
    </div>
    {submitError && (
        <div className="error-box" style={{ marginTop: 8 }}>
            <pre style={{ whiteSpace: "pre-wrap" }}>{submitError}</pre>
        </div>
    )}
    {submitResponse && (
        <div style={{ marginTop: 8 }}>
            <pre style={{ whiteSpace: "pre-wrap" }}>{submitResponse}</pre>
        </div>
    )}
</section>