When Laya Is the Better Choice for a Raspberry Pi

Laya is Convai Innovations' open-weight decision model. It answers typed questions over a state and returns probabilities. Its counterpart is Jev, TypeSafe AI's System One model, which is hosted and priced per input token.

Both models return a typed answer, so neither needs a parser. The difference on a Raspberry Pi is where the weights live.

What a Raspberry Pi has to hold

Laya's three checkpoints are a 421M-parameter ModernBERT-large model, a 322M-parameter mmBERT-base model for 100+ languages, and a 421M-parameter checkpoint fine-tuned for typed-decision workflows.

Measured under the independent laya-mlx runtime on an M3 Max, peak allocation is 943.6 MiB for the 421M checkpoint and 687.6 MiB for the 322M one. The published downloads are about 808 MB and about 647 MB. That footprint sorts the Pi family:

laya-mlx is an Apple-silicon runtime, so it does not run on a Pi. On ARM Linux the two paths are the Python SDK (pip install laya, Python 3.10 or newer) and a community ONNX export that ships an laya_int8.onnx build for CPU runtimes.

The AI HAT, and what it actually accelerates

The HAT is the part people assume settles the question, so it is worth naming what each board is designed to run.

The Raspberry Pi AI HAT+ carries a Hailo-8L (13 TOPS, $70) or a Hailo-8 (26 TOPS, $110). It switches the Pi 5's PCIe link to Gen 3.0, and it does not add memory to the host. Its product brief describes the target workloads: object detection, semantic and instance segmentation, pose estimation. In Raspberry Pi's own comment thread, customers objected that this accelerator line cannot serve LLM workloads.

The Raspberry Pi AI HAT+ 2 is the GenAI board: a Hailo-10H at 40 TOPS INT4 with 8 GB of dedicated LPDDR4X memory, priced at $200. Its brief names large language models and vision-language models as the target, and the onboard RAM exists so the model's working set does not cross the PCIe link.

Neither board lists Laya, and neither runtime is a drop-in for it. Hailo devices do not run an ONNX file directly. The Hailo Dataflow Compiler converts a TensorFlow, TFLite, or ONNX model into a Hailo Executable Format (HEF) binary, and that compiler runs on x86 Linux, not on the Pi. The published model zoo for the Hailo-10H covers LLM families such as Llama 2/3, Qwen2/2.5/3, and Phi-2, and the Hailo-8L/8 branch covers classification, detection, and segmentation. Laya is a bidirectional encoder with a custom decision head that scores each option at its own marker token, so it sits outside both catalogs. Running it on a Hailo part is a custom compile rather than a download.

Two further constraints bear on the HAT. On a Pi 5 the accelerator and an NVMe SSD share one PCIe lane, so attaching the HAT costs the fast storage. And on raw decode throughput, independent testing has found the Pi 5's own CPU within striking distance of the Hailo-10H on some prompts; the accelerator's published wins are power, time to first token, and sustained throughput rather than peak speed.

When Laya on the Pi is the better choice

If the board is a Pi 5 with 4 GB or more, four properties follow from running the model on the device.

The link stops mattering. Jev is a hosted API, so a device with no dependable link cannot use it. Laya keeps deciding during an outage, at a remote site, or on a board that never gets a SIM. The ONNX Runtime documentation lists offline operation, privacy, and cost as the reasons to inference on the device.

Latency becomes local. A hosted call carries a round trip; a local call does not. Jev has been measured independently at 236 ms to 276 ms p50, and TypeSafe reports 70 ms to 500 ms end to end. Laya's own card reports 193 ms to 464 ms per request on a preloaded CPU, a figure that is not Pi-specific. No published measurement puts Laya on a Raspberry Pi, so the Pi number has to come from your own board.

The cost stops scaling with events. Jev charges per input token and the fee follows the tokens the device sends. Laya is self-hosted at $0 per decision under Apache 2.0.

State stays on the board, and the weights are yours. Jev sends the state to TypeSafe's servers; Laya computes locally. Laya's Apache 2.0 weights can also be fine-tuned and flashed, while Jev's documentation states it "is not fine-tuned or LoRA-adapted with customer data". On the same 140 labels, Anthus measured a full fine-tune of Laya at 0.896 against 0.870 for Jev with its feedback layer, on one constructed corpus.

The other side: when Laya on a Pi is not the better choice

The same footprint that qualifies a Pi 5 disqualifies a smaller board, and the memory floor is not the only cost.

A constrained board cannot host it. A Pi Zero 2 W has 512 MB, which is below the measured peak. A 1 GB or 2 GB Pi 5 leaves too little once the OS and the runtime are resident. If the product is built on a Zero, or on a microcontroller such as the ESP32 with 520 KB of SRAM, then Laya cannot run on the board at all, and a hosted API is the only path to a typed decision. A local rule set is the third option, and it is the right one when the mapping from input to action is deterministic.

The device is not the whole bill. Laya is free to license and not free to operate. The weights need a place in the image, the outputs need temperature calibration before the probabilities can be trusted, the fine-tunes need a training loop, and the inference stack needs maintaining for the life of the product. Jev moves all of that to TypeSafe.

A good internet connection removes the main advantage. If the Pi has a reliable link, Jev is a call with no image to flash and no runtime to tune, and it leads on the things a decision model is judged by:

• Accuracy before you have labels. Anthus measured Jev at 0.768 against Laya's 0.722 zero-shot on its corpus, and Laya's base checkpoints sit below the majority-class baseline on typed-decisions until they are fine-tuned.
• Wide option sets. Jev scores 0.870 on Banking77 against Laya's 0.425 at default settings.
• Long inputs. Jev lists 64k tokens per request, while Laya's English checkpoint holds 512 tokens and truncates overlong input without an error.
• Calibration out of the box. Jev's raw expected calibration error is 0.144 against Laya's 0.213.

A well-connected Pi 5 with a small, stable schema may therefore pay for a local model and get nothing back for it.

Choosing between the two

The condition that decides it is the link. A HAT changes the power budget and the headroom, and it does not remove the compile step that stands between Laya and a Hailo part.

flowchart TD
    Q1{Does the Pi have 4 GB of RAM or more?} -->|No| H[Use a hosted API: Jev]
    Q1 -->|Yes| Q2{Is the link dependable?}
    Q2 -->|Yes| Q3{Small, fixed schema?}
    Q3 -->|Yes| H
    Q3 -->|No| L[Use Laya locally]
    Q2 -->|No| L

Work with CortexTech

Whether the answer for your board is Laya, Jev, or a rule set, it turns on the free RAM and the link. CortexTech builds the working prototype: we size the model against your board, benchmark it on the hardware you will ship, and report which path holds before you commit to production. Picking a decision model for a Raspberry Pi or another edge device? Contact us and we will measure it on your hardware.

Sources

• Laya — 33ms Multilingual System 1 Decision Engine — Convai Innovations
• convaiinnovations/laya — Hugging Face
• Mattepiu/laya-onnx — ONNX export with int8 build — Hugging Face
• mizorewww/laya-mlx — benchmark table — GitHub
• Jev vs Laya: Same Labels, Same Questions, One Variable — Anthus
• Models — TypeSafe AI documentation
• jev-benchmarks — independent Jev latency — GitHub
• Introducing the Raspberry Pi AI HAT+ with up to 26 TOPS — Raspberry Pi
• Raspberry Pi AI HAT+ 2 — Raspberry Pi
• Introducing the Raspberry Pi AI HAT+ 2 — Raspberry Pi
• Hailo Model Zoo — GitHub
• GenAI Models, Hailo-10H — Hailo Model Explorer
• Every LLM that runs on Hailo-10H — CodeSOTA
• Raspberry Pi AI HAT: ONNX to HEF Conversion — Cytron
• Deploy ML Models on IoT and Edge Devices — ONNX Runtime
• Raspberry Pi — Wikipedia