Vision and video
Computer vision, object detection, image classification, OCR and video analytics. Define camera count, input resolution, frame rate, lighting conditions and acceptable errors.
Embedded product engineering
Embedded AI & Edge AI Development
Determine how AI fits your product, then build the processing architecture around the real hardware.
Processor selection begins with the workload, not the part number. We connect model requirements with embedded software, electronics, camera and display interfaces, power and thermal design, and the mechanical, environmental and lifecycle needs of your product.
Start with the result the product needs to deliver. The right design may use no AI, a small model on a microcontroller, CPU inference or specialized acceleration. Dedicated AI hardware is a decision to justify.
01 / Engineering decisions
AI Workload Definition
Describe the decision, detection or output the system must produce before selecting its processor.
Audio and language
Audio processing, speech recognition, keyword detection, natural-language processing, local LLM inference and generative AI. Separate response-time, context, memory and output-quality requirements.
Sensors and equipment
Anomaly detection, predictive maintenance, sensor classification and sensor fusion. Identify sampling rates, observation windows, operating conditions and how a result affects the product.
A keyword detector, a multi-camera video system and a local language model can require very different architectures. Establish representative data, acceptance criteria and a conventional processing baseline where useful.
02 / Engineering decisions
TinyML: inference on a microcontroller
At the smallest end of embedded AI, a compact model may run directly on an MCU without a GPU or NPU. Candidate applications include vibration monitoring, predictive maintenance, keyword or gesture recognition, simple image classification and sensor anomaly detection.
Consider TinyML where power, cost, fast boot or predictable behavior matter. Budget model storage, working RAM, signal preprocessing and inference time alongside interrupts, communications and control.
03 / Engineering decisions
CPU, GPU, NPU, DSP, FPGA and accelerator selection
No processing architecture is universally best. Compare model compatibility, latency, energy, thermal limits, interfaces, cost, lifecycle and development effort.
CPU inference
A useful baseline and often sufficient for modest workloads. Reuse the application processor where measured performance and resource headroom meet the requirement.
GPU acceleration
Parallel compute can suit vision and larger models. Evaluate supported operations, memory movement, runtime dependencies, sustained power and cooling.
Integrated NPU
A neural processing unit may efficiently run supported networks. Check the compiler, supported operators and precision, model partitioning and CPU fallback costs.
DSP acceleration
A digital signal processor can combine signal processing with supported inference kernels. Assess data types, optimized libraries, scheduling and shared workload demands.
Dedicated AI accelerator
An external or specialized accelerator can add inference capacity. Include host integration, PCIe or other transport, data-copy overhead, drivers and supply/support lifecycle.
FPGA implementation
Programmable logic can implement tailored pipelines and I/O. Weigh latency and parallelism against implementation effort, resources, verification and toolchain maintenance.
Heterogeneous SoCs
A system-on-chip may combine CPU, GPU, NPU, DSP, ISP, video engines and other specialized processing blocks. Evaluate the blocks actually present and usable together, their shared resources and the vendor’s software support.
Choose an MCU, MPU, SBC, compute module or custom processor board after the architecture requirements are clear.
04 / Engineering decisions
Model Requirements
Define model architecture and size, memory footprint, training versus inference responsibilities, required inference rate or frames per second, maximum latency, accuracy, batch size and real-time requirements. For language workloads, also define input/context length and response targets.
Record preprocessing and postprocessing, representative data, error costs and expected operating conditions. A model that meets a desktop benchmark may behave differently after conversion, deployment or changes in the input environment.
Training, adaptation and on-device inference are separate resource and development decisions. Agree the model source, data responsibilities and acceptance evidence before implementation.
05 / Engineering decisions
Numeric Precision & Optimization
Evaluate FP32, FP16, BF16, INT16, INT8 and INT4 where the model, runtime and hardware support them. These formats differ in range, precision and execution support; an advertised format does not guarantee every operation runs in that format.
Quantization, pruning and model compression can reduce model storage and memory traffic. Lower precision can also reduce inference time and energy when efficient kernels are available. Conversion overhead, unsupported operations or unsuitable hardware can erase those gains.
Compare optimized models with the reference using representative validation data. Record accuracy, failure cases, latency, peak memory and power. Calibration or retraining may be required; the acceptable tradeoff comes from the application.
06 / Engineering decisions
Measure application performance, beyond TOPS
Advertised TOPS is a peak arithmetic figure, not a prediction of complete product performance. Comparisons must state numeric precision, counting conventions and any sparsity assumptions.
Evaluate model architecture, accelerator utilization, memory bandwidth, conversion quality, supported operators, software toolchain and driver maturity. Measure the complete path from input capture through preprocessing, inference, postprocessing and output.
Benchmark sustained operation at the required power and temperature, with other system functions active. Report latency distribution, throughput, memory use and energy alongside application accuracy.
07 / Engineering decisions
Memory Architecture
Plan RAM capacity, bandwidth, model storage, cache, on-chip SRAM and external memory such as LPDDR. Include weights, intermediate activations, input/output buffers and the operating system and application working sets.
Shared or unified memory may reduce explicit copies, but CPU, GPU, NPU and other engines still compete for bandwidth and capacity. Review synchronization, buffer ownership, memory access and transfers between accelerators.
High theoretical compute performance can remain unused when data movement is the bottleneck. Measure memory pressure and concurrent camera, display, networking and storage traffic.
08 / Engineering decisions
Edge, cloud or a hybrid architecture
On the device
Consider local inference for bounded latency, offline operation or keeping selected data local. Account for device compute, memory, energy, thermal design and update responsibilities.
In the cloud
Consider remote compute where model size or processing needs justify it and connectivity is acceptable. Evaluate data transfer, privacy controls, recurring cost, service latency and availability.
Hybrid
Partition acquisition, filtering, inference and review across device and cloud where appropriate. Define behavior during disconnection, synchronization, version compatibility and recovery.
Evaluate latency, connectivity, privacy, data volume, bandwidth, reliability, offline operation, power, compute requirements and recurring cloud cost together. Edge AI is one architectural option, not an automatic preference.
09 / Engineering decisions
Hardware & Environmental Constraints
Fit compute into the product’s power budget and thermal envelope. Evaluate passive versus active cooling, mechanical packaging, board area, airflow, mounting and the effect of sustained load.
Capture industrial temperature, vibration, shock, EMC, reliability and service-life requirements. Include component availability, production quantities, lifecycle commitments and replacement paths in the platform decision.
Verification plans link these requirements to test conditions and the intended configuration. Prototype performance does not establish environmental qualification or regulatory approval.
10 / Engineering decisions
Interfaces that connect the complete product
Capture and sense
MIPI CSI and other camera interfaces; UART, SPI, I²C and sensor interfaces. Check bandwidth, synchronization, signal integrity, cabling and driver support.
Display and communicate
MIPI DSI, HDMI, DisplayPort and other display interfaces; Ethernet, CAN and USB. Budget display, network and control traffic alongside inference.
Connect processing blocks
PCIe and appropriate internal transports connect processors, storage and accelerators. Define transfer latency, throughput, recovery and power sequencing.
11 / Engineering decisions
Software Ecosystem & Deployment
Compare bare metal, FreeRTOS, Zephyr and embedded Linux around the complete system. Include driver availability, scheduling, boot time, security, updates and long-term support.
Evaluate TensorFlow Lite / LiteRT, TensorFlow Lite Micro / LiteRT for Microcontrollers, ONNX and ONNX Runtime, PyTorch export/deployment workflows and vendor AI SDKs as appropriate. Check the exact model, operator set, conversion tools, runtime version and target support.
Build hardware abstraction and clear component interfaces where they improve portability. Identify dependencies on vendor compilers, binary drivers, kernels and SDK versions, and retain reproducible model-conversion and software builds.
Plan signed or otherwise controlled update delivery, rollback, diagnostics, model/version compatibility and data handling according to the product’s requirements.
12 / Engineering decisions
Heterogeneous Processing Architecture
Distribute work according to the processing engines available and the latency, memory and power budgets. One possible pipeline is:
- Camera / sensor
- ISP or signal processing
- NPU / GPU / AI accelerator
- CPU application logic
- Display / network / storage / control output
The CPU still coordinates the product.
An accelerator usually does not replace the CPU. The CPU may manage the operating system, application logic, communications, device drivers, user interface, control logic and system orchestration while the accelerator executes supported neural-network operations.
13 / Engineering decisions
From requirements to production
Model deployment and acceleration decisions belong inside the embedded product-development process.
- Requirements definition
- AI workload analysis
- Processor architecture
- AI acceleration decision
- OS / RTOS / bare-metal decision
- Memory architecture
- I/O architecture
- Power & thermal design
- Hardware platform selection
- Model deployment strategy
- Software architecture
- Prototype
- Benchmarking
- Verification & validation
- Production transition
These decisions develop together. Prototype measurements can send us back to the model, memory, platform or cooling choice before the production configuration is committed.
Engineering outputs, agreed to the project
Typical outputs include requirements and architecture records, a processor/platform trade study, model-deployment plan, prototype integration, benchmark and validation evidence, reproducible builds, and production/handover information. Define acceptance criteria, data and model ownership, maintenance responsibilities and support scope early.
Bring the product requirement.
You do not need to choose the processor, RTOS, compute module or accelerator first.
FPD.DEV provides embedded system architecture and engineering for products that may combine AI, machine learning, computer vision, signal processing, displays, sensors, communications and control. We connect the model to the electronics, software, interfaces and physical product, through verification and production deployment.
Tell us what the product should accomplish, what data or model is available, and the timing, power, environmental and lifecycle constraints.