Overview
Computer Vision in Fulcrum lets you run machine learning inference on photos directly within a Fulcrum app — entirely on-device and offline on mobile. This allows data collectors to automatically fill in sections of a form based on what's in their photos.
This feature is only available with Elite and Enterprise plans. Check out our plans page for more information.
Fulcrum currently supports Object Detection - finding and locating multiple objects within a single photo, each with its own label and bounding box.
How It Works, at a High Level
You bring a trained object detection model (your own or a third-party one) suited to your use case.
The model is uploaded to your Fulcrum app as a reference file.
A Data Event is configured to run inference automatically whenever a photo is captured on that form.
The inference output — labels, confidence scores, and bounding boxes — can be used with
SETVALUE()to automatically populate designated fields on the record.
This runs on-device, fully offline, on the Fulcrum mobile app. It is not supported in the web Record Editor.
Setting It Up
Computer Vision is configured per app, using Fulcrum's Data Events. Setup involves uploading your model as a reference file, then writing a short Data Event that listens for new photos, runs inference against your model whenever one is captured, and uses SETVALUE() to write the results to a field.
For the full technical implementation — including model requirements, configuration parameters, and code examples — see the developer documentation for INFERENCE().
Running Computer Vision Inference
After setup is complete, computer vision runs on-device and offline on the Fulcrum mobile app whenever photos are added to the designated photo fields within the form. The inference output can be used with SETVALUE() to populate the designated field(s) automatically.