Pushing Fulcrum Data to CARTO With Webhooks

Use Fulcrum to collect data in the field and display it in real-time using CARTO

The standard Fulcrum workflow is as follows:

Design and build your custom data collection form
Optionally import existing data into your form
Collect new data and/or update existing data
Export your data out of Fulcrum for use in other applications

While this workflow provides a lot of flexibility in how you ultimately use the data you’ve collected with Fulcrum, it does involve several steps, which introduces a delay in the timeliness of your data.

Real-time data

Imagine a project where you are using Fulcrum to collect information on the condition of buildings after a major disaster or storm. Your job is to get trained surveyors on the ground inspecting the level of damage so they can report back on the severity of the event and mobilize the necessary resources. The event may have also caused an evacuation of the area and residents are eager to return to their homes. Time is of the essence and you need to make your information available to as many people as possible as soon as it is collected.

You could manually export a GeoJSON file out of Fulcrum every so often and script a Leaflet map to display the data points, but that can be time-consuming, and requires some programming knowledge. You could also leverage the Fulcrum API to pull the data directly into your map, bypassing the export step, but this still requires some coding. Fear not, Fulcrum Webhooks are here to save the day and turn you into a local hero!

Using CARTO to visualize your Fulcrum Data

CARTO is a fantastic service that provides intuitive, web-based tools for GIS and mapping. Like Fulcrum, CARTO harnesses the power of PostGIS for spatial data processing. Where Fulcrum excels at mobile data collection and management, CARTO excels at dynamic data visualization and mapping.

Using Webhooks to Push Your Fulcrum Data to CARTO

The plan is to set up a CARTO visualization and have a Fulcrum webhook pushing record changes directly to the CARTO table behind the viz. I’ve written a fairly simple PHP script that will be used as the webhook endpoint. This script can be modified with a text editor, simply replacing the CARTO and Fulcrum info with your own API keys, table name, data fields, etc. Once you’ve modified and tested the script, you can drop it on any webserver with PHP and cURL configured and add the URL as a new webhook on the Fulcrum organization settings page.

The process

Build your Fulcrum App and Export a CSV to Build Your CARTO Table

I used the Damage Report app from Fulcrum’s App Gallery.
Make note of the Data Namevalues for your fields, as these are used in the webhook script.
Once your app is configured the way you want it, add at least one data point for testing and export purposes.
Export the data out of the app in CSV format.

Create a New CARTO Table

Log into your CARTO account and add a new table by clicking the “New Table” button.
Under the “Local file or URL” tab click “Select a File” and browse out to your exported CSV file.
Once the import is complete, preview the table and map and delete any columns you don’t need in CARTO.
DO NOT delete the fulcrum_id field!
You may remove the latitude & longitude fields as the coordinates are written to the CARTO PostGIS geometry field (the_geom ).

Configure the Webhook Script

Download the fulcrum-cartodb-webhook.php script and open it in a text editor.
Fill in the CARTO & Fulcrum info for your accounts.
You can find the Fulcrum form ID by opening the Fulcrum web app in your web browser and copying everything between data/ and # in the URL bar.
Edit the custom form field variables with the data name values (starting at line 151) from your Fulcrum app (these should be the same as your CARTO field names if you didn’t modify them).
Modify the SQL statements for creating, updating, and deleting your CARTO records as necessary.
Copy the fulcrum-cartodb-webhook.php file onto your web server and make note of the directory/URL.
Create the webhook reference in Fulcrum by adding the URL in the “API Webhooks” section in your organization settings page.

Testing the Webhook

This webhook captures record.create , record.update , and record.delete events, writes the changes to CARTO, and spits out cartodb.sql and payload.json files to your server for debugging or further inspection.
To test it out, create a new record or edit an existing record in Fulcrum (via the web, mobile, or import).
Changes should be immediately reflected in CARTO. You may have to refresh the map view by zooming or panning and the table view will only update after refreshing your browser.
If you have trouble, you can inspect thecartodb.sql and payload.json files that should have been written to your server.
Try copying the cartodb.sqltext and pasting it into CARTO’s SQL pane for additional error information.

See the full code example of the PHP script.

Wrap up

Hopefully, this guide exposed you to some of the powerful capabilities made possible by Fulcrum webhooks. With real-time access to your data records and push notifications for every change made, you now have complete control of your data lifecycle.