Table of Contents

Webhooks setup

Alex Cota Updated by Alex Cota

You can use webhooks to set up workflows that require quick response times. Webhooks are useful for providing automatic notifications when an event happens in Labelbox, such as label updates, reviews, creations, etc. Every time a webhook is triggered, Labelbox will send a POST payload with the relevant information. A notification will be recorded to capture the request and response itself, along with all relevant metadata about the POST. Webhooks can be enabled organization-wide or on specific projects you choose.

This tutorial walks you through how to set up a simple webhook workflow with Labelbox.

Before you start

  • Install flask
  • Install ngrok

Setup steps

  1. Create a mock server that represents the endpoint for Labelbox to hit. Copy this code into your text editor and save as server.py. This is your server file. The secret can be anything you wish and should be specified when you create your webhook in step 4.
from flask import Flask, request
import json
import hmac
import hashlib
app = Flask(__name__)

@app.route('/')
def hello_world():
return 'Hello, World!'

# Make sure this is the same secret you provided in the webhook creation
secret = 'example_secret'

@app.route('/webhook-endpoint', methods=['POST'])
def print_webhook_info():
payload = request.data
computed_signature = hmac.new(secret, msg=payload, digestmod=hashlib.sha1).hexdigest()
if request.headers['X-Hub-Signature'] != 'sha1='+computed_signature:
print('Error: computed_signature does not match signature provided in the headers')
return 'Error', 500, 200

print('=========== New Webhook Delivery ============')
print('Delivery ID: %s' % request.headers['X-Labelbox-Id'])
print('Event: %s' % request.headers['X-Labelbox-Event'])
print('Payload: %s' % json.dumps(json.loads(payload.decode('utf8')),indent=4))
return 'Success'

if __name__ == '__main__':
app.run(host='0.0.0.0', port=3000, debug=True)
NOTE: The request header, X-Hub-Signature, is used to verify that notifications are coming from Labelbox. When you create the webhook and the request body payload, this header is created using the secret you provide. A SHA-1 hash is created using the body and encrypted using the secret as a key. You can verify the webhook notification by creating your own SHA-1 hash using the payload and secret. Validating it is the same as in the header.
  1. Run the file from your terminal and when you visit http://0.0.0.0:3000 you should see “Hello, World!”.
python server.py
  1. Because Labelbox cannot send messages to an endpoint on your local server, you can either deploy this server or use ngrok to provide a public HTTP proxy to your local endpoint.
ngrok 0.0.0.0:3000

You should get a public endpoint that looks like this: http://a5625d01fdab.ngrok.io.

  1. Run this createWebhook mutation in the API explorer. Append /webhook-endpoint to the end of the URL.
mutation CreateWebhook {
createWebhook(data:{
project:{
id:"<PROJECT_ID>"
},
url:"http://a5625d01fdab.ngrok.io/webhook-endpoint",
secret:"example_secret",
topics:{set:[LABEL_CREATED, LABEL_UPDATED, LABEL_DELETED]}
# topics:{set:[REVIEW_CREATED, REVIEW_UPDATED]}
}){
id
}
}

Fields:

  • id (OPTIONAL) indicates a specific project for which notifications should be sent. If None, notifications are sent for all events in your organization.
  • url (REQUIRED) is the URL to which notifications should be sent by the Labelbox server.
  • secret (REQUIRED) is the secret you specified in your server file in step 1.
  • topics (REQUIRED) indicates list of topics this webhook should get notifications for. Must include at least one topic:
    • LABEL_CREATED - user presses the "Submit" button in the editor.
    • LABEL_UPDATED - user presses the "Save" button in the editor.
    • LABEL_DELETED - user deletes a label via the activity table, deletes the dataRow that a label is associated with, or deletes a dataset which contains a dataRow that a label is associated with.
    • REVIEW_CREATED - user submits a review in queue-based review or open review.
    • REVIEW_UPDATED - user updates a review in open review mode.
  1. Test this out by going to your project and create or modify a label to trigger the webhook. You should see the webhook response in your terminal. The following is a sample webhook response for a LABEL_CREATED event.
=========== New Webhook Delivery ============
Delivery ID: ckfgcfsjk09jn06624cy7gb26
Event: LABEL_CREATED
Payload: {
"benchmarkAgreement": null,
"project": {
"name": "Image project",
"deleted": false,
"updatedAt": "2020-09-15T21:19:52Z",
"id": "ckf4fx97mel7z0729qm72dhyv",
"createdAt": "2020-09-15T21:00:27Z",
"description": "Image editor demo"
},
"deleted": false,
"agreement": null,
"label": "{\"objects\":[{\"featureId\":\"ckfgcff8n01v50z9n54gr2a8a\",\"schemaId\":\"ckf4g0be000fm0y3e8oavh4fv\",\"title\":\"Tree\",\"value\":\"tree\",\"color\":\"#e0ff00\",\"instanceURI\":\"https://api.labelbox.com/masks/feature/ckfgcff8n01v50z9n54gr2a8a\",\"classifications\":[{\"featureId\":\"ckfgcff8n01v60z9neim61644\",\"schemaId\":\"ckf4g0beu00fw0y3ehrrv3u75\",\"title\":\"Is the tree healthy?\",\"value\":\"is_the_tree_healthy?\",\"answer\":{\"featureId\":\"ckfgcff8n01v70z9n2f1p1opu\",\"schemaId\":\"ckf4g0bfe00fy0y3ef8puftut\",\"title\":\"Yes\",\"value\":\"yes\"}}]},{\"featureId\":\"ckfgcfnm901m00y7y9kjybi3p\",\"schemaId\":\"ckf4g0be000fm0y3e8oavh4fv\",\"title\":\"Tree\",\"value\":\"tree\",\"color\":\"#e0ff00\",\"instanceURI\":\"https://api.labelbox.com/masks/feature/ckfgcfnm901m00y7y9kjybi3p\",\"classifications\":[{\"featureId\":\"ckfgcfpyd04al0y8ncvdoc9wl\",\"schemaId\":\"ckf4g0beu00fw0y3ehrrv3u75\",\"title\":\"Is the tree healthy?\",\"value\":\"is_the_tree_healthy?\",\"answer\":{\"featureId\":\"ckfgcfpyy04am0y8n19b0favh\",\"schemaId\":\"ckf4g0bfe00g00y3e1l0c3tc8\",\"title\":\"No\",\"value\":\"no\"}}]}],\"classifications\":[]}",
"secondsToLabel": 9.581,
"dataRow": {
"rowData": "https://storage.labelbox.com/ck52rvx0mpw4g0766wgqvjtl9%2Fb56e90f7-0e9d-05f5-cd5e-37cf32f314d5-red-zeppelin-IjXxL75E1Io-unsplash.jpg?Expires=1602132968&KeyName=labelbox-assets-key-1&Signature=w07YrjJqdRdiMo7pgF2DEFYQogo=",
"externalId": "red-zeppelin-IjXxL75E1Io-unsplash.jpg",
"updatedAt": "2020-09-15T21:01:19Z",
"deletedAt": null,
"id": "ckf4fyd7i0itb0bo5gjgjavwf",
"createdAt": "2020-09-15T21:01:19Z"
},
"user": {
"id": "ck52rvx1kqazb07700mq27x4l",
"email": "alexandra+1@labelbox.com"
},
"updatedAt": "2020-09-24T04:56:08Z",
"dataset": {
"name": "lawn-drone-view.jpg",
"deleted": false,
"updatedAt": "2020-09-15T21:01:19Z",
"id": "ckf4fycvwf10f0811b6p2cj3b",
"createdAt": "2020-09-15T21:01:19Z",
"description": ""
},
"id": "ckfgcfsbi00003h5ziwnd8qt7",
"createdAt": "2020-09-24T04:56:08Z"
}

127.0.0.1 - - [23/Sep/2020 21:56:08] "POST /webhook-endpoint HTTP/1.1" 200 -

Modify webhooks

To view the GraphQL mutations for creating, updating, and deleting webhooks, see our GraphQL docs on Webhooks.

Was this page helpful?

Model-assisted labeling (import annotations)

Queue system

Contact