Batch Processor Overview

Batch Processor

Provide a means to process a file of many address records through the LightBox goecoder at the same time. Batch processing is significantly more efficient due to the dramatic reduction in network latency caused by individual geocoding requests. Our recommendation is to use Batch Processing for workloads outside of interactive geocoding or autocomplete use cases.  Multiple files can be uploaded and processed simultaneously.  Each file or job can have a unique or consistent configuration of the data attributes to be retrieved.  Batch processes are secured as they are only accessible to the user creating the uniquely identified job, each job has a private job identifier.  Batch jobs can be started, canceled and checked for the current process status including an estimated time to complete

The process involves:

• Obtaining a secure and unique upload URL and a job token to a secure cloud storage

• Uploading the CSV file to a specific job token

• Start the geocoding process

• Use the job token to monitor the job

• From the job token, retrieve a secure download link to the completed CSV file

• Download the CSV file

Features

• Obtain a secure upload and download link for the csv files

• A secure job token, is used to monitor the job with secure access to the job’s files

• The ability to start or cancel a job

• Supports very large files > 5GB

• Flexible field output

• API Health check endpoint

• Job processing status with estimated time to complete

• Batch Processor Documentation

• Batch Processor Release Notes