How does 6sense import data from Salesforce?

Prev Next

Overall Flow:

Screen_Shot_2021-06-28_at_2.20.01_PM.png

Technical Details

6sense uses Force Partner API and Salesforce WSC (web service connector) library, which provides a connector to connect to salesforce’s web service. Following are the steps we follow to sync/import data from Salesforce.

1. Get the Salesforce Object Schema

  • We perform a describe operation (describeSObject()) using the SOAP API on an object we want to get the schema for. All the field names that the 6sense’s integration user has access to, will be part of the response here.

  • A customer can apply restrictions to certain fields in their Salesforce instance if they don’t want them to be synced by 6sense.

  • Internally, describeSObject() method calls the services URL for the customer’s Salesforce instance (e.g. for 6sense we use https://6sense.my.salesforce.com/services/Soap/u/41.0) and pass an XML payload that contains the name of the object (e.g. Ac count) and the type of operation (i.e. describeSObject)

  • We just need read permissions on the objects to be able to call the describe API.

2. Create Job and Batch

  • Depending on the timeframe we are syncing data for, we either create a single or multiple salesforce job and a batch within it. 6sense uses Bulk API Query to query the object data.

  • To create a job we use the following endpoint:
    https://<instancename>.salesforce.com/services/async
    /<api_version>/job

  • To create a batch within a job we use the following endpoint:
    https://<instancename>.salesforce.com/services/async
    /<api_version>/job/jobId/batch

  • Each batch we create has a SOQL (Salesforce query language) query attached to it which specifies the field names, timeframe, and the object name that batch is responsible to gather data for. Example of SOQL query:
    SELECT <field names> from FROM contact WHERE ((CreatedDate >=
    2021-06-01T00:02:40+0000 AND CreatedDate < 2021-06-02T00:01:
    43+0000) OR (LastModifiedDate >= 2021-06-01T00:02:40+0000 AND
    LastModifiedDate < 2021-06-02T00:01:43+0000));

3. Syncing Data

  • Once a batch is created, Salesforce begins processing it. The time it takes to process the batch will depend on the timeframe in the query and hence our process will be periodically poll the status of the batch using the following query:
    https://<instancename>.salesforce.com/services/async/<api_version>/job/jobId/batch/batchId

  • Once the batch processing is successful, we use the following endpoint to retrieve the result:
    https://<instancename>.salesforce.com/services/async/<api_version>/job/jobId/batch/batchId/result