External Services Integration Strategies

In today's world, the systems you are developing are not isolated, they need many different integrations with other external services. From Authentication Systems (using SAML, LDAP, OIDC) to Data and Processing Systems (Malware Analysis, CRMs, Ticketing Systems).

Although the number of actual systems to integrate with is vast, luckily the Use Cases and types of integrations are manageable. Here we will discuss popular Use Cases and list integration strategies to accomplish them.

Use Case 1: Transfer the User from your System to the External Service

In this scenario, the User is working in your system and is now required to access specific functionality in an External Service. The following steps must happen: the service must be located, the user must be authenticated, and the right interface/data must be presented.

1.1 Service Locator Integration

Your system understands how to find the specific service, the user does not need to remember the location of the external service he wish to access. Thus if the user indicates to your system that he wants to be transferred to the external system (e.g. clicks on a "mail" icon), then your system resolves the required hostname and protocols of the requested service that is applicable to that user.

1.2 Authentication Integration (SSO)

Each service has by default its own authentication mechanism. When Authentication Integration is in place, the user does not require to provide his credentials to authenticate himself in the External Service. Your system will guarantee (through the same authentication mechanism or otherwise custom code) that the user will be "pre"-authenticated.

1.3 Context Integration

Most of the time, the user is looking a specific data inside your system, and then would like to be transferred to the external system having the same context. Think about a case listed in your Case Management System, and you would now like to be transferred to the Customer Relationship Management system to see the reporters details. With Context Integration the user has no longer to navigate himself to this page (context).

Use Case 2: Access the Data in the External Service from your System

The External Service contains Data that your System needs to have access to for processing.

2.1 Extract-Transfer-Load Integration

In database systems theory this approach is also called Data Warehousing. All the data is copied from the external service into your system. Your system operates on the local copy.

2.1.1 Data Push Integration

The external service is responsible for publishing its data into your system. The benefit is that no changes are required to your system when the external service is upgraded.

2.1.2 Data Pull Integration

Your system is responsible for extracting the data from the external system. This is a great solution in case the external system is beyond your control.

2.1.3 Hybrid Integration

In a hybrid approach both your system and the external service are combined responsible for obtaining the data in your system.

2.2 Query Integration

In database systems theory this approach is also called Data Integration. The major drawback with Data Warehousing is that data is often not realtime synced. Also changes to data schemas required a reload of the data. This is where Query Integration comes into play.

Query Integration is great to have on-demand access to data sets. A unified query language is presented in your system, and the queries it generates are translated to native queries for the external service. The external service returns in native format the results, and your system translates those native values to your systems data model. There is no need to have a data copy of the external service in your system.

Use Case 3: Make Functionality of the External Service available in your System

Many times your System needs to coordinate specific functionality with an External System. For example for the detonation of a Malware Sample.

3.1 API Integration

The functionality is accessed directly through the native APIs that the External Service is exposing. This means hard-coupling between the External Service and your System.

3.2 Connector Integration

First an Interface is defined for the needs of the business logic of your System. Next an Adapter is written that transforms the native API of the External Service to the Interface. This is made available in a Connector. This allows your System to switch from External Service by swapping the Connector (loose-coupling).

Use Case 4: Control your System from the External Service

Here we would like to be able to control the system from within an external service.

4.1 Client Integration

An extension or plugin is written and deployed inside the external service. This extension makes use of the exposed APIs of your system to control the state of your system.

Use Case 5: Inter-Operability between External Service and your System

This means that your System and the External Service can be controlled from both places and the state is reflected in both. This can be achieved by combining the API Integration or Connector Integration with the Client Integration.

Conclusion

There are a number of popular Use Cases and integration strategies. Having a good understanding helps making the right decisions for your software. Often implicit decisions are made on integrations, and your system becomes more hard-coupled and less extensible then needed.