If you’re not excited about data integration, you don’t get it

Digital companies are laser-focused on customer experience. Creating data-driven, actionable insights is one of their biggest weapons in the war to win and keep customers, and this comes from being able to combine disparate data sets – data integration.

It’s not for the faint-hearted. The table below outlines the high-level challenges.

Volume Companies generate immense amounts of data from sources including their networks, services and customers, as well as gather it from other sources like social media	Variety The data comes in many formats, is often incomplete, siloed, incompatible – for instance, structured and unstructured.	Veracity Data should be a ‘single source of truth’ – the cleansing needed to get it to that point is not necessarily simple.	Velocity We know the speed that data is produced at rises all the time. The question is but how fast a firm can act on it in real time or close to it.
Value derived from integrating data to gain insights, improve CX, efficiency, profitability & TTM

Digging down a bit deeper, although we have many data tools, cloudification and cloud native capabilities having a profound impact on data integration, their use is not an automatic guarantee of success.

One example is that some pre-configured data integration solutions require a great deal of additional work for even if only small amounts of customization are necessary to fit into a company’s systems and meet its needs more exactly. Partnering with a company that specialises in data integration can save an immense amount of time, effort and money, and provide a shortcut to better outcomes, in this instance because it can help firms to assess the suitability of pre-integrated solutions.

There are other great tools and technologies that a data integration partner can help companies get the most from.

The art of API management

APIs provide a pragmatic, secure way to expose data internally as well as to ecosystem partners, but to gain their full positive impact they need to be managed properly, through mechanisms like gateways, a developer portal, a publisher portal and an authentication server. This is to ensure they are used consistently across the organisation (customization is the enemy of replication) and access to the right people at the right time and in the right way.

Here’s a readymade way of doing it: The DigitMarket^TM API Manager from Torry Harris Integration Solutions (THIS), described by Ovum (now Omdia) as “a single, comprehensive package providing enterprises with a ready set of tools to create and manage digital platforms and a secure channel to share and monetize data as they proceed through their digital transformation journeys.”

Top management needs to be involved from the conception of the vision and stay involved because data integration can have implications for just about any part of a business and consistency, and a universal comprehension of purpose and the plan does not happen by itself. All transformation efforts need buy-in from the workforce, which only comes from understanding what’s going on and why it’s necessary.

Its On the radar report also said, “DigitMarket^TM allows easy onboarding of partners and curation of content and can function as a marketplace for both physical and digital products (such as APIs) for the enterprise and its partners. DigitMarket^TM is vertical agnostic (although vertical-specific templates are available) and can be white-labeled and configured to allow enterprises flexibility in how they market the platform.”

Why in-memory data grids are hot

Even better, THIS leverages its expertise for the benefit of its customers through collaborations with other market leaders, such as Gigaspace, which offers in-memory data grids (IMDG). They represent a massive step forward for data integration: an IMDG is a set of networked or clustered computers that pool their random access memory (RAM) so applications can share data with other applications running in the cluster.

They are designed to process data at extremely high speeds, and for building and running applications that are too large to run on a single server. They are especially useful for applications that carry out a lot of parallel processing on large data sets.

They can transform data architecture because they allow firms to migrate legacy data integration platforms to cloud-based services through the means of a cloud-based warehouse, that also supports traditional tools – so the customer gets the best of both worlds. Gigaspace also provides tried and tested frameworks like InsightEdge SmartDIH and Smart Cache.

Virtualized data for concrete success

Another example of THIS working with a market leader is its collaboration with Denodo, which offers a data virtualization platform. It can be used to enable API-based or traditional data queries for analytics and business intelligence.

Why is virtualizing data on a platform a good idea? Because it provides access to data across disparate systems, protocols, data formats and structures through a single layer. It provides real-time secure data access and access control for data across an enterprise for internal applications. The virtualized data can be securely exposed through an API gateway for third party consumption.

An ongoing study by Forrester Research has established these numbers:

Are we on track?

Another really important aspect of data integration that a specialist data integration company can help with is to define the desired business outcomes from data integration, which should never be technology driven. Once the vision is complete, companies need a strategy to implement it, which must include ways to assess and measure success, so that the course can be adjusted as necessary.

Data integration must be part of the wider business plan and transformation effort because it can have implications for just about every part of a business: It is not only about operations and IT. Just one example is to consider what Marketing could achieve if it could pull together data from multiple in-house and external sources to build as complete a profile of customers as possible to tailor products and services for them, which creates customer satisfaction and loyalty. Happy customers spend more!

Of course Marketing needs to get new features, products and services to market faster to make the fullest use of customers’ profiles and remain competitive. Faster time to market also relies on data integration in the shape of enabling higher levels of automation which boosts greater operational efficiency, and also helps drive profitability.

Successful data integration is one of the most important digitalization elements, right across the board. What are you waiting for?