It’s common within a continuous testing environment for companies to rely on a device lab to allow remote testing at any time and from any location. It’s a good way for DevOps teams to keep pace with the constant need to validate new releases across multiple devices, preserving quality while maintaining a testing practice that is scalable and cost efficient.

Why use a real device lab?

There are many alternatives to the approach of using a device lab for testing – for example using emulators and simulators to mimic real world environments. While those can be quite powerful, they lack the realism of an actual device, often with limited options for things such as software and hardware configs.

At Lynchpin we configure, deploy and support complex analytics and data ecosystems to support customer insights for digital products and services. Especially within B2C environments we find that we have to ensure quality and fidelity of analytics technologies such as Google Analytics, Adobe Analytics across an increasing number of platforms and devices.

Deploying analytics across these platforms is a complex business. Just as it’s vital to ensure the customer experience and brand experience is consistent across customer touchpoints, so too is the need to ensure consistency and quality of metrics that measure how customers interact across these services. Without that consistency, quality and accuracy, valuable data science projects such as marketing attribution, pricing analytics, content effectiveness and segmentation can become constrained by poor quality data.

Our approach

Investing in a robust process for testing analytics is an important and core component in a healthy analytics ecosystem. At Lynchpin, our engineers use a variety of tools and technologies to build scale and effectiveness for managing complex analytics setups. For example, we make use of Selenium and Appium to automate (many!) customer journeys to then understand what data is sent to Adobe or Google for subsequent processing in their analytics platforms. We then use our own tooling to quantify any anomalies and ensure that data from devices matches expected values from the original analytics specification. Very quickly multiple journeys across multiple devices, browser versions becomes a huge volume of data to analyse.

We are fortunate to work with many brands who value our approach – and that means we need to think about supporting our clients’ needs across increasingly capable technology. In building analytics testing resilience into what we do, we learned quickly that there is a need to deploy analytics onto real devices, where we can safely and securely manage the entire testing process, giving our clients rapid results to match their equally rapid app or website updates!

Ensuring resilience

It can be quite the challenge to get analytics onto some exotic devices. In another blog, we’ll talk through some of the tip and tricks for building a scalable multidevice analytic deployment ecosystem. It can be an equal challenge to then find solutions to intercept network traffic to then provide the comport our clients need to verify analytics implementation vs spec. When it comes to app analytics, multiple stake holders, rapid (urgent) release cycles and changing requirements often combine to form an intricate recipe for analytics. Once the mix is in the oven, its then too late to make changes to ensure analytics is baked in properly. We’ll talk about our techniques to build resilience into the sub processes related to network traffic analysis in another blog – stay tuned!

What we provide

In practical terms, we’ve invested in building our own analytics device lab. We’ve got an array of devices for which there are very specific ways to embed and to extract analytics! For our clients, this means to have a complete end to end capability so that we can provide certainty on the deployment of analytics, that the right data points are collected and therefore that the right metrics and KPIs are available for stakeholders to analyse.

Benefits of a device lab for analytics:

  • Scalability
  • Speed of execution
  • Coverage across the entire data lifecycle from spec design to deployment and testing
  • Data quality and consistency
  • Reliability
  • Device freshness – adding devices to our workflows is straightforward

Some of our devices:

  • Vast range of iOS devices (phones and iPads)
  • Vast range of Android devices (phones, tablets, media streamers)
  • Games consoles (including PlayStation and Xbox devices)
  • Set top boxes (including Freesat, Freeview, YouView etc)
  • Amazon devices
  • Roku devices
  • Vast range of smart TVs covering various OS types (e.g. LG, Sony, Samsung, Hisense)
  • Chromecast

If you’d like to find out more about our end-to-end Adobe Analytics and Google Analytics support for omni-channel, omni-device analytics please get in touch!