Image by Author

Introduction

In the last decade, Big Data technology has been extremely disruptive with open source playing a dominant role in shaping its evolution. It has led to a complex ecosystem where new frameworks, libraries and tools were being released pretty much every day, creating confusion as technologists struggled to understand the intricacies of the systems. In 2015, I made an attempt in addressing that (see original article) by demystifying the space. Well a lot has changed in the last 5 years with new ecosystems like Spark and Flink that have disrupted the 1st generation systems, advances in AI and ML and…


CDP Functional Overview

Introduction

Customer Data Platforms(CDP) have recently received a lot of attention as they are becoming an essential part of the digital strategy for an enterprise. This article provides an introduction to CDPs to help demystify their functionality and usage. Basically, a CDP is a marketer-managed which creates a persistent, unified customer database that is accessible to other systems. In general, it is built or bought infrastructure operated by marketers who act on real-time signals that the customers provide to in-turn deliver delightful personalized experiences back to the same customers.

Marketer-managed means that marketers can operate a CDP with minimal assistance from…


AI/ML Landscape

Back in 2015 I had written an article on 100 Big Data papers to help demystify landscape. On the same lines I thought it would be good to do one for AI. The initial part is about the basics and provides some great links to strengthen your foundation. The latter part has links to some great research papers and is for advanced practitioners who want to understand the theory and details.

AI is a revolution that is transforming how humans live and work. It is a broad concept of machines being able to carry out tasks in a way that…


In Part 2 we looked at how Personalization is a message or content that is relevant to the individual user, built on top of Globalization and interacts closely with Experimentation. In Part 3, we will look at how we have built out a Personalization platform that scalably solves for the scenarios we covered in Part 1.

There are 5 foundational blocks to Personalization namely Globalization, Experimentation , ML , Profile and Tracking & Instrumentation. Globalization and Experimentation are covered in detail in prior blogs, ML and Profile are below, while Tracking will be covered in a subsequent blog.

ML Platform

ML is…


In Part 1 we looked at some of the scenarios on how we Personalize to create customer delight in their interactions with us. In Part 2 we will look at a functional framework in how we go about solving for those scenarios.

Let’s start with the basic definition of Personalization. Personalization enables suggestion of message or content that is relevant to the individual user, based on user’s implicit behavior (Optimization) and/or explicitly preferences (Customization). Optimization uses implicit interests and learns what you like from your actions. Customization on the other hand is driven by explicit preferences. Optimization usually is at…


  • By Anil Madan Platform Engineering, Mayank Gupta Product Management Personalization

Our mission at Intuit is to Power Prosperity Around the World for small businesses, self-employed and consumers. We want to generate more money, more time, and more confidence for 50+ million people. By leveraging our data we want to build delightful experiences and unlock the power of many for the prosperity of one. A key capability that powers our mission is Personalization.

In this three part series we will walk you through how we solve in QuickBooks Online (QBO) for customer delight through Personalization. In Part 1, we will look…


In Part 3 we covered the execution engine that supports serving thousands of concurrent experiments to millions of users. In Part 4 we will look into the basics of experimentation analytics.

Overall Evaluation Metric (OEC)

If you recall from Part 2, OEC refers to what measure, objective or goal we trying to achieve. It is a metric used to compare the response to different treatments.

Business Metrics

For QuickBooks, the primary business metrics are:

  • Acquisition — conversion of a prospect to a subscriber.
  • Engagement — how often is the customer using the product and what is the time spent in each visit.
  • Retention — total number…


In part 2 we covered the design and setup of experiments to support thousands of concurrent experiments. In Part 3 we will look into the execution engine that serves experiments.

Logical Architecture

Let’s look at the logical architecture — steps 1 to 3 are design time, while steps 4–10 are runtime, when the user interacts with the product.

  1. The experimenter goes into the tool and sets up an Experiment, defines treatments and optionally factors (for factor based experiments) — this is what we covered in part 2.
  2. The configuration is stored in the Experiment Config database.
  3. It is pushed out to an…


In Part 1 we looked at how Intuit’s culture of design thinking has evolved to embrace rapid online experimentation. In Part 2 we will look into Design of Experiments.

Any experiment is a continuous process of design, execute and analysis. Let’s take a closer look at each.

Fig 1: Experimentation lifecycle flywheel

Design

There are some basic concepts in experimentation

  • Controlled Experiments — A controlled experiment is an experiment done under controlled conditions. E.g. a prospective user is shown an alternative experience to increase sign-ups.
  • Factor (or variable) — A variable that can changed independently (cause) to create a dependent response (effect) . Factors generally have…


This is a 4 part series on experimentation at Intuit. Part 1 covers the culture of experimentation @Intuit while Parts 2 to 4 cover how we have built a powerful and scalable experimentation platform to power data driven decisioning.

Culture

Our mission at Intuit is to Power Prosperity Around the World for small businesses, the self-employed and consumers. To achieve this we use design thinking to deeply understand our customers and solve their problems through Design 4 Delight (D4D).

D4D is ultimately about evoking positive emotion throughout the customer journey by going beyond customer expectations in delivering awesome product experiences.

There…

Anil Madan

Vice President, Data Platforms Walmart

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store