Data Innovators: Tal Shaked

To kick-off a series of conversations with Data Innovators, here's an abridged version of a recent sit-down between WisdomAI CEO, Soham Mazumdar, and Moloco Chief Machine Learning Officer, Tal Shaked.

Founded in 2013, Moloco is a machine learning company focused on performance advertising. Moloco's mission is to empower businesses of all sizes to grow through operational machine learning. Moloco, one of Business Insiders Hottest Ad Tech companies of 2023, recently announced a secondary transaction that valued the company at more than $2B.

‍Let’s start by describing your current role. 

‍I'm the Chief Machine Learning Officer at Moloco [Note: Moloco stands for Machine Learning Company]. I intentionally chose not to be the Chief AI Officer because, as I see it, ML is the real engine behind the scenes, that ultimately drives the end user experiences, which humans experience as artificial intelligence (AI).  

At Moloco, I manage teams spanning machine learning, data science and infrastructure. As the CMLO, part of my role is figuring out what it means to build and organize a company centered around machine learning.  

What does one have to keep in mind when they approach Machine Learning?

‍Quality. Machine learning is first-and-foremost about solving quality problems. 

Starting with a product example, when a user asks a chatbot a question, how do we rank possible responses from best to worst? It's fuzzy, and it's a quality problem. This is one of the most important lessons that I learned while working on machine learning at Google. These quality problems are fundamentally different from more traditional infrastructure and deterministic engineering problems. Traditional engineering problems that involve deterministic workflows and more predictable user interactions generally focus more on scale, latency, correctness, accessibility, and other aspects that can be planned for and controlled during design. That's not the case when using machine learning to solve quality problems. Instead, it's all about defining your objectives, managing and understanding the data, evaluating the performance of the product over time, and developing the appropriate framework to iteratively debug and improve the learning system across a diverse set of teams and users.  

Establishing an ML-first culture that incorporates all these elements into how we develop products is a core part of my role. This spans shaping our vision, hiring and organizing our teams, building the right performance products, and figuring out the best ways to sell and integrate our technologies so that our partners can grow their businesses as quickly as possible. 

You've been working in ML for 20+ years. What else have you learned along your journey, starting with our time together at Google? 

‍A lot of what I've learned started during our Google days. When we joined [Note: Tal and Soham both joined Google's Search team around the same time in late 2004, the month of Google's IPO] there was a lot of focus on search quality as an art, ensuring that folks with deep domain expertise were engineering the features and scoring functions necessary to rank the documents which show up when you search on Google. I was young, naive, and stubborn - I'd just dropped out of my PhD program - and wondered why we thought humans could build a better ranking system for web search than a machine learning system. We had around 50 people spending all their time writing thousands of lines of if-else functions and other complex logic to decide how to rank search results. Why write such a complex ranking algorithm that is hard for people to understand if we can just have machine learning do it in a simpler and better way? 

I led a small team that spent 3 years plugging away at this problem, concluding that without a doubt, machine learning systems could indeed rank results better than humans on just about every metric except one very critical one: can you explain WHY a particular result showed up and can you tell me what I need to change to fix a bad result. 

Machine learning systems are always going to make mistakes, and therefore you need a quality mindset to develop and improve software and products that use machine learning. When you look at ML-powered products such as WisdomAI, chatbots, and self-driving cars, you’ll find an amazing engine that has some superhuman capabilities, but also the tendency to make mistakes that a human will never make. This gets back to the difference between ML and AI: how do you build a user experience on top of those superhuman machine learning capabilities so that people interacting with those artificially intelligent products have amazing experiences despite the non-human behaviors from the underlying machine learning engine? Unlike self-driving cars, for performance advertising and web search, if you provide a low quality recommendation to a user, it's not the end of the world. Working through all these challenges is what I mean by quality work. 

What makes machine learning different from more traditional software engineering? 

‍As a discipline, ML engineering is a superset of software engineering. When you develop ML-powered products, you need to do everything that you do for typical software development, and then a lot more. You have to think about data in a completely different way. In addition to developing complex data pipelines, the data needs to be high-quality and organized in different ways to support a variety of unique ML scenarios. This includes large offline training jobs, continuous learning in order to adapt as the world changes, and real-time situations to enable the learning system to behave appropriately to the unique characteristics of any given situation. ML systems produce models that interact with the world and humans in non-deterministic and unpredictable ways. This makes it difficult to align training objectives that produce models with the evaluation criteria that people ultimately care about when they interact with ML-powered products. As a result, it is difficult to test, debug, and ultimately measure the quality of an ML system. This is what we learned 20 years ago and what the entire world is learning today as part of the ML and AI revolution. Here are some examples: 

• With a self-driving car, is the ride too jerky or sufficiently smooth? Can the car make left turns and manage high-traffic situations with the same efficiency as a human? 
• With a chatbot, does it sound creative, too dry, racist, or biased? Does it surprise me with its answers or does it sound robotic? Could those surprising answers be examples of hallucinations which is a well-known problem?

All of these examples illustrate quality problems that are different from typical software engineering. Traditional software systems often focus on more deterministic and predictable requirements such as  scale, latency, and correctness given certain inputs and assumptions about the world. ML-powered systems interact with humans in more dynamic ways and have the capability to continually learn and improve, which changes how we develop such software. 

ML engineering is a new discipline that expands upon software engineering to include data science, modeling, dynamic learning systems, and quality. You and I understand this because we grew up in Search Quality and Ads Quality teams at Google, but many people working on ML and AI today haven’t had such experiences with ML engineering and working through quality problems. 

What's your favorite recent example of using AI to solve problems? 

‍I love a story I heard about a dog and his owner. The owner took his sick dog to the vet, and test results indicated that the dog had an incurable illness and needed to be put to sleep. The owner was understandably upset with this diagnosis, and decided to feed the test results to a chatbot to get a second opinion. The chatbot agreed with the vet that the most likely outcome is that the dog has an incurable illness and needs to be put to sleep. However, the chatbot also suggested a couple other very rare and treatable conditions that could explain why the dog was sick. The owner then took his dog to another vet for a second opinion, guided by the chatbot's alternative diagnoses, and discovered that the dog indeed had one of those rare, treatable conditions. This illustrates how a chatbot with superhuman abilities to process test results and medical literature was able to save the dog’s life. 

I also read a similar article (link here) about a mother using ChatGPT to diagnose her 4-year-old son’s condition after seeing 17 doctors who couldn’t figure it out. This illustrates how GenAI can operate like a superhuman doctor who can explore corner cases to help with finding diagnoses that other doctors may miss. The amazing thing is that these technologies and capabilities are getting better every day. 

Can you give us an example of how Moloco is using ML in a real world context?‍

I’m excited about a recent partnership between Moloco and Scopely that has helped Monopoly GO! become the #1 mobile board game worldwide. Scopely created performance advertising campaigns with Moloco across Android and iOS that drove more than 1 million installs in the first month and many more since then, helping Monopoly GO! surpass 150 million downloads a couple weeks ago. Moloco uses operational ML to continuously train models that determine what ads to show on mobile devices. These models make predictions in tens of milliseconds for more than 100 million instances per second, processing more than 500 billion requests per day. This enables Moloco to find the most profitable users for Scopely across the world and show them ads for Monopoly GO! at the right time and for the best cost. Very few companies in the world, especially at the size of Moloco, have real-time ML systems that operate at this scale. 

Why are you excited about what solutions like WisdomAI can offer?

‍I think about it in terms of the amount of time it currently takes to produce data insights. Let's say we're in an exec meeting and somebody asks a question. In fact, let's say it's the CEO who asks a question. Maybe it's some metric for a specific country's users or how some metrics have been changing over the past 6 months. Those simple questions can become projects and then maybe a week later the answers come back. Then often the exec wants to dig deeper and the iterations can go on for weeks. If anybody could answer these questions on the fly it would be like magic. And it would speed everything up. 

Dashboards work ok, but they're static and provide limited capabilities for real-time iteration, especially for non-experts. If you can build dynamic dashboards that adapt to the questions being asked, that could be such an improvement. Over the last 20 years, Google Search evolved from a simple search box that returned a static set of results to a dynamic, personalized experience that helps users discover new information and content in a variety of formats spanning maps, videos, movie times, Q&A, related searches, and so much more. I see a future where WisdomAI personalizes data exploration and insights to enable every user, from a new employee to the CEO, to be more data-driven, efficient, and productive.

Thanks, Tal, it’s always a pleasure to hear your thoughts on AI and technology.