Warehouse analyzing data
Article

DeepRacer Fireside Talks: How to build an effective data science team to fulfill business goals

Mary Willcock is a data scientist manager on our Baker Tilly Digital team with experience in advanced data analytics solutions. In the first fireside chat of Baker Tilly Digital’s DeepRacer series, Mary talked with Anne Marie Webber, director of modeling and analytics at CUNA Mutual Group, a mutual insurance holding company in Madison, Wisconsin.

Mary Willcock:

Anne Marie, describe your current position.

Anne Marie Weber:

My current position combines my training in economics and advanced analytics. I focus on how we can leverage models in an entrepreneurial setting, so that there's financial value for the business. My group brings an almost startup mentality lens. Not just asking if there's predictive value to a model, but will the entire system provide value given the cost that it will take to stand it up.

Mary Willcock:

Where in your organization do you sit and how does your team connect with the rest of its IT resources?

Anne Marie Weber:

I sit within an analytics team that serves the marketing side of our direct-to-consumer business. We've created an operating model where we can serve other areas if we have bandwidth for it. We have a dedicated IT team that directly supports our work in TruStage, our direct-to-consumer life insurance product. We work through their manager to help prioritize various efforts.

CUNA Mutual has both a B2B side and B2C side. Since consumer data stretches the entirety of the enterprise, it’s been essential for us to think about building these data systems that will benefit a broader group of people. CUNA Mutual has created an enterprise consumer data team who we regularly partner with for creating data assets.

We focus more on the data science. Both teams have data engineers, but our data engineers are focused more on preparing datasets for the data scientists, especially for new projects where there is a longer data discovery phase. We found that it's valuable to have someone focused on the data side, and someone focused on the modeling.

Mary Willcock:

Why should we even do data science in the first place?

Anne Marie Weber:

If we think about data science as prediction, we make predictions in our everyday lives all the time. Data science is prediction on steroids. Our human brain does some things well and computers and machine learning do other things well.

Data science is often done best when it's a triaging model that says some consumers can be automatically pushed through the process, and other consumers need a human for review. I think about things from an insurance lens. Using data science effectively means that an underwriter who may have handled 1,000 cases a year can now handle 2,000 or 3,000 cases per year. There is a benefit in having human and machine work together.

When I think about data science within a business context, I think about it adding value from the prediction side and in optimizing the amount of brainpower that humans use. This is no different than any other digital transformation that's happened in the last 50 years, like going from typewriters to word processing: we were able to change the way that we did our tasks. We're still doing underwriting, we're still doing claims, we're still doing customer service. We're just working in a different way and we can handle higher volumes because of it.

Mary Willcock:

So how do we take data science and drive our business with it? What kind of team do you start with and how do you prove to a business they will get value out of it?

Anne Marie Weber:

Start with a small team – under five people – to tackle use cases that are low-hanging fruit. This team should focus on creating pilots and proof of concepts that are as low cost as possible, and ideally, that the modeling team is running themselves. It's difficult to ask, “Can I have $1 million to do this thing?” But if you do a pilot and you say, “If we did this in real time, we would have stopped $10 million worth of fraud. Can I have $1 million?” you likely will get the funding you need. 

Mary Willcock:

Who is the ideal data scientist for this small team? 

Anne Marie Weber:

The ideal data scientist has expertise in programming and basic computer science; statistics, math and modeling; business; and communication visualization. Since that ideal person likely doesn’t exist or would be too expensive to hire, the next best thing is to cover those four areas within a team setting.

When you're designing a team that's small, you're going to need people who wear a lot of hats. You want a well-rounded team that knows a little about a lot to be able to find and implement low-hanging fruit use cases. You may want to mix it up in terms of how many people are focused on modeling and how many are focused on data engineering. Then the manager of that team can help with prioritization and help with the strategy around the use of the models, and also be able to coach the team about how to work with the business.

Having a mix of talents on a team helps you avoid the scenario where a data scientist creates a model that can predict something, but a business-focused person on the team notes that the model and output isn't actually useful for the business.

The secret behind this small team isn’t about what they do with the data, it’s how they explain what they did to the business. People that can communicate well do well in startup teams.

Mary Willcock:

Can you talk more about an instance where the model is correct, but the business user says it doesn’t work for the business?

Anne Marie Weber:

I have two examples. One where it was obvious to everyone that the model wouldn’t help the business, and one where the conclusion was more subtle.

 In the first example, the team was working on a claim satisfaction model and they got to the piloting stage. They wanted to see if the model predicted whether a customer would be satisfied with their claims experience after it had wrapped up. The action was to call the customer to see how they were doing. The model worked: they found very angry customers who took advantage of the call to request to cancel all of their policies. So the model found angry customers, but they couldn't find an action that actually did anything of value. The entire process must have some value to the business.

The more subtle example involved a call center. They were looking to predict customer satisfaction from the call itself. You can get satisfaction scores from call centers, but they rely on customer surveys, which may be right after the phone call is completed or may take place the next day. There's not a high response rate from those, and you usually end up with skewness in terms of angry customers, but also skewness in terms of the demographics of who actually picks up the phone in the middle of the day. They were looking for a way to be able to predict customer satisfaction for every single call.

They were able to create a model, which the data scientists said, on aggregate, worked. The business leaders didn’t care about the aggregate; they wanted to know whether individual call center agents were doing a good job.

Part of the solution was giving the business only as much technical information as they actually needed. So we created some Tableau reports that showed the confidence interval. And then we trained a pilot group of managers on how to use that confidence interval. Basically, if there were 300 to 500 calls per rep for a given time period, the confidence interval was small enough that you could trust the score. The other thing that we were able to train the managers on was that although the score accuracy itself was noisy at the call level, the ranking worked well. The team was able to say, “Instead of just randomly listening to calls, we can help you find a person’s worst 10 calls.” So thinking creatively about the use of the output of the model and what can be used for the business context.

Mary Willcock:

Walk us through what it looks like growing a team from five people to 20.

Anne Marie Weber:

Data science teams can be decentralized, centralized or hybrid. The data science teams can be embedded, or decentralized, within various business units, which might be great for a large enterprise with many different business areas. In a smaller corporation, it may make sense to have your data scientist team fully centralized, where you're able to leverage having all those people together, making sure that all the models can be peer reviewed, and making sure that there are model risk management practices in place and everything is being handled consistently.

The hybrid approach is where you have a center of excellence, but you also have some embedded data science teams. You're able to have those embedded teams in areas of the business that are large enough to have a backlog that would support them. For any other area of the business that doesn't have an embedded team, you leverage a centralized team, and then they manage the things like the model risk management practices. They might have a dedicated machine learning team for advanced use cases partnering with the embedded data science teams.

Let's say, for example, you had an original team of five, and you're expanding to 10 to 15. It would be important to have three roles: data scientists, data engineers and analytics strategists. As you expand the team, it may even make sense to add a portfolio manager who would have a good idea of bandwidth, because then you might have multiple data science teams with multiple managers. If you're now talking about a team of 30 or 40 people, you might need some more administrative staff. 

Mary Willcock:

As we think about growing teams, project initiation and then project maintenance, how do you differentiate each to ensure success?

Anne Marie Weber:

Data scientists or anyone doing modeling always likes the new and shiny. Whether it's a new technique or a new project, these are very creative people who are always trying to investigate and there's a lot of energy. To keep those folks engaged, it's important to make sure that they each have a good mix of projects: models that need refreshing, tests to pilot, research papers to write, things of that nature.

And then there are new projects that everyone wants to get their hands on. There's some amount of risk to new projects. People get excited about them, but not all projects will be successful. So it's important for leaders to ensure that the team is happy with the types of work that they're doing, and align the work to those opportunities because then people bring their best selves to the workplace. If they're excited to do the work, if they're able to add skillsets, they can continue to move up.

It's important to know what they want to be doing in the next three to five years so that they can cultivate the skills that they want to cultivate. They will bring their best selves to work every day and will want to stay within the company. Which is important, because these are highly sought-after people, and they will get poached, especially now that people can work from anywhere.

Mary Willcock:

Tell me a little bit about your experience with advancement paths for data scientists.

Anne Marie Weber:

We have four levels of data scientists. I think it's important to have different levels of skillsets. People can move up within the company as they build skills in data science, communication, strategy or project management. I try to make sure that they have opportunities to utilize those skills and show that there is a career path. The longer a person stays with the company, the more value they add, because they’re used to data systems and understand the business.

Mary Willcock:

How does data science fit into digital transformation and what are some key components to that digital transformation?

Anne Marie Weber:

We are in a wave of digital transformation within life insurance. We're one of the last industries to digitally transform. We still use fax machines. There are many things that are paper-based, especially in a fully underwritten world.

There are other areas of our economy that haven't gone through the digital transformation that the rest of the economy has, but consumers are hungry for it. They get two-day delivery from Amazon, and they expect that kind of service everywhere else. From a consumer expectation standpoint, we have to grow and change, because the first person within our industry to do so will have an advantage over everybody else.

When we have this tremendous amount of technological change, data science is part of it. In order to fully think through use cases that are transformative, you're talking about cloud technology and prediction and maybe 5G and, maybe some blockchain if you're getting fancy. But it gets very complicated very quickly.

If you are thinking about dipping your toe into data science, and you don't have prediction as part of your core business, it's not too late. I think it's important to start building buy-in with your leadership as soon as possible, figure out a way to make that umbrella wide and bring people in so that it's not scary. Then it becomes an approachable tool, like our computers are now.  All of these new technologies take time to get used to, and we will get used to these as well.

Focus on what prediction can do for the business. Answering that question is foundational to being able to get buy-in and building that team, starting small, starting with the low-hanging fruit. If you can get sort of that scrappy startup-style team that can wear a lot of hats and be focused on business results, there's a decent amount of success to be had.

Learn more about Baker Tilly Digital’s inaugural DeepRacer league

arrowCreated with Sketch.
Chicago great lakes shoreline
Next up

Great Lakes regional M&A update: H2 2020