This episode of our Happy Hour series highlights a Black Swan Event. Lee and Eric, both in Data & AI practice, discuss Black Swan Events and what they are and how it affects predictive models.
E: Hi, I’m Eric Jacobsen from Catapult Systems and I’m here with Lee Harper and we’re here to talk about a little bit about Black Swan events. So Lee what is a Black Swan event, and I guess I’ve heard it mentioned a few times in the media with COVID-19 being one and why is it one? What is the definition? How does this fit into that sort of paradym?
L: So Black Swans are rare. Most are gonna be white, so Black Swan events are events that are very much outside of the normal. Um way of things. So that can be in positive and negative directions. So a classic example
of this is the 2008 Great Recession where those large amount of economic contraction and that meant that conditions were very different to how I would normally expect them to be in. Of course, right now with COVID-19 were all under or maybe under quarantine. Right now the varying kinds and so. Life is very different. How would normally be outside of this sort of time period.
E: So I guess given the space you work in, I’m curious what is the consequences of events like this to predictive models of various sorts?
L: So it’s highly likely that most people’s printed models that were working fine among or two ago and now aren’t going to be returning good predictions. There are still technically work. You can still get predictions that will be much
less accurate than they were right. You can imagine that right now. Models might be predicting an increase in sales of ice cream, but I’m guessing that we don’t go out that much, so we probably aren’t buying as much ice cream as you would otherwise be. Buying at least not here in Texas.
E: Speak for yourself.
L: Fair enough. But some. So the kind of one liner here is that most people’s pretty models are now going to be wrong, or at least less accurate than they were before.
E: So is there is there anything that people who are working with those type of models can can do about it either in the short term or longer term?
L: So in the short term, it’s actually often very difficult because machine learning models use historical data to predict what might happen in the future. And of course there is no historical precedent for where we are right now, so it becomes
very difficult to train models on relevant data. People might try and look at mirror events. For example, you know this is more similar to the Great Recession than two months ago, so you might try and do some data from back then, but there’s
no guarantee it’s going to work. And then in the longer term, you know, many people when we return to normalcy might well actually just remove this chunk of time from their datasets and they come to train their machine learning models. It depends on the application area. Of course, but it’s very common to do that with oh 810 and just remove it from your modeling because it can skew the results when life returns to normal.
E: Interesting, so are there. I know there are a lot of different types of predictive models in machine learning models out there being employed for different. Purposes, are there different types that are more or less affected by Black Swan events?
L: So anything that’s behavioral-based or that’s really looking forward and trying to forecast? They’ll probably affected by these kind of events pretty dramatically, but then there are other kinds of models. For example, in natural language processing or computer vision which are going to be so affected. Language doesn’t change dramatically over the course of a few months, and what we see. Also, is it changed a lot over the course of a few months? Unless of course there zombies come and we can’t recognize those.
E: It’s really interesting. Thank you very much.
L: You’re very welcome, thank you.