Catapult’s Data & AI team launched their weekly Happy Hour Chats series. The idea behind these less than 5-minute vlogs is to answer some of the most prevalent questions in our industry today while we unwind from a busy week. Every Friday, the Data & AI team will share their knowledge on topics that may be of interest to you.
Today’s Topic: Azure Synapse: Part 1
Talent: Edgar Cuellar, Lead Consultant, featuring Sid Atkinson, Director of Data & AI
S: “Welcome everyone! My name is Sid Atkinson, the Director of our National Data & AI practice and with me today is…
E: “Hello. I’m Edgar and I’m an architect in the data & AI practice.”
S: “So Edgar, one of the things that we want to talk today about so far… a lot of our customers hearing about Azure synapse, so what at the high level can you tell us what this is, what’s Microsoft coming out with and what should our customers know about it?”
E: “Absolutely. Recently Microsoft made announcements over the Synapse product. The great thing about it for end users is that this empowers them to get full access to their daily environment without having to go in there and look at one individual file at a time. They introduced this concept of service query on demand, where the user can just go in there and start running queries, all of their data in their data lake without having to create any separate environments for that.”
S: “Awesome. So, if you were to start with the end users, what does this mean for end users that are going to be accessing these massive systems?”
E: “So the great thing for them is that it gives them the ability to very quickly get insights into what they’re doing. In addition, when they wish to engage with data engineers, currently, what they have to do is they have to go in and launch a separate environment, create pipelines, sources target mappings, etc. The great thing about the Synapse Workspaces is that the data engineers are in the same workspace as their end users and they can collaborate closely with them and create all their orchestration pieces within that same synapse workspace.”
S: “So, we’re seeing a nice simplification of all these different products and capabilities for both the engineers and for the end users. So, what about the data scientists?”
E: “Right now as a data scientist you typically go into a notebook and start running some piece of code that you want to do some wrangling, build out some models. Typically, is a separate environment than where your data engineers and where your end users are doing. The great thing about the Synapse workspaces is that they have introduced the concept of the spark pools in there. What that essentially means is you can go in there as the data scientists in that same synapse workspace and start creating your notebooks right there without having to create different Azure resources, or different harbor environments or virtual machines. You just go into the same workspace and start doing it work there. And start building out your models.”
S: “So we’re seeing, thematically, is a nice consolidation of a lot of these platforms and tools into one space so we can kind of get to one workflow to rule them all in a way.”
S: “So what should we tell our customers and the people that are watching that for those who have existing data and things that they’ve deployed to Azure today, what does this mean and what should they consider?”
E: “The great thing for them if you’re an existing customer utilizing a data warehouse, you can continue using your data warehouse. You just take advantage of some of these extra capabilities as you need them. But if your projects are currently running you have the ability to maintain your existing data warehouse and then leverage the database enhancements that you’ll be getting. In addition, this is going to allow you the opportunity to have more options to collaborate more closely across different parts of the organizations whether they are data scientists or end users?
S: “Sounds like a lot of topics that we have that we can dive into So what do you do today? Will talk about next?”
E: “I think the next step would be to get more into detail on the actual data warehouse itself. And the new capabilities that will be introduced along with the other Synapse functions that are coming out as well.”
S: “Great. Well, thank you Edgar.”
Next up: Azure Synapse Part 2 – Watch here.
Want to check out more Catapult’s videos? Visit our You Tube Channel.