Examine how Wintellect built the WintellectNOW video service on top of Azure - YouTube
YouTube
Primary Language: English
Description:
My Recent Work
You did not do any work on this video
Subtitles in: English
Start End Subtitles
00:00:00 00:00:03 So, I'm gonna introduce the speakers in a second.
00:00:03 00:00:07 This is a kind of a more informal kind of fun tour of,
00:00:07 00:00:11 one like now, which is our online training service.
00:00:11 00:00:14 Similar to Skillsoft at window.com, and Pluralsight.
00:00:14 00:00:17 A couple of housekeeping notes.
00:00:17 00:00:18 One is, Jeffrey and
00:00:18 00:00:23 I almost always 99 percent of the time do deep dive technical talks.
00:00:23 00:00:26 So, when we do a case study, I'm gonna take you through some of
00:00:26 00:00:30 the history, and some of what this product is, and value propositions.
00:00:30 00:00:32 So in other words, kinda marketing slides.
00:00:32 00:00:34 And the reason I'm telling you this is,
00:00:34 00:00:35 is that's totally out of our comfort zone.
00:00:35 00:00:37 Because we train ourselves not to do any marketing and
00:00:37 00:00:39 sales, cuz developers hate that.
00:00:39 00:00:42 But I wanted to talk you through how we came to build this thing.
00:00:42 00:00:45 How we got here, the reasons we did it and what it is.
00:00:45 00:00:48 It probably will only take ten minutes or so, maybe fifteen,
00:00:48 00:00:51 do a demo of it and then we'll dive in with Jeffrey to some of
00:00:51 00:00:55 the architecture key points and I'll explain why we thought it might be
00:00:55 00:00:58 interesting to do this talk as we go through it.
00:00:58 00:01:00 The second housekeeping point is when we submitted this,
00:01:00 00:01:03 we submitted for a small a 45-minute breakout session, and
00:01:03 00:01:05 they gave us 75 minutes.
00:01:05 00:01:08 So you're gonna have at least a half an hour of free time to hang out and
00:01:08 00:01:10 do email or ask Q&A at the end of this talk.
00:01:10 00:01:12 We're not gonna try to stretch this out to 75 minutes.
00:01:15 00:01:18 So, my name is Todd Fine, I'm CEO of Wintellect.
00:01:18 00:01:20 I'll talk about the company in a second.
00:01:20 00:01:26 You have my information there if you wanna email me or Jeffrey later.
00:01:26 00:01:28 And much more important than me is Jeffrey Richter so
00:01:28 00:01:29 you wanna- >> All right.
00:01:29 00:01:30 >> Introduce yourself?
00:01:30 00:01:33 >> Sure, so I'm Jeffrey Richter, I'm a Software Engineer, I've worked
00:01:33 00:01:37 full time at Microsoft now in the Azure Service Fabric Team.
00:01:37 00:01:38 But I'm also a co-founder of Wintellect and
00:01:38 00:01:41 I've work at Wintellect for many, many years in my life.
00:01:41 00:01:44 Most people are familiar with me from the various books that I've
00:01:44 00:01:48 authored over the past 20 or so years, I'm down 18 books in total.
00:01:48 00:01:52 I put the 3 most recent book covers up here on the slide and then,
00:01:52 00:01:55 at the bottom is my Wintellect email address and
00:01:55 00:01:59 my LinkedIn information, if anybody wants to be friend me there and
00:01:59 00:02:01 my Twitter handle is there as well.
00:02:01 00:02:06 >> Okay, and hopefully you're familiar with Jeffrey.
00:02:06 00:02:07 He's one of the reasons I came to the company.
00:02:07 00:02:11 He's kind of a legend in the Microsoft and architecture arena.
00:02:11 00:02:14 And it's one of the reasons why I wanted this talk because this is
00:02:14 00:02:17 a product where Jeffrey was not working with any other company,
00:02:17 00:02:18 he had no managers.
00:02:18 00:02:21 And he had free reign to build whatever he wanted,
00:02:21 00:02:24 in any architecture he wanted exactly the way he wanted.
00:02:24 00:02:27 Which is why we thought it would be a little bit of fun taught because
00:02:27 00:02:29 when you see some product out there or
00:02:29 00:02:33 some cool piece of software, it could be small, it could be big,
00:02:33 00:02:34 it could be enterprise, it could be cloud.
00:02:34 00:02:35 As a developer,
00:02:35 00:02:38 I always kind of go I wonder how they actually built that.
00:02:38 00:02:39 Technology changes every six months here.
00:02:39 00:02:40 I wonder how they actually did it.
00:02:40 00:02:43 And some of the things that Jeffrey did when he built this are kind of
00:02:43 00:02:45 interesting and definitely unique.
00:02:45 00:02:48 Which is one of the couple reasons we thought it would be interesting
00:02:48 00:02:49 to do this talk.
00:02:49 00:02:50 Okay.
00:02:50 00:02:51 Very simple agenda.
00:02:51 00:02:54 I'm just gonna go over a little bit of the history of how we got here.
00:02:54 00:02:56 How we decided to build this service.
00:02:56 00:02:59 And do a quick product demo, tell you a little bit about the service.
00:02:59 00:03:02 Then Jeffrey will jump in for the bulk of the presentation and
00:03:02 00:03:04 talk about some of the key pieces of the architecture.
00:03:04 00:03:06 And if there's any questions,
00:03:06 00:03:09 we'll probably have 45 minutes to answer your questions.
00:03:11 00:03:13 Okay, so how did we come to build this?
00:03:13 00:03:17 And I will demo it in a few minutes.
00:03:17 00:03:20 But just so you know who Wintellect is, we are a Microsoft partner of
00:03:20 00:03:23 the bulk of our work nowadays with modern web development and
00:03:23 00:03:28 that kind of stuff isn't really on the Microsoft platform per say but
00:03:28 00:03:30 we have three lines of business, we have a consulting business,
00:03:30 00:03:33 which is application development architecture.
00:03:33 00:03:36 We have a hard core developer training business
00:03:36 00:03:40 which is in person training at companies around the world.
00:03:40 00:03:43 And then in 2013, we added a third line of business which is our
00:03:43 00:03:45 on-demand WintellectNOW service that we're talking about today.
00:03:46 00:03:49 That is just taking our developer training and
00:03:49 00:03:51 putting it online for scale.
00:03:53 00:03:58 So here's a really simple high level kind of timeline.
00:03:58 00:04:00 Wintellect was founded in 2000.
00:04:00 00:04:03 At that time, we were only an instructor led training company,
00:04:03 00:04:05 very hard core.
00:04:05 00:04:06 I always considered ourselves one of the top two or
00:04:06 00:04:09 three in the country, if not the world, along with
00:04:09 00:04:12 competitors like DevelopMentor and that kind of thing.
00:04:12 00:04:15 We always knew each other very closely in the industry.
00:04:15 00:04:18 So we initially were that kind of top tier developer hard core
00:04:18 00:04:19 training company.
00:04:19 00:04:22 We added consulting in 2005-2006 which is really
00:04:22 00:04:24 application development.
00:04:24 00:04:27 And then in 2013 as I said we added this third
00:04:27 00:04:28 service called WintellectNOW.
00:04:30 00:04:34 So now I'm gonna talk about how we kind of came to make the decision
00:04:34 00:04:35 to build this.
00:04:35 00:04:40 We had been for 13-14 years Microsoft Corp.'s
00:04:40 00:04:44 number one outside training vendor for their internal developers.
00:04:44 00:04:47 I don't mean for just people that were developers in the Microsoft
00:04:47 00:04:49 stack, I mean for Microsoft's developers.
00:04:49 00:04:52 We put somewhere between 35,000 to 50,000 of their developers
00:04:52 00:04:53 through our training.
00:04:53 00:04:55 We were a required set of a curriculums for
00:04:55 00:04:57 some of the subsets of engineers,
00:04:57 00:05:00 like people in the OS team had to take some of Jeffrey's courses.
00:05:00 00:05:02 And every single one of those years,
00:05:02 00:05:04 we had the highest instructor ratings of any other vendor.
00:05:04 00:05:07 So, we had this big presence with Microsoft.
00:05:07 00:05:11 We had this huge reputation in hard core developer training, and
00:05:11 00:05:14 in 2012, we saw a couple things.
00:05:14 00:05:16 One is we saw the market that you all see right?
00:05:16 00:05:21 Online training, MOOCs, all this kind of stuff, you see Lynda.com,
00:05:21 00:05:23 you see the rise of Pluralsight that we all know.
00:05:23 00:05:25 We know those all those guys.
00:05:25 00:05:27 We know the owners of Lynda.com and
00:05:27 00:05:33 we know Aaron Skonnard from Pluralsight's so we're on that mix.
00:05:33 00:05:35 We saw the market happening.
00:05:35 00:05:37 We said maybe we should kind of do something ourselves.
00:05:37 00:05:39 We have this great brand.
00:05:39 00:05:42 And at that same time, Microsoft who was again our number
00:05:42 00:05:44 one customer for instructor led training.
00:05:44 00:05:46 We were going all over the world.
00:05:46 00:05:46 We still do.
00:05:46 00:05:49 In Denmark and Beijing and India, and.
00:05:49 00:05:52 Everywhere, I mean, millions of dollars worth of training for
00:05:52 00:05:55 Microsoft Corp developers around the world.
00:05:55 00:05:58 They said they saw the same marketing trends that we did,
00:05:58 00:06:01 and they said, can you take all your training and
00:06:01 00:06:04 make an online version available, so we can scale it even more?
00:06:04 00:06:05 And even more cheaply.
00:06:05 00:06:09 So our biggest customer of our training wanted to take it online.
00:06:09 00:06:12 We started thinking about taking it online.
00:06:12 00:06:16 We had that industry reputation, and we had that industry expertise and
00:06:16 00:06:18 everything really just lined up.
00:06:18 00:06:19 We were already doing consulting in AFTA.
00:06:19 00:06:21 We knew how to produce content.
00:06:21 00:06:22 We knew how to train.
00:06:22 00:06:24 We knew how to present.
00:06:24 00:06:25 And because we are software developers,
00:06:25 00:06:26 we knew how to build a platform.
00:06:26 00:06:27 It just all made sense.
00:06:27 00:06:29 So we said, let's do it.
00:06:29 00:06:31 And we really kind of did this all on our own,
00:06:31 00:06:33 we didn't take any investor funding.
00:06:33 00:06:36 We just wanted to kind of extend what we already have been doing.
00:06:36 00:06:39 And that's how we came to make the decision to start WintellectNOW.
00:06:41 00:06:43 So it's a charitable slide, right?
00:06:43 00:06:45 But just to be clear about what this is.
00:06:45 00:06:48 This is an online, developer training-focused service,
00:06:50 00:06:54 works worldwide, scales to hundreds of thousands or millions of users.
00:06:54 00:06:55 Which Jeffrey will talk about.
00:06:56 00:07:00 And it's also become more and more integrated with Microsoft.
00:07:00 00:07:04 So WintellectNOW is already offered as of November of last year
00:07:04 00:07:08 as part of the new, or new back then, Dev Essentials program.
00:07:08 00:07:11 Which is kind of that free MSTN tier
00:07:11 00:07:13 type of thing that people get worldwide and
00:07:13 00:07:17 it's a benefit you get for free if you sign up through that program.
00:07:17 00:07:20 It's now also being offered parts of it in subsets on
00:07:20 00:07:22 Microsoft Virtual Academy, and there's more and more of those
00:07:22 00:07:27 partnerships things that are coming on with different Microsoft groups.
00:07:27 00:07:31 Because although Microsoft has a lot of 100, 200 level content,
00:07:31 00:07:34 they don't have very much 300 and 400 level content.
00:07:34 00:07:38 And almost everything we have is just very deep.
00:07:38 00:07:42 Premium level, hard core, developed by experts training.
00:07:42 00:07:44 So, they're asking for more and more of that.
00:07:44 00:07:46 There's some bigger things coming that I think even you don't know
00:07:46 00:07:49 about right now, so I feel I'll tell you afterwards.
00:07:49 00:07:53 So that relationship with Microsoft is expanding.
00:07:53 00:07:56 The services successfully makes money.
00:07:56 00:08:00 But it's also not a Pluralsight or when [INAUDIBLE] bringing in tens of
00:08:00 00:08:03 millions of dollars a year and a million users.
00:08:03 00:08:07 So it's this kind of a thing we use on the side that's profitable but,
00:08:07 00:08:09 we've never really made it,
00:08:09 00:08:12 invested the money to have it kinda take off in a national type basis.
00:08:14 00:08:15 I don't know why that's important.
00:08:15 00:08:17 It's just you that you may have heard of it, you may have not.
00:08:17 00:08:21 But it's equally or I would say equally equal or
00:08:21 00:08:23 much better, than some of those other services out there in
00:08:23 00:08:25 terms of the quality of the content.
00:08:25 00:08:27 And I don't know if I will have time to show you that but
00:08:27 00:08:28 you should check it out.
00:08:28 00:08:32 And if you wanna check it out, I gave a promo code,
00:08:32 00:08:36 you can go right to the site, go sign up, go to the promo code screen
00:08:36 00:08:39 and you get, I don't remember how much I put in there, a month or
00:08:39 00:08:41 three months or something for free.
00:08:41 00:08:43 And you can check out the entire service, all facilities,
00:08:43 00:08:46 all videos, for free.
00:08:46 00:08:49 And if you have any trouble with that, look up the slides or
00:08:49 00:08:52 talk to me later and you can email me and say I couldn't get in or
00:08:52 00:08:54 something happened and I'll get you in and you guys can check it out.
00:08:57 00:08:58 Okay.
00:08:58 00:09:02 So what were the goals of this when Jeffrey decided to build it?
00:09:02 00:09:05 We wanted to build it on the latest platform
00:09:05 00:09:09 because there's a lot of cons for not being first in doing this.
00:09:09 00:09:12 But one of the benefits is you can start with the latest technologies.
00:09:12 00:09:14 So we wanted to build it on latest technologies.
00:09:14 00:09:16 We wanted to build the whole thing on the cloud.
00:09:16 00:09:18 And we chose Azure.
00:09:18 00:09:22 We wanted it to be, and this is getting to Jeffrey's area, very,
00:09:22 00:09:23 very highly optimized for cost.
00:09:24 00:09:27 That was one of the key design features.
00:09:27 00:09:30 And very, very scalable to millions of users.
00:09:30 00:09:31 And that was the focus I think.
00:09:31 00:09:32 Is that correct Jeffrey?
00:09:32 00:09:33 >> That's correct.
00:09:33 00:09:34 >> So- >> Very cheap.
00:09:34 00:09:36 I wanna run this thing, real cheap.
00:09:36 00:09:39 >> [LAUGH] >> Yeah and it's funny because we've
00:09:39 00:09:42 gone into some of those other companies that I've mentioned,
00:09:42 00:09:46 talked to them about various things, about sharing content,
00:09:46 00:09:50 licensing content, buying content, and we've met the owners and
00:09:50 00:09:52 CEOs and we've seen what they pay for this stuff.
00:09:52 00:09:57 And some of them are on internal racks, some of them are on AWS but
00:09:57 00:10:01 all of them pay Order of magnitude or two more than our system cost,
00:10:01 00:10:03 just because it was built for
00:10:03 00:10:07 that from the very beginning from the ground up.
00:10:07 00:10:10 Very simple, highly cost optimized, highly scalable, and
00:10:10 00:10:11 Jeffrey will talk about that.
00:10:12 00:10:14 There's my marketing slide, guys.
00:10:14 00:10:17 It actually is a marketing slide [LAUGH] that I put into this,
00:10:17 00:10:20 but you might ask what's different about it?
00:10:20 00:10:22 On the one hand, not much, right?
00:10:22 00:10:25 You can go to UniMedia you can go to any of these four, five, six,
00:10:25 00:10:30 eight online training services and they all share a lot of things.
00:10:30 00:10:35 We did design this or built this in a few different ways that may or
00:10:35 00:10:36 may not matter to you.
00:10:36 00:10:39 One is a lot of the content here that I told you was required
00:10:39 00:10:42 curriculums for Microsoft's Operating System engineers,
00:10:42 00:10:46 meaning people working on the OS, or people working on Azure.
00:10:46 00:10:50 I mean, Jeffrey's done dozens of deep Azure classes to Microsoft.
00:10:50 00:10:53 John Robbins, the Bugslayer columnist from MSDN is one of our
00:10:53 00:10:55 founders and owners, as well.
00:10:55 00:10:59 That entire series of classes that Microsoft hardcore engineers
00:10:59 00:11:01 gets is on here.
00:11:01 00:11:02 And you won't get that from other places.
00:11:02 00:11:05 Now we don't have 5,000 videos, we have 4,500 at the moment,
00:11:05 00:11:09 and it keeps growing every month, but
00:11:09 00:11:12 a lot of those videos seriously are unique.
00:11:12 00:11:15 We also said we can't compete with Lynda.com or
00:11:15 00:11:20 Pluralsight in terms of scale when we're doing everything ourselves and
00:11:20 00:11:25 they're getting 100, 200 million in an outside of investor funding.
00:11:25 00:11:29 We've consciously priced this to be the best value price, meaning you're
00:11:29 00:11:32 getting the same quality content videos, but it's much cheaper.
00:11:32 00:11:35 Because we're not going to try to beat the other guys and
00:11:35 00:11:36 take them head on.
00:11:36 00:11:40 And the rest is really just typical marketing stuff, so I'll stop there.
00:11:40 00:11:41 I'm about to demo it so
00:11:41 00:11:45 is there any questions before I give you a really quick five minute demo?
00:11:45 00:11:48 Oop, okay.
00:11:48 00:11:53 And you'll be happy to know that after that it gets technical again.
00:11:53 00:11:57 For those of you guys who haven't seen it, let me put on my,
00:11:57 00:12:00 find my old people glasses.
00:12:00 00:12:06 Put this on and let's start here.
00:12:06 00:12:08 I'm just going to login.
00:12:08 00:12:10 I'm going to make this really brief.
00:12:10 00:12:14 It's not that there's tons of incredibly unique things to show you
00:12:14 00:12:17 here, but I at least want you to know what you're looking at so
00:12:17 00:12:21 when Jeffrey's showing you stuff, you can kind of relate it visually.
00:12:24 00:12:25 So you log on.
00:12:25 00:12:27 Each person has a dashboard.
00:12:27 00:12:29 You can setup queues and favorites and bookmarks and
00:12:29 00:12:34 say I want to do this whole core series, in this order.
00:12:34 00:12:36 And you just go to that Dashboard.
00:12:36 00:12:38 You can see your video history etc., etc.
00:12:38 00:12:40 You can go to a Course Library.
00:12:40 00:12:41 There's various ways to look at it.
00:12:41 00:12:43 You can just kinda look at the whole thing.
00:12:43 00:12:49 You can filter it by author like Jeffrey Richter.
00:12:49 00:12:52 And let's see how good the Internet is here.
00:12:52 00:12:55 Let's click on one, see if this works.
00:12:58 00:13:00 I can't hear it but.
00:13:00 00:13:04 >> [INAUDIBLE] >> I don't know if you guys can hear
00:13:04 00:13:05 any of this.
00:13:05 00:13:07 Put your microphone next.
00:13:07 00:13:11 >> Currently flowing of the main tread of my application.
00:13:11 00:13:13 >> You can't hear that at all can you?
00:13:13 00:13:19 >> You can get manage thread ID so we could see
00:13:19 00:13:24 the ID of the thread that is executing this
00:13:24 00:13:30 function and we see it has an ID [INAUDIBLE].
00:13:30 00:13:33 >> So it's a typical online training service.
00:13:33 00:13:34 There is discussion forums,
00:13:34 00:13:38 there is bookmarks, there is tables of contents, you can skip around.
00:13:38 00:13:41 There is, let me stop this so I can hear myself.
00:13:41 00:13:43 There is related code and
00:13:43 00:13:46 samples when they exist that you can download.
00:13:46 00:13:47 It could be a PowerPoint, it could be code samples, that kind of thing.
00:13:49 00:13:52 Other typical stuff you would normally see.
00:13:52 00:13:54 There is series of content.
00:13:56 00:13:58 MVC five fundamentals.
00:13:58 00:14:00 Here's that mastering.NET debugging.
00:14:00 00:14:03 You might have seen mastering.NET performance tuning.
00:14:03 00:14:04 It's not all .NET.
00:14:04 00:14:08 There's tons of PHP, and Facebook, and Linux, and
00:14:08 00:14:09 all kinds of other stuff on there.
00:14:09 00:14:13 But if you want to see a series, and the reason I'm bringing this up is
00:14:13 00:14:18 that this is, I would say a unique series from John Robins,
00:14:18 00:14:21 who is one of the top debugging experts literally in the world.
00:14:21 00:14:26 Works with the Microsoft debugging teams with Visual Studio.
00:14:26 00:14:29 He has his book, Mastering .NET Debugging, his book,
00:14:29 00:14:32 Mastering Bugging with C++, all this kind of stuff.
00:14:32 00:14:35 This is an example of an entire course series and
00:14:35 00:14:37 some unique content.
00:14:37 00:14:38 And this is much less important,
00:14:38 00:14:41 there's the instructors which you can check out.
00:14:41 00:14:45 We always try to highlight the instructors to give
00:14:45 00:14:47 them some marketing props.
00:14:47 00:14:50 And there's an admin section which you would not see, but
00:14:50 00:14:54 that lets us look at all kinds of accounts, users, promo codes.
00:14:54 00:14:55 I mean,
00:14:55 00:15:00 I might wanna go here and be careful of what I show you this time.
00:15:00 00:15:01 Let's look at.
00:15:01 00:15:02 Let's just bring up an account.
00:15:02 00:15:04 MVP, let's see if that works.
00:15:07 00:15:12 If you're an MVP, we give free accounts out to MVPs worldwide.
00:15:13 00:15:16 Let's see.
00:15:16 00:15:19 All right, I just clicked the wrong button, I'm sorry.
00:15:23 00:15:24 I don't know what's going on there.
00:15:24 00:15:27 I just had it up, let me show you, there it comes up.
00:15:27 00:15:31 There's 1,150 in this one account, a little over 1,000 people.
00:15:31 00:15:33 And I gotta be careful what I click on because if I
00:15:33 00:15:36 click on the wrong thing you'll see everybody's email addresses.
00:15:36 00:15:40 But there's a bunch of things for a business portal in here.
00:15:40 00:15:43 None of which really applies to much to what Jeffery is gonna show you,
00:15:43 00:15:45 cuz he's gonna show you the architecture internals.
00:15:45 00:15:49 But I think I have one other thing I wanted to show real quickly here,
00:15:49 00:15:52 and this is that same MVP account.
00:15:54 00:15:58 And there should be a bunch of user administration stuff.
00:15:59 00:16:01 And logs and things like that.
00:16:01 00:16:04 There's all kinds of reporting that our customers can get.
00:16:04 00:16:08 For instance, let's pick on one of these.
00:16:08 00:16:12 You can get graphs of video history, which people are watching videos so
00:16:12 00:16:15 that, if you're an enterprise buying this stuff, and
00:16:15 00:16:18 you can download Excel spreadsheets and all kinds of stuff.
00:16:18 00:16:21 You can say, hey, I'm paying Wintellect all this money because
00:16:21 00:16:25 you Josh, said you wanted to get this video and study it, but I see
00:16:25 00:16:28 you didn't watch any videos last month, so I'm taking you out and
00:16:28 00:16:31 I'm putting in somebody else who's gonna watch it.
00:16:31 00:16:34 This is all kinds of kind of business related functionality in
00:16:34 00:16:38 there as well, which doesn't really apply to what we're gonna talk
00:16:38 00:16:41 about, but that kind of gives you, I guess, a four minute,
00:16:41 00:16:43 five minute overview of what the system is.
00:16:43 00:16:45 Anything else you wanted me to hit in the demo before you show it?
00:16:45 00:16:47 Is there anything else you would want to show?
00:16:47 00:16:50 >> No, I think that's the bulk of the features that are interesting.
00:16:52 00:16:57 >> I think the things that were, hard for us or hard for Jeffrey,
00:16:57 00:17:00 maybe not hard, but he had to think about, was it's like anything.
00:17:00 00:17:03 You can make this work, you guys can design this.
00:17:03 00:17:05 You have ten people who were using it.
00:17:05 00:17:08 But what if a million people are using it?
00:17:08 00:17:10 And what about the storage for that?
00:17:10 00:17:12 What about the streaming for that?
00:17:12 00:17:14 What is the cost gonna be for
00:17:14 00:17:18 bandwidth when you are streaming to 100,000 people?
00:17:18 00:17:21 And it must work worldwide in the end, Australia,
00:17:21 00:17:22 everything like that.
00:17:22 00:17:23 What are the storage costs?
00:17:23 00:17:25 We're on the cloud?
00:17:25 00:17:27 If you're storing stuff in blobs, or tables, or
00:17:27 00:17:30 whatever, it always starts out really cheap.
00:17:30 00:17:34 But then when you get to that scale, even in Azure, it starts to add up.
00:17:34 00:17:35 And that's what Jeffrey's gonna talk about.
00:17:35 00:17:38 That's what he was considering among many other things.
00:17:38 00:17:42 Memory performance, storage performance, cost optimizations and
00:17:42 00:17:45 that was the key focus I think for you when you were building it.
00:17:45 00:17:45 >> You didn't.
00:17:45 00:17:46 >> Jeffery really didn't.
00:17:46 00:17:47 >> Why did we not have to build ourself.
00:17:47 00:17:49 >> What can we leverage.
00:17:49 00:17:53 >> Jeffery, you didn't really have any at least minimal preservation
00:17:53 00:17:57 with the front end really or those portals of logs except the support
00:17:57 00:17:59 it from the back end, is that correct?
00:17:59 00:18:01 >> Right, so I architected the system.
00:18:01 00:18:04 I focused heavily on the back end which is what I'll be talking about
00:18:04 00:18:06 in the architectural slides, and
00:18:06 00:18:10 then we had some contractors who did the front end, making it look nice.
00:18:10 00:18:13 I'm not a HTML user interface person.
00:18:13 00:18:15 But we make sure that all the features
00:18:15 00:18:18 propagated through from the back end to the front end of course.
00:18:18 00:18:19 And then we redid the UI.
00:18:19 00:18:24 We hired a full time employee and then we redid the UI with Joel. And-
00:18:24 00:18:24 >> And
00:18:24 00:18:26 Microsoft has hired away from us a few months ago.
00:18:26 00:18:27 >> Yes, and then- >> [LAUGH]
00:18:27 00:18:28 >> But now Jeff is
00:18:28 00:18:29 doing a lot of UI stuff.
00:18:29 00:18:30 >> Right.
00:18:30 00:18:33 >> But the back end architecture, my understanding is it's been pretty
00:18:33 00:18:36 much, I mean it works really well, it's been running that way for
00:18:36 00:18:38 a couple of years other then a couple of tweaks here and
00:18:38 00:18:40 there over time.
00:18:40 00:18:41 It's largely good.
00:18:41 00:18:44 There was a major rearchitect between version one and version two.
00:18:44 00:18:46 Once we started getting feedback from customers,
00:18:48 00:18:50 when I was working with the frontend guy and we're thinking,
00:18:50 00:18:53 we have to improve the user interface for this experience.
00:18:53 00:18:57 And doing that really meant that we should change things on the backend.
00:18:57 00:19:00 There was a point in time where we did a pretty significant
00:19:00 00:19:02 rearchitecture of how we stored things in data storage.
00:19:02 00:19:06 And then we had a migrate from one set of Azure tables to the new
00:19:06 00:19:08 format for the Azure tables.
00:19:09 00:19:13 And we did it the cheap way which was by having downtime
00:19:13 00:19:16 without trying to do it with and being up the entire time.
00:19:16 00:19:19 Over a weekend we went down to couple hours while we did that
00:19:19 00:19:21 migration and then we came back up.
00:19:21 00:19:24 Other than that it runs and I don't if it had any other outages or.
00:19:24 00:19:26 >> We, I don't, we've had an outage.
00:19:26 00:19:31 >> I don't think we've ever had an outage due to anything on
00:19:31 00:19:33 the app itself.
00:19:33 00:19:36 It would just be like your normal, if Azure was down for a minute or
00:19:36 00:19:38 some CDN thing would happen.
00:19:38 00:19:39 >> Yeah.
00:19:39 00:19:42 >> Nothing that's even worth talking about, meaning it wasn't ever-
00:19:42 00:19:44 >> I remember a certificate problem.
00:19:44 00:19:48 John Robbins went in had to get a copy of our certificate from our
00:19:48 00:19:50 certificate provider.
00:19:50 00:19:55 When they did that they revoked the previous one.
00:19:55 00:19:57 And we didn't realize that.
00:19:57 00:19:59 There were some people who couldn't connect for
00:19:59 00:20:00 a while until we figured that out.
00:20:00 00:20:03 Then we had a do something to fix this certificate.
00:20:03 00:20:05 >> I wrote a blog post about it if you want to know about that,
00:20:05 00:20:07 it's on the Wintellect website.
00:20:07 00:20:10 >> But that was years ago and everything keeps [CROSSTALK]
00:20:10 00:20:11 >> I'm just being honest,
00:20:11 00:20:14 there's never been an architecture problem with performance or
00:20:14 00:20:17 outages due to anything you did Jeffrey.
00:20:17 00:20:21 [LAUGH] It's always been some Azure or a network issue, or as you
00:20:21 00:20:26 know the focus on optimization and scale sometimes took over
00:20:26 00:20:30 from the how do we maintain this thing user perspective.
00:20:30 00:20:34 But again, the backend has survived completely intact.
00:20:34 00:20:36 >> So another thing for
00:20:36 00:20:39 me was I was learning a lot about Azure at the time.
00:20:39 00:20:42 So this was, what, 2013 we're saying was the year [INAUDIBLE]?
00:20:42 00:20:42 >> 2013, we were building it. Yeah.
00:20:42 00:20:43 >> So
00:20:43 00:20:46 I had been working with Azure prior to that, and I'd been working with
00:20:46 00:20:48 some customers, and I'd been working with some people at Microsoft.
00:20:48 00:20:50 But I never really architected anything
00:20:50 00:20:53 By myself from scratch entirely.
00:20:53 00:20:55 >> Let me [INAUDIBLE] just for
00:20:55 00:20:59 30 seconds, because if you guys don't know Jeffrey,
00:21:01 00:21:05 he's a super star in that Microsoft arena, in terms of architecture.
00:21:05 00:21:07 And I'll just mention two things he worked on.
00:21:07 00:21:09 He worked on some of the internals of building the CLR.
00:21:09 00:21:10 >> Sure. >> You're working right now on
00:21:10 00:21:12 the internals of service fabric.
00:21:12 00:21:13 >> Sure.
00:21:13 00:21:16 >> So when Jeffrey's working he gets to pick and
00:21:16 00:21:18 choose a lot of projects for what he wants to work on.
00:21:18 00:21:19 Whether it's multi,
00:21:19 00:21:21 what's threading in the garbage collector, or whatever.
00:21:21 00:21:27 And he's a master of that internal code optimization.
00:21:27 00:21:29 We're not talking about building a UI or
00:21:29 00:21:30 look at this cool electron app.
00:21:30 00:21:32 >> [LAUGH] >> We're talking about architecting
00:21:32 00:21:34 for optimization, speed, and performance.
00:21:34 00:21:37 And that's almost your entire background.
00:21:37 00:21:40 So just to make sure people know where you're coming from,
00:21:40 00:21:43 cuz it's a pretty premium rare skill set.
00:21:43 00:21:44 It's not my skill set.
00:21:44 00:21:46 I was a regular developer.
00:21:46 00:21:47 I would develop for the line of business parts.
00:21:48 00:21:52 >> Well, I grow up with TRS-80, it had 4k of RAM, and 4k of ROM.
00:21:52 00:21:53 That's all we had.
00:21:53 00:21:55 [LAUGH] And so I had to learn to make everything work,
00:21:55 00:21:59 and fast with that kinda really low end hardware at the time.
00:21:59 00:22:02 So when this project came along, I was really excited about it, and
00:22:02 00:22:05 I knew it is an opportunity for me to learn a lot about it, and
00:22:05 00:22:08 you know since it is my company intellect that we were gonna benefit
00:22:08 00:22:11 from it I really tried to invest heavily in it, tried to keep
00:22:11 00:22:16 costs down, architect for scale cuz we had no customers initially.
00:22:16 00:22:18 We're hoping to get a lot of customers in the future.
00:22:18 00:22:20 We don't want to re-architect it and
00:22:20 00:22:23 re-write it as those customers come on board.
00:22:23 00:22:24 We want them to have a great experience.
00:22:24 00:22:27 So these were all the things I was thinking about as I was
00:22:27 00:22:28 building out the system.
00:22:28 00:22:29 Furthermore, we had to have
00:22:29 00:22:31 a lot of conversations about the business model.
00:22:31 00:22:32 What are we gonna charge?
00:22:32 00:22:35 What kinda features, like the queues and the bookmarks?
00:22:35 00:22:39 And we had gotten feedback from people who used competitor sites,
00:22:39 00:22:43 like Pluralsight, that enterprises wanna pay for so many seat months,
00:22:43 00:22:45 like 100 seats per month.
00:22:45 00:22:49 And then move their employees in and out of those seats, to watch videos.
00:22:49 00:22:51 And then they want to be able to track what those employees
00:22:51 00:22:51 have watched.
00:22:51 00:22:53 So we talked to a bunch of these customers and
00:22:53 00:22:55 we gathered all this feedback and
00:22:55 00:22:59 input, and then we used all of this to go and architect this system.
00:23:00 00:23:04 All right, so I'm gonna walk through the name
00:23:04 00:23:07 Architecture key points of this system now.
00:23:07 00:23:12 So we have a video library xml file, cuz few years ago xml seem
00:23:12 00:23:15 to be more popular than Jayson and so it still xml file to this day.
00:23:15 00:23:19 And this is just an enormous xml file
00:23:19 00:23:23 that is read when ever we run on Azure in cloud services.
00:23:23 00:23:27 So whenever a web roll boots up, so we can scale in and
00:23:27 00:23:31 scale out dynamically using the Cloud services model in Azure.
00:23:31 00:23:34 Whenever a virtual machine boots up and it goes and joins our cluster,
00:23:34 00:23:40 if you will, upon boot it goes and reads this video library xml file.
00:23:40 00:23:43 This file contains in it our entire library information.
00:23:43 00:23:45 So the name of all the videos, the names of all the instructors,
00:23:45 00:23:48 which instructor is responsible for that video.
00:23:48 00:23:50 The subsections within the video.
00:23:50 00:23:52 The title of the subsection.
00:23:52 00:23:56 The time within the video where that subsection begins.
00:23:56 00:23:58 Links to any of the collateral material that Todd showed you like
00:23:58 00:24:00 where the slides can be downloaded.
00:24:00 00:24:03 Demos can be downloaded and so on.
00:24:03 00:24:04 All of that is in this file.
00:24:06 00:24:09 And as I said, it shows red by each virtual machine when it boots up.
00:24:09 00:24:11 And so once they read it,
00:24:11 00:24:15 then they will periodically go back to see if this file changes, and
00:24:15 00:24:20 we store this file Blob storage, in Azure Blob storage.
00:24:20 00:24:23 And one of the reasons, like a lot of host people would ask me,
00:24:23 00:24:25 why didn't we put this in a SQL database?
00:24:25 00:24:27 Why didn't we put this in an Azure table?
00:24:27 00:24:29 Why didn't we use some other kind of store for it?
00:24:29 00:24:32 Why did you use something so simple as the XML file?
00:24:32 00:24:35 Because now we can't do searches and things like that.
00:24:35 00:24:37 Well first of all, the amount of information in here
00:24:37 00:24:38 is not that much.
00:24:38 00:24:39 It's really metadata.
00:24:39 00:24:43 Certainly not the video content itself is not in this file.
00:24:43 00:24:46 And by having this as separate file like this, if we ever decide that
00:24:46 00:24:49 let's say Azure is having problems and we wanna move to some other
00:24:49 00:24:53 cloud service like Amazon, the file is gonna move to trivially.
00:24:53 00:24:56 Whereas if we put in let's say sequel Azure or an Azure table,
00:24:56 00:25:00 then we have to write code to migrate this data over.
00:25:00 00:25:03 This can also be read atomically, where's because we're uploading new
00:25:03 00:25:06 videos all the time and we're changing things in here.
00:25:06 00:25:10 So because blob storage is a atomic store right you,
00:25:10 00:25:13 when you upload something new it won't really raise the old thing
00:25:13 00:25:15 until the new thing is up there and its entirety.
00:25:15 00:25:18 We don't have to worry about race conditions that are happening.
00:25:18 00:25:21 Whereas if you had things that were being added to a database or
00:25:21 00:25:23 a table, you might be adding records for
00:25:23 00:25:26 the sections of a video while something else is
00:25:26 00:25:28 reading those sections of the video at the same time.
00:25:28 00:25:31 And then you end up with some corrupted state where
00:25:31 00:25:35 maybe something has read but it doesn't know all of the sections on
00:25:35 00:25:38 a particular video because the timing was just off.
00:25:39 00:25:42 Another reason for the is that we have mobile clients now, like you
00:25:42 00:25:47 can download an iOS Client App for WintellectNOW and on the client app
00:25:47 00:25:49 you wanna all the videos that are possible in the sections.
00:25:49 00:25:52 While the Client App can periodically go back into blob
00:25:52 00:25:54 storage and just read this file.
00:25:54 00:25:57 And then we can put a nice user interface on top of it to show you
00:25:57 00:26:00 that, as opposed to a chatty interface going back and forth to
00:26:00 00:26:04 some kind of database to get all the different pieces of information.
00:26:04 00:26:05 So this is much faster,
00:26:05 00:26:09 it's atomic, we can move it between Cloud services.
00:26:09 00:26:12 And blob storage is actually cheaper than Azure tables as well and
00:26:12 00:26:14 so that's another benefit.
00:26:14 00:26:19 Todd I don't know what you're doing [LAUGH] all right fine.
00:26:19 00:26:22 All right.
00:26:24 00:26:27 >> So the whole thing is responsive web it works
00:26:27 00:26:31 any device and there's client applications.
00:26:32 00:26:32 >> Yeah.
00:26:32 00:26:34 >> Versions of it.
00:26:34 00:26:35 For iOS, Android, and Windows.
00:26:35 00:26:38 So that kind of goes back to what you were saying.
00:26:38 00:26:40 So you could download these when you're a subscriber.
00:26:40 00:26:42 Watch them in an airplane while you're sitting there.
00:26:42 00:26:45 I just wanted to get that out because you were mentioning.
00:26:45 00:26:47 >> Sure. >> How easy it is to separate the UI
00:26:47 00:26:48 and the rest of it.
00:26:48 00:26:50 So let me start this up again for you.
00:26:50 00:26:51 >> Okay.
00:26:51 00:26:52 >> Okay. >> All right, now don't move.
00:26:52 00:26:53 >> I won't.
00:26:53 00:26:55 >> All right. [LAUGH] Then each virtual machine
00:26:55 00:26:59 will periodically go back to blob storage approximately every five
00:26:59 00:27:02 minutes or so, and will ask if there's a new version of this file.
00:27:02 00:27:05 And usually there isn't a new version of the file, in which case
00:27:05 00:27:11 it's just one quick IO request which were billed for the IO request, but
00:27:11 00:27:15 it's making IO requests in Azure storage is really, really cheap.
00:27:15 00:27:17 I forget what the number is now, but it's incredibly cheap.
00:27:17 00:27:19 So once every five minutes, I don't know,
00:27:19 00:27:21 maybe adds up to like a quarter a years.
00:27:21 00:27:22 It's like twenty-five cents a year.
00:27:22 00:27:24 It's really cheap to do that.
00:27:24 00:27:25 So if the file hasn't changed,
00:27:25 00:27:28 which is the usual case, then there's nothing to download, and
00:27:28 00:27:31 then the VM just uses all the meta data it currently has.
00:27:31 00:27:34 But, if it makes the request and the library has changed
00:27:34 00:27:38 because we've uploaded some new files, new videos, to the system.
00:27:38 00:27:42 Then, it will download the new video library xml file in its entirety.
00:27:42 00:27:44 We will parse the xml, and
00:27:44 00:27:46 then we do an atomic replace of it within the VM.
00:27:46 00:27:49 And so not all VM's are absolutely in sync with each other, but
00:27:49 00:27:52 it's an eventually consistency model and
00:27:52 00:27:54 it tends to work really well, and usually within five minutes.
00:27:54 00:27:56 All the VM's are going to be up to date anyway.
00:27:57 00:28:00 And then as I mentioned, it's client downloadable, this is for
00:28:00 00:28:03 the client side apps that Todd was just showing you a moment ago.
00:28:03 00:28:05 >> We didn't have to migrate anything, we didn't have to say hey
00:28:05 00:28:07 what's the data storage on this little device we're doing,
00:28:07 00:28:09 we just had an xml file, it just worked.
00:28:09 00:28:12 >> Yes, in all these different places.
00:28:12 00:28:16 Okay we do give cheaper pricing to people in certain countries like
00:28:16 00:28:20 India for example and so, when people log into the system,
00:28:20 00:28:22 we have to know what country they are coming from.
00:28:22 00:28:26 And when they watch the video we need to know what country they're
00:28:26 00:28:29 coming in from that request to go watch a video.
00:28:29 00:28:32 So, we have to go from the customers IP
00:28:32 00:28:34 address to determine what country they are in.
00:28:34 00:28:36 And there's numerous ways to do that.
00:28:37 00:28:40 There are web services out on the Internet where
00:28:40 00:28:42 when the person is logging in, we get their IP address and
00:28:42 00:28:45 then we can make a web request to this other service,
00:28:45 00:28:48 pass in the IP address, and it'll tell us what country it is.
00:28:49 00:28:52 And that was one thing we strongly considered, but
00:28:52 00:28:54 there were several reasons why we ended up ruling it out.
00:28:54 00:28:58 The first was that there's a latency involved with that.
00:28:58 00:29:00 So, we'd be making this request to another service,
00:29:00 00:29:03 I don't know how long it's gonna take for that service to come back.
00:29:03 00:29:04 Or what if that service is down?
00:29:04 00:29:06 Then, do we not let the person log in?
00:29:06 00:29:08 Do we not let them watch videos?
00:29:08 00:29:10 And at the top, there's that latency thing.
00:29:10 00:29:14 So, we didn't like that.
00:29:14 00:29:17 So instead, there are other services where you can go and
00:29:17 00:29:21 you can get an IpToCountry comma-separated file.
00:29:21 00:29:24 And we can download this file periodically from these other
00:29:24 00:29:26 services, and so we do that.
00:29:26 00:29:30 Like once a month or so we go and we grab this file from some service,
00:29:30 00:29:34 we take the file in its entirety, and we put it up in Blob storage.
00:29:34 00:29:37 Again that file is read by each of our virtual machines at boot time.
00:29:37 00:29:40 So we have a mapping in memory of all the IP address ranges to what
00:29:40 00:29:44 country those things are in, and then once a day we re-read
00:29:44 00:29:47 this file, because we don't have to keep it up to date all that often.
00:29:47 00:29:48 But really, as I said earlier,
00:29:48 00:29:50 we really only update that file once a month.
00:29:50 00:29:55 But the VM's poll for a change once a day.
00:29:55 00:29:57 And as I said at the bottom, it is seldom updated.
00:29:57 00:30:01 So now we don't have a dependency on an external service Each VM can
00:30:01 00:30:02 operate in parallel,
00:30:02 00:30:06 there's no latency issue because the file is there on the VM.
00:30:06 00:30:09 So we can just easily look up anything that we need to convert
00:30:09 00:30:13 an IP to a country for some of our internal logic related to billing.
00:30:17 00:30:21 The promo code, Todd showed you that you could log into the system and
00:30:21 00:30:24 for you guys you can use a promo code of Ignite2016.
00:30:24 00:30:27 So when people register with the system they have an option of
00:30:27 00:30:28 putting in a promo code.
00:30:28 00:30:30 The promo code might give them discounted pricing or
00:30:30 00:30:33 might let them use the service for free fro some number of hours or for
00:30:33 00:30:36 some number of months or something like that.
00:30:36 00:30:39 We have an Azure table that has promo codes in it and
00:30:39 00:30:43 we have some administrative tools where a person like Todd can go and
00:30:43 00:30:44 say I'm going to a night conference.
00:30:44 00:30:47 I need to generate a promo code, this will be the name of it, and
00:30:47 00:30:50 anybody who signs in with this promo code hears the features
00:30:50 00:30:52 that they get to use with that code.
00:30:52 00:30:55 And then that information goes into an Azure table when a person goes
00:30:55 00:30:59 registers and they have a place where they can put in a promo code.
00:30:59 00:31:01 We go to the Azure table, we look that up then we know how to
00:31:01 00:31:05 initialize their account with how much free time they have and so on,
00:31:05 00:31:07 what videos they can watch, etc.
00:31:11 00:31:15 As videos are being watched, we pay royalties to the authors who
00:31:15 00:31:19 produced the videos and we do that once a quarter.
00:31:19 00:31:22 So, at the end of each quarter, we need to look up and see for
00:31:22 00:31:25 this video, how many views or how many minutes of it were watched?
00:31:25 00:31:29 For this video, how many minutes of it were watched and so on?
00:31:29 00:31:30 So we have another Azure table for that,
00:31:30 00:31:35 which is our video statistics table and as people are watching videos,
00:31:35 00:31:38 we are periodically updating a counter in this table.
00:31:38 00:31:41 So it's, we're just going right to a particular video and
00:31:41 00:31:44 we're just adding one to a calendar property in there.
00:31:44 00:31:46 So the information we keep in the table,
00:31:46 00:31:48 I try to keep it really small and sweet.
00:31:48 00:31:49 Because the more information you have,
00:31:49 00:31:54 the more expensive it's gonna be to pay for that table year in and
00:31:54 00:31:57 year out, cuz we're charged for gigabytes per month on it.
00:31:57 00:32:00 Also, we wanna make it small, because we wanna update it
00:32:00 00:32:03 very quickly, so you'll have to read the current number, add one, and
00:32:03 00:32:05 write the number back out.
00:32:05 00:32:09 And then, Azure tables use an optimistic concurrency model.
00:32:11 00:32:15 So we have to have retry loops in our code and things like that, but
00:32:15 00:32:18 the likely hood of multiple people watching the same video.
00:32:18 00:32:21 And we're trying to update it at the same time is very unlikely.
00:32:21 00:32:23 So there's not a lot of contention on it, which means the performance
00:32:23 00:32:27 of it is very good and the costs again are low.
00:32:27 00:32:29 Because retries will also cost you money.
00:32:29 00:32:33 We want to try to reduce retries as much as possible in the code,
00:32:33 00:32:35 it cost you time and it cost you money.
00:32:35 00:32:38 The real bread and butter of the service though are the user accounts
00:32:38 00:32:42 and there is several things that are associated with a user.
00:32:42 00:32:46 So first off, a user has there login information and we use and
00:32:46 00:32:48 email address to log you in.
00:32:48 00:32:51 All the passwords information is completely encrypted, we don't store
00:32:51 00:32:56 any financial information on this site, we never even ask it of you.
00:32:56 00:32:58 So, we don't have credit card, we don't have social security number.
00:32:58 00:33:03 We don't have, I've focused a lot on the security aspects of this
00:33:03 00:33:07 to make sure that if we are, I guess assume it will get compromise.
00:33:07 00:33:10 So if we get compromised, what do we have we can lose?
00:33:10 00:33:14 And the best answer was, let's not have anything,
00:33:14 00:33:16 if we just don't have it, then there is nothing that can be lost.
00:33:16 00:33:19 So the most critical thing you have would be your email address
00:33:21 00:33:24 that's all that we even ever store anywhere.
00:33:24 00:33:28 And then a user belongs to an account and the account is the unit
00:33:28 00:33:31 of billing, an account can pay for multiple people.
00:33:31 00:33:35 Like an enterprise, could say, I'm buying 100 seat/months, and then,
00:33:35 00:33:37 they put those users in and
00:33:37 00:33:39 out of those seats, over the course of the month.
00:33:39 00:33:42 So, a user belongs to an account and we actually have a feature where
00:33:42 00:33:45 a user can belong to multiple accounts.
00:33:45 00:33:47 Like they might have a personal account, and
00:33:47 00:33:51 then maybe they work for a company, and then the company can pay for
00:33:51 00:33:54 them, and they can suspend their personal account.
00:33:54 00:33:57 And then when the user leaves that company, sometime in the future they
00:33:57 00:34:00 quit, then they can reinstate their personal account.
00:34:00 00:34:05 So associated with the user we have all the videos they've watched and
00:34:05 00:34:07 that's associated with the user, not with the account.
00:34:07 00:34:10 Because, as I said, you might have your own personal account, you watch
00:34:10 00:34:14 these 10 videos, then maybe your company is paying for you.
00:34:14 00:34:16 But you still watch those 10 videos, you want to be able to see your
00:34:16 00:34:19 history you leave that company, you'd still like to be able to see
00:34:19 00:34:22 the full history of everything you've watched.
00:34:22 00:34:23 This was a big change that happened for
00:34:23 00:34:26 us when we architected version two.
00:34:26 00:34:27 The way that we had done counts and
00:34:27 00:34:31 users in the past really wasn't working very well, and so
00:34:31 00:34:34 this new layout, which is substantially better.
00:34:34 00:34:37 Actually, the code is cleaner, the concepts are cleaner, we can present
00:34:37 00:34:41 the UI cleaner, and really we added more features to it as well.
00:34:41 00:34:44 So we were addressing customer and enterprise needs, so
00:34:44 00:34:47 associated with each user there's an Azure table
00:34:47 00:34:50 that has all the videos that were watched by that user.
00:34:51 00:34:53 And the user is the partition key in that.
00:34:53 00:34:56 And so we would never have contention on the same partition key
00:34:56 00:34:59 unless the single user was logged in multiple times.
00:34:59 00:35:03 And using the service actively from multiple different locations
00:35:03 00:35:05 which is very unlikely to happen.
00:35:06 00:35:09 So, our partitioning scheme is good that keeps our scability,
00:35:09 00:35:12 our performance really high and it also keeps our cost low and
00:35:12 00:35:15 there wouldn't be any re-trys on that access of the data.
00:35:15 00:35:17 Associated with the user is the bookmarks.
00:35:17 00:35:19 So, as your watching the video,
00:35:19 00:35:22 they have the ability to put a bookmark say like ten minutes into
00:35:22 00:35:24 the video, here's what I found that was really interesting.
00:35:24 00:35:26 [INAUDIBLE] 20 minutes into this video,
00:35:26 00:35:28 here is another thing I found interesting.
00:35:28 00:35:29 So they can add these bookmarks, and
00:35:29 00:35:32 they can type some text with it to remind them of what it is.
00:35:32 00:35:35 Then when they go to their profile page, they can see all the bookmarks
00:35:35 00:35:38 they've ever set, set and then they can click on it.
00:35:38 00:35:39 It will go right to that video,
00:35:39 00:35:42 right to the location in the video and start playing from there,
00:35:42 00:35:45 if they wanna refresh their memory as to what they've been seeing.
00:35:45 00:35:47 So, the bookmark information for
00:35:47 00:35:52 the user is also stored in an Azure table and then users have a queue
00:35:52 00:35:54 of what videos they want to add to the queue.
00:35:54 00:35:57 And the queue can be ordered, like they prefer to watch this video
00:35:57 00:36:00 next, and then this one next, and this one next.
00:36:00 00:36:02 Because Azure tables are ordered
00:36:02 00:36:05 by alphabetically by what you put in the table.
00:36:05 00:36:08 But we wanted to give the user the ability to order the contents of
00:36:08 00:36:13 their queue it wasn't a good match to put queues in an Azure table.
00:36:13 00:36:16 So what we ultimately ended up doing was we put the queue information
00:36:16 00:36:17 into a blob.
00:36:17 00:36:22 A blob is just a you know a sequence of bytes but our code that we write
00:36:22 00:36:27 can completely controls the order of the information that's in that blob.
00:36:27 00:36:30 Also blobs are cheaper than tables as well and
00:36:30 00:36:33 blobs can be more easily moved to other services like Amazon
00:36:33 00:36:36 where they've tried to move contents of a table somewhere else.
00:36:36 00:36:39 Then becomes really challenging because Amazon's tables don't look
00:36:39 00:36:41 like Azure's tables and so on.
00:36:42 00:36:45 So we try to avoid tables as much as possible, quite frankly, and
00:36:45 00:36:49 we try to use blobs a lot because, again, they're cheaper.
00:36:49 00:36:51 They're faster, and
00:36:51 00:36:55 they're easier to move to some other place if we ever decide to move.
00:36:55 00:36:57 The thing you get with tables that you don't get with blobs,
00:36:57 00:37:00 though, is searching and filtering.
00:37:00 00:37:02 And if you need to do a lot of searching and
00:37:02 00:37:04 filtering, then tables is a better choice,
00:37:04 00:37:07 but realize that you're gonna compromise a little bit on cost.
00:37:07 00:37:10 A little bit on performance, and a little bit on the ability to
00:37:10 00:37:12 move your data around from service to service.
00:37:14 00:37:17 Now as I said,
00:37:17 00:37:21 each user is associated with one or more accounts.
00:37:21 00:37:25 An accounts actually have or sponsors if you will for one or
00:37:25 00:37:27 more user within the account and
00:37:27 00:37:29 the account is really the unit of billing.
00:37:29 00:37:32 So there's an identity for the account like who is the manager for
00:37:32 00:37:35 the account and you and within the company you can
00:37:35 00:37:39 actually have multiple people who are managers of an account.
00:37:39 00:37:42 And then there is billing information in there.
00:37:42 00:37:44 We have some customers that pay monthly by credit card,
00:37:44 00:37:47 we have some customers that just send us a purchase order for
00:37:47 00:37:48 maybe a year in advance.
00:37:48 00:37:52 We have some customers that use PayPal, for
00:37:52 00:37:56 example, so all that information is stored in an Azure table.
00:37:56 00:37:58 And I'll talk more about what that actually looks like and
00:37:58 00:38:00 how we designed the table a little bit later on.
00:38:05 00:38:07 So that's the bulk of it but
00:38:07 00:38:10 then there's these administrative tasks, too.
00:38:10 00:38:13 Whenever you're running some kind of cloud service, you have all these
00:38:13 00:38:16 virtual machines that are doing various things at various times.
00:38:16 00:38:21 And sometimes things do go wrong and sometimes things go right, but
00:38:21 00:38:23 you still would like to have a log of what happened.
00:38:23 00:38:27 For example, if somebody in the company pulls a person out of a seat
00:38:27 00:38:28 and adds another person into the seat.
00:38:28 00:38:30 What if that person calls us up and says, hey,
00:38:30 00:38:33 I used to have a seat at this company and I don't anymore.
00:38:33 00:38:36 Well, they could call us and we could say, well on this date,
00:38:36 00:38:39 this date this person on your company took you out of the seat and
00:38:39 00:38:40 put this other person in the seat.
00:38:40 00:38:44 And, we need to have records of that information so that we could this
00:38:44 00:38:47 kind of customer support for people and track what's going on.
00:38:47 00:38:52 So, logging is a critically important part of any kind cloud
00:38:52 00:38:54 based service.
00:38:54 00:38:59 And we have created a whole logging architecture that focuses on
00:38:59 00:39:02 lots of different ways of producing logs.
00:39:02 00:39:04 And we have all different kinds of logs and
00:39:04 00:39:06 they're produced in different ways.
00:39:06 00:39:09 I have slides on that a little bit later on but then the logging
00:39:09 00:39:12 thing was critically important for many reasons.
00:39:12 00:39:15 Then of course we want to do it fast, we want to do it cheaply and
00:39:15 00:39:20 we want to do it typically best effort not necessarily guaranteed.
00:39:20 00:39:24 So there are chances that in Azure, a node could go down and
00:39:24 00:39:27 we may lose some log information there but it's, fortunately,
00:39:27 00:39:30 it's very unlikely that that happens.
00:39:30 00:39:34 But it is possible, so the logs are really done in a best effort.
00:39:34 00:39:36 And usually we don't have any failures, but
00:39:36 00:39:37 occasionally it's a chance that we will.
00:39:37 00:39:40 And we have to realize that when a customer calls us up and
00:39:40 00:39:43 we're looking at a log there is a chance that some information may be
00:39:43 00:39:45 missing from the log.
00:39:45 00:39:48 If you really wanted to have logs that were absolutely consistent
00:39:48 00:39:49 everywhere its a lot more effort.
00:39:49 00:39:51 It slows things down, and
00:39:51 00:39:54 then what happens if there's a bug in your logging infrastructure,
00:39:54 00:39:57 then you want to log that, then you have a real big problem.
00:39:57 00:39:59 So those are the kinds of things that we thought about.
00:39:59 00:40:03 So we log promo code usage so, for example, after Ignite maybe a month
00:40:03 00:40:06 or two from now, Todd could look up and see us.
00:40:06 00:40:10 So how many people registered using the Ignite 2016 promo code?
00:40:10 00:40:12 And we're expecting all of you to do that in this room, so
00:40:12 00:40:16 we're gonna get six maybe, or so.
00:40:16 00:40:18 But you can give it out to friends and family, so
00:40:18 00:40:19 maybe a little bit higher than that.
00:40:19 00:40:21 But this gives us ability to track also,
00:40:21 00:40:23 when we go to different trade shows.
00:40:23 00:40:26 At this trade show, we had this many people registered.
00:40:26 00:40:28 At this trade show, we had this many people register.
00:40:28 00:40:31 And that helps our marketing people know where to invest more energy,
00:40:31 00:40:33 in which trade shows to go to in the future.
00:40:34 00:40:35 So, it's useful for that.
00:40:35 00:40:38 And then the account management which was the example I gave earlier
00:40:38 00:40:39 on this date.
00:40:39 00:40:41 Your administrator took you out of the seat and
00:40:41 00:40:44 put somebody else in the seat or whatever on this date.
00:40:44 00:40:47 We successfully billed your credit card but on this date,
00:40:47 00:40:50 we we're unsuccessful with billing your credit card
00:40:50 00:40:53 where we can give that information to anybody who calls us up.
00:40:53 00:40:55 Those are ongoing logs, by ongoing logs,
00:40:55 00:40:58 it means we never really delete the information.
00:40:58 00:41:00 We're just constantly adding to it,
00:41:00 00:41:03 we usually put them in files that are on a month by month basis.
00:41:03 00:41:06 So this is the log for January, this is the log for February,
00:41:06 00:41:07 this is the log for March.
00:41:07 00:41:10 And then we just kind of keep them in perpetuity
00:41:10 00:41:14 since they're in blob storage and Azure blob storage is really cheap.
00:41:14 00:41:18 We can keep a lot of information in there in a very cost effective way.
00:41:18 00:41:21 And we only tend to really go back and read the information
00:41:21 00:41:23 if a customer calls us up and they need some kind of support.
00:41:23 00:41:26 Or if we wanna do some data mining, over it then we would go and
00:41:26 00:41:29 read it, other than that, it's mostly archival data.
00:41:29 00:41:32 We have periodic logs with garbage collection.
00:41:33 00:41:37 These are logs that we do roll over, we have some retention on them,
00:41:37 00:41:39 we'll keep the last week's worth.
00:41:39 00:41:41 Or the last seven days worth, let's say it that way.
00:41:41 00:41:44 And then we will delete anything older than that because we don't
00:41:44 00:41:48 want these logs to grow without bounds because we wanna keep our
00:41:48 00:41:52 costs low of storing this stuff in Azure storage.
00:41:52 00:41:55 So we have a daily log which is done for any errors.
00:41:55 00:41:58 Any of our code that causes unhandled exceptions and
00:41:58 00:41:59 causes the program to crash and fail.
00:41:59 00:42:02 We wanna capture those, of course, so that we can go into the code and
00:42:02 00:42:04 make the code better.
00:42:04 00:42:08 So every day, the first exception that's thrown on that VM for
00:42:08 00:42:11 the day, that will go and create this log file.
00:42:11 00:42:13 And then we'll accumulate exceptions with stack traces and
00:42:13 00:42:15 other information in these.
00:42:15 00:42:18 And then our developers can go and look at these log files and say,
00:42:18 00:42:20 okay, this is what's going on.
00:42:20 00:42:23 We get the stack trace information and we can go and fix it and
00:42:23 00:42:24 we store these in blob storage.
00:42:26 00:42:28 Some systems, like even Azure,
00:42:28 00:42:33 they normally save these kinda stuff in Azure tables.
00:42:33 00:42:35 I don't like using Azure tables for
00:42:35 00:42:38 this kind of stuff because it's slow, it's more expensive.
00:42:38 00:42:41 You can go to the table and then you can say let me search for
00:42:41 00:42:43 things in the table.
00:42:43 00:42:48 But you almost never do that and so doing the logs made it faster and
00:42:48 00:42:51 cheaper and we did it on day by day basis.
00:42:51 00:42:51 So we could save and
00:42:51 00:42:55 say, yesterday there are no errors at all that occurred, that's great.
00:42:55 00:42:57 Or maybe the last week, there's nothing there, right?
00:42:57 00:43:00 And then, the idea is to just make that be as small as possible.
00:43:00 00:43:03 And then, we also have the account user watching information.
00:43:03 00:43:06 So every time users are watching, we record for
00:43:06 00:43:09 the account that is sponsoring that user.
00:43:09 00:43:12 This user on this day, watched these many minutes of this video, or
00:43:12 00:43:15 this user on this day just watched this.
00:43:15 00:43:17 And this allows the administrators of an account,
00:43:17 00:43:19 to be able to look at the information.
00:43:19 00:43:22 And say over the last month, Jeffery Rictor watched these videos for
00:43:22 00:43:24 this period of time.
00:43:24 00:43:27 So the administrator of the company can see if they're
00:43:27 00:43:29 getting a good return on investment on the product.
00:43:29 00:43:33 That information is being stored in coma separated files and
00:43:33 00:43:35 then we add it to the user interface.
00:43:36 00:43:40 Web pages that can go and download that and then can show graphs and
00:43:40 00:43:43 charts, I showed you some of that earlier.
00:43:43 00:43:46 Also, those CSV files can simply be downloaded out of blob storage.
00:43:46 00:43:48 We create a shared access signature for it,
00:43:48 00:43:51 if you know what that is for blobs.
00:43:51 00:43:54 And then we give a personal link on the webpage when they click that
00:43:54 00:43:55 link, it goes right to blob storage.
00:43:55 00:43:59 Bypassing our website entirely, that increases our scalability and
00:43:59 00:44:00 lowers our cost.
00:44:00 00:44:03 The CSV file gets downloaded right from blob storage down to
00:44:03 00:44:05 the administrator's machine.
00:44:05 00:44:07 And then they could load that up in Excel or
00:44:07 00:44:09 some other tool of their choice.
00:44:09 00:44:12 And produce whatever charts or diagrams they want to analyze it,
00:44:12 00:44:14 to make sure that they're getting a return of investment.
00:44:14 00:44:17 Or if the people on a certain team have watched the required videos to
00:44:17 00:44:21 really be successful for working with certain technologies that,
00:44:21 00:44:23 that team within the company seems to use.
00:44:27 00:44:30 Okay, and we didn't wanna build everything.
00:44:30 00:44:32 And if we had to build absolutely everything,
00:44:32 00:44:37 including the video streaming services and the transcoding and
00:44:37 00:44:39 a content delivery network.
00:44:39 00:44:41 It's really too much for,
00:44:41 00:44:44 we really only had two developers at any time working on the system.
00:44:44 00:44:46 Me on the backend and somebody else doing the frontend.
00:44:49 00:44:52 And we're not experts in a lot of these other things like payments and
00:44:52 00:44:54 sending out emails and so on.
00:44:54 00:44:57 So we found a bunch of third party services that were out there.
00:44:57 00:45:00 Now, this does mean that we're taking a dependency on these.
00:45:00 00:45:03 So some of them are free, some of them we have to pay for.
00:45:03 00:45:05 Some of them, the service might be down,
00:45:05 00:45:07 what do we do if the service is down.
00:45:07 00:45:10 Some of them, the latency might be high, we wanna make sure that our
00:45:10 00:45:13 end users have a good performance experience when using our site.
00:45:13 00:45:16 So we did a lot of analysis on these various
00:45:16 00:45:19 companies that were providing these various services.
00:45:19 00:45:20 We compared their prices,
00:45:20 00:45:23 we compared their service level agreements with one another.
00:45:23 00:45:27 And then we ultimately, came back with a set of them that we need so
00:45:27 00:45:31 we have an external service that we use for storing our videos, and for
00:45:31 00:45:32 transcoding them.
00:45:32 00:45:35 And they are connected up to a content delivery network which
00:45:35 00:45:38 allows the videos to stream globally, across the earth.
00:45:38 00:45:41 So that all of our customers have a good streaming experience when
00:45:41 00:45:43 watching the videos.
00:45:43 00:45:44 We teamed up with a bunch,
00:45:44 00:45:48 a few different payment services like Stripe and PayPal.
00:45:48 00:45:51 So there are customers have different ways of paying for
00:45:51 00:45:52 the servers.
00:45:52 00:45:54 And that means if they have a relationships with these
00:45:54 00:45:57 other services, that will store other credit card information.
00:45:57 00:46:01 Which means we are not storing their credit card, ao we also did it for
00:46:01 00:46:02 security reasons as well.
00:46:02 00:46:04 Using this external services,
00:46:04 00:46:08 we need to send emails out to people when you register on the service.
00:46:08 00:46:11 Or when a person at a company puts you into a seat,
00:46:11 00:46:14 we wanna send you an email saying, you've nowbeen put into a seat, so
00:46:14 00:46:15 you're allowed to watch videos.
00:46:15 00:46:18 Or when you get removed from a seat, we wanna send you an email saying
00:46:18 00:46:20 you're not allowed to watch videos anymore.
00:46:20 00:46:24 The person took you out of the seat and the person can add you and
00:46:24 00:46:25 you don't even have an account on the system.
00:46:25 00:46:28 So we will email you and say, hey, this person just put you
00:46:28 00:46:31 into a seat, click on here to go create your account.
00:46:31 00:46:35 We'll add you to that, create your user account if you will.
00:46:35 00:46:37 Then we'll add you to a company account, and
00:46:37 00:46:39 then you can start watching videos.
00:46:39 00:46:41 So we needed email support for all of that,
00:46:41 00:46:45 plus periodic marketing blasts and things like that, we have to do.
00:46:45 00:46:47 So we use a third party company for
00:46:47 00:46:50 that, we use a third party company for user support.
00:46:50 00:46:51 So if people have problems on the website,
00:46:51 00:46:52 something isn't working correctly.
00:46:52 00:46:54 They wanna offer a suggestion or
00:46:54 00:46:57 feature that we can add to the service, so
00:46:57 00:47:00 there's a third party service that we use, UserVoice.
00:47:00 00:47:03 Where they can go and enter that information in, it goes and
00:47:03 00:47:05 creates tickets that emails people within Wintellect.
00:47:05 00:47:08 Like our marketing people to hear that kind of stuff and
00:47:08 00:47:10 then we can consider writing those features over time.
00:47:10 00:47:13 And then whenever you watch a video, there's a forum for each video.
00:47:13 00:47:16 Where people can go and have a discussion about what they saw on
00:47:16 00:47:18 the video or maybe asked other questions.
00:47:18 00:47:21 And the author of the video can contribute to the forum and
00:47:21 00:47:24 answer questions that people have when they watch their content.
00:47:24 00:47:27 And we use a third-party service for the forum stuff as well.
00:47:30 00:47:33 So if a lot of these go down, certainly if email goes down that
00:47:33 00:47:36 means we can't send you email for a day or two untill it comes back up.
00:47:36 00:47:40 If the forums go down that means you can't contribute to the forums, but
00:47:40 00:47:42 you can still watch videos, right?
00:47:42 00:47:43 And these were all kinds of things that I had to
00:47:43 00:47:45 think about when designing the service, so
00:47:45 00:47:48 that none of these things were on the critical path.
00:47:48 00:47:48 Now, obviously,
00:47:48 00:47:51 the video service goes down, you can manage your account.
00:47:51 00:47:56 But you don't have a hard time watching a video over there.
00:47:56 00:47:59 And if payment goes down and we can't let's say,
00:47:59 00:48:02 bill your credit card, we do have a grace period where
00:48:02 00:48:05 we will let you continue to watch for a day or two.
00:48:05 00:48:08 And then we will assume that we're gonna try billing again, so
00:48:08 00:48:11 we have to have some retry logic in there for that.
00:48:11 00:48:13 But again, we want you to have a good experience, so
00:48:13 00:48:15 if the payment service seems to be down.
00:48:15 00:48:17 We'll extend you some grace period,
00:48:17 00:48:20 we'll keep trying to bill before we shut your account off.
00:48:20 00:48:24 If it turns out that the billing service is up but your credit card
00:48:24 00:48:27 has expired, then of course we'll shut you down, immediately.
00:48:27 00:48:29 And the next time you log in,
00:48:29 00:48:31 we won't let you watch videos but it will tell you why.
00:48:31 00:48:33 And then you can give us a new expiration date for
00:48:33 00:48:35 your credit card or change it.
00:48:35 00:48:37 And then we will start the process again and
00:48:37 00:48:38 then you can start watching videos again.
00:48:38 00:48:43 Right, so, these are all things we had to think about in order to
00:48:43 00:48:47 provide the service in a scalable, inexpensive way, and
00:48:47 00:48:48 be feature rich.
00:48:49 00:48:54 All right, so to go into a little bit more of the details now,
00:48:54 00:48:57 the real bread and butter as I said, is the way we manage users and
00:48:57 00:48:59 accounts in the system.
00:48:59 00:49:01 That's really the most important part to us,
00:49:01 00:49:03 everything else is kind of icing on the cake.
00:49:03 00:49:07 But you need to be able to log in and we need to be able to
00:49:07 00:49:10 be billing you for the service cuz we wanna make money with it.
00:49:10 00:49:14 So we have this user table, which is an Azure table, and
00:49:14 00:49:17 the partition key of the table is a person's email address.
00:49:17 00:49:19 That's what people use to log in to the system.
00:49:19 00:49:25 So there's gonna be one row in the table for a particular person, and
00:49:25 00:49:27 the row key is just an empty string, quote, quote.
00:49:27 00:49:30 So we don't use the row key for anything at all.
00:49:30 00:49:36 Then our new user in our system, we create a unique user ID for you.
00:49:36 00:49:38 That is a GUID, that is stable.
00:49:38 00:49:42 So you can change your email address over time but
00:49:42 00:49:45 your user ID is a static value that will never change.
00:49:45 00:49:48 So anything else that we need to know about you, like your queue,
00:49:48 00:49:51 your bookmarks, your video watch history.
00:49:51 00:49:54 All of that information is keyed off the static user ID.
00:49:54 00:49:58 In version 1 of our service, it used to be that we would find those
00:49:58 00:50:00 things by looking up your email address.
00:50:00 00:50:01 So, they were keyed off the email address.
00:50:01 00:50:03 And then, when a person change their email address,
00:50:03 00:50:06 we couldn't find those things anymore.
00:50:06 00:50:09 So, we introduced this in order to make that change.
00:50:09 00:50:12 So, you're able to change your email address which is your way of logging
00:50:12 00:50:16 in, but everything else is keyed off the static value that will never
00:50:16 00:50:17 change over the lifetime.
00:50:17 00:50:21 That really simplified things pretty dramatically for us.
00:50:21 00:50:25 And then, a user can be an administrator of zero or
00:50:25 00:50:27 many accounts.
00:50:27 00:50:30 So, there's just an AdminOf column here, which has an array of
00:50:30 00:50:34 Account IDs which could be empty, so you're not an administrator of any
00:50:34 00:50:37 accounts at all or you could be the administrator of multiple accounts.
00:50:37 00:50:40 Within a single company lets say, and
00:50:40 00:50:44 then the user can be in a grantee of zero or more accounts,
00:50:44 00:50:47 which means your aloud to watch videos if this account
00:50:47 00:50:51 is granting you access which means they're basically paying for you.
00:50:52 00:50:55 So, as long as the account is still not in arrears,
00:50:55 00:50:58 it's still valid their payments are up to date.
00:50:58 00:51:00 If you are a grantee of the account,
00:51:00 00:51:02 then you are allowed to continue to watch videos.
00:51:04 00:51:05 Then we have the account table.
00:51:05 00:51:09 The account table is the list of accounts that we have.
00:51:09 00:51:12 The partition key is an account ID.
00:51:12 00:51:14 The row key is an empty string.
00:51:14 00:51:17 So again, there's really only one entry in here for each account.
00:51:18 00:51:19 Then there's,
00:51:19 00:51:22 who are the users who are the administrators of the account?
00:51:22 00:51:25 So that's an array of email addresses there.
00:51:25 00:51:29 And then, we have, so these are the people who can change who's being
00:51:29 00:51:31 granted access within an account.
00:51:31 00:51:35 And then, we have a grantees column, which is an array of email addresses
00:51:35 00:51:38 for the people who are being granted access to the account.
00:51:38 00:51:39 Now, if a person's paying for
00:51:39 00:51:42 let's say 100 seat months, that means that we'll never let more
00:51:42 00:51:47 than 100 grantee emails be in that column at any given time, right.
00:51:47 00:51:47 If they're paying for
00:51:47 00:51:50 1,000 then we would let no more than 1,000 be in there.
00:51:50 00:51:51 And our business logic and
00:51:51 00:51:57 our code manages that to make sure that that constraint remains true.
00:51:58 00:52:00 Then we have payment properties.
00:52:00 00:52:03 So, some accounts as I said are being paid for with credit cards,
00:52:03 00:52:06 some accounts are being paid for with a purchase order,
00:52:06 00:52:07 some accounts are being paid for by different things.
00:52:09 00:52:11 The way we ultimately settled on this,
00:52:11 00:52:14 I'm really happy with I think this worked out really great for
00:52:14 00:52:18 us and it's really taking advantage of a no SQL Database here.
00:52:18 00:52:21 You wouldn't be able to do it quite as easily with a relational
00:52:21 00:52:24 database like a SQL database, but that was really
00:52:24 00:52:29 easy to solve this using a no SQL Database which is what tables are.
00:52:29 00:52:32 And then, we need to map to the Azure table information
00:52:32 00:52:34 into dot net C# classes.
00:52:34 00:52:36 By the way, all this stuff is written in C#.
00:52:36 00:52:40 So, in C#, we have an abstract to base class which is called
00:52:40 00:52:44 an account, and this has some common properties for all accounts.
00:52:45 00:52:48 Like maybe the expiration date, for example.
00:52:48 00:52:50 Then, that's an abstract base class.
00:52:50 00:52:53 Then derived off of that abstract base class,
00:52:53 00:52:56 I have three other classes which are sealed classes.
00:52:56 00:52:58 Well, and actually there's several more than three, but
00:52:58 00:53:00 I'm only showing three here on the slide.
00:53:00 00:53:03 So, we have a credit card account, for those people who are paying with
00:53:03 00:53:06 credit cards, and that's derived from account, and then that adds
00:53:06 00:53:09 some additional properties that are specific to a credit card.
00:53:09 00:53:12 Again, we don't store credit card numbers in our service anywhere.
00:53:12 00:53:15 But we would have maybe an expiration date for
00:53:15 00:53:17 when your billing has stopped.
00:53:17 00:53:20 And you could prepay us with the credit card for a year.
00:53:20 00:53:23 Then we know that you're going to stop 12 months in.
00:53:23 00:53:25 Kind of thing. So, we would store that in here.
00:53:25 00:53:29 Then, we have a direct debit account that would be derived from the same
00:53:29 00:53:32 base class, and it might have some properties in it for direct debit.
00:53:32 00:53:34 And then, we have a purchase order account, and
00:53:34 00:53:36 that's also derived from the same base class account, and
00:53:36 00:53:39 that would have some properties for PO props.
00:53:39 00:53:42 And if we ever added any additional billing things in the future, and
00:53:42 00:53:45 actually we do have some, we have well that's a credit card one.
00:53:45 00:53:48 We have PayPal and we have some other ones like we have free
00:53:48 00:53:51 accounts is another one of our account types that we have.
00:53:51 00:53:54 So, if we ever added additional types of billing we just creates
00:53:54 00:53:57 a new class, derive it of the account base class and
00:53:57 00:54:00 then we add whatever properties are specific for it.
00:54:00 00:54:05 And then, we take that account object, we can then take the various
00:54:05 00:54:08 properties of it including the base class properties.
00:54:08 00:54:11 And then, we put that into the Payment Properties column.
00:54:11 00:54:12 So then, when we go and
00:54:12 00:54:17 read an account out of the table, we can see what properties are there.
00:54:17 00:54:23 Then we know to deserialize it into the correct class over here,
00:54:23 00:54:26 and we're storing the data in the table, exactly what we need.
00:54:26 00:54:29 We don't have empty columns for things that we don't need.
00:54:29 00:54:31 We can go and create new account types in the future that have new
00:54:31 00:54:34 properties that we haven't even thought of today, and
00:54:34 00:54:36 those will be efficiently stored in the table and
00:54:36 00:54:39 efficiently retrieved from the table, so it'll be cheap.
00:54:39 00:54:41 It's easier for us to add new accounts,
00:54:41 00:54:45 and this worked out phenomenally great for us, so we're really happy
00:54:45 00:54:50 with this architecture and the way that this is working.
00:54:50 00:54:52 And this was all done in our version two.
00:54:52 00:54:55 Our version one was not good, [LAUGH].
00:54:55 00:54:58 Our version one stuff, the way we handled payment.
00:54:58 00:54:59 We had all the properties for
00:54:59 00:55:02 every kind of billing that we had, all in every row.
00:55:02 00:55:04 Some of them would be euros, some would be minus one,
00:55:04 00:55:05 some would be empty.
00:55:05 00:55:08 And it was much harder to go and look things up.
00:55:08 00:55:09 It was also easy where we could go and
00:55:09 00:55:13 use tooling to scan over the table, and we could say so
00:55:13 00:55:15 how many credit card customers do we have,
00:55:15 00:55:18 how many purchase orders customers do we have, and it's pretty easy to
00:55:18 00:55:22 do a query across that account table to get that kind of information now.
00:55:23 00:55:26 This worked out, there's so many benefits of doing it this way,
00:55:26 00:55:27 it really worked out well.
00:55:29 00:55:31 So adding and removing accounts.
00:55:31 00:55:34 This is some of the logic that we have for doing that.
00:55:34 00:55:36 We have a cardinal rule.
00:55:36 00:55:38 That whatever we wrote code anywhere in the system,
00:55:38 00:55:42 the cardinal rule was, an account never trusts a user.
00:55:42 00:55:44 That's the default.
00:55:44 00:55:47 That means that you are by default not allowed to watch a video.
00:55:48 00:55:51 Under any given account that you're a part of, unless,
00:55:51 00:55:55 that account is not in arrears, the payment status is good.
00:55:55 00:55:57 So that's our first rule.
00:55:57 00:56:00 Then, we have to disprove the default
00:56:00 00:56:02 by making sure that the account is not in arrears.
00:56:03 00:56:07 So this is the logic that we use for having an Administrator add or
00:56:07 00:56:09 remove a user to a particular account.
00:56:11 00:56:14 First, and because tables is an eventual consistency model,
00:56:14 00:56:16 we can't do this with transactions.
00:56:16 00:56:19 For consistency, we had to write this code in a certain order.
00:56:19 00:56:21 It has to be done in a certain order.
00:56:21 00:56:24 And it has to be replayable, so that it's idempotent as well,
00:56:24 00:56:27 because it might get replayed back multiple times due to
00:56:27 00:56:31 the eventual consistency models that tables provide.
00:56:31 00:56:34 So, we update the account, that means we add the user to
00:56:34 00:56:38 the array of users for a particular account.
00:56:38 00:56:40 Imagine somebody at a company is saying Jeff can
00:56:40 00:56:41 now go watch a video.
00:56:41 00:56:44 So, we're adding Jeff to this account,
00:56:44 00:56:46 saying that Jeff's allowed to watch videos.
00:56:46 00:56:49 And then, we send an email notification to the user saying,
00:56:49 00:56:51 you have just been added to this account, so
00:56:51 00:56:53 you're now allowed to watch videos.
00:56:53 00:56:56 In the email we have a URL that they can click on
00:56:56 00:57:00 to basically accept that they have been granted into this and
00:57:00 00:57:03 they can click on that multiple times, right?
00:57:03 00:57:05 Cuz what's to stop you from clicking on that multiple times.
00:57:05 00:57:07 And again, our code had to be item potent,
00:57:07 00:57:10 to make sure that if you clicked on it multiple times, you were really
00:57:10 00:57:13 only added to the account just once, not multiple times.
00:57:14 00:57:17 And an admin can also resend this email if necessary.
00:57:17 00:57:20 So, if a person says, hey, I never received the email,
00:57:20 00:57:22 then they can tell the admin person who said hey,
00:57:22 00:57:23 you told me you were gonna put me in the seat.
00:57:23 00:57:24 You never did.
00:57:24 00:57:27 Then the person can go back to our portal, click a button, and
00:57:27 00:57:31 we will resend the email to that person with the link in it, so
00:57:31 00:57:32 that they can go and be added to the system.
00:57:32 00:57:37 Right, so we had to think through all these workflow scenarios
00:57:37 00:57:39 to make sure that everything was item potent throughout the system.
00:57:41 00:57:43 That was the hard part, the actual writing of the code to make it all
00:57:43 00:57:45 work was actually pretty simple and
00:57:45 00:57:47 straight forward, but thinking through all the scenarios and
00:57:47 00:57:51 make sure you got it right was the tricky part to it.
00:57:51 00:57:52 So then, the user is gonna go and
00:57:52 00:57:55 click on that link that's inside the email.
00:57:55 00:57:57 Now, that's gonna take them to our website.
00:57:57 00:58:00 We're gonna look up the user, but they may not exist, right?
00:58:00 00:58:02 The admin could've put them in by their email address.
00:58:02 00:58:04 That's what people use to log in to the system, but
00:58:04 00:58:06 we may not have a user account for them yet.
00:58:06 00:58:09 So, if we don't have, then we take them to the registration page,
00:58:09 00:58:12 and they start to fill out the registration there.
00:58:12 00:58:15 And then now, that they're a new user,
00:58:15 00:58:17 we have to then associate them with the account.
00:58:17 00:58:20 So, we make sure that that happens next.
00:58:20 00:58:22 And if the account knows the user,
00:58:22 00:58:24 then we will add the account to the user.
00:58:24 00:58:26 So in other words, when the person registers,
00:58:26 00:58:30 we check to see if the account that they're trying to register with,
00:58:30 00:58:32 has still granted them access.
00:58:32 00:58:37 Cuz another scenario is the administrator grants access to Jeff,
00:58:37 00:58:40 and before Jeff ever clicks on the link, the administrator took Jeff
00:58:40 00:58:43 out of the seat and put Fred in the seat instead.
00:58:43 00:58:45 So now, when Jeff goes and clicks on the link,
00:58:45 00:58:48 should they be allowed to have access or shouldn't they?
00:58:48 00:58:51 And the answer is they should not because the administrator has
00:58:51 00:58:52 removed them from a seat, and
00:58:52 00:58:54 they are not allowed to watch videos anymore.
00:58:54 00:58:55 Right, so there's all these race conditions,
00:58:55 00:58:58 we had to think through the scenarios and
00:58:58 00:59:00 we had to write the code accordingly.
00:59:00 00:59:01 So, when a person signs up,
00:59:01 00:59:06 we check to make sure that the admin still is granting them the seat and
00:59:06 00:59:09 if they are, then we will add them, so they can watch videos.
00:59:09 00:59:12 If not, then we will remove them from the account and
00:59:12 00:59:13 that operation has to be item potent,
00:59:13 00:59:18 so it can be replayed back multiple times, with no ill effect.
00:59:18 00:59:23 That's the kind of thinking that I need to employ,
00:59:23 00:59:25 as I was building various parts of the service.
00:59:25 00:59:28 And we use this kind of thinking in various other locations
00:59:28 00:59:30 throughout the service as well.
00:59:30 00:59:32 Again, the actual code is not hard to write.
00:59:32 00:59:35 It's the thinking through all the scenarios and the race conditions
00:59:35 00:59:38 and the item potency if this happens multiple times.
00:59:38 00:59:41 That's the hard part about thinking all of that through
00:59:41 00:59:45 to make sure that the service is really reliable and robust.
00:59:45 00:59:48 So, by writing the code this way it solved all these problems.
00:59:48 00:59:51 The user never clicks an email for whatever reason.
00:59:51 00:59:53 Maybe they just never got the email.
00:59:53 00:59:55 Well, the administrator can go back and resend it, okay.
00:59:55 00:59:57 The user clicks the email multiple times.
00:59:57 01:00:00 We had to make sure that code was item potent, and
01:00:00 01:00:03 we had to make sure that That's the third one now.
01:00:03 01:00:06 The user clicks the email after another administrator removes
01:00:06 01:00:07 permission.
01:00:07 01:00:09 Then we can't just give the person permission,
01:00:09 01:00:11 we have to make sure they don't get it cuz the administrator's not
01:00:11 01:00:13 allowing it anymore.
01:00:13 01:00:18 And a malicious user changes the URL to an account not granting access
01:00:18 01:00:20 that ends up being treated as a buff.
01:00:21 01:00:24 All right there’s nothing stopping somebody from creating their
01:00:24 01:00:25 own URL and clicking on it.
01:00:25 01:00:29 All right so we had to make sure the URL itself was secure and
01:00:29 01:00:30 that it made sense.
01:00:34 01:00:36 And we solved all those pretty easily.
01:00:36 01:00:40 All right so now, you're a user, you're logged into the system and
01:00:40 01:00:41 your using the system.
01:00:41 01:00:44 So there’s some information that we store on your behalf.
01:00:44 01:00:48 We have this Watched Video Table, this is a list of all the videos you
01:00:48 01:00:51 have watched, it's really for your video history.
01:00:51 01:00:54 So the table looks like this in Azure, we have the partition key,
01:00:54 01:00:57 which is the user ID, so these are associated with you.
01:00:57 01:01:01 The row key is the video ID, so we know for
01:01:01 01:01:05 which video you have, which means there's only one row in the table,
01:01:05 01:01:07 for any given user, for any single video.
01:01:09 01:01:11 There's the timestamp, the date time,
01:01:11 01:01:14 which was put in there by Azure, so that's out of our control, but
01:01:14 01:01:18 really it indicates the last time in this row when the table was updated.
01:01:18 01:01:21 Then we have the offset within the video where you were,
01:01:21 01:01:23 this allows you to click on this history.
01:01:23 01:01:24 It will go right to that video, and
01:01:24 01:01:27 pick up right from where it left off, the offset in the video and
01:01:27 01:01:29 then you can continue watching from that point.
01:01:29 01:01:32 And then we also store in here how many minutes of the video
01:01:32 01:01:33 you have watched.
01:01:33 01:01:36 And of course, you can watch a video five times over,
01:01:36 01:01:39 from beginning to end and then we will have the total
01:01:39 01:01:42 number of minutes in here that you have watched of the video.
01:01:42 01:01:46 These are grouped by user, within the table.
01:01:46 01:01:51 So we need to go and when you log in, and say, okay,
01:01:51 01:01:53 Jeff has just logged in, what's his history.
01:01:53 01:01:55 We can go right to the table and
01:01:55 01:01:58 say, give me all the rows with a partition key for Jeff.
01:01:58 01:02:00 And then it will go right to that location in the table and
01:02:00 01:02:02 give me the history for Jeff.
01:02:02 01:02:06 So it's very fast to be able to grab that history for a particular user.
01:02:06 01:02:10 It's easy to update a specific video, so as you're watching a video
01:02:10 01:02:13 and we need to update it, we know who you are, the partition key, and
01:02:13 01:02:16 we know which video you're watching, the row key.
01:02:16 01:02:18 So we know exactly what row in the table to go to to do the update.
01:02:18 01:02:19 So that's blazingly fast.
01:02:21 01:02:25 But with the timestamp, we could say, so how, what people.
01:02:25 01:02:26 Let's do a scan over the table.
01:02:26 01:02:29 Which videos have been watched in the last day?
01:02:29 01:02:33 And then we can use a timestamp column to scan the entire table.
01:02:33 01:02:35 And we can see what videos have been watched most
01:02:35 01:02:36 recently in the last day.
01:02:37 01:02:40 And then the client side application (meaning the website),
01:02:40 01:02:43 where if you're using our mobile app, if you would like to store
01:02:43 01:02:48 your history in terms of most recently watched as opposed to.
01:02:48 01:02:49 By default this will be alphabetical,
01:02:49 01:02:51 by the name of the video.
01:02:51 01:02:53 But if you would like to watch it in time-stamp,
01:02:53 01:02:55 then you can easily store by time.
01:02:55 01:02:59 Instead, and usually you're history is a couple hundred videos at most.
01:02:59 01:03:00 Maybe ten videos,
01:03:00 01:03:03 100 videos, whatever you've watched on the system.
01:03:03 01:03:07 So having the clients do actual other sorting for the end users
01:03:07 01:03:11 benefit is a very cheap thing to do because we return a severe subset of
01:03:11 01:03:13 the data back to you, and there's not that much of it.
01:03:15 01:03:16 Now, we have a Bookmark Table.
01:03:16 01:03:20 As I mentioned, as you're watching a video, you can click on it and
01:03:20 01:03:22 say, I would like to bookmark this location within the video and
01:03:22 01:03:25 then you could then add some text that's meaningful for you.
01:03:25 01:03:28 So, we have a Bookmark Table, again, the PartitionKey is the user,
01:03:28 01:03:31 the RowKey is the video.
01:03:31 01:03:34 Then we have the timestamp put in, put there by Azure.
01:03:34 01:03:36 And then for the bookmarks, it's just one column.
01:03:36 01:03:38 And we just make it an array, a byte array.
01:03:38 01:03:42 And inside that byte array, we store the offset of the videos.
01:03:42 01:03:45 Within that particular video and some note, right,
01:03:45 01:03:48 which is your textual note that you'll have there.
01:03:48 01:03:52 So the benefits of this is, again, they're grouped by user and video.
01:03:52 01:03:54 Just like they are for the watched video table.
01:03:54 01:03:58 So it's easy when you log in and you go to your profile page, we can just
01:03:58 01:04:02 quickly go to the table and say get me all the bookmarks for Jeff.
01:04:02 01:04:03 And if you go to a particular video,
01:04:03 01:04:07 we can say give me all the bookmarks for Jeff on this particular video,
01:04:07 01:04:10 that would just be one row on the table.
01:04:10 01:04:12 So, blazingly fast to look this stuff up.
01:04:12 01:04:14 And to go and add new entries to it.
01:04:14 01:04:16 So it's easy lookup or fast lookup.
01:04:16 01:04:19 And again, we can do filtering to the last number of days.
01:04:19 01:04:20 I added a bookmark in the last week.
01:04:20 01:04:23 Can you just show me the bookmarks I added in the last week.
01:04:23 01:04:25 It's trivially simple for
01:04:25 01:04:27 us to have the client side apps do that kind of filtering.
01:04:29 01:04:32 And then the last thing we have for a user is the video queue.
01:04:32 01:04:34 And that's a blob container.
01:04:34 01:04:35 We don't use tables for this.
01:04:35 01:04:37 We use a blob container for this.
01:04:37 01:04:41 And within the container, the blob name is the user ID.
01:04:41 01:04:45 So each user on our system has a separate blob that has binary data.
01:04:45 01:04:47 The reason again why we don't use a table for
01:04:47 01:04:50 this is because tables always sort their information.
01:04:50 01:04:54 But we wanna let the user have control over the order of
01:04:54 01:04:55 the elements that are in their cue.
01:04:55 01:04:57 Which video do you wanna watch next,
01:04:57 01:04:59 which video do you wanna watch last, and so on.
01:04:59 01:05:02 So, we decided a table's not good for that.
01:05:02 01:05:04 We're gonna use a Blob for that instead.
01:05:04 01:05:07 And then we in our code, we write the code,
01:05:07 01:05:08 where the user does the ordering, and
01:05:08 01:05:11 then we will write the information as the Blob in the desired order.
01:05:11 01:05:12 Of the user.
01:05:12 01:05:15 So the queue items are manipulated all together.
01:05:15 01:05:17 Since that all of the queued items are in a single blob.
01:05:17 01:05:19 They are always manipulated all together as a single unit,
01:05:19 01:05:21 which is good.
01:05:21 01:05:23 When you add a new item to the queue we can just append to the end of
01:05:23 01:05:25 the blob, so that's really simple.
01:05:25 01:05:28 We have to read the blob, append to the end, write it back out.
01:05:28 01:05:28 That's easy.
01:05:30 01:05:33 We can easily remove something from your queue that's in the middle.
01:05:33 01:05:34 Of it, we just read the blob,
01:05:34 01:05:38 pull it out, write the whole blob down and then, we can do,
01:05:38 01:05:41 showing whatever we want to show or order whatever we want to show, but
01:05:41 01:05:43 we'll have the client applications, the websites or
01:05:43 01:05:47 the mobile apps, go and do that for you just by download.
01:05:47 01:05:50 They can download your queue from blob storage and
01:05:50 01:05:52 the client apps can do whatever they want to with it.
01:05:52 01:05:55 So again, I think a really good architecture here.
01:05:55 01:06:00 Very cheap, scales really well for us.
01:06:00 01:06:02 I mentioned earlier about the logging infrastructure.
01:06:03 01:06:05 Wow, I'm really using a lot of time, Todd.
01:06:05 01:06:07 How about that?
01:06:07 01:06:10 And that's really, when you're building anything for
01:06:10 01:06:12 the cloud, it's really worth spending a phenomenal amount of time
01:06:12 01:06:14 on the logging infrastructure.
01:06:14 01:06:19 It can help you in so many different ways, if you do it right.
01:06:19 01:06:21 So, we use blobs not tables.
01:06:21 01:06:24 Again, blobs are faster, blobs are cheaper.
01:06:26 01:06:29 You can't do filters, but you can download the blob,
01:06:29 01:06:30 which can be a CSV file.
01:06:30 01:06:33 And then you can load it into other things like Excel,
01:06:33 01:06:35 and then you can do filters.
01:06:35 01:06:38 So that's typically why we did it.
01:06:38 01:06:41 The logging infrastructure is best effort, as I said.
01:06:41 01:06:43 So there's a chance that if a VM goes down,
01:06:43 01:06:44 we're gonna lose some data.
01:06:44 01:06:46 But that's really always true.
01:06:46 01:06:49 So as long as the VMs are up and running, we're really good at
01:06:49 01:06:52 having all the log information and having none of it get lost.
01:06:52 01:06:54 The logging is done asynchronously.
01:06:54 01:06:56 That is, whenever we want to log something,
01:06:56 01:06:59 we call a piece of code that says, this needs to be logged.
01:06:59 01:07:02 But we don't put that code in the user's path.
01:07:02 01:07:04 Because we don't want something wrong with the logging
01:07:04 01:07:07 infrastructure or latency to affect the end user experience.
01:07:07 01:07:10 So if an administrator adds a person,
01:07:10 01:07:12 grants them access to the system,.
01:07:12 01:07:14 we write that to a log file.
01:07:14 01:07:16 But we do that asynchronously.
01:07:16 01:07:20 And we let the result come back right away instead.
01:07:20 01:07:23 So that they're not waiting for the logging to happen.
01:07:23 01:07:25 That gives a quicker response time in the application.
01:07:25 01:07:27 We have two different kinds of logs.
01:07:27 01:07:29 The first is an ongoing log.
01:07:29 01:07:31 This keeps the last 55,000 entries.
01:07:31 01:07:33 The reason, or 50,000 entries.
01:07:33 01:07:36 The reason why is we use block blobs in Azure.
01:07:36 01:07:41 And block blobs and Azure support up to 50,000 blocks in a single blob.
01:07:41 01:07:45 So we just keep upending new blocks to the end of the blob, and once
01:07:45 01:07:49 it hits 50,000, we delete from the beginning and we upend to the end.
01:07:49 01:07:53 So we always store the last 50,000 entries in here.
01:07:53 01:07:56 That means we'll lose earlier stuff, but if we
01:07:56 01:07:59 lose it we probably don't care about it cuz it was 50,000 entries ago.
01:07:59 01:08:00 When we lost it.
01:08:00 01:08:03 So we use this for our promo code usage, and for
01:08:03 01:08:04 people who are registering on the system.
01:08:04 01:08:07 We have a record of the last 50,000 people who have registered on
01:08:07 01:08:10 the system, you know, very easily.
01:08:10 01:08:13 And then beyond that, we can throw some of that data away,
01:08:13 01:08:14 so we're not paying for it all.
01:08:14 01:08:16 But we'll of course have their user accounts and
01:08:16 01:08:19 everything else in the system, that's permanent.
01:08:19 01:08:22 And account management operations, when users get grant access or
01:08:22 01:08:24 revoked access, those are being written out too,
01:08:24 01:08:29 and again we saw 50,000 of them, that 50,000 per company, by the way,
01:08:29 01:08:30 for that particular thing.
01:08:30 01:08:32 So if a company has a problem,
01:08:32 01:08:35 we can look at the last 50,000 admin operations that have occurred But
01:08:35 01:08:38 we just don't let it grow without bounds, right.
01:08:38 01:08:42 We don't want things to grow without bounds because it's very expensive
01:08:42 01:08:45 to do that, and then most of that data just sits there and
01:08:45 01:08:49 you never access it, but you end up paying for it on a monthly basis.
01:08:49 01:08:52 And then the other kind of log we have is a periodic log, and
01:08:52 01:08:55 I wrote a bunch of code to make these things really easy to write
01:08:55 01:08:59 inside of our code base You can have like a daily periodic log, or
01:08:59 01:09:01 a weekly, or a monthly, or a quarterly.
01:09:01 01:09:05 And then you can tell it how many of these, like for daily,
01:09:05 01:09:08 you can say this is the log over the last seven days.
01:09:08 01:09:12 And then it will purge old ones and only keep the last seven days worth,
01:09:12 01:09:14 again, it's a cost cutting measure to do that.
01:09:15 01:09:18 And also it kind of reduces the amount of data that you have to look
01:09:18 01:09:20 over if you need to look over some data.
01:09:20 01:09:23 So you're not spending a lot of time perusing data that's,
01:09:23 01:09:26 you know, two years old in the system.
01:09:26 01:09:29 So we use this for errors handling, unhandled exceptions and
01:09:29 01:09:31 that's in a daily log.
01:09:31 01:09:34 And we keep the last months worth, thirty days worth,
01:09:34 01:09:35 of the daily logs of exceptions.
01:09:35 01:09:39 An we have a count user transcript like the user watching history and
01:09:39 01:09:43 this is done on the monthly basis for the billing cycle.
01:09:43 01:09:47 Then an admin can go and say over the last month I wa billed for this.
01:09:47 01:09:49 What videos were watched over the last month?
01:09:49 01:09:50 Who in my company watched them?
01:09:50 01:09:52 How many minutes of each one did they watch?
01:09:52 01:09:56 All that information's available for them to download.
01:09:56 01:10:00 As I was writing all of this stuff, I tried to write it in a,
01:10:00 01:10:02 I worked on class libraries a lot in my life,
01:10:02 01:10:06 and I was on the .NET Framework team at Microsoft for eight years.
01:10:06 01:10:08 As a consultant there.
01:10:08 01:10:12 And during that time, I worked on a lot of the class libraries that your
01:10:12 01:10:14 .NET programmer use today.
01:10:14 01:10:18 So I got in a habit of trying to, when I write code,
01:10:18 01:10:21 I try to write it in a reusable fashion, let me built a class
01:10:21 01:10:24 library that can be reused in lots of different scenarios.
01:10:24 01:10:28 So as I was working on this, I ended up building a document
01:10:28 01:10:31 called Jeffrey Richter's Guide to Working with Azure Storage Tables,
01:10:31 01:10:33 cuz I learned a lot about Azure storage tables.
01:10:33 01:10:36 Good ways to use them, bad ways to use them.
01:10:36 01:10:39 Ways that'll hurt you if you try to version your data in the future.
01:10:39 01:10:42 Ways that'll make it really easy for
01:10:42 01:10:45 you to version your data in the future.
01:10:45 01:10:48 Cost cutting measures, performance measures and so on.
01:10:48 01:10:51 So, I wrote this really big document about that, and
01:10:51 01:10:54 that document has been endorsed by Microsoft's Azure Storage Team and
01:10:54 01:10:55 it's available for free.
01:10:55 01:10:58 You can download it from this link over here.
01:10:58 01:11:03 And then, I go into what the guide's purposes are down below.
01:11:03 01:11:06 Some of Microsoft's recommendations are actually not good
01:11:06 01:11:08 recommendations at all, that will really get you into trouble.
01:11:08 01:11:11 I followed them, myself, and I got burned by them.
01:11:11 01:11:14 So I explain what recommendations are bad and why, and
01:11:14 01:11:16 what to use instead in this document.
01:11:16 01:11:17 I show good patterns and
01:11:17 01:11:20 practices related to working with Azure tables, and
01:11:20 01:11:26 then I created a class library with a bunch of utility code which is
01:11:26 01:11:31 all open source, and available on, you can download the DLL on NuGet.
01:11:31 01:11:33 And then, the source code is available as well and
01:11:33 01:11:35 it has a lot of helper classes in it.
01:11:35 01:11:39 So for blob features, I have logging, the on going logs and
01:11:39 01:11:41 the periodic logs.
01:11:41 01:11:44 We can just say, I need to have a daily log and keep the last seven
01:11:44 01:11:47 of them and then has APIs where you can call the write to it.
01:11:47 01:11:48 And then it will just do it synchronously and
01:11:48 01:11:49 update the logs for you and
01:11:49 01:11:52 it will do the garbage collection to get rid of the old logs.
01:11:52 01:11:55 So that's all code you wouldn't have to write if you use this class
01:11:55 01:11:57 library that I have.
01:11:57 01:12:00 The periodic elector, sometimes, you need to pick one machine,
01:12:00 01:12:03 elect one machine to do like run a report.
01:12:03 01:12:07 All right, so we have 10, let's say, machines running in Azure and
01:12:07 01:12:10 on Saturday night at midnight, we wanna produce a report.
01:12:10 01:12:13 So we want to elect one of the 10 machines to be the one to produce
01:12:13 01:12:14 the report.
01:12:14 01:12:18 So I have code in this library that will form an election
01:12:18 01:12:19 across those ten machines.
01:12:19 01:12:22 So one of them we chose as the elected winner and
01:12:22 01:12:23 that one will generate the report.
01:12:23 01:12:26 And if that machine goes down while it's producing the report,
01:12:26 01:12:30 then,one of the other nine machines will become the elected winner, and
01:12:30 01:12:32 it will go and produce the report.
01:12:32 01:12:35 So, that kind of resiliency which is really hard to do,
01:12:35 01:12:39 we needed that and so I wrote it in this reusable class library.
01:12:39 01:12:42 For a table, I have a bunch of code in there that will do backup and
01:12:42 01:12:44 restore of the table.
01:12:44 01:12:47 So we do backup our tables periodically, we do this for
01:12:47 01:12:50 disaster recovery, we also do it in case there's a bug in our code or
01:12:50 01:12:52 in case we get hacked.
01:12:52 01:12:56 We can go back to a previous version of the table and we can go and
01:12:56 01:12:56 restore it.
01:12:56 01:12:58 So that was also important for us.
01:12:58 01:13:02 We have encoding here to help with the optimistic concurrency and
01:13:02 01:13:05 could make it easier to create a filter for doing a query against
01:13:05 01:13:08 the table or make it easier for right code to go through the various
01:13:08 01:13:11 segments of enumerating information that's in the table.
01:13:11 01:13:15 Sometimes, we wanted to replace the names of properties in a table,
01:13:15 01:13:18 or delete a property to every entry in a table,
01:13:18 01:13:20 or add a property to every entry of the table, or
01:13:20 01:13:23 have some helper code in there that does all of that.
01:13:23 01:13:27 We have a pattern for extensible entity schemas, this gives you that
01:13:27 01:13:30 versioning, easy way to version things in the future and
01:13:30 01:13:33 a way to take a collection of items, serialize them into a byte array so
01:13:33 01:13:36 you can put them into a single property of a table.
01:13:36 01:13:38 We use that for our bookmarks.
01:13:38 01:13:40 We use that for our, well, a similar thing we use for
01:13:40 01:13:43 our queue, the user's queue of what is they wanna watch.
01:13:43 01:13:46 And all this is downloadable from that link down there at the bottom.
01:13:46 01:13:48 All, right, I think we're done.
01:13:48 01:13:49 >> Is that it? >> I think so.
01:13:49 01:13:50 >> Well, let me make
01:13:50 01:13:53 two quick points then cuz we're [INAUDIBLE] on time.
01:13:53 01:13:57 One thing that's fascinating when you do a real product, when I say
01:13:57 01:14:01 real product, something that's out there in commercial production, and
01:14:01 01:14:03 you look at this system.
01:14:03 01:14:05 I look at it everyday.
01:14:05 01:14:08 And it looks like, hey, you sign in, there's a user and
01:14:08 01:14:09 there's videos and there's a library.
01:14:09 01:14:10 It looks like nothing.
01:14:10 01:14:14 But every single little bit of user accounts, or payments,
01:14:14 01:14:16 we didn't talk about promotions.
01:14:16 01:14:18 But what if somebody cancels a payment here?
01:14:18 01:14:21 The payment options we tried, we had a lot of lessons learned.
01:14:21 01:14:24 We tried PayPal, we tried different things, we found all kinds of-
01:14:24 01:14:25 >> Like people in Germany won't
01:14:25 01:14:26 use PayPal.
01:14:26 01:14:29 >> Well, there's customer issues like that, but there's also
01:14:29 01:14:32 programmatic stuff, and how you access and how you report on it.
01:14:32 01:14:35 Every single little thing, I remember you, and Joel, and
01:14:35 01:14:38 me on a white board once just talking about users and accounts.
01:14:38 01:14:39 And it was spaghetti,
01:14:39 01:14:42 I mean, it was just everywhere because of what we wanted to do.
01:14:42 01:14:45 We wanted to have accounts that were just a single user, simple.
01:14:45 01:14:47 But what if you're a business account, and
01:14:47 01:14:48 you can assign multiple users,
01:14:48 01:14:51 then we had all kinds of different things like accounts or something?
01:14:51 01:14:52 >> I can't even remember.
01:14:52 01:14:53 >> Yeah, that's right.
01:14:53 01:14:55 >> Where people wanted to buy blocks of, I don't wanna get into it.
01:14:55 01:14:56 >> [LAUGH] >> But
01:14:56 01:15:00 we had seven billing types of accounts at one point.
01:15:00 01:15:03 It's just the challenges are just fascinating to deal with.
01:15:03 01:15:04 The last point I wanted to make,
01:15:04 01:15:07 because it was the whole point of the architecture optimization,
01:15:07 01:15:10 is what is this thing cost, right?
01:15:10 01:15:14 When we first started, and we've learned from version 1 to version 2
01:15:14 01:15:17 with accounts and payments and PayPal and Stripe and all kinds of
01:15:17 01:15:21 things, it cost us roughly 1500 to 3000 a month, I think.
01:15:21 01:15:22 Something along those lines.
01:15:22 01:15:24 We've gotten that down to almost less.
01:15:24 01:15:28 It's just a few hundred to several hundred a month at this point.
01:15:28 01:15:30 And that was through choosing different CDNs and
01:15:31 01:15:35 server companies that stream this in different ways.
01:15:35 01:15:36 And we got one,
01:15:36 01:15:39 we had to make some trade offs in the video players and stuff.
01:15:39 01:15:43 But it's basically costing us a few hundred a month to service thousands
01:15:43 01:15:44 and thousands and thousands of users.
01:15:44 01:15:48 And that's all in, excepting investments in videos,
01:15:48 01:15:50 meaning hiring people to do the videos.
01:15:50 01:15:53 You take that out and this is a few to several hundred dollars a month,
01:15:53 01:15:54 all on Azure.
01:15:54 01:15:58 Paying for the Azure storage, the hosting, the VM, everything.
01:15:58 01:16:00 It's just several hundred dollars a month and
01:16:00 01:16:04 I can guarantee that none of our competitors are within that order or
01:16:04 01:16:06 two orders of magnitude of that number.
01:16:06 01:16:08 So, that's servers architecture and
01:16:08 01:16:11 that's some of the good things that came out of all this.
01:16:11 01:16:11 So we're done.
01:16:12 01:16:13 Any questions you have,
01:16:13 01:16:15 I can't believe we- >> I know.
01:16:15 01:16:17 >> You and I were like, what are we gonna do?
01:16:17 01:16:18 We're gonna disappoint the audience.
01:16:18 01:16:20 We'll have a 30 minute conversation.
01:16:20 01:16:22 >> Apparently, I can ramble on a lot more than I thought.
01:16:22 01:16:24 >> Yeah, so any questions?
01:16:24 01:16:25 >> Anything you'd like to talk about? Yeah?
01:16:25 01:16:27 >> [INAUDIBLE]
01:16:27 01:16:29 >> We do not use Azure
01:16:29 01:16:30 queues anywhere in the model.
01:16:30 01:16:34 That is correct, but, the, what's that?
01:16:34 01:16:35 >> Cost?
01:16:35 01:16:36 >> No, it had nothing to do with cost.
01:16:36 01:16:38 In fact, I think the costs are pretty good.
01:16:38 01:16:40 I mean there's a little bit higher latency on that but
01:16:40 01:16:42 there was just never a need for
01:16:42 01:16:44 it in anything that we're doing to do any kind of killing.
01:16:44 01:16:47 >> Don't you have like a press [INAUDIBLE] user to
01:16:47 01:16:49 click on the link to activate services?
01:16:49 01:16:56 >> We send that to SendGrid, it's the service that we use for e-mail.
01:16:56 01:16:58 And then they have to deal with emailing it.
01:17:00 01:17:01 Yes and that's it.
01:17:01 01:17:02 And so, if they never get the email,
01:17:02 01:17:04 the administrator has to log back into our site.
01:17:04 01:17:07 And they click on us that says re-send it and then we will re-send
01:17:07 01:17:10 it to SendGrid and then they will go and send the email again.
01:17:10 01:17:14 Maybe they had the email incorrect or for whatever reason it got lost.
01:17:15 01:17:16 But yeah, we don't use Azure queues,
01:17:16 01:17:18 just not because there's anything wrong with them.
01:17:18 01:17:21 I've used them in other things that I've built, but
01:17:21 01:17:23 in here, there was just no need for them.
01:17:24 01:17:25 You have a question?
01:17:25 01:17:26 >> Yeah.
01:17:26 01:17:33 >> [INAUDIBLE] >> Yeah.
01:17:33 01:17:43 >> [INAUDIBLE] >> Sort of.
01:17:43 01:17:47 So the question is do we have a way of knowing distinct minutes of
01:17:47 01:17:49 a video that a user is watching.
01:17:49 01:17:51 Yeah, we do have a way to do that.
01:17:51 01:17:53 And actually with the way we solve that, I think,
01:17:53 01:17:55 is actually kind of clever.
01:17:55 01:17:59 So we take a video, let's say it's an hour long, and we divide it down
01:17:59 01:18:02 into, I can't remember the exact number [CROSSTALK]
01:18:02 01:18:02 >> Three minutes.
01:18:02 01:18:05 >> Three minute chunks, maybe a three minute chunks.
01:18:05 01:18:10 And if you're watching anywhere in the first three minutes of it,
01:18:10 01:18:12 we set bit zero of an integer to a one.
01:18:12 01:18:15 And if you watch it over and over and over again,
01:18:15 01:18:17 we just keep setting that bit to a one.
01:18:17 01:18:19 And if you seek somewhere in a video and you're in some other
01:18:19 01:18:23 three minute segment, so we do it with a three minute resolution.
01:18:23 01:18:26 You're in that, then we will go and update this integer and we'll set
01:18:26 01:18:28 that bit for that three minute section of the video to a one.
01:18:29 01:18:33 Then, we can look at all the videos that people have watched.
01:18:33 01:18:36 We grab these 64 bit integers or maybe there're, I can't remember,
01:18:36 01:18:38 if it's 32 bit or 64 bit integer.
01:18:38 01:18:39 But then, we can look at all the bits that are set.
01:18:39 01:18:42 And we would multiply each bit set by three minutes.
01:18:42 01:18:44 And we know how much you've watched to sync,
01:18:46 01:18:47 given the three minute granularity.
01:18:48 01:18:49 That's how we solve that.
01:18:50 01:18:51 Although bookmarks are exact.
01:18:51 01:18:57 >> [CROSSTALK] part of my company and I love these features.
01:18:57 01:18:57 >> Okay.
01:18:57 01:19:00 >> [INAUDIBLE] I did follow the same procedures.
01:19:00 01:19:01 >> You did the same thing?
01:19:01 01:19:02 >> The same thing.
01:19:02 01:19:03 >> Okay, great, so great minds think alike.
01:19:03 01:19:04 >> [LAUGH]
01:19:04 01:19:06 >> [INAUDIBLE] >> What's that?
01:19:06 01:19:10 >> Have you also done [INAUDIBLE] >> Okay, I don't know, possibly.
01:19:12 01:19:14 But we arrived on our own, we didn't look up.
01:19:14 01:19:18 But I would imagine, it is a common thing for people to do when.
01:19:18 01:19:21 And that's what we ended up doing as well.
01:19:21 01:19:25 So again, it's very cheap but it always stored in integer 32 bit or
01:19:25 01:19:27 64 bit for you to video per user.
01:19:27 01:19:31 And then we just get it, flip a bit, and
01:19:31 01:19:33 if it hasn't changed, we don't even have to write it back, right?
01:19:33 01:19:35 If the bit was a one, and we're making it a one,
01:19:35 01:19:36 we don't even have to write it back.
01:19:36 01:19:38 But if it was a zero, and we make it a one,
01:19:38 01:19:39 then we can write it back out.
01:19:39 01:19:44 >> [INAUDIBLE] >> Play the video from email?
01:19:44 01:19:47 >> [INAUDIBLE] >> From Vimeo.
01:19:47 01:19:57 >> [INAUDIBLE]
01:20:00 01:20:01 >> Vimeo is what you saw.
01:20:01 01:20:03 Is that what you're asking, what the video player is?
01:20:03 01:20:03 >> Yeah.
01:20:03 01:20:05 >> So we've used a few.
01:20:05 01:20:08 We've tried three or four over the years.
01:20:08 01:20:10 >> That's been our biggest pain point I would say.
01:20:10 01:20:12 >> It has been one of our biggest pain points.
01:20:13 01:20:16 Cuz we wanted an all HTML5-type editor.
01:20:16 01:20:20 So I won't mention any of the older ones, but the last two that we used,
01:20:20 01:20:23 and this is one of those challenges and tradeoffs where Viddler for
01:20:23 01:20:25 awhile, if you've heard of Viddler, and Vimeo.
01:20:26 01:20:29 And Viddler started to get more and more and
01:20:29 01:20:31 more expensive as we scaled.
01:20:31 01:20:35 And they wanted to charge more in a premium model of service.
01:20:35 01:20:39 And despite the premiums we were paying, in some parts of
01:20:39 01:20:44 the country like areas of India or some area of Australia, we'd
01:20:44 01:20:50 just get this flood of complaints that we could not figure out.
01:20:50 01:20:52 And none of it was us, it was the CDNs we were using.
01:20:52 01:20:55 So we tried video player after video player.
01:20:55 01:20:57 And the reason it's a bunch of a tradeoff
01:20:57 01:20:58 is each video player's different.
01:20:58 01:21:00 Some people like one, some like the other, some have back and
01:21:00 01:21:05 forth and video playback speeds and some dont.
01:21:05 01:21:07 The controls look different.
01:21:07 01:21:11 We have actually Viddler and Vimeo in place right now, but
01:21:11 01:21:14 we're moving at the moment almost all to Vimeo,
01:21:14 01:21:19 because their performance has been better all-around globally.
01:21:19 01:21:22 We have almost no complaints of any kind anymore,
01:21:22 01:21:23 no matter what location.
01:21:23 01:21:27 And they're by far the best value and price.
01:21:27 01:21:30 So that few hundred to several hundred dollars a month that I told
01:21:30 01:21:32 you, that's including all the bandwidth requirements, so
01:21:32 01:21:34 I wasn't excluding that.
01:21:34 01:21:36 That's literally what it costs us to host this thing.
01:21:36 01:21:39 And we have competitors that are paying seven figures a year for
01:21:39 01:21:41 their hosting services.
01:21:41 01:21:45 So basically I'm saying, you accomplished all your goals.
01:21:45 01:21:46 >> [LAUGH] >> [LAUGH]
01:21:46 01:21:47 >> I'm very proud of it.
01:21:47 01:21:48 I worked on it for a long time.
01:21:48 01:21:50 [LAUGH] >> Did that answer your question?
01:21:50 01:21:51 >> And it was fun, it was a fun project.
01:21:51 01:21:53 >> We are using Vimeo and Viddler.
01:21:53 01:22:00 >> [INAUDIBLE]
01:22:00 01:22:03 >> He's saying, how can you jump to
01:22:03 01:22:06 a location within Vimeo, into a video using Vimeo?
01:22:06 01:22:10 >> [INAUDIBLE] >> Yeah.
01:22:10 01:22:10 >> Don't we get- >> So
01:22:10 01:22:12 I'm not involved in this part of it- >> [LAUGH]
01:22:12 01:22:13 >> Cuz this is in the user
01:22:13 01:22:15 interface, it's not in the backend.
01:22:15 01:22:17 But I know what we did with Viddler is with the player,
01:22:17 01:22:20 we would get notifications where you are in the video.
01:22:20 01:22:22 >> Yeah, there's an API.
01:22:22 01:22:23 >> So our JavaScript would get that.
01:22:23 01:22:25 When it's running in the browser,
01:22:25 01:22:27 our JavaScript would receive notifications from the player,
01:22:27 01:22:29 you're here, you're here, you're here, you're here.
01:22:29 01:22:33 That JavaScript code would call into my backend code to say, this user,
01:22:33 01:22:35 this video, is at this location.
01:22:35 01:22:38 I go to the table, grab their integers, flip the bit.
01:22:38 01:22:39 If the bit is changed, I write it back.
01:22:39 01:22:42 If it didn't change, I throw it away.
01:22:42 01:22:43 And that's all I had to do.
01:22:43 01:22:46 >> Yeah, all the players have an API to interact with them that way.
01:22:46 01:22:49 >> I'm assuming the Vimeo player does the same thing.
01:22:49 01:22:50 Again, that's on the JavaScript part.
01:22:50 01:22:52 So if they're still calling me in the back-
01:22:52 01:22:54 >> [LAUGH]
01:22:54 01:22:55 >> I don't know what notification
01:22:55 01:22:59 they've hooked into for the Vimeo player to be able to call me.
01:22:59 01:23:00 >> If I remember quickly, it's very similar.
01:23:00 01:23:01 There's always some,
01:23:01 01:23:05 what's your location in the video thing going on, that you can access.
01:23:05 01:23:06 >> And when the user hits pause or
01:23:06 01:23:10 stop, the browser also sends me a notification saying for
01:23:10 01:23:13 this user, at this video, they're at this location.
01:23:13 01:23:14 So when they go to the history,
01:23:14 01:23:17 we can continue from where they left off.
01:23:17 01:23:19 We also update as they're watching it, too.
01:23:19 01:23:23 But it's better if the person hits stop or pause on the video,
01:23:23 01:23:25 because then we know exactly where they are.
01:23:25 01:23:28 Other than that, we don't send constant stream of notifications
01:23:28 01:23:31 from the browser to the back as they're watching the video.
01:23:31 01:23:35 It's too much network traffic, and we pay for all those updates.
01:23:35 01:23:39 So it's on a, I don't know, once a minute or something.
01:23:39 01:23:42 So if they go back, if they just leave the webpage and
01:23:42 01:23:45 come back, we'll get them close, within a minute or so
01:23:45 01:23:47 of where they were, but it won't be exact.
01:23:47 01:23:47 If you hit pause or
01:23:47 01:23:50 stop, then we'll know exactly where you are on the video.
01:23:52 01:23:53 Okay, I think we're- >> Anything else?
Embed
Copy and paste the embed code above
Share
Copy and paste the embed code above