325. Chris Lapp Of Cisco On “Anything & Everything” With AV Networking

In Episode 325, guest co-host Aram Piligian joins Andy to sit down with Chris Lapp, developer of the new Open Fabric Studio AV network management software and senior specialist solutions engineer at Cisco and ask all about anything and everything AV networking! This episode is sponsored by Allen & Heath and RCF.

This is the first of a two-part conversation where everything from switch configurations, the differences between Dante, AES67, and AVB/Milan, managed versus unmanaged switches, whether or not to use DHCP, and more was on the table.

Chris Lapp leads the design and delivery of mission-critical media and data center infrastructure at the intersection of broadcast, IP networking, and AI. With a background that spans live production, large-scale data center architectures, and modern software-defined systems, he helps organizations build platforms that are performant, resilient, and ready for what’s next. Chris’s work focuses on translating complex technical requirements into practical, scalable approaches that support real-time media workflows and high-stakes operations. In addition to his work with Cisco and developing Open Fabric Studio, Chris also serves as the vice president of membership for the Society of Motion Picture and Television Engineers (SMPTE).

Episode Links:
About Chris Lapp
Open Fabric Studio
jetfx (Personal Audio DSP + Guitar NAM)
Episode 325 Transcript

NOTE: Mike Green, the artist who performs “Break Free” that opens every episode, has released a new album — Hang The Moon: Part One — available on all streaming platforms as well as DSPs that support spatial audio. Mikegreenmusic.com will direct folks to the vinyl release or allow them to purchase digitally. And, Mike is hitting the road with Whitney Tai for “The Record Store Tour” starting May 23 in New Orleans. Find out more here.

Connect with the community on the Signal To Noise Facebook Group and Discord Server. Both are spaces for listeners to create to generate conversations around the people and topics covered in the podcast — we want your questions and comments!

Also please check out and support The Roadie Clinic, Their mission is simple. “We exist to empower & heal roadies and their families by providing resources & services tailored to the struggles of the touring lifestyle.”

The Signal To Noise Podcast on ProSoundWeb is co-hosted by pro audio veterans Andy Leviss and Sean Walker.

Want to be a part of the show? If you have a quick tip to share, or a question for the hosts, past or future guests, or listeners at home, we’d love to include it in a future episode. You can send it to us one of two ways:

1) If you want to send it in as text and have us read it, or record your own short audio file, send it to signal2noise@prosoundweb.com with the subject “Tips” or “Questions”

2) If you want a quick easy way to do a short (90s or less) audio recording, go to https://www.speakpipe.com/S2N and leave us a voicemail there.

Transcript

Episode 325 - Chris Lapp, Part 1

Note: This is an automatically generated transcript, so there might be mistakes--if you have any notes or feedback on it, please send them to us at signal2noise@prosoundweb.com so we can improve the transcripts for those who use them!

Voiceover: You’re listening to Signal to Noise, part of the ProSoundWeb podcast network, proudly brought to you this week by the following sponsors:

Allen & Heath, whose new dLive RackUltra FX upgrade levels up your console with 8 next-generation FX racks – putting powerful tools like vocal tuning, harmonizing, and amp simulation right at your fingertips. Learn more at allen-heath.com

RCF and TT+ AUDIO.... Delivering premium audio solutions designed for tour sound and music professionals for over 75 years.  Visit RCF at RCF-USA.com for the latest news and product information.

Music: “Break Free” by Mike Green

[00:00:58] Andy Leviss: Hey, welcome to another episode of Signal to Noise. I'm your host, Andy Leviss. And with me this week, I do not have my usual better half Sean Walker 'cause he is, uh, up to his neck in, uh, some trade show stuff this week. So, uh, subbing in for him as the end-to-end to my transparent is your friend and mine.

Mr. Aram Piligian, how are you doing? Aram?

[00:01:21] Aram Piligian: Hey, how's it going everyone? Great to be with you all once again.

[00:01:25] Andy Leviss: So, uh, yeah, you're, so you're in a hotel? Is that what you, is that what you were

[00:01:30] Aram Piligian: I'm actually, uh, I'm just working on an event up in, uh, New York this

[00:01:34] Andy Leviss: Nice. Up, up

[00:01:35] Aram Piligian: figured I'd pop in and do some podcast things

[00:01:38] Andy Leviss: Nice, up up, uh, vaguely in my neighborhood, or actually literally up in my old neighborhood. Um, but, uh, I, I had this half temptation to like do a, a spontaneous game of what's the coolest thing in arm's reach, but I feel like you're in a hotel, so that's just setting you up for failure on that one.

[00:01:54] Aram Piligian: not necessarily

[00:01:55] Andy Leviss: okay. I mean, 'cause we haven't done it in a while, so we could

[00:01:59] Aram Piligian: so while we haven't done it in a while, I do, I have my pelican very close by near me. Uh, so the coolest thing actually that is literally within arm's reach is, uh, my, in your case. Uh, so I have a pair of, uh, JH Laylas that I love. They're fantastic. They come in this metal round container that makes a lot of noise like a bell.

Honestly, when you go to open it and close it, it's kind of noisy. Uh, so I actually got a small pelican for it and then I got one of my friends to 3D print an insert for the pelican that fits the in ears, uh, in a way that holds them safe and keeps them warm and cozy and,

uh, is a lot less noisy to open when you're in a quiet environment trying to put in your in ears.

So

that's the coolest thing I got on me.

[00:02:55] Andy Leviss: nice, I have like a Facebook, uh, post that comes up as a memory every once in a while where I refer to the loudest piece of Velcro on Broadway, which there was a, there was a battery for an, an amplifier for like a built-in wireless, you know, prop piano on Beautiful. When I sub backstage on that and the battery was velcroed in and it was the quietest moment of the show when I had to like, retrieve it from the piano every night.

It was, I'm like, I so self-conscious about it. Um, I say I've finally been decorating my office, so I've got like some show posters up behind me and, um, my, I didn't go, so it was, I guess it was my second show as an associate designer on Broadway was, my name is Lucy Barden at the Freedman Theater for, uh, Manhattan Theater Club, and I forgot to get a show poster for it.

And when I was doing the next show there, I found this, which, let's see, can I. Tilt so you guys can see it, which is the late seating policy poster from the lobby of the show, which I thought was like so perfectly on-brand for me and my A DHD that I was like, can I take that? And they were like, yeah, no problem.

So that is now hung up in place of pride along with a couple other show posters there. So that, I don't know if it's really the coolest, but it might be the coolest for me.

[00:04:09] Aram Piligian: So I feel, I feel like we've stalled enough to give our guests an opportunity to figure out something that might be interesting that's within Arms reach. don't you introduce our guest there, Andy?

[00:04:21] Andy Leviss: So joining us today, we have a, a friend of mine, Chris Lapp, who we have been trying threatening cajoling to get onto the podcast for, I don't know what, Chris, a year and a half a while. 'cause

[00:04:36] Chris Lapp: I think, uh, I think about about two and a half years at this point now.

[00:04:40] Andy Leviss: It might be. Yeah. No, 'cause it was, yeah. 'cause we first met like a couple jobs ago for me, um, which we probably shouldn't talk about exactly how we met.

[00:04:48] Chris Lapp: No.

[00:04:48] Andy Leviss: murky details there.

[00:04:49] Chris Lapp: Uh, no, no worries. Uh, so the coolest thing in my arms reach, which technically is just out of arms reach, I was trying to grab it over here, is, uh, uh, I recently got myself, uh, a Jetson or a nano, I'm not sure if you guys are familiar with those, or like the little tiny raspberry pi, like, uh, Nvidia, GPU supercomputers.

Um, so it has like a whole eight gigs of, uh, unified memory, and I've been using it to essentially make an AI effects pedal for my guitar.

Uh, so

that's what I've been experimenting with. So that's the coolest thing, just out of arm's, reach to my left here. Um, but,

[00:05:22] Andy Leviss: Met Metric arms Reach is

[00:05:24] Chris Lapp: yeah. Yes. Right. I'm, I'm in Canada, so it's a metric arm.

Yeah. Yeah.

[00:05:28] Aram Piligian: The, the coolness of that definitely, uh, expands its reach to be within your arm's length. So I think you're good.

[00:05:36] Andy Leviss: No, I, I think we'll fly with that. So, yeah, so I'll give the quick, like, horrible explanation of who Chris is and why we, we wanted him on the show, and then I'll let you kind of dive in and tell us a little more of your story. But, uh, Chris, uh, is, broadly speaking, if I had to categorize him, is, um, one of the smartest folks I know about AV networking, clocking, PTP, any of those things.

Dante, 21, 10, you name it. Um, by day he works, uh, dealing with all that stuff at, uh, Cisco, a small switch manufacturer that's kind of obscure, but some of you may have heard of. And, uh, when he is not doing that, he's also just launched an open source project called Open Fabric Studio that I'm not even gonna try and encapsulate it.

We'll just get into it in a little bit and I'll let you explain it better so that I don't butcher it. But also a very cool AV networking switch config. Uh, project going on. So I know. Chris, why don't you start off and give us a little bit about like, kind of where you came from and how, how you got here.

[00:06:39] Chris Lapp: Yeah. Sounds good. So it all started back, uh, so I grew up in a small town, uh, in, uh, Canada, in Ontario. And the day that I turned, I think it was either 13 or 14, I had this like, revelation. I, I was watching the Matrix and I was like, you know, I really want to get into the engineering of this stuff. Like, I don't wanna be on, on camera, I don't wanna do any of this creative stuff.

Like, I wanna make this stuff work. And I had a teacher at the time who was like, well, you know, you should just get into TV engineering somehow. We'll figure it out as, as we go along. So from there I started doing, um, you know, audio live, sound reinforcement for school, plays around town and stuff like that.

Working with like a local, uh, audio company. And then, um, I finally found that there was a, a TV program school relatively close by as I was finishing high school. And I went there, a little disappointing. It was more of a production program, but they actually kind of let me do my own thing and like do the engineering of the shows. And then I had a mentor there, um, who kind of pointed me in the right direction after that program. But while I was at that program, uh, which was in Hamilton, Ontario, I started working at nightclubs, doing live sound reinforcement. I started doing lighting. Uh, when I turned 18, my boss at a theater that I worked at at the time actually got me to go get my pyrotechnics license. And so I've also been doing pyrotechnics for a long time. My whole life

has been about show business, like show business engineering. So once I, uh, you know, started doing all that, um, I finally finished that program. I went to Calgary to another program called S sat, which was specifically broadcast engineering. My first full-time job, I'm gonna say full-time because I'd been doing many jobs in show business. Um, you know, across the time, uh, my first full-time job was at Bell Media in Toronto, in Canada, which is one of the largest television networks in, uh, Canada. And I was in charge of 89 master control television stations there, um, and designed new ones as well.

Um, and then from there I got kind of bored. And so the day after I launched, uh, in, if you're in Canada, you'll know TSN 4K, uh, which is essentially Canada's equivalent to ESPN. Like the day after I launched it, I handed in my resignation and decided I was gonna go do something new. Um, so I went over to a company called Everts Microsystems. Uh, I started doing some IP video stuff for them 'cause that was starting to become a very popular thing. Um, and then my wife was, uh, well, got married while I was at Everts. You know, my wife said, you know, you need to stop traveling so much. And I said, well, okay. And I got a new job, got poached actually for a integration company based in New Jersey called Diversified. I was their only Canadian employee, uh, for a very long time. And I became their specialist for all IP systems, whether it be video, audio, av, et cetera. Um, and then my wife said, you need to stop traveling more. So I had to find another job where I stopped, started traveling less. And um, it just so happened, I'm not gonna get into the, the, the craziness of the details, but something happened on a Day at Diversified, where the, you know, the corporate entity got me really upset.

And that same day, uh, my old boss at Cisco reached out to me and he said, Hey Chris, we have a job and we'd like you to take it. Perfect timing. My wife was mad at me for traveling that day. I was mad at the company and I went to work at Cisco. And lo and behold, here I am at Cisco. My job here is senior specialist solutions engineer for cloud and AI infrastructure and media, and an enter media and entertainment. That's a mouthful. I just tell people to call me Crisco. So that's my, my nickname.

[00:10:06] Andy Leviss: Dig

[00:10:07] Chris Lapp: That was a good explanation of where I came from.

[00:10:09] Andy Leviss: Yeah, no, that's, that's awesome. And I mean, I, I think even if I had that job title, I wouldn't be able to keep it straight. So it's about, I'm watching it, we're watching him on the video, like almost counting out his fingers, getting all the parts of it. Uh.

[00:10:22] Chris Lapp: Yeah, it's, it's, uh, you know, I had to have three lines on my business card just to, just to say all that so.

[00:10:28] Aram Piligian: No, I

love that. I, I'm really curious. So there's not a lot of people that get to experience reaching, uh, kind of the, an apex of being within a company. Uh, and what, and I'm kind of referring to your experience with that broadcaster you, you were with, where you're like, Hey, I'm designing all these studios, I'm implementing this, I'm engineering this. I am at the top of my game within this company, and it's not enough. And, or maybe even not enough isn't the word. It's just, it's not what I'm looking for. And, um, I would love to hear more about your experience with like, taking that, you know, Hey, the human in me needs something different than what the, like technician in me is achieving right now.

[00:11:20] Chris Lapp: Yeah, that's an, that's an interesting kind of way to think about it and look at it. So I always tell people that, you know, I'm never looking for a job, but I'm always looking for opportunities. So if an opportunity ever approaches me and I think, you know, that would be frigging cool. I always try and take it a little harder now, you know, I have three kids now, so, you know, I can't just be like, oh, I'm just gonna move to Europe, you know, outta nowhere. Um, but if if something is going to present me with, you know, no negative effects, but always positive effects, I always try and take that leap now. Cisco, every time I've said, Hey, I wanna do something cool, they're always just like, just do it. No problem. Just do whatever you want. And like, I have all this, you know, freedom internally to innovate and create things and work on really cool projects. Um, I wasn't getting that at other places. Um, also at the TV station, if you've ever worked in broadcast engineering, it's shift work and that kind of sucks when you have kids. So I wanted, you know, something a little more, I wouldn't call it stable, but more predictable. Um, but uh, you know, that that's kind of the way it was.

But I also have crippling A DHD. And so anything that can like, make my brain think really hard is always good. So I always try and move towards that next thing.

[00:12:37] Andy Leviss: Man, I was just talking with one of my coworkers at YAMA the other week 'cause a meme popped up that was like that. Like the, like the tech support is so great for like a DHD folks because it's both that like the problem solving, the puzzle solving, it's a new challenge every hour, every day. And you get that dopamine rush and like it doesn't go away.

'cause as soon as you solve that one, then you're onto the next person. I was like, I feel really attacked right now. But also that explains a lot.

[00:13:04] Chris Lapp: Yeah. I, and it, it's actually cool 'cause like when I was at my old jobs where I was, I was kind of like, at Everts especially, I, I was considered like the bazooka, the one you would fire up problems to make sure that they got fixed.

Um, I never had to take my A DHD medication because the, the problem solving was my medication.

Once I moved to Cisco and I had more of like a desk oriented job, impossible to handle without like some level of control on the dopamine that I would, I would get, but the amount of times that I can sit down and like solve problems that are two years away and think about that, like it's it's kind of humbling to be able to put my knowledge towards that instead of just like day-to-day minutiae.

[00:13:44] Andy Leviss: Yeah, it's, it's, and it's, it's just, it's, it's really cool for me, like talking through like those similar journeys. 'cause that's obviously, like folks know, that's the same reason I made the pivot back into the support side was the, I've got a little one at home and, and yeah, the traveling of like freelancing is, is too much.

And yeah, that predictability, the ability to work from home and all that was like just the right combination of factors on top of the like, yeah, built in, built in dopamine, hit, let's do it. Uh.

[00:14:12] Chris Lapp: Yep.

[00:14:14] Andy Leviss: I mean, I don't know, like Aram, if, if you have any other questions on like the, the general path of stuff or if we wanna just like, dig into the nerd stuff and, and, and ask all the, all the tricky, uh, networking questions, uh,

[00:14:26] Aram Piligian: Oh, I mean, so I, I'm just curious, you know, you, you said you're working cloud and AI infrastructure, media and entertainment. So what if, if, if someone asks you, like, so what do you actually do? What, what does that entail? You know, what's your like elevator pitch for that? You know, what, what is, what is your day to day?

What are the, what are the things that are important to you?

[00:14:51] Chris Lapp: I, I wish I had a really good answer for that, uh, about my, what about what my actual day-to-day was. Um, I, I guess I could describe it as, so in the Cisco portfolio umbrella, um, I handle everything under the Nexus 9,000 line, everything under the UCS line, everything that as it touches data center security. Everything as it touches, data center software, everything as it touches the iso valent umbrella, which is our like, um, kind of, uh, CNI, uh, Kubernetes ecosystem. But then because I do the media and entertainment stuff and on PTP, I also have to touch Catalyst almost every single day. I have to touch the endpoint ecosystem as well.

So when we integrate with vendors like Q Sys or Yamaha or Sure. You know, a lot of the people at Cisco don't know how that stuff works. And so I'm usually involved because I have a very, like, intimate understanding as to how those ecosystems work. So I'm, I basically touch almost every Cisco product except collaboration, which is kind of funny 'cause you know, they're more AV than almost any other Cisco product.

But I don't touch them.

[00:15:55] Aram Piligian: That makes sense. So you're a very, it's, it is very vertical. You go from all the way the base level, like someone walking into a room and dealing with some Cisco stuff and, you know, for their AV install all the way up to, you know, 3, 4, 5, 6 layers deep to an advanced. Uh, you know, advanced implementation where that person that's in the room is pretty much an endpoint for something way more complex that's going on.

[00:16:26] Chris Lapp: Yeah. Yeah, pretty pretty much. The other thing that I, I basically do is, you know, at Cisco we have, I mean, it's easy to describe us all as pre-sales when we're in the architecture role, because we're technically talking to customers before stuff's sold, after stuff's sold, when stuff's broken, when stuff needs to be kind of re-architected. Um, but I also, I call myself to be in charge of the technical distillation of these systems. So as we're talking to CTOs and CEOs and SVPs and VPs who don't necessarily understand. The end to end way that everything's supposed to work. It's my job to distill that down for those level of people so that they understand, you know, how this whole system was put together. So that's the other part of the job. On top of that, like super, super low level, I also have to bring it back and do the super, super high level. I,

[00:17:15] Aram Piligian: That's awesome. How did, how did you get, uh. Acclimated to both, you know, it sounds like you came from sort of that lower level, the technical background. What was the experience of, uh, gaining the, you know, layer upon layer knowledge to get up to have those conversations with the major stakeholder kind of experience for you?

[00:17:41] Chris Lapp: oh, so it kind of started, so my, my oldest daughter at the time was kind of about four or five years old when I started to have to go between the levels. Um, and I actually, I started to teach her how to solder. So my, my daughter was soldering by the time she was five. And I mean like soldering surface level mount soldering components on boards.

[00:18:01] Andy Leviss: nice.

[00:18:02] Chris Lapp: but I would explain this to her and like if I can explain something to my four and 5-year-old daughter, I can explain something to anybody. And so I would go through this process of trying to explain to her like how things work and stuff like that. So it was this very intimate process of seeing what worked and how could I distill information down.

And it doesn't matter whether you're talking about networking or Kubernetes or CNI systems or kernel bypass systems or soldering, if you can explain it to a four-year-old or a five-year-old and they can understand it, you can explain any information to anybody as long as you have the key details.

[00:18:40] Andy Leviss: No, that, that's, that's some, I'm like, I'm sitting here sinking into that right now.

[00:18:44] Chris Lapp: Yeah. Andy, I expect your kids to be soldering. Like, you know, you have one kid, one kid right now, right? One

just one kid right now. So. Once that kid is five, like

we're gonna have a soldering competition, man.

Like

[00:18:53] Andy Leviss: Sounds good. Sounds good.

It sounds like some, some families go for like glass blowing for Christmas ornaments. Just, we'll do like the, the soldering, uh, LED flashy kits.

[00:19:05] Chris Lapp: Yeah. Like Christmas ornaments, man, you

should see my house at Christmas.

[00:19:09] Andy Leviss: I can only imagine. Uh,

[00:19:13] Aram Piligian: I mean, if, if someone's kids wants to take care of DB 25 and W one for me, I'm

happy to let them do that.

[00:19:22] Andy Leviss: Oh, all the W one and the, I mean, the tiny hands would be so much better for like, straightening the pins without snapping them. If you could teach 'em control. Wait a second. Or we might be onto something here.

[00:19:31] Chris Lapp: and most of them can see better. And they don't have shaky hands yet. You know, you know,

it's just, it's a perfect combo

[00:19:37] Andy Leviss: I, I, now we just gotta work on peeling back some child labor laws and I think we got a business venture here.

[00:19:42] Chris Lapp: as long as it's your own

kid.

[00:19:43] Aram Piligian: family.

[00:19:44] Chris Lapp: labor. Yeah.

[00:19:45] Aram Piligian: Exactly. I.

[00:19:47] Andy Leviss: Loophole's. Gotta love 'em. Um, so yeah, I don't know if we want to dig into. 'cause there's two aspects I really want to get into. Like, I certainly want to talk about open fabric and what that is and where you're going with it, but I also feel like this is an opportunity for like all the like silly networking questions that like we have and that folks have that like we can actually have somebody who like actually understands the answers as opposed to like 90% of us audio folks who like mostly understand it, but don't quite, um, I mean, I, I don't know, I guess the, like a good place to start might be like Chris, like you kind of interface with folks on all levels with all sorts of these network problems.

Like particularly in like most of our listeners are audio focused, so we're mostly into like Dante a VB Milan kind of corner of the world. Um, are there like common like mishaps or missteps you see come up like time and time again that like you think are worth like clearing up to folks maybe as a, as a point to start off from?

[00:20:47] Chris Lapp: Yeah, so we can, we can probably break that down to two levels. So if we start at just like a network that is dedicated to av, right? Like let's, let's just use that as kind of the base point, which most people is what, what most of them are dealing with. Um, unless you're using Dante, you should never turn off IJP snooping. Please don't ever do it. And even when you're using Dante, my suggestion is leave it on because the moment you add one other thing to that network, it's gonna break and it's not gonna be a good time. And you're gonna call somebody like me and we're gonna be like, well, it's doing exactly what you told me to configure it as.

So the network's working, right? Um, the other thing is that most of the problems that people come to us with that are like being blamed on the network are not actually network problems, right? The network is doing what it's supposed to do most of the time, unless it's configured incorrectly. But most endpoints do not handle what you've told the network to do properly. I'm gonna pick on Dante a little bit. Not that Dante's bad or anything like that, but it's what I see most often. Um, and we can compare that with like, ultimos versus, versus Brooklyns, right? In the, in the Dante world is that a Brooklyn is significantly better than an Ultimo, and the CPU in a Brooklyn is significantly better than the CPU in an Ultimo. And you can put a Brooklyn and Ultima on the same network, and the Ultima will randomly reboot and the Brooklyn won't. And that gets blamed on the network. Like, oh, the network is, you know, failing the, the, the ultimo that's going through clock elections, et cetera. It's like, actually, let's just imagine you have 100 devices on the network. Each of those is using PTP, that Ultimo and that Brooklyn have to handle roughly a thousand messages per second with just a hundred devices on the network. And they have to go through and say, I need this message. I don't need that message. I need this one, this one's important, this one's not. I have to choose which one's to discard. The higher end CPU can do that a heck of a lot better than the lower end CPU. And what usually ends up happening with the lower end CPU is when it runs out of buffer or you know, cycles reboots itself. And that's when you see a clock election happen. 'cause the device comes back online, it does a clock election.

You're like, oh, my clock's failing. No, the device just literally just restarted its network stack because it can't handle what's going on. The network cannot do anything to solve that ever. And nothing we ever do. No, no amount of QOS you apply. No amount of traffic shaping will ever solve that fundamental problem of you are only as strong as your weakest link. And 90% of the problems I solve are problems like that. The weakest link is causing the application layer to break down.

[00:23:23] Andy Leviss: That make, that makes sense. Um, I know one thing we mentioned in there that I feel like is a term that everybody listening who's dealt with Dante is at least a little familiar with, but probably doesn't fully understand, is IGMP snooping. Do you wanna give us like the quick, kinda like crash course for folks who might not, who like know that's a thing they're supposed to enable but don't understand what it's actually doing or what they're doing when they set it up?

[00:23:46] Chris Lapp: Yeah, sure. So, so IGMP, there's three versions of it. Version one, version two, version three. Uh, if you can control it, you should be version three on the network. Okay. Version three on the network is backwards compatible with version two, but version two on the network, not backwards compatible or forwards compatible.

Technically speaking with version three, so if the network is V three, you account for all cases, so always configure your network as V three. When the device wants to join a multicast, basically what it does is send out a joint request. Usually, um, in AV we see what we call the network world star gs, which means join any device that's sending this out at this multicast address. We also have the concept of joining an sg, which is source and group saying, I want this specific group from this specific device. So you can specify it that way as well. That's more in the version three. And then when we get to multicast routing, PIM source, PIM source specific multicast. But that's a whole two hour conversation.

We can talk about another day about pim. Um, but IGMP is essentially the device signaling to the network. I would like to join this multicast when the

[00:24:51] Aram Piligian: And so, and I, I want to jump in right here

to to recap the things that in a Dante network, multicast is used for in a very vanilla kind of way. Like, hey, I have my couple basic, I have my console and a stage box. You know, multicast, even if you haven't configured specific multicast streams, multicast being used for device discovery especially, and uh, and if you do need multicast for, uh, multiple destinations, et cetera. But basically it is something that is required for the devices to see each other on the network.

[00:25:31] Chris Lapp: To see each other an a clock because PTP also uses multicast and it needs to be IGMP.

Right? So it's the, yeah, it's fundamental. And I always, I, it's a true, I see people all the time like, oh, well I'm not using multicast. Well, you are, whether you like it or not, because you're, you're fundamental functions run on multicast.

'cause it's many to many. Um. When a device is a transmitter to multicast, it also has to send out a message to the network to say, Hey, I am the owner of this source and you can let other people know that I have the source. And that message has to go to the query or the M router on the network. And so the query kind of keeps state of who owns what sources on the network.

So when you send your join request, it goes up to the query and the query says, yeah, for the traffic out to these guys who are asking for this flow. 'cause I know where those flows exist. Now one thing that lots of people don't know about IGMP and about layer two multicast is that although you're saying I wanna join this multicast IP address, you're actually saying to the network, I want to join this, this multicast Mac address. And we do not use multicast Mac addresses the same way that we use IP addresses. When you join a Multicast Mac address, there is actually 32 IP addresses that match one single Mac address. So if you overlap those IP addresses, you could join one group but accidentally get 32 and oversubscribe the port because there's no, no control of that.

And we call that multicast Mac aliasing in the network world. But when the, uh, you know, the IETF and I know them got together to, to make multicast, the guy who did it, I can't remember his name, but he basically asked for a bunch of multicast Mac addresses. They said no. And so essentially they gave him a much smaller range and the only way that he could do it was to essentially only use the lower 23 bits of the IP address to formulate the MAC address.

So there is 32 groups per Mac address and multicast. So if you're doing an IG MP joint and you get double the amount of bandwidth that you expect because you have two groups that match the Mac address,

[00:27:38] Andy Leviss: And is there, is there any, like, how would, how would you go about fixing that?

[00:27:43] Chris Lapp: M layer three multicast all the way. So that's why going up 10 levels when we do ST 2110. Any high bandwidth multicast, we actually do routed access all the way to the port. So it's a single tiny little subnet on tiny little subnet on every single interface that devices are connected to, to get away from that problem.

[00:28:03] Andy Leviss: Got it. Um, yeah, I mean, I feel like there's so many terms, like we could just easily just take like each answer of this and like go down the glossary of like what we're talking about. Because I mean, I know like one thing you mentioned that, that I, I feel like we should, um, like kinda clarify for folks is there is that big, um, mis misunderstanding I guess, that folks have where they're like, well, I'm not using multicast.

And that's, there's, there's that labeling terminology thing. 'cause like if you don't understand that underlying nature of like all the clocking and all this other stuff and discovery that happens in. Multicast, it's easy to think in Dante that, like when you say multicast, you mean multicast flows of audio, which are a subset of the multicast there.

And yes, are a thing that folks generally only use when they're very specifically using it to solve a certain problem or for a certain purpose. But just because you're not using that, like Chris said, doesn't mean you're not using multicast in your Dante network. If you have a Dante network, you're using multicast all over the flipping place.

Um, uh, BI had like another IGMP question off of that, but it totally went outta my brain right now, so I'm sure it'll come back to me in like 10 minutes in the middle of another, uh, of another subject there. Um.

[00:29:21] Aram Piligian: So I guess from a, from a user standpoint, from a, Hey, I'm a, I'm a random audio guy, you know, I, I am, or I, I'm a random audio human. I, I have experience using audio over IP in small to medium setups and the, you know, I've configured a switch or two here or there. Uh, I haven't done anything that's big enough to have to worry about massive multicast and getting down into individual subnets for individual endpoints, anything like that.

You know, when, when do you see, uh, where's the point where someone needs to really start digging into, Hey, this my, uh. Investigation into this, my education into networking needs to go more than just the audio world and into, you know, kind of break from that like audio over IP training bubble of the, here are all the safe things that will keep you happy in a Dante world into like, hey, now I really need to be more of a network engineer.

I need to go beyond just the audio related basics and really be competent with some higher networking concepts. 'cause it sounds like the, you kind of hit a point where, you know, the, your Dante level three isn't really gonna give you the tools you need anymore to, uh, get into these more complicated setups and be able to be effective in deployment and troubleshooting.

[00:31:08] Chris Lapp: Yeah, so I, I think there's two things that, that affect that, and we will talk about the easy one first, and that's scale. Um, and it's hard to define scale. Um, it's really your, as I said earlier, your weakest link. And it's hard to know what that device is unless you've done extensive testing yourself or you choose to believe people's data sheets. Um, I tend to not believe data sheets, and I tend to always test things myself as, uh, we were talking about earlier. I have literal piles of devices in this room. I have tested every audio endpoint and every video endpoint probably that exists in this world. And I can tell you exactly what breaks them. Um, that's, that's kind of why I'm so good at my job is 'cause I sit down and I do that testing in my spare time. Um, people will literally ship me their devices to tell them what breaks. Like, uh, it's at that level. Um, but if once you know that, like, that scale factor, it's, there's things to solve it. Number one is using multicast routing instead of layer two multicast. With IGMP, you could use multicast routing to solve some problems, but the first thing that always breaks is clocking. And it doesn't matter what protocol you're using, there's some level of clocking and it will always break, and it usually breaks somewhere between 50 devices and 200 devices. The only way to solve clocking problems is to use PTP version two and use boundary clock. If anybody tells you to use transparent clock, throw their knowledge away because transparent clock doesn't solve scale. Transparent clock can be useful in some situations, but it does not solve scale. Scale can only be solved by boundary clock. With boundary clock and PTP, you are actually infinitely scalable from a clocking perspective. Like you, you, there is no limitation. Like, if you meet that limitation, let me know, but you, you won't hit it.

[00:32:49] Andy Leviss: So

[00:32:49] Aram Piligian: what's the difference?

[00:32:52] Chris Lapp: So, okay,

[00:32:54] Andy Leviss: yeah, I was about to

[00:32:54] Aram Piligian: Short version you got? yeah,

[00:32:56] Andy Leviss: folks at home who just made the Scooby Doo confuse sound. So yeah, define boundary clock in.

[00:33:02] Chris Lapp: so spark's, no spark's no version. With no PTP awareness whatsoever, you just send the multicast PTP out as multicast. With PTP transparent clock, we kind of half intercept that message, adjust the timestamps in it and send it back out. But we're still sending it out as multicast in boundary clock.

We absorb the multicast into the control plane of the switch so it no longer travels. IGMP. It no longer travels. Payment becomes no different than the switches talking together over like their hello messages or CDP neighbors or whatever else, right? And then we retransmit new messages on the other side.

So the conversation is really from port to whatever's connected to that port

and nowhere else.

[00:33:41] Aram Piligian: almost independent verification of clock.

[00:33:44] Chris Lapp: it basically independent to that one single interface and whatever's connected to it. So. Because you don't have that like all to all mechanism of everyone flooding each other's CPUs, you, you just infinitely scale.

[00:33:58] Andy Leviss: Got it. And then I guess the other terms we, we were throwing at were like. PTP and there's different versions of of PTP. Do you wanna give the quick rundown on at least where like audio folks are running into the different versions and maybe what, what the differences are?

[00:34:12] Chris Lapp: Yeah, sure. For, for the most part, like in the audio world, there's PT P version one, PTP version two, and a little bit of 2.1 Now, although we're still just using two because 2.1 is not ready yet for, for endpoints. Um, version one is it's Dante, for Dante is the only thing that uses version one outta the industry. Um, the only difference really between version one and version two is some enhancements on what we call the best master clock algorithm and how they handle that decision making process between each other among the ability to add things like transparent clocks and boundary clocks to the network to increase that scale.

So. Version two, although it's not compatible with version one, um, is the better way to go. If you have a protocol that uses version two, you should just be using version two and turning off V one on the Dante side if you have the option, because trust me, it's, it's just better version two all the way. Um, I, I did wanna touch on another part.

I, I, we talked about, you know, when do you have to go take that, that second step to networking And the other reason why you should take that second step in your networking knowledge is anytime you have to integrate with a corporate network, if you have to run AV across a corporate network and you wanna actually know how things are happening, you have to get into things like security, SGT tags, which is security group tags traveling across the network.

You have to get into overlay technologies like vxlan, E-V-P-N-S-D-A networks. It's a whole complication that actually adds a ton of crazy complications onto realtime protocols like audio. 'cause when you're going across something like EVPN, it encapsulates the packets as it travels across the network and that increases delay. So understanding those technologies when you go into a corporate network is extraordinarily important because packets aren't packets anymore. Packets are packets that turned into different packets that re became new packets. On the other side, it's like a whole new world. So trust me, like if you are integrating with a corporate network, you need somebody who understands that level of networking because it's going to be complicated the first time you do it after you've done it once.

No problem. The first time is hard.

[00:36:12] Aram Piligian: I love that. That's super helpful. 'cause you know, the, so much of the way audio networking, you know, for the last, I mean we could say 10 plus years at this point, is, is uh, it's simple. If you're alone, uh, you, it's simple. If you are in your, you're isolated network, doing your own isolated network things, uh, you know, it's easy to, it's easy to get it up and running and, uh, get it to a usable point.

But once you need to start integrating, then it becomes. The problem's not quick. So, you know, I, I agree that, you know, that's exactly where people need to start, uh, either increasing their knowledge level or bringing in people that can do that. And, uh, yeah, that's, that's a really great point. I love that.

[00:37:01] Chris Lapp: Yeah. And, and it always ends up coming back down to scale, right? Like the one thing I always tell people, my famous sentence that everyone quotes me on, is I always say it works until it doesn't. Right? And if, if you're a system integrator installing a system and you, you know, you can send audio back and forth between two locations, that's great. Can you send 5,000 audio streams back and forth between two locations? 'cause that's a whole different story, right? Scale matters when you're testing things, especially in complicated networks, because it's more bandwidth, it's more replication, it's more m fb scale. And m fb is like the actual hardware in the switch, like creating those multicast routes. Like these are all things that. You won't hit those limitations or those, those patterns unless you actually test at scale. So scale is important.

[00:37:44] Aram Piligian: So I want to get to something, something else in a bit, but one, one thought that I had leading into this is one of the things actually that I found as someone who. Was learning the networking process with audio stuff and you know, Dante specifically, but I've also spent a lot of time dealing with Ravenna stuff, et cetera, is testing is actually kind of hard unless you have a lot of hardware and a lot of IO to throw it something to, you know, push a whole chunk of audio down the line. Um, is, is there anything you found kind of helps with the testing process, uh, to ensure that like, Hey, I've built this thing. I'm not a hundred percent confident that it's stable, but I want to be able to stress test it without ha needing, you know, 400 preamps to actually push down the cable.

[00:38:43] Chris Lapp: So there's a, a few things you can do, right? Like there's lots of open source SDKs out there, things like even ff eg that can create very good A is 67 very good multicast, right? Um, we have a line of switches in our nexus line that can actually recreate new multicast based off an original multicast. So you can create thousands of new flows just from a single flow to test scale as well. There's tons of things that you can do. It just, you need the thing to do it with. So like I use FFM PEG on a, uh, it's, here's one right here actually, this is within Armory reach. You guys can see it. It's a tiny little $200 Chinese mini PC that I bought that has a, a core I nine in it. But the important part is the nick in it is actually PTP hardware aware. I can actually do hardware timestamping, PTP and have very, very, very precise audio flows coming outta this. And they're 2.5 gig nicks, so I can have like 300 per interface on this machine, uh, 300 streams generated. So something like that, like obviously today with the memory prices is probably not a couple hundred dollars anymore.

But you know, same idea like having something and being able to do some software engineering to develop something or going on my repo and downloading my tools, um, you know, is something you could do as well.

[00:39:58] Aram Piligian: So you can, you can pretend that that is a massive console essentially, and

[00:40:02] Chris Lapp: Yeah, actually it, could be a massive console. Yeah.

absolutely. I mean, almost every audio console these days is software, right? I mean, what's to stop you from making your own audio console?

[00:40:14] Aram Piligian: So what, uh, so that kind of sets us up for like, what are some of the other tools that I, I hear that you're working on to, uh, help with, uh, all, all, all of these to audio things and getting everyone's, uh, gear working together better.

[00:40:33] Chris Lapp: So, um, as Andy knows, you know, I'm, I'm a guy who likes to solve problems that not everyone necessarily has, but if I see a problem, I, I tend to, to fix it or at least try and get somebody else to fix it if I'm not, you know, up for fixing it myself. But, um, one of the problems that Cisco had is that, you know, we have lots of really good network orchestration tools. Um, we have Catalyst Center, we have Meraki, we have, you know, nexus dashboard, but none of those tools are really built for what I would call like an air gap. AV network, something that you're running on stage in a touring environment, you can't connect to the internet. It just doesn't exist. And also, I would hedge to say that nobody actually has a network orchestration tool that you can start up in five seconds.

Like it just doesn't really exist. Um, so I was like, I'm a, I'm a guy who's been on tour, like I want, I want something that people can actually use. Um, so I decided to make my own, um, originally I was gonna call it Catalyst Fabric Studio and, you know, to, to kind of like stay within the Cisco umbrella. But then I decided that, you know, I'm gonna make that step, I'm just gonna call it Open Fabric Studio. Um, and Open Fabric is going to be the software umbrella that it lives in. This is the studio application underneath the Open Fabric umbrella. And what it really aims at doing is taking complex network configuration and just simplifying it. I'm not trying to boil the ocean. I'm not trying to do complex security configs.

It's really meant for those like point and shoot AV networks that just need to work, um, to the point where PTP across the whole network is literally two clicks and then you hit Deploy and PTP, it does a whole check to say like, all these switches, support PTP, here's Chris's best practice template. Just deploy it, make sure PTP works and it just does it.

And like it's, instead of spending five and a half hours and people calling me and saying, Chris, PTP doesn't work. You can just click this button. PTPI guarantee you is going to work. Um, I've gotten the boot time of the application, so if you're familiar with Docker, it's a docker application, so it just spins up a couple containers. Um, once you've done the initial install and it pulls the images, if you stop it and then restart the containers, it's literally 4.6 seconds to start up the application. Start to

finish insane speed. I don't, I mean, I wasn't even aiming for that number, but it's really good. So it's just unbelievable what I've been able to achieve, um, just by using like very simple like open source technology.

[00:43:08] Andy Leviss: So, and then are you just like, you're pointing it at a list of switches, are you connecting it to like raw switches and it's configuring them like ground up, including ips or like how, if I've got a pile of switches that I wanna like build into an AV network and I'm gonna do it with OFS, like what's.

What's the start of that process look like?

[00:43:26] Chris Lapp: Yeah. So today, at the moment, um, as of, as of this moment that we're speaking, um, it supports Catalyst 9,000 and Nexus 9,000 switches on, on the Cisco side. Um, mostly because that's what I have here in my lab to, to check against. Um. All you need on the switch at the moment is for it to have a control IP address and to have, uh, YANG and NETCONF enabled. And on the repo it tells you how to enable that to make sure it's good. Then tells you to get a coffee. 'cause the first time you enable it, it takes a little bit on the switch. Um, once it has that, you just point the application at the IP address and punch the password and the username in, and it will then, whatever you need to do, VLAN config, et cetera, you can just do from the application. If it has a config on it already, anything that matches the intent of what the application tries to configure, it will import that as well. So if you have VLANs configured already, it will brownfield them into the system. Um, I've, uh, I just released a test version for some people to test for me. Um, the, the code to control NETGEAR switches as well, the four two fifties and the four three fifties. So that'll be coming to general release soon. Uh, it's just in testing at the moment, but same idea, you know, just rest API config to the, to the NETGEAR switches.

[00:44:35] Andy Leviss: That's awesome. So like you could do like a mixed network of like Netgear and, and Cisco models and, and not have to worry about the, okay, are they talking the same flavor of IG and p, are they talking

[00:44:45] Chris Lapp: Yeah, I'll, the application will take care of all that for you. It will

not let you mix IgM P plus and IgM P, and it'll all be a standards based config not gonna lie to you. I'm not gonna do anything that's not standards based, but it'll be a standards based config. The IgM P stuff will match. The PTP stuff will match.

Everything, will check and balance to make sure that what you're trying to enable is actually possible on the hardware.

[00:45:07] Andy Leviss: That's really cool. That's, I'm, I'm excited to see where this goes and, and how it grew as and expands and, and since you're kind of teasing that there might be other applications within like the Open Fabric framework...

[00:45:19] Chris Lapp: another one. There's another one already on its way.

[00:45:21] Andy Leviss: nice, I'm, I'm excited to see what that is.

[00:45:26] Andy Leviss: Hey, everybody, future Andy here! So, this conversation was super awesome, and we got really indepth into a lot of everybody's burning questions on Dante and general networking stuff--AVB, AES 67, DHCP, managed vs unmanaged, all that. So instead of letting this episode get super duper crazy long, I'm gonna divide it up into two parts. So this is part one, and we're gonna pause it here for a bit, and in a few days keep your eyes on the feed, and we'll have part two of this conversation with Aram and Chris Lapp! In the meantime, thanks to Allen & Heath and RCF for helping keep the lights on in the virtual studio and--as Sean would say if he were here this week--that's the pod!

Music: “Break Free” by Mike Green