03.11.2019       Выпуск 306 (28.10.2019 - 03.11.2019)       Вопросы и обсуждения

Podcast.__init__: Accelerating The Adoption Of Python At Wayfair

Audio

Читать>>




Экспериментальная функция:

Ниже вы видите текст статьи по ссылке. По нему можно быстро понять ссылка достойна прочтения или нет

Просим обратить внимание, что текст по ссылке и здесь может не совпадать.

Summary

Large companies often have a variety of programming languages and technologies being used across departments to keep the business running. Python has been gaining ground in these environments because of its flexibility, ease of use, and developer productivity. In order to accelerate the rate of adoption at Wayfair this week’s guest Jonathan Biddle started a team to work with other engineering groups on their projects and show them how best to take advantage of the benefits of Python. In this episode he explains their operating model, shares their success stories, and provides advice on the pitfalls to avoid if you want to follow in his footsteps. This is definitely worth a listen if you are using Python in your work or would like to aid in its adoption.

Announcements

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. And for your tasks that need fast computation, such as training machine learning models, they just launched dedicated CPU instances. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
  • Having all of your logs and event data in one place makes your life easier when something breaks, unless that something is your Elastic Search cluster because it’s storing too much data. CHAOSSEARCH frees you from having to worry about data retention, unexpected failures, and expanding operating costs. They give you a fully managed service to search and analyze all of your logs in S3, entirely under your control, all for half the cost of running your own Elastic Search cluster or using a hosted platform. Try it out for yourself at pythonpodcast.com/chaossearch and don’t forget to thank them for supporting the show!
  • You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Dataversity, Corinium Global Intelligence, Alluxio, and Data Council. Upcoming events include the combined events of the Data Architecture Summit and Graphorum, the Data Orchestration Summit, and Data Council in NYC. Go to pythonpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today.
  • Your host as usual is Tobias Macey and today I’m interviewing Jonathan Biddle about his work to encourage and empower Wayfair engineers in their use of Python

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by describing the mission statement for you and your team at Wayfair?
    • What is the origin story for how your group got started?
      • How and where was Python being used within Wayfair at the time?
  • What are the primary languages that are used throughout Wayfair?
    • What is involved in the selection process for a language and technology stack for new projects within Wayfair?
  • Can you describe how and why you work with different groups throughout Wayfair?
  • What are some of the common misconceptions or barriers that you encounter when working with other engineering and product teams about how and where Python will be useful?
  • How large is your team currently and what is the length of a typical engagement?
    • How has the scale and scope of your work changed since your group was first formed?
  • How many different product teams have you worked with at this point and what are some of the notable outcomes?
  • What are some of the most challenging aspects, both technical and organizational, of educating other engineers on when and how to use Python?
  • Can you share some examples of engagements that you would classify as a failure?
    • What lessons have you learned from those situations?
  • What advice do you have for other groups or organizations who may be considering or actively launching similar initiatives?

Keep In Touch

Closing Announcements

  • Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you’ve learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Click here to read the raw transcript...

Tobias Macey

0:00:12

Hello, welcome to podcast, the podcast about Python and the people who make it great. When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at the node. With 200 gigabit private networking, scalable shared block storage, node balancers, and a 40 gigabit public network all controlled by a brand new API, you'll get everything you need to scale up. For your tasks that need fast computation such as training machine learning models, they just launched dedicated CPU instances. They also have a new object storage service to make storing data for your apps even easier. Go to Python podcast.com slash lindo, that's LI and OD today to get a $20 credit and launch a new server and under a minute, and don't forget to thank them for their continued so Part of this show. Having all of your logs and event data in one place makes your life easier when something breaks unless that something is your Elasticsearch cluster because it's storing too much data. Chaos search frees you from having to worry about data retention, unexpected failures and expanding operating costs. They give you a fully managed service to search and analyze all of your logs from s3 entirely under your control all for half the cost of running your own Elasticsearch cluster or using a hosted platform. Try it out for yourself at Python podcast.com slash chaos search and don't forget to thank them for supporting the show. You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest and machine learning and data analysis. For even more opportunities to meet listen and learn from your peers you don't want to miss out on this year's conference season. We have partnered with organizations such as O'Reilly Media, cranium, global intelligence, Alex CEO and data Council. Upcoming events include the data orchestration summit and data Council in New York City. Go to Python podcasts.com slash conferences to learn more about these and other events and take advantage of our partner discounts to save money when you register today. Your host, as usual is Tobias Macey. And today I'm interviewing Jonathan Biddle about his work to encourage and empower way for engineers and their use of Python. So Jonathan, can you start by introducing yourself?

Jonathan Biddle

0:02:22

Sure. My name is Jonathan Biddle. I lead a team at wayfarer called developer acceleration, you can think of our team is essentially trying to help other teams become more effective. So I consider us a bit of a meta team in that regard. And large part of that would be how we kind of drove a lot of Python adoption over the past. I'd say about two and a half years at this point.

Tobias Macey

0:02:42

And do you remember how you first got introduced to Python?

Unknown

0:02:45

Yeah, pretty clearly actually. So I'm a bit of a comment definite call myself a self taught programmer and the first part of my programming career was essentially taking over maintenance of some Perl scripts and for a newspaper company. And I just taught myself that organically for a while and eventually got introduced to Python by soap, who we were using for our, they had a product called ZM. It was powering a bunch of our websites. And I started to get interested in Python about 2006. And then really, really got into it in 2007 when I discovered Django, and I just fell in love with Python at that point. I think one of the things that stands out to me with Python is its emphasis on readability. And especially with Django, I could almost like into it, the API and a lot of cases, it seems so easy to get into and guide you towards best practices.

Tobias Macey

0:03:33

And you mentioned that now you're working in the team to help other engineers and teams at wayfarer discover and use Python for their particular projects. I'm wondering if you can start by describing a bit about the overall mission statement for you and your team at way fair and some of the origin story about how your group got started?

Unknown

0:03:53

Sure. Yeah, definitely. So we consider ourselves like effective in our mission. If engineers Whichever about over 2000 of it way fair and growing, if they're focused on actually delivering, like business value and focus on their, their the task at hand and not fighting with like standing up blogging, or figure out how to get a project running in production or anything along those lines, we want to get them as quick as possible onto like the meat of their problems and, and pave those roads. So that's like, really what we're trying to do is reduce friction and get people adopting best practices to be really effective. And how we got into that is, it's kind of interesting is a little happenstance little. So maybe I should describe the scene at the time. This is a just shy of three years ago, Python had 12 apps running away fair, and give or take. And it was getting to a point where those applications really didn't have like a dedicated team behind them. And I happen to be in the meeting, we were talking about this. And as we were describing, like what this team would be at the time was leading a very Java centric team. I heard what they're asking for and I'm like, Oh my god, that sounds like that. My dream job to like figure out how to support and grow this Python ecosystem at wayfarer. So a friend of mine and organization who came from the same startup as me, we got together we talked about, like how we could actually do this. And we we just got into it and had to figure out like how to create an actual like Python platform team from essentially absolutely nothing and try and grow adoption. And you mentioned that at the time you first started the roughly 12 apps being built and maintained away fair, they were using Python and morning if you can characterize some of the different ways that it was being used, and some of the other primary languages that have been common throughout warfare. So there's a lot of PHP at warfare. There's also I'd say Java, obviously javascript.net. And Python was a bit of an edge case at the time, it was used predominantly to production as a data science model of some type. It came in, I believe that the very first Python app at wayfarer was essentially a way of serving as a data science model to The e commerce experience. So you can imagine like your NM PHP app on the front end. And behind the scenes, it's proxy that to a Python services, a flask, a flask service in the background. So that was like how Python first came to the organization. It's grown quite a bit since then, since we started to like, really make it easier and easier to use. But yeah, at the beginning was really about serving data science models, to various types of systems.

Tobias Macey

0:06:26

And for programs and projects that are engineering focused that way fair, what's involved in the overall selection process for determining which languages are a good fit and what the overall technology stack is? Is that something that is sort of designed by committee or is each team left to their own devices to figure out what works? Is there sort of an approved list of technologies that can be used? I'm just curious how that manifests that wayfarer, specifically.

Unknown

0:06:52

Yeah. So I'd say it's a bit of some of those, mainly the ladder to generally teams have fair degree of freedom to figure out what as the best fit to purpose. You know, we encourage everyone to be very pragmatic about that. Some languages have different, say, characteristics and strengths and and drawbacks, maybe really strong open source libraries, that would be a good fit. But really, we want to trust teams to kind of use their best judgment. But we do have like a kind of a list of technologies, bit of a tech radar, if you will, of things that are paved and battle tested. And we know we're going to like production is fine. And we tend to like ask people to choose from that list, which you know, ideally, that list is ever changing as we adapt new technologies and figure out you know, what is a something to lean more into? Or maybe we don't way from?

Tobias Macey

0:07:44

And then in terms of how you engage with the different groups and engineering teams throughout wayfarer, I'm wondering if you're focused primarily on Greenfield projects, or do you also embed with existing product teams to figure out whether to either incorporate Python bits of tooling or to replace bits of their application. I'm just wondering if you can sort of characterize some of the decision factors that go into who you work with and for what particular end goals?

Unknown

0:08:12

Yeah, maybe actually, if it's helpful American, walk that back a little bit the so we engage with teams in a variety of ways, which we got into a little happens naturally. So when we created when we first got the platform team off the ground, we I, you know, as all my Python experience, as it may be, I actually hadn't worked out a Python team at the company before. So I didn't really know like, what the pain points were, what problems we actually needed to solve to be really effective. So we had this idea of like, well, what if we just spent a decent time, you know, just working with teams on their projects. So imagine like one person on the back end, trying to like, smooth out points of friction and one person going out and just like pairing with teams on what their problems are seeing where the friction is, and then bring that down. Back to the team also maybe like giving someone some, you know, expertise on their project to learn from. And so we ended up like leaning into that pretty hard. It worked really well. And we developed a program out of it called og. So we're engineering. So you can imagine half of the platform team, roughly spending between like two and three quarters a year, working with teams on projects as our as part of those teams. So that's, that's probably our largest investment. And where we take we tend to spend a lot of times like planning that out setting expectations and those, those engagements last about a quarter and then we walk off you can walk all the way back to light touch things such as ad hoc support office hours, where we encourage people to come up in and chat with us or just dig in with the team on our on a random question. The auxiliary engineering one though, in particular, we've, we've got a lot of value and mileage out of that. And it's become a pretty integral part of how we how we run our platform teams internally

Tobias Macey

0:09:59

and Are there any analogous groups throughout wayfarer that are serving to support other technology stacks or languages? Or is this something that's unique to Python?

Unknown

0:10:09

Well, it used to be a neat Python. It was something that was going well enough that they, you know, it, we decided to apply the same exact model to other languages. So I think about a year and a half, if I'm remembering this correctly, after starting Python platform, we spun up PHP platform. And we have been trying to build out largely the same type of team and overall at the same model and that ecosystem, and we're about, I think, just about a year into building out a similar ecosystem over there.

Tobias Macey

0:10:37

And for the groups that you work with closely. Can you talk about some of the just overall day to day of how your team works with the other engineers and some of the sort of the, like the typical engagement and maybe some of the common points of friction or confusion that exists with people who are familiar with other languages tried to adopt Python?

Unknown

0:10:59

Yeah, definitely. So maybe I should start by, I'll just say here. Here's what the general lifecycle of one of these engagements looks like. We spend about a, I think about three or four weeks trying to find a good group to engage with we we asked, we asked him to basically solicit us for one of these. So it's generally a team approaching us saying, hey, we'd like you to come and partner with us on this MVP or this or this critical project, they would do that because you essentially you as a team asking for that get a very experienced software engineer on your team to help you with a with a project. So that's very attractive. And we we tend to look at these in three dimensions, business impact, technical impact, and team culture fit. So we're essentially trying to hopefully get like a little over what we can take on in terms of bandwidth and choose the one that is highest leverage or impact across those dimensions. Once we find that we do a round of expectation and goal setting where we just make sure that we're well aligned. We know what success looks like. We know what pitfalls we're going to we want to avoid. And then the engagement lasts about a quarter give or take a few weeks. And during that time, we we do weekly retro shows, we do a project retro at the end, it's pretty intense and ambitious. And we're trying to bring a lot of change into the team, while we're also trying to achieve these really ambitious business goals. And during that entire time, we're also learning a ton firsthand about our own platform. My favorite example of this would be if you go and work with a team, and they have a new engineer that just joined the company, maybe a junior engineer, their senior, your ecosystem for the first time you see them struggle on something. And it's really hard to feel that pain. Like as maybe someone who's very familiar with Python, or specifically Python at wayfarer, you see that and you say that's an opportunity for improvement. We make sure that our engineers have like 20% time to just jump on those things without having to go through any process. And you know, if you fix it that week, and you get back to the person say hey, so I struggle with this. With this maybe documentation or automation of help, that feels great. And that person sees that, you know, hey, I, you know, being vocal and just like sharing my pains and going through with the team lead to actual meaningful change very quickly, in terms of failure modes that we try to avoid, or where we've seen where we struggled, the there is really I think of like three projects that come to mind. One was, more or less on us. We we undersized pretty significantly the the challenge of migrating something from two to three. And we still had a really big impact. It was just that that process took up a lot more time than we thought it would, mainly because there was a lot of dependencies that also had to be brought over in that migration to get over the finish line. And the other two challenges we've faced were largely, I would say, externalities, one team lost a few key sponsors on their side of things right after the project got off, and the other one, the engineers that are supposed to work with us on project had to stay on a previous project because it was having some issues getting over the finish line. And, you know, maybe wouldn't want to sabotage that project to to just to get the new one up and running, I probably wouldn't be the right thing to do. So some of those are we have lessons learned in general, the main takeaway we've had is if we do you see an issue with him being able to engage the team effectively, and some of the failure modes hit the surface, we want to be pretty aggressive about calling it early on to the engagement. So because at the end of the day, our team is super large, it's, I think it's like five or six people. So you want to be very thoughtful about where you're where you're spending your time. And if you don't have that full buy in from the team, you know, calling it really generally is the right thing to do.

Tobias Macey

0:14:42

And in terms of identifying people who would be strong candidates to work in your team to help proselytize and and educate people on proper uses of Python and how to engineer at appropriately. What's the sort of selection criteria figuring out if they would be a good fit and some of the overall skill set either technical or otherwise, that you found to be most effective.

Unknown

0:15:08

Yeah, definitely. So it's, I would say it's a, it's a pretty interesting team to be a part of, we deal with a lot of ambiguity, it's a lot oftentimes not as clear cut, as, you know, make this change to hopefully make, you know, the product more effective. Sometimes you really have to figure out where the points of friction is or the ambiguity in your own platform. So we'd like people who like are feeling really comfortable just figuring it figure out that ambiguity space and and our Okay, navigating it. Also, we found that it's like, obviously, somebody that enjoys helping someone else, either through education or training or pairing to become more effective because we are a meta team like people who really feel a sense of reward and seeing others become more effective and successful that that tends to pair real Well, with with these types of with our platform team in these types of engagements in general, I'd say our team generally skews pretty senior as a whole. And we, we treat when we try to attract the best Python engineers in the organization to this team so that we can figure out how to replicate their knowledge as effectively as possible across the entire organization.

Tobias Macey

0:16:25

And then within your team, I'm wondering if you have found that there are some other useful training materials or sort of product scaffolds that you can use to hit the ground running when you're working with other groups or useful tools or libraries that you've been able to leverage either within projects that are helpful for you internally to have sort of reusable patterns or tools, again, particularly helpful for some of the other teams that you've worked with.

Unknown

0:16:55

So that's, that's, I'm glad you asked that because this is this is one of those funny stories. Were very early on, we had this example project that we had written for, like, Hey, here's how you can structure flask and, you know, use blueprints and get the best, like project structure versus like maybe, you know, throwing everything into a couple files unstructured, like, you know, which is almost like, I think by default, if someone's coming into class, they tended to create like these, like, Oh, just a monolithic files that just had everything and that that's what we were seeing some of so as we as we helped give some guidance there. We then realized, well, people are like copying and copying this template or not template, but example project. They're just like replacing their own stuff in it. cookie cutter is an awesome framework for this, why don't we just build some cookie cutter project templates for people to use? And at the same time, we were thinking, Oh, everyone has to figure out logging and metrics. Why don't we write some libraries and write a really well, so they're extremely intuitive to us. We'll call those like core libraries. And that'll mean like, oh, if you're creating your project, like Here's a logging library it just make solves it all for you, you can get up and running. And you know, logging is kind of a just good to go. Same with metrics, and a few others. So the combination of those two ended up incredibly powerful. The teams are just like clone a cookie cutter project. And, you know, as we spin up Docker, whatever it was, we started to add into the template, like they just get all that for free. And that cut through so much boilerplate for the teams that they really became a core strategy for the platform where we're also building that up in other ecosystems now as well just like core libraries, plus a cookie cutter template. And I think it like you can really see the mark that it left on the organization because generally if you walk around and ask some about project templates, they get called cookie cutters, just because that's the framework that we happen to use. And I think not a lot of people realize that that was the name of the framework versus like what we were just calling the project templates. So cookie cutters, private quite a well used

Jonathan Biddle

0:19:01

temp lighting system that way for these days?

Tobias Macey

0:19:05

Yeah, so it's it's got the band aid problem where the brand name has become synonymous with the actual

Tobias Macey

0:19:14

And then as far as some of the particular types of projects that you've been engaged with, I'm wondering if there are any sort of common patterns or categories of things that you have found that other teams have been asking for help with or if it's sort of all across the map and just some of the interesting or unexpected types of projects that you've been engaged with there.

Unknown

0:19:35

There's really been a lot we don't we typically don't use Python like and in the customer facing experience like the the e commerce experience, that's that's very much like a very massive PHP application. But behind the scenes, it's used all over the place and oftentimes what you'll you'll end up seeing is a team has maybe I seen another team be really effective. Say Standing up a, a Python application to consume and leverage like Kafka queues, for example, and they see that go really well. And they are. They're interested in that. But they might not have the existing experience on the team to just like go with it taken on themselves. So that's one thing that we've definitely supported a lot where there seems to be like Python is a great use for this problem. But we don't have people today who really understand the language. It'd be awesome to have someone come from the platform team, just like partner with us for a quarter to get up to speed. I think a lot of our big success stories look something like that.

Tobias Macey

0:20:42

And then you mentioned that, at this point, you're up around five or six engineers when you started, it was yourself and one other individual so I'm wondering how the overall scale and scope of your work has changed and some of the evolution of your stated mission and some of the ways that you're Trying to make yourself known within the organization so that people are even aware that you exist as a resource for them.

Unknown

0:21:07

Yes, we actually got started with three people. And early on, it was very much like you can imagine you inherit this platform that's kind of been more or less built out over volunteer time from other teams. And we've pretty much start in like, like, behind the curve, underwater stage. And we had to figure out how to, you know, put out some existing fires but also looking at like, what do we what do we think this platform can be in the future? So right out the gate, what we did was we we lean into, like 20% time, so every Friday was pretty much like self directed forward looking. So no matter more or less how underwater we felt like we were, we were at least carving out a day a week to really look at the future. And then we tried to strike a balance between, I would say, you know, just just kind of like pushing the platform forward and Making sure like what we're doing is actually providing values to teams and it feel like for about a year or so, it was a lot of iteration on that. We we started to grow the team, we had another person joined who helped us get the engineering program off the ground really where we started like for formalize it, and it felt like the the value we were providing teams and the leverage we are giving them was starting to become more and more recognized. And within a year and a half actually, like the team had developed a brand you saw like auxiliary engineering become something that not just engineers are talking about, but product people were talking about because they're like, wow, you know, this, this platform team came over with our and they work with our team and I suddenly like saw a lot of really strong outcomes. They were moving so fast and and some of it was because we were bringing expertise, but also being close to infrastructure, we could help them cut straight through. Maybe some of the stuff that might be confusing if the first time you have to interact with it. Today, I would say it's it's actually a pretty integral part of wayfarers tactical strategy is to figure out how to how to grow this platform focused effort in a way that really, really maximizes how much time engineers spend on delivering, you know, or focusing on their, their core projects and minimizing how much time they spend on toil to use a word and, and things that are ultimately like, not directly related, but generally they're like, have to be done. Again, like maybe standing up, or figure out how to like get like the logging stuff integrated. We just want to keep people focused on the real the meat of our problems. And I think that's at this point, just really widely recognized and the branding and marketing side of it internally, is like I felt like we've kind of grown past that being an issue at this

Tobias Macey

0:23:54

point. And then for the teams that you're working with, particularly those who don't have Any in house experience using Python, I'm wondering what you have found to be some of the common stumbling blocks or points of confusion or pain points that they've encountered while trying to adopt Python and build things with it or incorporate it into their existing infrastructure and products.

Unknown

0:24:21

Python has a pretty good testing culture these days. And I actually didn't recognize this until I had a chance to compare it but you know, I was kind of one of those one of those Python engineers that really liked and, and, and kind of like looking at type annotations and, you know, kind of embracing them so I'm like, really, I kind of stuck it out for like, though, I like my my dynamic systems. And, and what I didn't realize is that I was relying on very strong test suites to give me some of the same assurances that you know, like a static read typed compiled language would give you so when people are coming from like a Java or.net similar language they, you know, they've kind of squint at Python a little bit and go like, like, what do you Yeah, there's the I think we've all been engaged in this conversation before, but get into the understand like, Yeah, right, you know, you Python has type annotations. Now you can kind of should lean into those, in my opinion. But your test suite is also like Python has an amazing set of tools for writing some really, really, like high value tests not not tested just give you like, an uptick in your coverage, but actually, like, hopefully break at some point and save your butt at some point. So that's one that we really try to like, like help people figure out code review is just appear, you know, has nothing to do with Python. But we've noticed like spraying some code review best practices is a pretty big lift for teams. We tend to advocate that like code reviews. Like if you if you have good lifting pipelines, like you pull style feedback, how to code reviews, if you treat code reviews is like really urgent like even more emotion than writing your own work, then like you don't have things getting bottlenecks and code review and like teams are generally amazed at how fast they start to move once they remove that bottleneck. And I would say one one thing I shout out to jack Diedrich is the his stop writing classes talk. That's, that's one of my personal favorite Python talks. It changed the way I write Python. When I first saw people again, coming from like a, like a very Java like language tend to want to like put one one class in one file. And, you know, if you watch that talk it you know, getting teams think about Python as like, well, you don't actually need to use a lot of classes. You know, your files are there to help you organize things, not necessarily using the same way that you see in Java. bringing some of those changes into the team tends to be a contrarian point. But one that like once you get someone else to really see the value of like concise code that's very easy to read and contribute to it. generally takes.

Tobias Macey

0:27:05

And then in terms of the projects that you have worked on and built up, what are some of the notable successes? And what are some of the cases that you've run into where you ended up having to abort the engagement because of either issues at the technical level or organizational pushback or just a poor team fit?

Unknown

0:27:28

Yeah, we've we've had quite a few wins. We I think that there's something done somewhere between 12 and 15 of these engagements, a number of them have exceeded even exceeded the goals of the engagement and we were into like, stretch goals. By the end. We've worked on systems ranging from computer vision, our our pricing systems, finance, we have a lot of our search tech is Python, and we've worked in that space. You know, there's there's a pretty interesting thing. From around like how you actually move products to customers houses and we've we've worked with that we've recently even built out systems that are part of a pretty critical workflow the the order processing pipeline actually takes a completed order and does all the necessary things with it. So we've worked with quite a few, I would say there's there's actually only been one engagement where we ended early. And that was, that was one where ultimately like the the team that was we were supposed to pair with wasn't they were stuck working on a different project and weren't able to, to engage and everyone to think about the priorities of these engagements. We explicitly set them as priority number one is transform the team in some way. So make them more comfortable working with Python as a technology or some imagine like some type of behavioral change that we want to stick past the engagement. That first is probably Number one, number two is actually like effectively hitting the project goals. And that's very explicitly order in that way. Because if, like if the if the auxiliary engineer is just taking on all the hard work themselves, that's actually not like for us, that's a failure, that's not going to actually have a lasting impact on the team, the way we would want. So we made in this case, likely, it's important that we kind of like, pull the ripcord because we're ultimately not able to hit number one, if no one's there working with us on the project.

Tobias Macey

0:29:28

Another thing I'm wondering about is if there have been any common patterns as far as the groupings of languages that the teams you're working with, have been familiar with, and if you have seen any patterns as far as language camps that have refrained from working with you, because they're happy enough with their own feature set and they don't feel the need to explore Python.

Unknown

0:29:52

It's a really interesting question. I haven't I don't think I've actually thought about that before. I don't I think we see a pretty broad mix. I think Maybe, you know, if I had to, it's really hard because at the end of the day, PHP is very prevalent language of the organization. So we do see a fairly large number of PHP apps who are like maybe switching to Python for something. But that's that might actually be a proportional to how much PHP is used. So I don't think I have a great answer there. But I do think it's a general mix. And we haven't actually, at least not to the point where we've said, Oh, people who are using.net refuse to write Python, not nothing like that has really stood out at any point for us.

Tobias Macey

0:30:33

Another thing that I'm curious about is because of the fact that you're working with people who are familiar with all these other different languages, if there are any design patterns or lessons that you have brought back into your work in Python and your team's work in Python, as a result of working with those other teams and some of the ways that they approach technology and system and software design.

Unknown

0:30:55

Yeah, I think so from a software and systems design perspective, nothing jumps out. I, that doesn't mean that that hasn't happened. But nothing's just coming to mind at the moment, but I definitely say we, it is a bi directional knowledge share. When we're engaged with the team, we learn a ton from the teams, especially around our own platform. And I'm sure we, you know, we consider it a goal to absorb best practices from teams. We don't view ourselves essentially coming into the team. As you know, some elitists is actually there's some attention behind what we call auxiliary engineering and avoid some of the names that tend to imply supremacy even if implicitly. So, yeah, we're definitely looking for like, what we can learn what we can learn our platform, our platform, and then how we can spread those best practices to maybe the next engagement or to ourselves, and I know that's happened quite a bit. The ones that come to mind are largely around you know, we We've identified some, you know, some rough patches that we weren't really fully aware of where it said, we work with the team, and they're like, oh, wow, you know, get it getting up and running with this one particular thing was a lot more difficult than we would have hoped. And we hear that in a retro. And we really encourage that type of feedback, because it's impossible for us to see it for the first time. And so when someone tells us that we can like say, oh, great feedback, and then we can prioritize smoothing that out, and we definitely, that becomes a huge factor in how we set our own priorities and goals.

Tobias Macey

0:32:32

And so given the fact that you have been using and working on building with Python, and encouraging the use of Python at wayfarer for a few years now, I'm curious how changes in the language and ecosystem of Python have changed the way that you approach it as a team, and also how the overall usage of Python at wayfarer has evolved over that same time period.

Unknown

0:32:56

So I think when we first got started, you know, there was just those handful of applications that you know we were supporting them that they're the 12 or so. And one of the things that we leaned in on very early was Docker eyes. Like Docker I called Docker eyes development environments and Docker ici pipelines, the applications themselves ran on a VM and production. But we wanted to make it easy for people to develop locally. And we decided like Docker was one of the best ways to make that development environment, fairly reproducible, and easy to engage with, that actually positioned us really well to adopt Cooper daddy's early on as we started to explore that. And Python was the first language to get into Kubernetes. And from when we started that today, we went from being the first to having over 115 Python services running and Kubernetes. And so I'd say like, we've seen the ecosystem grow tremendously. I don't remember how many people were in the Python Help channel when we got started. But I remember looking at this just the other weekend, it was, I think, over 500 people interest in that channel. And that's a channel use for like asking other teams that are using Python help questions in general, just getting like kind of cross team guidance or opinions on things. So we've seen quite a bit of growth. It's been really exciting personally,

Tobias Macey

0:34:15

for any other organizations or companies or groups who are interested in building a similar type of team or organization to encourage the use of Python and educate other engineers. What are some of the lessons that you have learned in the process or tactics that you found to be particularly useful or helpful or any other advice that you might give?

Unknown

0:34:38

Yeah, totally. This is a great question. Because I in retrospect, there was a lot of pitfalls that we tried to avoid that I think could like sabotage a team's effort as I try and navigate this. First of all, I think I get the impression that there's a lot of companies that have probably had some Python week into their ecosystem just because of the the data science traction that Python has. And like at some point, you have to like figure how to production eyes that Python becomes like maybe a pretty easy path to go down. So like, I'm willing to bet there's a lot of organizations that like, already have a little bit of Python and similar wayfarer for years ago to figure out, Okay, what do we do with this? How do we, how do we do this properly? I would say this will be by guidance, every ecosystem is going to be different. And what you want to do is, in my opinion, don't sit on the sidelines and trying to think of what the perfect solution would be, go and just work with the teams firsthand and see where they're struggling. take that to heart and try and make that easy. Figure out how you like, build some advocates, both on the engineering but also on the product side when they see like Python being used, you know, to maximize its strengths, which I would argue if someone asked me personally, why use Python. I think Python has this great story. Of being like, Sure, it might not be the most performant language and languages VO straight up benchmark. But Python is so fast to get up and running and iterate on. If you're if like speed to market is at all a concern, I think Python becomes an extremely strong option. In fact, if someone asked me very personally, I generally say that, like someone, someone has to convince me Why not to use Python because it's just so such a good general purpose language. And there are plenty of use cases where it's probably not a good fit. But I want to see that case made before I personally would jump for granted. I'm biased. I've been, you know, playing with Python since 2006 or so. But it's a it's such a strong language. I think there's a lot of organizations that will find that very, very appealing. And my final advice would be, make sure you view the team as never dependency OR gate keeping team. You're there to pave roads and help other people figure out how to become more effective. In fact, if you become a dependency at any point, there's a serious risk that you will become swarmed with that, that type of activity and you actually become incapable of moving forward in the platform. Let's say if, if you had everyone come through you before they like went out to production or something like that, like that backlog could build up really fast, potentially. And now you're spending all your time doing those activities instead of actually automating and innovating on things.

Tobias Macey

0:37:27

And are there any other aspects of your work on the engineering team or your experiences of helping other groups within way fair learn and use Python that we didn't discuss yet that you'd like to cover before we close out the show?

Unknown

0:37:42

Yeah, so there's, I would say there's one one thing that we did that was pretty cool early on. You can imagine a world where you have dozens and dozens of teams, all with maybe a couple of people doing something with Python. And early on, we felt was really important to bring all those teams together all those people using Python and we're here together and kind of build a sense of community. I think I find in general, at least if someone's involved in the in the Python community, it tends to tend to be very welcoming and supportive. And I find that that that trait also leaks onto the the people who use Python at work. But what we did is we we created an opportunity for teams to just come together and share what they were working on in Python. And then we gave out a bunch of bugs that were co branded with like a Python logo and a wayfarer logo, a little gimmicky, but you'd be really surprised I think how those mugs became a pretty sought after item where people would kind of like, see it on your desk and know that I got someone doing something in Python and you know, you go and talk to them. So there's I think there's a lot of times opportunities to build that community, especially if something is really dispersed layer and usage throughout the organization and not like there's like a center of excellence already for it. So thinking a little outside the box about stuff like I think it matters. And it really helps. So we saw it become really successful. We actually saw at one point someone like trying to figure out where the mugs came from an offering to like, buy someone for buy it for $10. And, and we just had a rule of like, if you want a mug, just come and talk to us, and we'll give you one. But yeah, definitely like try to find a way to build a sense of community, we did that it seems to really pay a lot of dividends for us. And I bet the same story will play out and pretty much any organization.

Tobias Macey

0:39:26

All right, well, for anybody who wants to get in touch with you or follow along with the work that you're doing, I'll have you add your preferred contact information to the show notes. And so with that, I'll move us into the pics. And this week, I'm going to share a podcast that was started by a listener of this show, who contacted me to say that, through listening to the episodes, it motivated him to start his own thing. So it's a show about learning Basie and statistics. So I'll add a link to that to the show notes for anybody who wants to check that out. And with that, I'll pass it to you Jonathan, do you have any pics this week?

Unknown

0:39:57

Sure. I'll throw out to That I, that we really like a bucket to the first one together, which is like pedantic and fast API. We really liked those. We have a lot of flask apps right now. And for a number of them, I feel like pedantic and fast API might be a bit of an upgrade. So we're trying to lean into those and make those a little easier to adopt internally. And so far I I personally really like that, those frameworks and I think they they seem like they have some pretty bright futures. The other one that I would throw out there is MK docs, if anyone hasn't checked it out, we you know, personally, it's been my favorite framework for getting documentation up and running, especially with like, some of the some of the themes that are out there like reuse material lot. We we made that easy to use. And we found that teams just started like, gravitating towards it like wildfire, because it makes it so easy to get some really, really attractive looking documentation up and running quickly.

Tobias Macey

0:40:58

All right, well, thank you very much for taking the time today to join me and share your experiences of working on the engineering team at wayfarer and helping other groups throughout learn and take advantage of Python. So I appreciate your efforts on that front and I hope you enjoy the rest of your day.

Jonathan Biddle

0:41:14

Thanks, Tobias. Thanks for having me. This is awesome.

Tobias Macey

0:41:20

Thank you for listening. Don't forget to check out our other show the data engineering podcast at data engineering podcast com for the latest on modern data management. And visit the site of Python podcasts. com to subscribe to the show, sign up for the mailing list and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email host at podcast in a.com with your story to help other people find the show please leave a review on iTunes and tell your friends and co workers

Liked it? Take a second to support Podcast.__init__ on Patreon!

Accelerating The Adoption Of Python At Wayfair 1





Разместим вашу рекламу

Пиши: mail@pythondigest.ru

Нашли опечатку?

Выделите фрагмент и отправьте нажатием Ctrl+Enter.

Система Orphus