DevShop Stories #11 - The Cost of Skipping User Acceptance Testing

TO THE GOOD STUFF

DevShop Stories #11 - The Cost of Skipping User Acceptance Testing

In this episode, we cover user acceptance testing. Which covers user an important part in the software development process. It involves the customer reviewing the software, comparing it to the requirement documents, and ensuring that it meets their expectations.

Listen on Apple

Listen on Spotify

Listen on YouTube

[00:00:00] Josh: Welcome to another episode of Dev Shop Stories. My name is Josh and I have Kai here with me. And today we're gonna share a story about what is a product owner. And so we have actually a couple of stories here that we wanna share of past experiences. We'll just call it company A and company B to keep the innocent, out.

so we're the first one, company A was a, a big project that we had. And what was kind of interesting was we weren't fully in charge in the sense of we didn't have project manager capability. We didn't have. Kind of a direct tie-in to the stakeholders that were, that were there. We kind of had to go through, a third party.

And that's how it actually started was it was a third party even outside of that organization that kind of came in and did PMing and kind of just told us what we did. And, we actually started following that and started working on what they told us to do and, and that that relationship was actually pretty good.

They, gave us concise tasks and we were able to work on them, and I think they were happy, but then things went awry. A kind of third-party PM group came out, and the company, the client actually brought in their own resources as project managers, Kai, you remember that?

[00:01:10] Kai: Yeah. Actually, it was one of my first projects, I was involved in and, I remember the whole transition time.

I remember there were a couple of weeks where it was a crapshoot, to be honest. A lot of things were being built but not being approved by one party or another. And the ultimate underlying feeling was we just needed direction. And so, when I stepped in, it was probably about halfway through the project and we were.

kind of just cleaning up everything that we've been given from that last build and we're looking to build, kind of the next steps.

[00:01:40] Josh: Right. And Kai, so your role in the, in the company is you do a lot of project management and a lot of kind of oversight and direction there. So in this instance, Kai was more of a scrum master. Was this kind of your capabilities and roles? Well, you still had to answer to the PMs that they had brought in new, new staff that were actually hired, right?

[00:01:59] Kai: Yeah, yeah, definitely. In fact, actually, when I first jumped on the project, they had hired a product owner slash manager, and then.

Two days after the hiring, he quit and left, and then we had to sit around for a couple of weeks and work with their CTO actually to keep things moving. But there was a little bit of not too much direction and then they hired someone else after.

[00:02:23] Josh: And so while you're working through this, there was a couple of times where we actually go try to do a two-week demo and, and try to show it to the executive team. Sometimes the executive team members were there. Sometimes they weren't able to attend. But you remember one instance where you actually were demoing this to their CTO and their you know, the head boss there, right?

[00:02:43] Kai: Yeah. So it was kind of a similar thing to that during that time period. Their CTO kind of guided us and they actually gave us documents and information about what we should build for that sprint. And so, we followed their direction. We had everything ready. We QA'd everything was working perfectly. And when we did the demo, at the very end, their main stakeholder, their C EO Looked at it and said, thanks so much guys.

This looks great, but why did you build it? This has nothing to do with, our strategy.

[00:03:07] Josh: Yeah. Yep. So, and then kind of what happened there was the CTO on the same call said, no, no, I think this actually is what we want to do. And, they kind of had like a little argument back and forth right in front of us and

[00:03:20] Kai: it was a kinda weird little kid watching their parents bicker and, yep.

Yeah, we just didn't know what to do. We built exactly what was to design and documents, but it just really highlighted the need for a product owner.

[00:03:34] Josh: Right? And, we're gonna take a moment here in a little bit to explain what is a product owner, what, what we kind of expect out and, and kind of six steps to becoming a successful product owner.

but continuing with the story. That project continued on. They actually got a couple of project managers on there. They worked with Kai and they had our dev team running and, and kind of got to the very end of the ready-to-launch. And again, we actually walked through kind of their back office, their admin site, and demonstrated again, here's what the project managers told us to build.

We built it exactly to the spec. we showed it again to the CEO. And what was his response?

[00:04:12] Kai: He was furious. Yeah. He was so mad at me cuz it wasn't what he wanted to be built. It was, it was a completely different strategy than what he had envisioned. but at the same time, when they came back and were asking to review designs and documents, they couldn't come up with a single point to us that we veered off and did our own thing.

[00:04:31] Kai: They can only blame their project managers for really, giving us the work that wasn't approved.

[00:04:35] Josh: And of course, their project managers threw us under the bus and basically were pretty quiet when, when the CEO exploded on us. Right? Yeah. So company B though is a little bit of a different story. in this case, we had built, a product for them, over a series of three to four months and essentially, worked with them, did the requirements, did UI design, showed it to the owners of the company, kind of walked through it was trying to spend a lot of time as much trying to extract what their vision was, what they want.

And we actually were going to launch it. And what happened was they said, Hey, we need to delay the launch by a couple of days cuz we need you guys to come up to our office and show us how to use it. Right?

[00:05:17] Kai: Yeah. So throughout our development process, we always do a demo. After every sprint and we kind of walk through the client, you know, what's being built, what features are coming up, and they, towards the end, started becoming less and less available.

I remember it was pulling teeth to try and schedule those demos, and even when we had those demos, it was usually pretty short of, Hey guys, this looks great. Let's keep going on with what we're doing. And so exactly to what Josh is telling here. I remember having a meeting where they asked us to come over to their office.

a couple of days post their launch date and say, Hey, can you come in and just teach us how to use the system? Because clearly, we missed a few things and we need to understand what's, how to make things work.

[00:06:01] Josh: Right, yep, so that kind of begs the question, what is a product owner and how could it have helped in these, you know, scenarios that we had here?

[00:06:09] Kai: I think the key thing here is to remember we're the development team. We're a development partner, meaning we, work with you and look at the strategy that you're giving to us and try to build a product to match that. if there's no strategy, if there's no coherence there, then we will struggle.

And so to that effect, the difference between a product owner versus even like our project managers, the product owner. Is from the client side, the one that's gonna be, kind of like the champion for the stakeholders, for the clients, for whoever's gonna see the product. And then the project manager's gonna be the guy that's gonna be executing.

On those and making sure we hit those timelines.

[00:06:46] Josh: Right. And in the case of a product owner, it usually is not the person that kind of came up with the idea. Right. It's, yeah. A lot of times you see these companies, you'll have like a creative that sits at the head of the company as CEO and he just gets, you know, he or she gets this wild hair and says, what if we built an app that did this?

You know, and, they're also in charge of so many different things that. Work of making it turn into a materialistic thing and, and actually execute on it, gets handed down to somebody else in, in that organization is ideally what we have. I mean, sometimes we have worked directly with that CEO level and got the idea, but that becomes extremely difficult because, they say, well make it look like Instagram, but add Tinder into it and add LinkedIn and might as well add Pinterest, you know, kind of thing.

And I've got this great idea. It's like, well, let's, let's boil that down a little bit more and a little bit more granular. And so, the product owner is somebody that's on the client's side that is gonna do those negotiations internally to try to help us answer and extract requirements out that we can actually build against.

[00:07:51] Kai: Exactly. and to that point, finding the right person is, critical. And there's gonna be six steps that we're gonna go over here on what makes a successful product owner. but within each one, it's a big job. And it's something that we as Redsky can kind of help but we can't do for the client. because we're not the company.

We don't know the strategy, we don't know the reason for the product. We only know what they tell us.

[00:08:14] Josh: Yep. So the six steps to a successful product owner. Number one is to identify the right person. So this person needs to be able to show up to all the meetings that we need to. They need to be able to answer those questions.

They need to go and actively reach out to the stakeholder and make sure that, their needs are being met. As well as when the dev team or the project manager on our side on Red Sky Engineering's side happens come across, they can, basically have a shared relationship. Like, Hey, you go do your stuff on your end.

We'll do our stuff on our end and then we'll let's meet again, you know? Right.

[00:08:49] Kai: Exactly. It's the counterpart to our project manager. which kinda brings us to step number two, which is organizing and prioritizing. So as the product owner, you're gonna be working with the stakeholders within, your company.

You're gonna be identifying the key features that's gonna make your product stand out. You're gonna identify the requirements and ideally work with our designer to get those initial designs out. part of this is understanding from a multitude of people that there's gonna be requirements coming from your marketing team or your finance or, from legal.

And it is, it's ultimately up to the product owner. To kind of boil these requirements down and then prioritize what's most important for this project.

[00:09:33] Josh: Yep. step number three is they need to have a buy-off from the stakeholders. So each round of a sprint cycle, so like a two-week cycle, they need to go and make sure that they have, essentially the requirements and the buy-off from the stakeholder.

These are the things that are most important that we work on this, this round, right? So they go out, they get those approvals. They also might want to go out and elicit customer feedback to kind of show ideas to the team and ensure everything is meeting the core needs of what this project needs.

[00:10:02] Kai: you know, had we had that in that first story, we would never have had that problem.

You know, just that communication between the product owner and the main stakeholders, right? It's huge.

[00:10:15] Josh: and in, in reality, I would venture to say that if they can't get that time from the stakeholders, they probably shouldn't be doing that project at all. I mean, it's just gonna be a lot of anger and resentment later on.

And this isn't what I planned on, this isn't what I, I envisioned and everything. But if, if you can't give the time to the stakeholder, does the stakeholder give the time to the product owner? It's just not worth even doing the project.

[00:10:39] Kai: Absolutely. You know you can dream up an idea and then tell someone through a, phone call and hope this person gets it Exactly right the first time. I mean, it's, it's very unlikely. So we'll continue with, our steps. So step number four set milestones. we've walked through the steps of we found the right person, we've organized the project, and we've got buy-off from the stakeholders. Setting the milestones is now deciding, all right, how do we want to approach getting this product out?

Do we want to do this in different phases? Is there a launch that's gonna require all the features or only some of the features? We recently just had a project that actually had a big part of their admin site, and decided that, Hey, we don't need this part for launch. We want to get this product out as quickly as possible so we can understand user behavior and then iterate, and then we can adjust our admin site however we need.

[00:11:35] Josh: Right. Step number five is execution. So one of the things that we insist upon at Red Sky Engineering when we're doing these is to have a biweekly demonstration, and that way we can make sure that everybody's still in sync. We can show demos, we can get the stakeholder excited, and we can make sure, essentially, that, the product owner's doing their job.

Because if we actually get to that two-week cycle and find out, That we have these blowups, as we had with company A. We can start reevaluating, is this the right product owner? which kind of leads into the next meeting that we normally have after a demo, which is kind of a, an executive meeting.

So here we bring in, you know, maybe the sales agent that helped secure the deal. We'll bring in the stakeholder and we'll just kind of say, all right, how are things going? Is this meeting your expectations? You know, are we performing? Let's solve these issues. Let's get, the hard conversations out of the way now so that it doesn't become, a massive blowup at the end.

And then everybody's fighting for who did what and who said what. Let's have these little touchpoints that kind of go along there.

[00:12:41] Kai: So yeah, I definitely have noticed a huge difference. I don't know if you have Josh. Well, since we've started implementing that, but it seems like ever since we have had executive touchpoints in addition to our regular meetings.

Have any little items that might have had the potential to blow up have been ironed out? And then the minute that those stop happening is when we start to notice some of these problems creep in, or, you know, differences, in ideas.

[00:13:05] Josh: Yep. And obviously, we're, reiterating the project timeline with them and that the objectives are met for that, with the client.

[00:13:12] Kai: Yes. And so rounding it out to the next step, which would be the launch. So at this point, we've executed it, and it's been demoed. You know, we're, we've reached our miles down. This launch phase is, is pretty heavy on the product owner, meaning that there's gonna be a lot of extra work for them. a lot of it's gonna be reliant on them getting feedback from their stakeholders, from their customers, making sure everything is up to date and ready for their launch.

So to kind of deep diver or dive deeper into this, this is gonna look like setting up a beta group. This is gonna be soliciting beta feedback from these people regularly, conducting your own user acceptance testing, and then any huge or even minor changes that need to go in the launch, planning that out so we have enough time to get that in.

And then retested prior to that launch.

[00:14:08] Josh: Yep, absolutely. So I'm gonna kind of reiterate these six steps with you guys right now. So step number one of becoming a successful product owner is to identify the right person. They, again, need to be able to attend meetings and all that kind of stuff.

Step two is they need to be able to organize and prioritize work. Step three. They need to have to get that constant buy-off from the stakeholder. So they need to be willing to go in and, and say, Hey, is this right? And be able to have the attention of the stakeholders. Step four is they need to be able to set milestones.

Five is they need to be part of the execution of this project going through from each phase to each phase. And then step six is they need to be able to facilitate a successful launch, which includes setting up beta testing groups and, and kind of post-launch behaviors, right?

[00:14:58] Kai: Yeah, absolutely.

[00:14:59] Josh: Well, I hope that gives you a good look at what a product owner is and how to be successful at it.

At Red Sky Engineering here, we help train product owners in their roles, cuz the better a product owner is on the client side, the better we will perform on the Red Sky engineering side. So thank you for listening to our stories. We'll be back next week with more stories, personal experiences, and advice on running a dev shop.

Thanks.

Also Check Out

Planning - The most Crucial Step In Your Project

RedSky discusses what you can expect when we take you on as a client. In this video, Kai Schulze goes over our Planning phase and explains to you our onboarding process.

DevShop Stories #10 - The Story of Why We Do Fixed Bid

In this story, Josh and McKay, explain the importance of having a fixed bid and a clear definition of what "done" means in order to prevent misunderstandings and frustrations.

What to Look for When Selecting a Software Development Shop

Software is ubiquitous in business. Whether it's to streamline processes, improve customer experience, or to gain a competitive edge, software is a critical investment for companies looking to grow and succeed.