Videos
Development - Where the Plan is Put Into Place
[00:00:00] Josh: Welcome to a special edition of Dev Shop Stories. My name is Josh and I have Tanner here with me. And today we're gonna go over the developer House of Horror Stories episode one. And so what we're gonna do here is we're actually gonna have a, a kind of a fun, lighthearted podcast where we're gonna go over a number of kind of interesting or, terrible stories.
And then these, some of these will come from personal experiences in our dev shop, and some of them will come from. Just reading posts online from reddit and such place. But this, this should be fun. So, I'll start off the first story here. this is a personal one where I worked at a company doing advanced radar algorithms and we had used a professor at a local university to help create these set algorithms.
Well, it was actually my task to take his MATLAB code, which is, you know, the code that he wrote, all these FFT and all these advanced algorithm analysis, and it was my job to take his code and convert it from MATLAB to C or C++, so it can run real time. What I didn't realize was how bad he was at coding.
Sure, it worked out. But his coding style was quite interesting in the fact that he used a single letter variable name for something that he's gonna use. So normally you'd name your variable, something that's, you know Related to ideally what you're, you're actually working on. But he actually chose, I'm just gonna, first thing I need to use it, I'm gonna use the letter A.
And then when he needed another one, he'd used the letter B and so on and so forth until he got all the way to the end of the alphabet at letter Z. And then he'd go back and his next one, it would be A1 and then B1 and it was extremely hard to understand what was going on cause you had no context and anything and you're, you're scrolling down to it and yes, you might think like, There's probably no way he's used 26 variables.
Well, he did everything in one function, so he didn't even use functions to call outside.
[00:01:48] Tanner: Seems super maintainable, so, you know.
[00:01:50] Josh: Yep. It was, it was a nightmare. So horror story number one. Tanner.
[00:01:55] Tanner: Yeah. That, that's awful. It's super maintainable code. this personal story here, we were hired to do some team augmentation and tried to implement code reviews into this process before they didn't have anything.
While in the midst of doing the code reviews, we noticed one of the other team's developers had been adding in checks with kind of the, the positive if clause or if portion as an empty body block, and putting the logic into the ELs portion. So he was really expecting his, if case, to fail and go into the L's block, where he would.
kind of the rendering. And this is like kind of what, what is this? You know, what, what are you doing? it's the, the most ridiculous pattern, of operation. He didn't know that he could invert the if check. and have just, just conditional rendering on stuff
[00:02:40] Josh: or, or maybe he thought like, at some day I'll add something into this top if area. Right.
[00:02:45] Tanner: Maybe. Yeah. Right. Like that. That's our hope. that he did it that way.
[00:02:49] Josh: Yeah. I mean, I'd hope that he's not coding in production code that, you know,
[00:02:53] Tanner: that's the worst, the worst part as it was. And it was all, yeah. And it was just this consistent pattern of this if block that would fail into an L's block.
Oh, it was bad. In that same instance, we had, that developer not wanting to type in their email and password, so he temporarily disabled the authentication portion and naturally forgot to take it out or put it back in rather, when he submitted his merge request at the time, the product owner was in a huge hurry to get the new set of features out.
So he kind of bypassed the code review process that we were trying to put in place and the QA team and the sandbox, and pushed it right to production. Needless to say, all you had to do was really click the button for login and you were instantly authenticated cuz there was no authentication into the e-commerce system. It was great.
[00:03:42] Josh: That is terrible, terrible, terrible .
[00:03:45] Tanner: Yeah, it was pretty rough.
[00:03:46] Josh: and we've all been there before, but The authentication system that just something that's you just would forget about and, and Oh yeah. And especially in like password managers that you have too, you know, they just auto fill in those forms for you directly.
There's, there's no need to do that kind of stuff anymore. Yeah. But then worst of all right, to just. Bypass all of those steps that, that should be in your process. Right?
[00:04:07] Tanner: Well not even go to sandbox, right? Like, Hey, prod, here we go. Here's this thing
[00:04:11] Josh: there. There's a great video on YouTube that we like to quote around in our dev shop, and it's, the developer, I'm a JavaScript developer kind of thing.
And, and he, he starts typing on code. Hes like, this isn't production code. He types it, but it will be tomorrow. Tomorrow. That's right. It's so good. Okay, well, this one comes from Reddit, so I'll just read this off. I was in charge of a redemption site for a very popular sports franchise video game on its release day.
I did not have appropriate permissions to my production database, so I was asked to build a few PHP pages, which when loaded, performed some really sketchy operations on the live database, including wiping This had been in place for QA and QA finished with only a few hours to go before launch.
Promotion began at 9:00 AM I was asked at 9:15 to wipe a database over my cube wall by my pm I misheard her and wiped production costing the company 16,000 in extra DLC Redemptions.
[00:05:06] Tanner: Yeah, that's I mean, isn't there, there're kind of a quote for that. if, dropping in your database?
[00:05:11] Josh: Oh yeah. Yep. what is that? It goes, so if having a cup of coffee doesn't wake you up in the morning, try dropping the production database instead. Yeah. You'd be like, cuz you know, instantly when you do that, you're like, oh, shoot, you know,
[00:05:24] Tanner: yeah. That is an instant, like, pucker effect. Like, oh man. What did I just do?
Yeah, pretty awful. okay. I'll take this next one. I think this one's also a Reddit one. told the client a rather large change would take three weeks. They didn't wanna wait, so they hired three additional developers. they thought they would learn the pretty large code base and make the changes in a week.
A month later, they were fired and the other, their devs and I did the changes in three weeks. I also undid every change they made and luckily it was all in get repo, so it wasn't a big deal. They were actually moderately okay devs, but had zero onboarding or support from me. They confidently told me m y assistance wasn't needed.
I literally offered to do it for free , so this is that like kind of classic, you know, nine women can have a baby in one month sort of thing. That mentality that always is out there.
[00:06:15] Josh: Yeah. You get the upper management, it's like, well we gotta go faster, so let's throw on more people and that's, that's ultimately gonna get it faster.
[00:06:22] Tanner: Yeah, that's, that's the thought is. If I just throw people against this thing, you know, it'll speed it up. It's like, well, only to a point. There's a whole lot of problems that are incurred and add additional, you know, stuff. It's awful.
[00:06:35] Josh: Oh, and we find that, I mean, if we hire on a new developer, we don't expect anything out of them for, a month possibly, or three, you know?
[00:06:41] Tanner: Oh yeah. There's a lot of onboarding and a lot of knowledge that has to be transferred, even if they're a really good dev. You know, it's like that ramp up time has to exist.
[00:06:50] Josh: well, this next one's kind of good, short and sweet. A coworker that I used to work with had superfluous conditional blocks to indent code and even documented.
So in other words, he purposely liked his code to be indented over sometimes and and multiple times. And so he would put a comment above that says, this is just here for indention. And he would say, if one, And then curly brackets. So purposely just wanted to push his code over, but there's no reason to actually have it there.
[00:07:16] Tanner: That's so bad. Yeah. Yeah. Good thing. It was a C-based language on it. It wasn't the python or something. That's funny. I had to manually slow down pages or page loads because. It was loading very fast and the fancy loading animation could barely be seen. So had some, some fancy Lottie or something that was trying to load and it was too quick, so they had to actually slow down their page loads.
[00:07:39] Josh: So management actually liked seeing a loading animation instead of the page just instantly loading. So he was, he was tasked. Purposely slow down your website so we can see that. Loading animation. next one. I left in a toast message I used when I was testing a frustrating issue.
Somehow I committed this and it went through the poll request review. Later someone was screen sharing with the product owners and it popped up !@#$% no ITO. Yeah, that'd be kind of embarrassing.
[00:08:07] Tanner: Yeah. Everybody's putting in some of those comments once in a while. Yeah. But you hope that you pull 'em out, . That's funny. All right. this one I was offered a very weird, but very well-paid WordPress project a few years ago. The previous developer had a fight with the boss.
So he quit. And from that day, the website didn't work anymore, even though the code looked perfectly. Okay. I cloned the production site so I could have a closer look at what was going wrong, and indeed nothing was working. But the code looked perfectly fine. There were thousands of errors on every page claiming that variables were unknown, except they clearly were declared and used sometimes on the line right above without an error.
High-disabled plugin. And stripped downward press to its lightest form, hoping to understand it all better, but still didn't see what the problem was. I decided to diff the files with those from the fresh WordPress install and noticed that some lines were showing as different, even though they looked exactly identical.
So I understood something here was fishy. I switched to hex and went, character by character, which can suck in U T F A by the way. And then I saw what was wrong. Some of the letters were not what I thought they were, but identical looking in U T F A equivalence. This person, had used his last, work week writing a script, a pretty smart one actually, that replaced letters in some variable names in some of the places where they were referenced, and he actually left the script containing the function and a reverse function to change it back well hidden.
in the assets directory. Well, that wasn't fun, but it was well paid.
[00:09:45] Josh: So that is, that is the ultimate like malicious compliance. Oh yeah. That guy was pissed. Yeah. Pissed. He's like, I'm gonna screw you guys . I know. And make it extremely hard. And that is, that is some clever stuff. You know?
[00:09:57] Tanner: I just want his script really
[00:09:58] Josh: You think about quitting Tanner? Yeah. Well just, yeah. And that's like you're saying, you would look at your code and be like, well, everything here is right. Why is not working?
[00:10:08] Tanner: Oh man. And those, those are like, that would be impossible to find almost. That would take so long.
[00:10:13] Josh: Yeah. That is, that is all right. Here's another one. I accidentally did an RM dot on the server and then clicked enter. So for those who don't know, RM remove and he's saying, remove the folder accidentally and clicked enter. before I could even think about what I had done, all of our websites were toast. I spent the entire weekend putting everything back together from backups.
[00:10:36] Tanner: Oh. So that is awful.
[00:10:38] Josh: Yeah. With Linux, know, some of these commands that you, you execute 'em, and they just instantly take off. You can do, for the RM example, there is a command line argument that you can question you. You're like, are you sure you really wanna do this? Or delete this file or delay this file, but I've done that myself.
Actually, one time I was gonna type rm dot slash and then like a path, and I just, I just typed RM slash and hit enter and it starts deleting from the root partition. And you can hit control Z. Whatever's gone is gone. You can, but whatever was there was gone. And so, yeah, basically a whole new operating system install again.
[00:11:13] Tanner: Yeah, that sucks. So I interned at a startup where the CEO was also the CTO, left work on Friday with fully functioning code. Came back on Monday to a full code change. The repo I was working on had been deleted and a new repo with new code had been made. I was required to start working on the new repo and try to integrate it with the existing dashboard.
[00:11:35] Josh: You gotta love that.
[00:11:36] Tanner: Oh yeah. If you wanna talk about micromanagement and kind of crazy, leave for the weekend and come back to a whole new set of code. Good luck. Mm-hmm.
[00:11:44] Josh: Yeah. Oh, maybe that's the guy that wrote the script that changed everything here. Makes sense now? Yeah, absolutely. All right, next one.
This was years ago in my first job outta college, working for a software company that provided managed e-commerce software that integrated with a retail point of sales system. This was before anything cloud. The client's POS system and database were onsite at their location. We were there setting up the e-commerce integration, which interfaced directly with the POS database.
The senior dev was at the council running some final setup commands. Their CEO, our CEO, and a bunch of other people were all there standing behind him watching as he worked. He needed to test something with a specific user, and I watched as he typed and executed the following SQL Command. Update customer set email equals test test.com.
Semicolon . For those who don't know, basically what that would've just done is updated their entire database of all their customers and set all of their emails back to test test dot
[00:12:46] Tanner: every single one.
[00:12:47] Josh: And it says the remainder of the day was spent helping the client restore their customer database from tape backups.
Ever since then, I always typed my wear condition before. Conditioning update statements.
[00:12:58] Tanner: Oh man, that is, that is so funny. I couldn't, I mean, I've done similar stuff. Not that bad. that's also why you set some unique constraints on things and try to architect your database a little bit properly.
[00:13:11] Josh: I'll always think twice when you do a, an update or delete. It's kind like the, you're working wood smithing and you always, you know, measure twice, cut once and, and same thing. And, and when it goes to databases.
[00:13:21] Tanner: Yeah. If I ever write a delete or an. 9 out of 10, I write the select against my wear condition just to sanity check my, you know, that, that data set.
[00:13:30] Josh: final one here. this one is actually not necessarily a horror story, but something that's just kind of a funny anecdote. Tanner, you wanna read this one?
[00:13:38] Tanner: Okay. So our build engineer has left for another company. The dude was literally living inside the terminal. You know that, type of guy who loves a vim creates diagrams in dot and writes wiki posts in markdown.
If something, anything requires more than 90 seconds of his time, he writes a script to automate that. we're sitting here looking through his, legacy code He has a shell. Smack my up. it sends a text message late at work to his wife. automatically picks a reason from an array of strings, randomly runs inside a Cron and the job fires off if there's active s SSH sessions on the server after 9:00 PM with his login.
I mean that's genius though, you know,
[00:14:21] Josh: random excuses for a while. I'm gonna be home at late and, and just, and it's automatically set to go off at 9:00 PM if he's still working there, which is crazy.
Yeah, that's awesome. Next one. Yeah, you can use so, so the next one is another file that he had on there called Kumar X. !@#$ hole sh. It scans the inbox for emails from Kumar. He's a DB, a DBA at our clients. Looks for keywords like help trouble, sorry, etc... If keywords are found, the script SSH is into the client server and rolls back the staging database to the latest backup, then sends a reply. No worries, mate. Be careful next time.
[00:14:59] Tanner: Hey, that's a, that's a genius. just, oh yeah. Automate his job. the next one is hangover. Another chron job that is set to specific dates, sends automated emails like not feeling well, gonna work from home, etc, adds a random reason from another predefined array of strings, fires if there's no inner active session. on the server at 8 45 in the morning.
[00:15:24] Josh: So got, he's got both his cases covered. So he's got his working late at, at night, you know, with his wife, or he's got his, you know, didn't make it in time in the morning.
[00:15:32] Tanner: Yeah. Yeah. I might, I might take that script.
[00:15:35] Josh: And then, so the the final one, says, and the Oscar goes to !@#$ coffee. Sh. This one waits exactly 17 seconds exclamation point, then opens a Telenet session to our coffee machine. We had no fricking idea that the coffee machine is on the network. It runs a Linux and has a TCP socket up and running, sends something like cis brew. Turns out that this thing starts brewing a mid-size half-cafe latte and waits another 24 seconds before pouring it into the cup.
The timing is exactly how long it takes. To walk to the machine from the dude's desk.
[00:16:11] Tanner: Man, that is awesome. How long it took to figure that out?
[00:16:15] Josh: Oh, I'm sure. All on company time.
[00:16:17] Tanner: Oh, absolutely. Yeah. I mean, he's, he's actively working on it apparently.
[00:16:21] Josh: Man. Yeah, so, well, we're gonna keep on doing these developer House of Horror stories in the future if you're liking them.
So again, thank you for listening to our story. We will be back next week with more stories, personal experiences, and advice on running a dev shop.
[00:16:35] Tanner: Thanks everybody.
.png)