Q&A: Nashville Software School CEO on how AI does, and doesn't, change coding – The Business Journals
Listen to this article 4 min
John Wark has been programming computers for 50 years. He’s never seen hype accelerate as fast as it has for today’s artificial intelligence systems that can generate computer code.
Those abilities now exist with tools such as ChatGPT — though Wark, the founder of the Nashville Software School, is confident they won’t replace software developer jobs “anytime soon.” (If you think he’s in denial, he has a response to that).
This interview has been edited for clarity and length.
NBJ: What’s your top-line takeaway as you size up ChatGPT and other AI tools such as GitHub Copilot?
Wark: They clearly already have some value, and they are going to change the work of software developers and data analysts and a lot of people in tech. But they’re not going to replace anybody. They’re not capable of that. These tools are not creative. They do not invent solutions.
They will lead to significant improvements in productivity for developers. But clearly, these tools need to be supervised by professional software engineers.
Some of these hype stories that run in some media, you get the CTO or CEO of an AI company saying we won’t need programmers in five years. That’s clearly — you can’t print that word, so I’ll just say it’s B.S. They’re just driving attention for their AI company and their valuations. This is not the first time AI has been hyped as killing programming jobs. But that is one of our challenges: The hype curve for this exploded in a way I don’t think we’ve ever seen.
You believe it’ll actually create more jobs? I think we’ll generate a lot more programming jobs in next five to 10 years than we’ll eliminate.
Think about the internet. It ultimately exploded the number of developer jobs, because it exploded the number of problems we could solve. It exploded new ways to solve old problems. Boom: Now we had e-commerce. Streaming. Online gaming. What are the new applications we haven’t even dreamed up yet that a new transformational technology will enable, and who will build those? It’s going to be software engineers and data analysts and data scientists.
Those people are professional problem-solvers, and we’ll still need professional problem-solvers to use these tools effectively. They’ll just not have to do all the grunt work, just like developers today don’t have to do the grunt work I did in the ’70s. It will replace the routine and repetitive stuff and allow us to use their creativity more, because we’re bogging them down less with the boring stuff.
What makes you so certain? There’s a new study [of almost 1 million developers on GitHub] which found that developers only used Copilot’s recommendation about 30% of the time. So that means 70% of the results were wrong, or inaccurate, or incomplete, or not useful. That isn’t something that will replace a software developer.
We’ve surveyed our alumni. About 60% are using these tools daily or weekly. One of the big responses we got was, “you can’t trust them.” They’ll say it took darn near as much time to check its code, to make sure it worked, as it would have just to write it themselves in the first place.
For these tools to generate good code, they need very specific instructions — much more specific than we’re used to giving development teams. Real developers fill in those gaps because they learn from experience. ChatGPT just does what it is literally asked to do, and it is 100% confident in its response.
Anyone who’s tried to generate real code from these things, you can’t trust them to generate clean code. Sometimes, that is user error, not asking good-enough prompts. That’s a skill we have to learn. But some of it just is, under the cover of all of its impressive conversational capabilities, it’s a statistical model based on analyzing hundreds of thousands of examples. If there are a lot of examples of what you’re asking it to do in that training database, then it’ll generate a reasonable set of code. If you said, “generate a WordPress plug-in,” there’s probably tens of thousands of WordPress plug-ins in that database. But if there’s not, or you’re asking it to do something new, it’s got nothing. It can fool you into thinking it knows more than it does.
At this point, some people are thinking “He’s just saying this because he runs a software school.” I’ve got my own bias, absolutely. I would hope people would question me on, aren’t you just in denial? Which I think is a fair question, and I would say, not based on a lot of this research that’s come out as people start to get real experience with these tools.
We’re so early in the deployment of this stuff. It doesn’t solve enough of the problem, enough of the time. We are not at a point where a user can say “build me an electronic medical records application” and get anything meaningful. We haven’t figured out where all the legal risks are, what the liability implications are if you depend on these things for answers.
Aren’t these tools only going to get more capable and more powerful, the more we use them? Isn’t it conceivable they’ll eventually be as good as a human programmer? The only way to detect which prompts you want to use, or detect that you’ve adequately specified the details so that you get good working code, is to be a software engineer. You have to know how to test the code, integrate the code, evaluate it to know, “This isn’t solving the right problem” or, “It’s 80% right, but there are some holes here.”
I have a strong belief it’ll be that way for quite a few years, even if the tools get better — because they will. But we have a long way to go. Is that in five years, 10 years, 40 years? That’s a great question. These are early days. Think about the last five transformational technologies someone’s been hyping. Google Glass? Yeah, that worked out. Blockchain? I don’t think that has hit the full potential everyone talked about. The internet? That one worked out, but look how long it took us to really adopt it, how much evolution had to happen before mainstream businesses used it in volume.
How are you thinking about this from an education standpoint? You are a school, after all. Oh boy howdy, I’m glad I am not teaching English classes this year. [For that] it’s like “here are a whole bunch of ways to cheat that we didn’t think of before.”
Students are curious, so they’re going to go out and try it. We’ve been wrestling with our policy on the use of these tools and how we incorporate them, and also what opportunities they open up for us. It’s an instructional aide, a learning tool, a productivity aide. But we won’t teach them to generate code with it until the very end of the bootcamp, or even in a follow-up to the bootcamp.
Local area tech employees
Growth of Nashville: The next hot spot
Massive transformation is on the horizon for Dickerson Pike.
2023 Most Admired CEOs
Tell us about a CEO who deserves to be recognized.
© 2023 American City Business Journals. All rights reserved. Use of and/or registration on any portion of this site constitutes acceptance of our User Agreement (updated January 24, 2023) and Privacy Policy (updated June 27, 2023). The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of American City Business Journals.
