Software Development: The Challenge of Deep Knowledge

Current software engineers often lack deep knowledge, resulting in widespread shallow coding practices.

### The Challenge of Deep Knowledge in Software Development
* 💻 "Be the change you want to see in the world" resonates deeply, emphasizing the **importance of maintaining high coding standards** for others to follow.
* 🛠 The barrier to entry in software development is low, resulting in a prevalence of **bad code**, which is **more common than good code**, highlighting the difficulty of **creating quality libraries and abstractions**.
* 📉 There is a lack of clarity about what constitutes a "**respectable engineer**," unlike in civil engineering, where **responsibility and expertise** are paramount.
* 🤖 The rise of AI-generated code poses risks, as many developers may **not understand or review the code**, leading to **potential complications in the future**.
* 📚 A significant issue in programming culture is the distinction between **deep and shallow knowledge**, with many developers lacking a **strong foundational understanding** of fundamental concepts.
* 🔍 Developers often skip learning core programming fundamentals, such as **JavaScript**, in favor of frameworks like **React**, which can lead to a **lack of understanding of underlying mechanics**, like the **virtual DOM**.
* 🌊 The influx of new developers since the programming boom around 2005 has resulted in a **flood of shallow knowledge**, as many focus solely on **learning APIs** without grasping the **deeper principles** behind them.
* 🔄 Ultimately, while the software industry continues to evolve, the challenge remains in **fostering a culture of deep knowledge and understanding** among developers to **improve the overall quality of code**.

so our friend Jonathan Blow is back, and he's going to blow us all away again with his hot take: current software engineers have no deep knowledge. I think there's a pretty serious problem that started happening in programming culture. I mean, it's been in programming for as long as I've been in, and this is beyond games even; this is everywhere. People don't have a clear picture of what knowledge is deep versus what knowledge is shallow, right? So, if you go to school and you're a good student and you learn the difference between n log n and n squared, right? That's deep knowledge that will apply in many cases, and it's something fundamental about computers, right? Um, on the other hand, if you go on Hacker News and read an article about somebody learning a specific JavaScript API and tutorial-ing it, right? That's very shallow knowledge because it's just, you're learning about arbitrary decisions that somebody made about how to interface with a thing that you're not even trying to understand, right? And

that but that knowledge. So, I have a couple of points here. One, everybody hates algorithms and data structures, but I think one thing you can't deny is that those topics are very fundamental. And that's why you see them applied in so many areas, like sliding window is not just something used to solve leak code problems, it's used in many places, data processing, rate limiting, a lot of places. But the bigger point, what he said, nowadays I see it manifested a lot when it comes to React developers, to be honest, because the first thing somebody does when they want to learn front-end development is they see React, that's the number one framework. But it's almost like they skip a step, they don't even learn JavaScript fundamentals. I don't know how you can possibly call yourself a React Dev and not know anything about the virtual DOM, for example, and why it helps like improve the performance of larger front-end apps. Sometimes I feel like most people don't even know why they're using a framework anymore, especially beginners, they just do it.

because everyone else is doing it. If you have a really small one-page app and you need a little bit of interactivity, why don't you just write vanilla JavaScript? Oh wait, they can't. They don't know how. They've never directly manipulated the DOM before, and it's kind of embarrassing, and I mean that in the nicest way possible. If you're a beginner, I'm not trying to pick on you. I'm just telling you, you should learn to walk before you can run, 'cause it's going to catch up with you in the long run. Same thing with open-source development. Before, it used to be you need to do LeetCode to get a job. Nowadays, it's you need to do open-source software development to get a job. And so, you'll see beginners asking, and people have asked me this question, How do I get into open source? and they barely know how to code. They haven't done personal projects. They learned one language, never learned Git. God help the projects that they end up contributing to, because it's going to be a mess. Now, I do see the

other side obviously understanding the virtual Dom is not going to necessarily allow you to actually build an application, but like high-level knowledge of JSX, how React works, that stuff actually will allow you to build an app. So that's like the first thing people are going to go for, which is understandable. So even though I'm ranting, usually there is a reason for everything. It became very popular since, let's say, since 2005, once this second boom of programming happened, and everybody realized, Oh, you can make a lot of money in programming, or whatever. All these people showed up, and they just want the job where they could do JavaScript, and so they just learn the API, right? And so there's been a deluge of shallow knowledge. I wonder if Jonathan Blow considers front-end developers lower tier, and for the record, I don't. I did front-end development. Front-end development can go surprisingly deep, but I thought that was an interesting take. Seems like he's calling quite a few people out. Um, it's a little bit more

complicated than that because part of the way that you learn deep knowledge is by looking at shallow things and reading the pattern underneath. Like if there's an API to do something, if it's a good API, then by looking at the API choices, you can understand the problem and the mechanics, even if you can't see the code. It's better if you could see the code inside, but if you can't, you can still see some things about the mechanics. And you know, one example of that is a graphics API. It's like, okay, they want me to put the vertex data into a buffer and lock the buffer and then not touch it. Why is that? Oh, it's going to the GPU and all this stuff, right? You can figure this out. On the other hand, if it's a bad... I'm not familiar with GPU processing, but there's a similar pattern when it comes to big data processing. After each step, the data itself gets locked. One of the reasons is it's happening in parallel. So I think what he says

definitely applies. Patterns are so much more important than like the actual smaller details. API, which most APIs are, it'll be full of stuff that's stupid or even cargo culting where they're copying a good API. So, my favorite example of this was when Microsoft did Direct 2D, which was taking Direct 3D, which was successful, and then trying to add font rendering and all that to that. They made an ARGB color, right? So, red, green, blue, and alpha. A lockable resource. So, to create one of these, you had to lock it and fill the color data and unlock it. And I'm like, you have no idea what you're doing. And I actually told Microsoft this, and they did not like my opinion on that. But they were just copying, Oh, Direct 3D has you lock things and fill them, and so let's do that here. And it was completely absurd. So, I think he's absolutely right that this kind of stuff happens at big tech companies where

you wouldn't necessarily expect bad abstractions to exist 100% even at Google. I've talked probably enough about that. For the record, good abstractions exist as well, and that's what most people will hear about. You don't really hear about the bad abstractions, but they're everywhere, unfortunately. So how to fix it? There just isn't much of an idea right now of what an actual respectable engineer is. You know, like if you think back to civil engineering, like people who build bridges or whatever, you think of people who are responsible and know things well and make sure they're right because it's important to be right. We don't really have that in software at all. It's completely dead as far as... and the funny thing is AI is probably not going to help that because, okay, got a bunch of AI-generated code, thousands of lines, well, it works. I'm not going to read it. If you ever need to have an actual developer go in there, I mean, unless the AI can just literally do

everything unless it's AGI. At some point, you might need a developer to go in there and fix the mess that the AI made. It's going to be pretty damn hard if it's nonsensical code, as I can tell. And part of the reason it's dead is because nobody can tell what's right because it's just a bunch of people arguing in a loud room, and like no real information gets through. And if anybody has an idea how to solve that, I would love to hear that idea. I don't know. The only way that I know to solve it is to just do better in an obvious way. But even that maybe doesn't win, right? Because all these just do better JavaScript Frameworks are still popular even though they're all, yeah, the same as each other, right? You have npms to just to pad things because no one wants to write code anymore. In the same way, the market can remain irrational longer than you could remain solvent. Um, I think software industry culture could remain stupid longer than

you're alive, so I guess what he's saying is, be the change you want to see in the world. Just make your code a high standard, and then others can kind of follow that. So, just don't create garbage packages. Given that it's like a free world and anybody can become a developer, it's pretty easy. The barrier to entry is relatively low. I don't think there is a solution to what he's saying. There's probably more bad code out there than good code, and it's not easy to create good code. To be fair, creating really, really good libraries that can be extended and used, and providing good abstractions, is hard. That's the problem. It's not that developers are dumb and they're just always shipping really bad code; it's hard to do what he's saying. Like, that's just possible. So, I'm doing what I can do, but I'm very conscious that I have a limited ability to affect this giant thing.

Programmers in 2024 have no Deep Knowledge

NeetCodeIO

Jonathan Blow's Critique of Software Engineers' Knowledge

Jonathan Blow argues that current software engineers lack deep knowledge, leading to a cultural problem in programming.

He distinguishes between deep knowledge, like understanding algorithm complexities, and shallow knowledge, such as learning specific API tutorials.

Blow criticizes big tech companies for copying successful patterns without understanding their fundamental principles, leading to ineffective and absurd decisions.

The Challenges of Learning to Code and Open Source Development

Many beginners struggle with basic coding skills, such as manipulating the DOM with vanilla JavaScript, which can hinder their progress.

The shift in job requirements from LeetCode to open-source development has led to beginners seeking contributions without a solid coding foundation.

The chaotic nature of open-source projects and the overwhelming popularity of similar JavaScript frameworks contribute to the ongoing challenges in the software industry.

The Popularity and Perception of Front-End Development

Front-end development has gained popularity since 2005, driven by the programming boom and the desire for lucrative jobs in JavaScript.

Many developers focus on learning the API, resulting in a surge of shallow knowledge in the field.

The speaker questions the perception of front-end developers as lower tier, arguing that front-end development can be surprisingly deep.

The Importance of Understanding Fundamentals in Programming

Deep knowledge is gained by analyzing shallow things, like API choices, to understand underlying mechanics.

Algorithms and data structures are fundamental and widely applicable, influencing various areas beyond coding challenges.

Many aspiring front-end developers skip learning JavaScript fundamentals and the virtual DOM, leading to a lack of understanding of the frameworks they use.

The speaker emphasizes the importance of setting a high standard for code to inspire others, but acknowledges the difficulty of consistently producing good code.

With a low barrier to entry for developers, bad code is more prevalent than good.

The lack of a clear definition of a respectable engineer in software contrasts with other fields like civil engineering, and AI-generated code may not solve the problem of poor abstractions.