There are hints to understand the direction of stock prices, and you can hear directly from companies through investor relations information. It's available in the fixed comment section. When designing AI chips, Jim Keller from Tenstorent mentioned that instead of using HBM, they could use graphics DRAM like LPDDR. Now, let me ask this at home: with the increased number of NPUs like those from Tenstorent, it might be challenging to see the demand for HBM grow significantly beyond expectations. It has to be the case, right? The important thing is that it doesn't matter how many of these chips come out; ultimately, only one needs to succeed. Throughout history, there have been many GPU companies, but now NVIDIA dominates the market, while AMD is left behind. This is because if one succeeds, the demand will follow accordingly.

It doesn't make sense for many chips like this to be released if even one customer experiences a significant increase in learning or inference speed, by dozens of times, with TensorTorrent. If that happens, then this kind of effort, which is essentially the programmer's struggle, becomes meaningful. Right now, when I run learning on an NVIDIA chip, I load the data and train the GPU. When the HBM data comes in, I simply call for it to load the data, and everything is automatically handled in the HBM. Then, when I ask the GPU to perform calculations, it transitions smoothly. I can multiply a million numbers, and the results come out immediately. I don't know how it works internally, but NVIDIA's chip manages the HBM well. However, if I want to use the TensorTorrent chip, I need to specify the data location myself.

First, I need to input the questions that the user will be asking. Then, I will specify the first layer of artificial intelligence, the second layer, the third layer, and so on. The NVIDIA chip is generally well integrated into this, so the question is how we can maximize its performance for specific purposes. We also need to present successful case studies. That sounds interesting. However, recently Jim Keller visited our country, and several companies, including Hyundai, are trying to attract investment, right? So, our large corporations are investing, saying that it's the new semiconductor era or the legendary semiconductor. But whether this will succeed or not is still uncertain. I think it makes sense if the target of the investment is planning to use this technology. The target of the investment...

For companies, such as Hyundai, it makes sense to invest in AI, especially if there are individuals who believe in the potential of this technology. Personally, I think that companies that are advancing in AI should invest in technologies like wormholes or tensor semiconductors. For example, when Tesla first released Hardware 3.0, they announced that their AI performance was significantly better compared to existing NVIDIA solutions. 

Now, suppose Hyundai is finally trying to create their own autonomous driving system. Let’s assume they are working on developing an autonomous driving box similar to Tesla's. What does this mean? It means they have decided on the type of AI that will control the vehicle. Although we can't be certain how well this will work, it is an important step forward.

I tried using the NVIDIA solution, and sometimes it may not work as they want. There are limitations to the performance that NVIDIA chips can provide. However, after testing the Keller chip, it seems like it could work based on the simulations. So, it might be usable. As for this part, there hasn't been anything concrete released yet, so I might not know everything. These companies are making astronomical investments, but they do it out of necessity, right? 

However, realistically, these chips require programs to be integrated. For example, if I have a program for NVIDIA, like a PyTorch program, and I use it here, all the interactions between the components happen automatically with my NVIDIA chip.

My program is finished when I write one, but this Tensor Torrent chip requires programs to be created for it as well. I need to specify the memory locations, and there is also a PyTorch program running outside of it. However, the programs that go into this chip are not in the domain of those who have been researching artificial intelligence; rather, they are closer to the field of embedded systems. For companies that have only been focused on traditional artificial intelligence, this may seem like a challenging task. Inside, there are elements that are quite complex, and it even contains a risk factor that is not commonly used. Therefore, only companies that are really good at programming should use this. There probably aren't many programs available to assist with this either.

Ah, so there is another type of program that needs to be developed already. There is a bit of a risk here; it's based on the risk factor. This is the buy risk. This can be utilized by companies that have experience with artificial intelligence and semiconductor technology. Purely artificial intelligence experts usually don’t deal with semiconductors; they just rely on them to handle everything. However, when trying to use these kinds of chips initially, they need to verify everything that runs inside. If we compare it to cars, it’s like someone who has driven an F3 car can drive an F1 car to some extent. In F1, all the controls are on the steering wheel, so someone with a bit of experience can drive an F1 car. But if a person who only drives a regular car suddenly gets into an F1 car, accidents can happen. Now, the next step for the Jim Keller chip is...

So, instead of the small RISC-V, we're adding a large RISC-V next to it. Yes, yes, yes, we've already seen this kind of design. If you look at the AMD MI 300, it combines the CPU and GPU together. Yes, it's the same concept. The next step is to change it to an AMD CPU with RISC-V architecture. What are they called? Yes, when you go to the granularity, there are additional components. With this, what functions are improved? The reason AMD created products that combine the CPU like this is that, in reality, tensor operations on the GPU alone aren't enough; the CPU needs to handle complex tasks as well when running artificial intelligence. For instance, certain activation functions and dictionary lookups are tasks that need to be done by the CPU when running AI algorithms. So, it can throw a request like, Hey, CPU, help me out for a moment, and then it receives a response.

I'm going to use RISC-V for that. But as expected, I can actually include x86 here. There's nothing wrong with that. If it succeeds, I believe there will definitely be people who will want to incorporate x86 into it. The reason is that the existing programs, for example, were built using NVIDIA chips, and the programs are designed to run on Intel x86 while communicating with NVIDIA CUDA cores. To use it as is, having x86 functionality is beneficial. Our combination is an Intel-NVIDIA combination, so it would make sense to include x86 as well. I think it's definitely possible; there's no reason it wouldn't work. 

Now, if RISC-V is successful in its ecosystem, it will have a significant impact. The meaning of using RISC-V here is quite different from using RISC-V in other contexts.

리스크 파라는 생태계에 솔직히 말하면 소프트웨어 생태계에 큰 보탬이 되지 않습니다. 왜냐하면, 이 안에는 몇십 줄, 몇 백 줄짜리 코드가 들어갈 가능성이 높기 때문입니다. 아주 복잡한 것이 들어갈 거라고 기대하기는 어렵습니다. 그냥 작은 방처럼, 예를 들어 20개 정도의 신호를 동시에 좀 효율적으로 처리하려고 하는 것입니다. 신호등 같은 역할을 하는데, 빨간 불이면 들어오지 말고, 노란 불이면 여기서만 데이터가 들어오고, 어떤 불이면 여기서만 들어와라 하는 식으로 통제하는 녀석이 들어갑니다. 이 안에 있는 것은 소프트웨어라고 부르기도 민망한 녀석입니다. 실제로 제가 아무리 좋은 소프트웨어를 여기에 넣어봤자, 누가 가져다 쓰기 쉽지 않은 것이죠. 여기서 돌아갈 프로그램은 실제로 일하는 프로그램들이고, 좀 복잡한 프로그램들이 여기서 돌아가겠죠. 고차원적인 문제를 다뤄야 하는 것입니다.

When the chip first came out, a major issue was who would create the programs that would be initially loaded onto it. In terms of performance, it might be similar; this one could be a bit better. However, as I mentioned earlier, modern CPUs, whether they are RISC or x86, are like different languages for humans. It’s just a matter of whether you use English, Korean, or Japanese. But, you know, languages also have their own advantages and disadvantages. For example, in English, the subject is followed immediately by the verb, making the action clear right away. When you say I love you, it’s immediately understood in context. However, in Korean, you say 나는 너를, which translates to I (subject) you (object). This structure can cause a slight delay in conveying meaning. There are similar differences in commands as well. But when we talk about CPUs...

What I hope for is that if we go all the way, we can do well in addition, subtraction, multiplication, and division. We should also be good at comparing two values and based on that, the program should give us the appropriate results. I expect us to do this quickly and accurately. In English, we use the term compare, which is 비교 in Korean. I'm not sure about Japanese, but that's the difference in terminology. It's really interesting. If it's fun, it might mean that I haven't fully understood it. For example, if someone says they understand quantum physics, but they actually don't, that’s a misunderstanding. I often feel like when I read newspaper articles, it’s just a jumble of words that don’t make sense. However, with a little more explanation from the side, I can now understand newspaper articles a bit better.

It seems that now, with Tensor Torrent chips, there is a somewhat extreme feeling in one direction, and there are many intermediate stage chips. AMD chips, for instance, can actually be seen as NVIDIA chips with just a different language. Other NVIDIA chips, that is, GPUs, are simply GPUs that haven't been integrated into the AI software ecosystem. Intel GPUs are the same; they are not significantly different from NVIDIA chips. Well, there are differences in design capabilities, and CPU communication might be a bit better, but essentially, they function as general-purpose chips. The forms are similar, and the data on those chips communicates with the chips and HBM, which allows them to run relatively well. The concept is similar; in a way, they could be seen as similar architectural structures, but...

Did it become an international sample? Yes, yes. Did it devour the ecosystem? That's the difference. So, it's just a matter of whether you proactively set up a mat and sat down in that spot or not. Ah, the Tensor Torrent chip couldn't set up the mat. It couldn't set it up at all. No matter how much Jim Keller is called the god of design, the elements that go into it will ultimately include multipliers, adders, memory controllers, and so on. Even if Jim Keller comes, he cannot suddenly increase the performance of the multiplier twenty-fold within the same silicon area. Everyone can design multipliers, and they can design adders and arithmetic units as well. Yes, everyone can design them. However, you cannot significantly improve those small components. Hmm, it's like I'm trying to make a refrigerator right now.

It's similar to creating a refrigerator that is ten times more energy efficient. While there may be differences in the global refrigerator rankings, there will still be a percentage difference, right? The reason is that the core components of a refrigerator, like the compressor, are all based on principles that are well understood. The arithmetic operations inside, like multiplication and addition, are also based on known principles. No matter how much of a design genius Jim Keller is, or how dominant he is in the semiconductor field, it's impossible to significantly improve components whose principles are already established. This is unavoidable. 

The floating-point arithmetic units, for example, have a history of over 30 years; they've been around for a long time. Improving each of these individual components is challenging. So, what kind of improvements are we looking for? This chip is designed because many of the arithmetic units created due to memory are underutilized.

I saw that while using artificial intelligence, it is possible to double the efficiency with the same number of operational units. So, how can we reduce the processing time? Jim Keller answered this question by saying that if we can control the data flow to a certain extent, we can ultimately reduce the number of back-and-forth operations. By doing this, we plan to perform all calculations of a data unit within the TPU and then send it to memory. From Jim Keller's perspective, HBM memory isn't keeping up, and he didn't like seeing the GPU idle. To avoid that, many companies are now using HBM. It's interesting, and I definitely understand it now. That's great! The thing is, people are really paying attention to semiconductor legends like Jim Keller.

He explained it several times through interviews, but I wonder how well people really understood it. I personally found it hard to grasp as well. Should I explain this? The understanding varies depending on the audience. We've been talking for almost two hours now, and it's quite confusing, isn't it? The reason is that I come from a liberal arts background, while the writer has experience in semiconductor development and artificial intelligence. He has been involved in the verification side, having seen a lot of verification software. Conversely, I have looked into some SOC firmware, so my knowledge is more aligned with verification. He has also worked in an AI startup, where it was necessary to handle semiconductors rigorously as a means of verification, rather than just looking at the manufacturing process.

While examining the software, I had to look into how it interacts with external chips. By the way, the book you published just before this was titled The Future of the AI Revolution. Yes, it has been a few months since its release, but it's still worth reading. Anyway, when I look at chips like Jim Keller's, there's something I always want to emphasize: what truly defines a good chip? This question has been around for a while. Jim Keller has successfully designed chips, and if we take a closer look, we can see that he worked on AMD's Ryzen design. The Ryzen itself was a competition to create a high-performance chip within the same ecosystem as Intel. This is because both Intel and AMD operate within the same software ecosystem.

x86 is a vast software ecosystem that has been established. So, if the design is successful, the chip will succeed as well. In terms of performance, it needs to run all existing Intel programs, right? And the same goes for Apple when they transitioned to ARM. When Apple made that shift, all the games and various apps that were running on existing smartphones had to work seamlessly. So, if the chip is well-designed, there are millions of potential buyers for it. However, the Tensor chip has a different challenge; the success of the chip itself is contingent on the success of the software that runs on it. This presents a much more difficult challenge. In a way, NVIDIA was founded in the late 80s, around '88 or '89.

NVIDIA has now emerged as the world's leading semiconductor company after struggling with Intel for about 30 years. It took them around that long to reach this level. Whether it was in the 80s or 90s, it doesn't make much of a difference, but it has indeed been nearly 30 years since the changes began. What transformed NVIDIA and Intel's standing? The answer is artificial intelligence. But who developed that artificial intelligence? Was it developed by NVIDIA? No, it was initially developed in Canada, primarily at places like the University of Toronto, after decades of research. The accurate theories were established there, and then those theories were applied to GPUs instead of CPUs. As a result, it became known that NVIDIA was the best for running artificial intelligence, and this reputation spread widely.

Torrent also has to go through a similar process. Ah, even without artificial intelligence, it can still be successful. These chips, yes, but I don't really know what they are. In fact, artificial intelligence can also run this quite well. If it's designed very well, the efficiency will be great. I can control all the data that goes into the cache, so the efficiency will improve significantly. But the problem is always whether raising the maximum performance by two times really makes our company twice as good. While you were speaking, I looked it up and found that NVIDIA was founded in 1993, which means I'm older than that. I need to correct this; otherwise, I’ll get comments right away. Ah, through writer Jung In-seong, I’ve heard a clear explanation today as well. Thank you very much. I think the humanities might be the most difficult field, right? I mean, writer Jung In-seong is actually…

You played the role of a bridge connecting the sciences and the humanities, and today was even better. It was the most difficult, to be honest. Yes, it was difficult, but it was okay. Well, you can just watch it twice; it's only about two hours long. But it is difficult. Is there really an easy way to approach this? It's like trying to chase away something that has already settled in. It requires a lot of research. So, if you realize that something isn't working out, you might think, I should try this instead, and then when the opportunity arises later, you can seize it. I believe that's a good way to approach it. Honestly, all these chips are created by looking at an uncertain future. However, we don't know what discoveries might happen in the field of artificial intelligence theory. The worst-case scenario is that development just stops at the current level. Right now, when ChatGPT gives an answer, it takes one second.

걸린다고 해볼 네, 근데 텐 스토렌 칩을 또 0.1초 만에 됐어요. 어, 근데 이거 사용자한테 영향이 있을까요? 그게, 그니까 안녕하세요. 뭐 이렇게 치면 그냥 대답이 나오죠. 여기 논문 요약해 줘야 하는데 1초 만에 대답 나오는 거랑 0.1초 만에 나오는 게 큰 차이가 있겠냐는 거죠. 그러니까 일반 사용자한테는 큰 의미는 없지만, 뭐 자율 주행이 됐든 아주 고도화되는 시스템에서는 그래도 비슷한 데서 대박이 나야 되죠. 그러니까 예를 들면 제가 어떤 로봇이나 자율 주행을 하는데, 이게 사례들을 쭉 봤더니 한 0.5초 내외로 의사 결정이 이루어져야 되는 게 엄청 많았다면, 이때는 반도체 자체 성능을 놓고 써볼 만하죠. 아, 근데 그런 발견이 없으면 왜 쓰죠? 막 이런 반응이 나올 수 있는 거고요. 이건 되게 아직은 모르는 곳이에요. 아, 너무 감사합니다. 

In testing it out, the ten-stolen chip was completed in just 0.1 seconds. But will this have any impact on users? For example, when

I had the pleasure of being with writer Jeong In-seong after a long time. Thank you. Oh, you worked hard. You really worked hard.

"쓸데없는 짓" '반도체의 신' 짐 켈러는 이꼴이 보기 싫었던 겁니다 / 정인성 작가 (4부)

"지식과 자산의 복리효과" 그 길을 복길이 여러분과 함께 걷습니다.
( 업로드 : 오전 11시45분, 오후 4시, 오후 7시 )
제휴 문의 : ystreet2021@gmail.com

인터넷 언론사 제호 : 와이스트릿 / 등록번호 : 서울 아53714 / 등록일자 : 2021.05.24
주소 : 서울시 영등포구 국제금융로8길 27-9 여의도 동북빌딩 910호
대표번호 : 02-6949-1701 / 팩스 : 02-6949-1702
이메일 : ystreet2021@gmail.com
발행인 : 이대호
Copyright © 와이스트릿  YSTREET ALL RIGHTS RESERVED. 와이스트릿의 모든 콘텐츠는 저작권법의 보호를 받으며, 무단 전재·복사·배포 등을 금지합니다.


"쓸데없는 짓" '반도체의 신' 짐 켈러는 이꼴이 보기 싫었던 겁니다 / 정인성 작가 (4부)

Scripts