Hey guys, this is Tosh. I'm going to show you a quick demo of how to set up readable streams using the OpenAI chat completion endpoint. So this is a little bit tricky, it took me some time to understand, but basically, we're going to

set this up using a React.js front end and a Node.js server. And I have a Next.js app here, and I'm going to walk you through the two files that we need. And I'll link to all the code at the end as well. So let me show

you a demo of how it works. So I'm sure you're familiar with ChatGPT, but we have a very simple app where we click this button, the submit button, and it streams the response on the front end, and we can get the response word by word.

So obviously, this is a much better UX than waiting till the very end until everything's sent. So there's two parts to this, a server-side and a client-side. So let's go over the server-side first in our Node.js endpoint. So I have a simple example here set up

for you, and I'll walk you through it step by step. We've basically imported the OpenAI Node.js library, and we've added our API key. And this is the endpoint here. It's pretty simple. We set up a chat completion, we use GPT-3.5 Turbo, we add a

system message, a user message, and importantly, we set the stream here to true, and we set the response type here to stream. And after we get the response, we process it all in this part of the code. And basically, what this is doing is it's

getting the response, it's getting the data from the response in chunks, and then here we're console logging the chunk. And I'll show you what that is here. The chunk is a buffer, which is basically a hexadecimal representation of the data that we actually need. So once we get the buffer

which is the chunk, we turn it into a string and we split it by the new line, and that's what payloads is here. So we can see that this is the payload, this data and then this object here. And once we get the payload, we loop

through it and if it includes the word "done", like this, which we can see at the very end, it has this "done" here, we're going to send a response to the server to the client that we're done, and then we're going to return

if that's not the case, then we further process that payload here, which basically just means we remove this data part, which is this. Let's see, we remove this, and we're left with an object which we parse, and then we just get the text from

data.choices.delta.content, which is this choices.delta, the content. And this is just a period here, but you can see he's one Z years, this one's coming, this one's the, and then we send the text to the client. And so that's basically all that

is, it's pretty straightforward. We're getting a buffer, turning the buffer into a string, looping over the string, getting the text, and then sending the text to the client. And let's go to the client folder now, which is page.jsx, and this is just simply React code, it's pretty short.

so all that's going on is we have a button I'm going to handle click and this is just using the fetch API to make a post request to that endpoint that I showed you on the server and then it awaits the response and it reads the response to this code here

so the response is being read it's piped through a text decoder stream which is native to the browser and just allows us to turn this into readable text and we process it here we console.log the value and then we set the value to the text

that we get and so that's basically it hopefully this was pretty straightforward and understandable but basically you just need two files a server endpoint and a React front end and I'll include a link to the code in the description as well so you

hopefully that was helpful and better understanding what's going on and how to stream

Streaming OpenAI Chat Completions Using React and Node JS

Tosh Velaga

OpenAI를 사용한 판독 가능한 스트림 소개 및 개요

Tosh가 React.js 프런트 엔드와 Node.js 서버로 OpenAI 채팅 완료 엔드포인트를 사용하여 읽을 수 있는 스트림을 설정하는 방법을 시연합니다.

Next.js 앱을 사용하며 나중에 제공되는 코드 링크와 함께 필요한 두 가지 파일을 설명합니다.

더 나은 UX를 위해 간단한 앱 응답을 단어 단위로 스트리밍하는 데모를 보여줍니다.

서버 측(Node.js)과 클라이언트 측의 두 부분에 대해 설명합니다.

OpenAI Node.js 라이브러리를 가져오고, API 키를 추가하고, GPT-3.5 Turbo를 사용하여 채팅 완료를 설정합니다.

stream`을 `true`로 설정하고 `response_type`을 `stream`으로 설정합니다. 청크 단위로 응답을 처리합니다.

청크를 버퍼(16진수 표현)로 설명하고, 문자열로 변환하고, 줄 바꿈으로 분할합니다.

페이로드를 반복하여 "완료"가 포함된 경우 완료 응답을 클라이언트에 보내고, 그렇지 않은 경우 추가 처리를 진행합니다.

데이터.choices.delta.content`에서 텍스트를 추출하여 클라이언트로 보냅니다.

클릭을 처리하는 버튼이 있는 간단한 React 코드.

fetch API를 사용하여 서버 엔드포인트에 POST 요청을 합니다.

응답을 읽고 텍스트 디코더 스트림을 통해 파이프하여 처리한 후 표시된 텍스트를 업데이트합니다.

프로세스를 요약하면 두 개의 파일(서버와 React 프론트엔드)이 필요합니다.

설명에 코드 링크를 약속합니다.