Hey guys, this is Tosh. I'm going to show you a quick demo of how to set up readable streams using the OpenAI chat completion endpoint. So this is a little bit tricky, it took me some time to understand, but basically, we're going to

set this up using a React.js front end and a Node.js server. And I have a Next.js app here, and I'm going to walk you through the two files that we need. And I'll link to all the code at the end as well. So let me show

you a demo of how it works. So I'm sure you're familiar with ChatGPT, but we have a very simple app where we click this button, the submit button, and it streams the response on the front end, and we can get the response word by word.

So obviously, this is a much better UX than waiting till the very end until everything's sent. So there's two parts to this, a server-side and a client-side. So let's go over the server-side first in our Node.js endpoint. So I have a simple example here set up

for you, and I'll walk you through it step by step. We've basically imported the OpenAI Node.js library, and we've added our API key. And this is the endpoint here. It's pretty simple. We set up a chat completion, we use GPT-3.5 Turbo, we add a

system message, a user message, and importantly, we set the stream here to true, and we set the response type here to stream. And after we get the response, we process it all in this part of the code. And basically, what this is doing is it's

getting the response, it's getting the data from the response in chunks, and then here we're console logging the chunk. And I'll show you what that is here. The chunk is a buffer, which is basically a hexadecimal representation of the data that we actually need. So once we get the buffer

which is the chunk, we turn it into a string and we split it by the new line, and that's what payloads is here. So we can see that this is the payload, this data and then this object here. And once we get the payload, we loop

through it and if it includes the word "done", like this, which we can see at the very end, it has this "done" here, we're going to send a response to the server to the client that we're done, and then we're going to return

if that's not the case, then we further process that payload here, which basically just means we remove this data part, which is this. Let's see, we remove this, and we're left with an object which we parse, and then we just get the text from

data.choices.delta.content, which is this choices.delta, the content. And this is just a period here, but you can see he's one Z years, this one's coming, this one's the, and then we send the text to the client. And so that's basically all that

is, it's pretty straightforward. We're getting a buffer, turning the buffer into a string, looping over the string, getting the text, and then sending the text to the client. And let's go to the client folder now, which is page.jsx, and this is just simply React code, it's pretty short.

so all that's going on is we have a button I'm going to handle click and this is just using the fetch API to make a post request to that endpoint that I showed you on the server and then it awaits the response and it reads the response to this code here

so the response is being read it's piped through a text decoder stream which is native to the browser and just allows us to turn this into readable text and we process it here we console.log the value and then we set the value to the text

that we get and so that's basically it hopefully this was pretty straightforward and understandable but basically you just need two files a server endpoint and a React front end and I'll include a link to the code in the description as well so you

hopefully that was helpful and better understanding what's going on and how to stream

Streaming OpenAI Chat Completions Using React and Node JS

Scripts