Run OpenAI Models Locally with Ollama and Node.js

Last week OpenAI released gpt-oss-20b and gpt-oss-120b – 2 advanced open-weight reasoning models which are available to download and run, either on your local machine or on a self hosted server.

In this guide I will show you how to download and run the model, then interact with it using Node.js
Please note I’m doing this on a Mac – if you are using Windows or Linux the setup and commands may differ.

To do this I’m going to be using Ollama. If you don’t already have this installed you can grab it here https://ollama.com/download

Once you have it installed, it will open up the GUI. You can close this as we want to use the CLI.

Next you need to download one of the models – I would suggest gpt-oss:20b. It is the smaller of the two but it’s perfect for Apple Silicon and will run fine aslong as you have atleast 16GB of RAM. If you have a machine with atleast 60GB of RAM give the gpt-oss:120b model a go, but please note it is quite big.

Open up a new terminal window and use the following command to downlaod the model:

ollama pull gpt-oss:20b

Once this is downloaded you can start chatting with it on the command line – just type:
(wait until you see the >>> to start chatting)

ollama run gpt-oss:20B

We don’t want to chat with the model on the command line. To stop Ollama running type:

/bye

The run command directly starts a chat session with a specific model. Instead we want to use the serve command as this starts the server in the background allowing us to interact with the model through commands or other applications.

ollama serve

On running the command you should see a relativley long command line output stating things like the servers IP address and the amount of memory availbale and used etc.

Now we want to write a basic Node.js script – open up your editor and create a new index.js file. Ollama does have its own NPM pacakge for interacting with any modes, but I’m going to OpenAPI’s package.

Open a new terminal in the directory you created your new index.js file. Install the package:

npm i --save openai

Copy and past the below into your index.js file:

const OpenAI = require('openai');

const client = new OpenAI({
  baseURL: "http://localhost:11434/v1", - local Ollama instance and port
  apiKey: "ollama" // dummy key - no real key needed as this is running locally
});

const main = async () => {
  const response = await client.chat.completions.create({
    model: "gpt-oss:20b",  // this matches your loaded model
    messages: [
      { role: "user", content: "Who are you and what can you help me with?" } // the message you want to send to the model
    ]
  });

  console.log(response.choices[0].message.content);
}

main().catch(console.error);

In this Node.js script I’m sending a very simple prompt to the model: “Who are you and what can you help me with?”. Feel free to adjust this.

Now in the terminal where you ran the npm i command, run you index.js file:

node index.js

Based on the power of your machine, you should see a response from the model. Please note that it may take a few seconds or maybe a few minutes to load.

Based on the prompt I ask, this is the resposne I got:

Hi there! đź‘‹

I’m **ChatGPT**, a conversational AI built by OpenAI on the GPT‑4 architecture. Think of me as a virtual assistant that chat‑talks like a human but brings a few special skills to the table.

---

## What I Can Do

| Area | What I Can Help With |
|------|---------------------|
| **Questions & Knowledge** | Quick answers, deep explanations, overviews of a topic, up‑to‑date (through 2023) info, trivia. |
| **Writing & Editing** | Draft emails, essays, reports, creative stories, poems, social‑media captions, technical documentation. |
| **Programming** | Code snippets in many languages, debugging hints, algorithm explanations, API usage examples. |
| **Math & Science** | Step‑by‑step problem solving, derivations, conceptual explanations, data analysis hints. |
| **Language Learning** | Practice conversations, grammar correction, vocabulary expansion, idiom explanations. |
| **Brainstorming & Planning** | Ideas for projects, marketing, product features, events, or creative writing prompts. |
| **Education Help** | Homework help, study guides, summaries of books or papers. |
| **Conversation** | Light chat, role‑play, emotional support (not a substitute for a professional). |
| **Translation** | Rough translations between many languages (for a polished version, a professional translator is best). |

---

## How to Make the Most of It

1. **Be Specific** – The more detail you give the better the answer.  
   *Instead of “Explain history,” try “Give a concise rundown of the Industrial Revolution’s key inventions.”*

2. **Ask for Examples** – If you’re learning something new, request code, diagrams (as text), or sample sentences.

3. **Iterate** – If the answer isn’t exactly right, tweak it or ask clarifying follow‑up questions.

4. **Safety First** – I try to avoid harmful, misleading, or sensitive content. If you encounter something off, flag it for review.

---

## What I Can’t Do

- Offer real‑time updates (news after 2023).  
- Provide personal financial, legal, medical, or safety‑critical advice.  
- Engage in physical actions or browse the web live.  
- Read or interpret images, audio, or video directly (unless described textually).

---

If you’ve got a project or just feel like chatting, let me know! I’m here to help. 🚀

If you’ve made it this far, you’ve successfully got a Node.js script talking to an OpenAI model running on your local machine.

If you got stuck at any point drop a comment below and I will do my best to help you.

This example is pretty basic, but from here the use cases get pretty wild. You could give the model access to local files for analysis, or even look at something like AWS Bedrock to host a model so your app can access it over the internet.

Comments