It’s early 2025, and you’ve likely dabbled with ChatGPT and the myriad of large language models making waves in both personal and professional spheres. They can be quite remarkable, yet the occasional hallucination and the perplexing inability to count the “r’s” in “strawberry” might leave you feeling underwhelmed. Still, you can’t ignore the sweeping impact AI has had on productivity, especially in business writing. Unfortunately, Microsoft’s Copilot is less than stellar (let’s be honest, it missed the mark).
If you share my preference for offline, local models—where control, security, and privacy reign supreme—prepare yourself for a cumbersome workflow filled with copy-pasting between your chat interface and Word. Not exactly the ideal scenario, is it?
Enter “GPT for Work,” which offers a plugin for MS Word and Excel, alongside integrations for Google Docs (though I haven’t explored those). Their business model is straightforward: enhance workflow integration and charge a premium on API call costs.
What sets this apart is their option to utilize local language models. This is a game changer, allowing you to leverage your custom setups, such as Ollama or a bespoke endpoint. At least, that’s the theory.
Overview
Our primary goal here is to streamline the content creation and revision process within Word. Imagine a model that understands your document, making updates swift, straightforward, and precise (granted your model is up to the task).
Personally, I value the ability to run offline, private models that I can customize. Here are two methods I employ:
Running the open-source Jan—despite its unfortunate name—similar to LM Studio but without the concerning monitoring rights in their terms of service.
- Jan operates a local API server on port 1337 by default.
- It requires manual startup.
- An API key is necessary (I use
12345—not particularly secure, but functional). - CORS Sharing can be enabled in “Advanced settings.”
Using Ollama via Docker on a powerful machine in my network, equipped with ample memory and a dedicated GPU.
- This setup is only accessible from my home network or a VPN.
- It boasts a wider model selection and superior speed compared to my laptop.
With these systems humming along, responding effortlessly to queries via Openwebui or my Jan application, I’m ready for the next step: installing GPT for Work, which is impressively smooth—kudos to the developers!
Just Kidding - HTTPS Required
The complications arise when we attempt to configure our Ollama-compatible endpoint. The GPT for Work documentation leans heavily on the LM Studio use-case, but it’s close enough to Jan:
- Download your desired model.
- Start the server.
- Enable CORS.
I tested this with both LM Studio and Jan, and apart from Jan requiring an API key, everything else aligned when verified with curl.
So, where’s the hiccup? GPT for Work mandates an HTTPS connection even for localhost—a rather convoluted requirement. Perhaps there’s a valid rationale behind this?
They recommend using nginx as a reverse proxy, which is a widely accepted solution and well-documented.
- Install via Homebrew:
brew install nginx - Generate a certificate:
cd /opt/homebrew/etc/nginx openssl req \ -x509 \ -newkey rsa:2048 \ -nodes \ -sha256 \ -days 365 \ -keyout lm-studio.key \ -out lm-studio.crt \ -subj '/CN=127.0.0.1' \ -extensions extensions \ -config <(printf "[dn]\nCN=127.0.0.1\n[req]\ndistinguished_name=dn\n[extensions]\nsubjectAltName=DNS:localhost,IP:127.0.0.1\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth") - Add the certificate to your keychain:
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ./lm-studio.crt - Configure
/opt/homebrew/etc/nginx/nginx.conf.
For ease of memory (and my own sanity), here’s how to manage nginx:
- Test configuration:
sudo nginx -t - Start
nginx:sudo nginx - Stop
nginx:sudo nginx -s stopor reload withsudo nginx -s reloadduring debugging.
Here’s a sample nginx.conf that works for me:
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen [::1]:1235 ssl;
listen 127.0.0.1:1235 ssl;
server_name 127.0.0.1 localhost;
http2 on;
ssl_certificate lm-studio.crt;
ssl_certificate_key lm-studio.key;
location / {
if ($request_method = OPTIONS) {
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization' always;
add_header 'Access-Control-Max-Age' 3600 always;
add_header 'Content-Length' 0;
add_header 'Content-Type' 'text/plain charset=UTF-8';
return 204;
}
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
proxy_pass http://10.0.50.2:11434;
proxy_http_version 1.1;
proxy_set_header Connection 'upgrade';
}
}
server {
listen [::1]:1236 ssl;
listen 127.0.0.1:1236 ssl;
server_name 127.0.0.1 localhost;
http2 on;
ssl_certificate lm-studio.crt;
ssl_certificate_key lm-studio.key;
location / {
if ($request_method = OPTIONS) {
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization' always;
add_header 'Access-Control-Max-Age' 3600 always;
add_header 'Content-Length' 0;
add_header 'Content-Type' 'text/plain charset=UTF-8';
return 204;
}
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
proxy_pass http://localhost:1337;
proxy_http_version 1.1;
proxy_set_header Connection 'upgrade';
proxy_set_header Authorization "Bearer 12345";
}
}
include servers/*;
}
With this configuration, I now have two Ollama-compatible providers at my disposal:
- Ollama running via Docker at
http://10.0.50.2:11434with CORS enabled indocker-compose.yml. - Jan running locally on my laptop at
http://localhost:1337.
I can effortlessly query models on my network machine with its powerful GPU or access models running locally.
The final setup, with LLM models operating “natively” within Word and Excel, is quite an achievement.