It’s early 2025, and you’ve likely dabbled with ChatGPT and the myriad of large language models making waves in both personal and professional spheres. They can be quite remarkable, yet the occasional hallucination and the perplexing inability to count the “r’s” in “strawberry” might leave you feeling underwhelmed. Still, you can’t ignore the sweeping impact AI has had on productivity, especially in business writing. Unfortunately, Microsoft’s Copilot is less than stellar (let’s be honest, it missed the mark).

If you share my preference for offline, local models—where control, security, and privacy reign supreme—prepare yourself for a cumbersome workflow filled with copy-pasting between your chat interface and Word. Not exactly the ideal scenario, is it?

Enter “GPT for Work,” which offers a plugin for MS Word and Excel, alongside integrations for Google Docs (though I haven’t explored those). Their business model is straightforward: enhance workflow integration and charge a premium on API call costs.

What sets this apart is their option to utilize local language models. This is a game changer, allowing you to leverage your custom setups, such as Ollama or a bespoke endpoint. At least, that’s the theory.

Overview

Our primary goal here is to streamline the content creation and revision process within Word. Imagine a model that understands your document, making updates swift, straightforward, and precise (granted your model is up to the task).

Personally, I value the ability to run offline, private models that I can customize. Here are two methods I employ:

  • Running the open-source Jan—despite its unfortunate name—similar to LM Studio but without the concerning monitoring rights in their terms of service.

    • Jan operates a local API server on port 1337 by default.
    • It requires manual startup.
    • An API key is necessary (I use 12345—not particularly secure, but functional).
    • CORS Sharing can be enabled in “Advanced settings.”
  • Using Ollama via Docker on a powerful machine in my network, equipped with ample memory and a dedicated GPU.

    • This setup is only accessible from my home network or a VPN.
    • It boasts a wider model selection and superior speed compared to my laptop.

With these systems humming along, responding effortlessly to queries via Openwebui or my Jan application, I’m ready for the next step: installing GPT for Work, which is impressively smooth—kudos to the developers!

Just Kidding - HTTPS Required

The complications arise when we attempt to configure our Ollama-compatible endpoint. The GPT for Work documentation leans heavily on the LM Studio use-case, but it’s close enough to Jan:

  1. Download your desired model.
  2. Start the server.
  3. Enable CORS.

I tested this with both LM Studio and Jan, and apart from Jan requiring an API key, everything else aligned when verified with curl.

So, where’s the hiccup? GPT for Work mandates an HTTPS connection even for localhost—a rather convoluted requirement. Perhaps there’s a valid rationale behind this?

They recommend using nginx as a reverse proxy, which is a widely accepted solution and well-documented.

  • Install via Homebrew: brew install nginx
  • Generate a certificate:
    cd /opt/homebrew/etc/nginx
    openssl req \
      -x509 \
      -newkey rsa:2048 \
      -nodes \
      -sha256 \
      -days 365 \
      -keyout lm-studio.key \
      -out lm-studio.crt \
      -subj '/CN=127.0.0.1' \
      -extensions extensions \
      -config <(printf "[dn]\nCN=127.0.0.1\n[req]\ndistinguished_name=dn\n[extensions]\nsubjectAltName=DNS:localhost,IP:127.0.0.1\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")
    
  • Add the certificate to your keychain:
    sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ./lm-studio.crt
    
  • Configure /opt/homebrew/etc/nginx/nginx.conf.

For ease of memory (and my own sanity), here’s how to manage nginx:

  • Test configuration: sudo nginx -t
  • Start nginx: sudo nginx
  • Stop nginx: sudo nginx -s stop or reload with sudo nginx -s reload during debugging.

Here’s a sample nginx.conf that works for me:

worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;

    server {
        listen       [::1]:1235 ssl;
        listen       127.0.0.1:1235 ssl;
        server_name  127.0.0.1 localhost;

        http2 on;

        ssl_certificate      lm-studio.crt;
        ssl_certificate_key  lm-studio.key;

        location / {
            if ($request_method = OPTIONS) {
                add_header 'Access-Control-Allow-Origin' '*' always;
                add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
                add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization' always;
                add_header 'Access-Control-Max-Age' 3600 always;
                add_header 'Content-Length' 0;
                add_header 'Content-Type' 'text/plain charset=UTF-8';
                return 204;
            }

            add_header 'Access-Control-Allow-Origin' '*' always;
            add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization' always;
            add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;

            proxy_pass  http://10.0.50.2:11434;
            proxy_http_version 1.1;
            proxy_set_header Connection 'upgrade';
        }
    }

    server {
        listen       [::1]:1236 ssl;
        listen       127.0.0.1:1236 ssl;
        server_name  127.0.0.1 localhost;

        http2 on;

        ssl_certificate      lm-studio.crt;
        ssl_certificate_key  lm-studio.key;

        location / {
            if ($request_method = OPTIONS) {
                add_header 'Access-Control-Allow-Origin' '*' always;
                add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
                add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization' always;
                add_header 'Access-Control-Max-Age' 3600 always;
                add_header 'Content-Length' 0;
                add_header 'Content-Type' 'text/plain charset=UTF-8';
                return 204;
            }

            add_header 'Access-Control-Allow-Origin' '*' always;
            add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization' always;
            add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;

            proxy_pass  http://localhost:1337;
            proxy_http_version 1.1;
            proxy_set_header Connection 'upgrade';
            proxy_set_header Authorization "Bearer 12345";
        }
    }

    include servers/*;
}

With this configuration, I now have two Ollama-compatible providers at my disposal:

  • Ollama running via Docker at http://10.0.50.2:11434 with CORS enabled in docker-compose.yml.
  • Jan running locally on my laptop at http://localhost:1337.

I can effortlessly query models on my network machine with its powerful GPU or access models running locally.

The final setup, with LLM models operating “natively” within Word and Excel, is quite an achievement.