<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Local Models on D. Sébastien Taggart</title>
    <link>https://www.sebastientaggart.com/tags/local-models/</link>
    <description>Recent content in Local Models on D. Sébastien Taggart</description>
    <generator>Hugo</generator>
    <language>en</language>
    <lastBuildDate>Sun, 04 Oct 2026 22:13:05 -0400</lastBuildDate>
    <atom:link href="https://www.sebastientaggart.com/tags/local-models/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Debugging Local LLM Access for GPT for Excel / Word</title>
      <link>https://www.sebastientaggart.com/writing/debugging-local-llm-access-for-gpt-for-excel--word/</link>
      <pubDate>Wed, 09 Jul 2025 10:08:51 -0400</pubDate>
      <guid>https://www.sebastientaggart.com/writing/debugging-local-llm-access-for-gpt-for-excel--word/</guid>
      <description>&lt;p&gt;It&amp;rsquo;s early 2025, and you&amp;rsquo;ve likely dabbled with ChatGPT and the myriad of large language models making waves in both personal and professional spheres. They can be quite remarkable, yet the occasional hallucination and the perplexing inability to count the &amp;ldquo;r&amp;rsquo;s&amp;rdquo; in &amp;ldquo;strawberry&amp;rdquo; might leave you feeling underwhelmed. Still, you can&amp;rsquo;t ignore the sweeping impact AI has had on productivity, especially in business writing. Unfortunately, Microsoft’s Copilot is less than stellar (let&amp;rsquo;s be honest, it missed the mark).&lt;/p&gt;</description>
      <content:encoded><![CDATA[<p>It&rsquo;s early 2025, and you&rsquo;ve likely dabbled with ChatGPT and the myriad of large language models making waves in both personal and professional spheres. They can be quite remarkable, yet the occasional hallucination and the perplexing inability to count the &ldquo;r&rsquo;s&rdquo; in &ldquo;strawberry&rdquo; might leave you feeling underwhelmed. Still, you can&rsquo;t ignore the sweeping impact AI has had on productivity, especially in business writing. Unfortunately, Microsoft’s Copilot is less than stellar (let&rsquo;s be honest, it missed the mark).</p>
<p>If you share my preference for offline, local models—where control, security, and privacy reign supreme—prepare yourself for a cumbersome workflow filled with copy-pasting between your chat interface and Word. Not exactly the ideal scenario, is it?</p>
<p>Enter &ldquo;GPT for Work,&rdquo; which offers a plugin for MS Word and Excel, alongside integrations for Google Docs (though I haven&rsquo;t explored those). Their business model is straightforward: enhance workflow integration and charge a premium on API call costs.</p>
<p>What sets this apart is their option to utilize local language models. This is a game changer, allowing you to leverage your custom setups, such as Ollama or a bespoke endpoint. At least, that’s the theory.</p>
<h3 id="overview">Overview</h3>
<p>Our primary goal here is to streamline the content creation and revision process within Word. Imagine a model that understands your document, making updates swift, straightforward, and precise (granted your model is up to the task).</p>
<p>Personally, I value the ability to run offline, private models that I can customize. Here are two methods I employ:</p>
<ul>
<li>
<p>Running the open-source <a href="https://github.com/menloresearch/jan">Jan</a>—despite its unfortunate name—similar to <a href="https://lmstudio.ai/">LM Studio</a> but without the concerning monitoring rights in their terms of service.</p>
<ul>
<li>Jan operates a local API server on port 1337 by default.</li>
<li>It requires manual startup.</li>
<li>An API key is necessary (I use <code>12345</code>—not particularly secure, but functional).</li>
<li>CORS Sharing can be enabled in &ldquo;Advanced settings.&rdquo;</li>
</ul>
</li>
<li>
<p>Using <a href="https://github.com/mythrantic/ollama-docker">Ollama via Docker</a> on a powerful machine in my network, equipped with ample memory and a dedicated GPU.</p>
<ul>
<li>This setup is only accessible from my home network or a VPN.</li>
<li>It boasts a wider model selection and superior speed compared to my laptop.</li>
</ul>
</li>
</ul>
<p>With these systems humming along, responding effortlessly to queries via <a href="https://github.com/open-webui/open-webui">Openwebui</a> or my Jan application, I&rsquo;m ready for the next step: installing GPT for Work, which is impressively smooth—kudos to the developers!</p>
<h3 id="just-kidding---https-required">Just Kidding - HTTPS Required</h3>
<p>The complications arise when we attempt to configure our Ollama-compatible endpoint. The <a href="https://gptforwork.com/help/ai-models/custom-endpoints/set-up-lm-studio-on-macos#enable-https-for-the-lm-studio-server">GPT for Work documentation</a> leans heavily on the LM Studio use-case, but it’s close enough to Jan:</p>
<ol>
<li>Download your desired model.</li>
<li>Start the server.</li>
<li>Enable CORS.</li>
</ol>
<p>I tested this with both LM Studio and Jan, and apart from Jan requiring an API key, everything else aligned when verified with <code>curl</code>.</p>
<p>So, where&rsquo;s the hiccup? GPT for Work mandates an HTTPS connection even for localhost—a rather convoluted requirement. Perhaps there&rsquo;s a valid rationale behind this?</p>
<p>They recommend using <code>nginx</code> as a reverse proxy, which is a widely accepted solution and well-documented.</p>
<ul>
<li>Install via Homebrew: <code>brew install nginx</code></li>
<li>Generate a certificate:
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="nb">cd</span> /opt/homebrew/etc/nginx
</span></span><span class="line"><span class="cl">openssl req <span class="se">\
</span></span></span><span class="line"><span class="cl">  -x509 <span class="se">\
</span></span></span><span class="line"><span class="cl">  -newkey rsa:2048 <span class="se">\
</span></span></span><span class="line"><span class="cl">  -nodes <span class="se">\
</span></span></span><span class="line"><span class="cl">  -sha256 <span class="se">\
</span></span></span><span class="line"><span class="cl">  -days <span class="m">365</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  -keyout lm-studio.key <span class="se">\
</span></span></span><span class="line"><span class="cl">  -out lm-studio.crt <span class="se">\
</span></span></span><span class="line"><span class="cl">  -subj <span class="s1">&#39;/CN=127.0.0.1&#39;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  -extensions extensions <span class="se">\
</span></span></span><span class="line"><span class="cl">  -config &lt;<span class="o">(</span><span class="nb">printf</span> <span class="s2">&#34;[dn]\nCN=127.0.0.1\n[req]\ndistinguished_name=dn\n[extensions]\nsubjectAltName=DNS:localhost,IP:127.0.0.1\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth&#34;</span><span class="o">)</span>
</span></span></code></pre></div></li>
<li>Add the certificate to your keychain:
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ./lm-studio.crt
</span></span></code></pre></div></li>
<li>Configure <code>/opt/homebrew/etc/nginx/nginx.conf</code>.</li>
</ul>
<p>For ease of memory (and my own sanity), here’s how to manage <code>nginx</code>:</p>
<ul>
<li>Test configuration: <code>sudo nginx -t</code></li>
<li>Start <code>nginx</code>: <code>sudo nginx</code></li>
<li>Stop <code>nginx</code>: <code>sudo nginx -s stop</code> or reload with <code>sudo nginx -s reload</code> during debugging.</li>
</ul>
<p>Here’s a sample <code>nginx.conf</code> that works for me:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-nginx" data-lang="nginx"><span class="line"><span class="cl"><span class="k">worker_processes</span>  <span class="mi">1</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">events</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="kn">worker_connections</span>  <span class="mi">1024</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">http</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="kn">include</span>       <span class="s">mime.types</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">default_type</span>  <span class="s">application/octet-stream</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="kn">sendfile</span>        <span class="no">on</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">keepalive_timeout</span>  <span class="mi">65</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="kn">server</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="kn">listen</span>       <span class="s">[::1]:1235</span> <span class="s">ssl</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">        <span class="kn">listen</span>       <span class="n">127.0.0.1</span><span class="p">:</span><span class="mi">1235</span> <span class="s">ssl</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">        <span class="kn">server_name</span>  <span class="n">127.0.0.1</span> <span class="s">localhost</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">        <span class="kn">http2</span> <span class="no">on</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">        <span class="kn">ssl_certificate</span>      <span class="s">lm-studio.crt</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">        <span class="kn">ssl_certificate_key</span>  <span class="s">lm-studio.key</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">        <span class="kn">location</span> <span class="s">/</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="kn">if</span> <span class="s">(</span><span class="nv">$request_method</span> <span class="p">=</span> <span class="s">OPTIONS)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Origin&#39;</span> <span class="s">&#39;*&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Methods&#39;</span> <span class="s">&#39;GET,</span> <span class="s">POST,</span> <span class="s">OPTIONS&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Headers&#39;</span> <span class="s">&#39;Content-Type,</span> <span class="s">Authorization&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Max-Age&#39;</span> <span class="mi">3600</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Content-Length&#39;</span> <span class="mi">0</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Content-Type&#39;</span> <span class="s">&#39;text/plain</span> <span class="s">charset=UTF-8&#39;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">return</span> <span class="mi">204</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">            <span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">            <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Origin&#39;</span> <span class="s">&#39;*&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">            <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Headers&#39;</span> <span class="s">&#39;Content-Type,</span> <span class="s">Authorization&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">            <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Methods&#39;</span> <span class="s">&#39;GET,</span> <span class="s">POST,</span> <span class="s">OPTIONS&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">            <span class="kn">proxy_pass</span>  <span class="s">http://10.0.50.2:11434</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">            <span class="kn">proxy_http_version</span> <span class="mi">1</span><span class="s">.1</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">            <span class="kn">proxy_set_header</span> <span class="s">Connection</span> <span class="s">&#39;upgrade&#39;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">        <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="kn">server</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="kn">listen</span>       <span class="s">[::1]:1236</span> <span class="s">ssl</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">        <span class="kn">listen</span>       <span class="n">127.0.0.1</span><span class="p">:</span><span class="mi">1236</span> <span class="s">ssl</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">        <span class="kn">server_name</span>  <span class="n">127.0.0.1</span> <span class="s">localhost</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">        <span class="kn">http2</span> <span class="no">on</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">        <span class="kn">ssl_certificate</span>      <span class="s">lm-studio.crt</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">        <span class="kn">ssl_certificate_key</span>  <span class="s">lm-studio.key</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">        <span class="kn">location</span> <span class="s">/</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="kn">if</span> <span class="s">(</span><span class="nv">$request_method</span> <span class="p">=</span> <span class="s">OPTIONS)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Origin&#39;</span> <span class="s">&#39;*&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Methods&#39;</span> <span class="s">&#39;GET,</span> <span class="s">POST,</span> <span class="s">OPTIONS&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Headers&#39;</span> <span class="s">&#39;Content-Type,</span> <span class="s">Authorization&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Max-Age&#39;</span> <span class="mi">3600</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Content-Length&#39;</span> <span class="mi">0</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">add_header</span> <span class="s">&#39;Content-Type&#39;</span> <span class="s">&#39;text/plain</span> <span class="s">charset=UTF-8&#39;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">                <span class="kn">return</span> <span class="mi">204</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">            <span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">            <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Origin&#39;</span> <span class="s">&#39;*&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">            <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Headers&#39;</span> <span class="s">&#39;Content-Type,</span> <span class="s">Authorization&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">            <span class="kn">add_header</span> <span class="s">&#39;Access-Control-Allow-Methods&#39;</span> <span class="s">&#39;GET,</span> <span class="s">POST,</span> <span class="s">OPTIONS&#39;</span> <span class="s">always</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">            <span class="kn">proxy_pass</span>  <span class="s">http://localhost:1337</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">            <span class="kn">proxy_http_version</span> <span class="mi">1</span><span class="s">.1</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">            <span class="kn">proxy_set_header</span> <span class="s">Connection</span> <span class="s">&#39;upgrade&#39;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">            <span class="kn">proxy_set_header</span> <span class="s">Authorization</span> <span class="s">&#34;Bearer</span> <span class="mi">12345</span><span class="s">&#34;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">        <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="kn">include</span> <span class="s">servers/*</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><p>With this configuration, I now have two Ollama-compatible providers at my disposal:</p>
<ul>
<li>Ollama running via Docker at <code>http://10.0.50.2:11434</code> with CORS enabled in <code>docker-compose.yml</code>.</li>
<li>Jan running locally on my laptop at <code>http://localhost:1337</code>.</li>
</ul>
<p>I can effortlessly query models on my network machine with its powerful GPU or access models running locally.</p>
<p>The final setup, with LLM models operating &ldquo;natively&rdquo; within Word and Excel, is quite an achievement.</p>
]]></content:encoded>
    </item>
  </channel>
</rss>
