Local AI Lab

Purpose

Build a small local AI environment that is useful enough to test real workflows and simple enough to understand without guessing what each layer is doing.

Problem

Most AI tooling becomes easier to use and harder to understand at the same time.

That is fine for shipping quickly, but not ideal for learning. I wanted an environment where I could see the moving parts clearly:

• the server
• the containers
• the services
• the model
• the workflow

The problem was not just getting a model to respond. The real problem was building a setup I could inspect, change, and document honestly.

Approach

The lab started small on purpose.

I used the Home Server Infrastructure Journey, Docker Compose, Ollama, and n8n as the first working stack. That gave me one runtime for local models and one orchestration layer for automations.

Instead of chasing a larger architecture immediately, I aimed for a narrow loop:

1. get the services running
2. confirm they can communicate
3. pass real content through the workflow
4. document what broke and what became clearer

That first phase was mostly infrastructure work. After the first few experiments, the constraint changed. The harder problem is now less about getting containers to talk to each other and more about designing prompts and outputs that are actually useful.

Architecture

Ubuntu Home Server
    ↓
Docker Compose
    ↓
+-------------------+     +-------------------+
| qurio-ollama     | <--> | qurio-n8n        |
| Ollama runtime   |      | workflow engine  |
+-------------------+     +-------------------+
    ↓
qwen2.5:3b

Supporting pieces around the main path:

• named volumes for model data and workflow data
• Docker networking between services
• Portainer for visibility and basic management
• the broader home server setup for backups, remote access, and maintenance discipline

Decisions

• I used Docker Compose because it keeps the stack reproducible and easier to reason about.
• I chose qwen2.5:3b because the server has limited RAM and the goal is learning, not pushing the largest model possible.
• I kept the first workflows narrow so problems would be easier to isolate.
• I treated the lab as infrastructure plus experiments, not as a one-off install.
• I kept the stack local so I could understand the system boundaries clearly before layering in more convenience.
• I now treat prompt structure as part of the build, not just something attached to the workflow at the end.

Progress

What is working now:

• Ollama is running locally in Docker.
• n8n is running locally in Docker.
• n8n can call Ollama over the internal Docker network.
• The workflow can pass content and receive structured output.
• Existing Qurio notes can be sent through the workflow and summarized locally.

What is still early:

• prompt design is still basic
• outputs are not yet saved automatically into Qurio content
• Markdown cleanup and structure are still workflow concerns
• monitoring and maintenance are still light

What has worked so far:

• Running the stack locally made the system easier to inspect and document.
• Docker Compose kept the infrastructure small and repeatable.
• n8n was sufficient as the orchestration layer for testing content workflows.
• qwen2.5:3b was strong enough for summarization, extraction, and early workflow experiments.
• The experiments proved the system incrementally instead of assuming success because the containers were up.

What has been difficult:

• The first difficulty was infrastructure, networking, and request formatting.
• After that, the harder problems moved up a layer into prompt wording and output design.
• Action items and structured responses become generic quickly if the prompt is vague.
• Markdown source content can include noise that the model does not always handle cleanly.
• A working pipeline is not the same as a useful pipeline. Utility depends on how well the workflow shapes the input and output.

The lab started by proving that the stack could run at all. It is now moving toward a knowledge workflow where the output should become reusable material inside Qurio instead of one-off model responses.

Lessons Learned

• Small infrastructure projects become easier to grow when the first version is narrow.
• Docker removed a lot of setup ambiguity, but it also forced clearer thinking about volumes, ports, and service boundaries.
• A local AI lab is more understandable when it is documented as a build, not just a list of commands.
• The useful question is usually not "does AI work?" but "which layer is failing right now?"
• The bottleneck has shifted. Infrastructure was the first problem, but workflow design is now the more important one.
• Prompt quality now matters more than container setup for the kinds of outputs I want from Qurio.
• The related notes Docker Basics, Ollama Basics, and n8n Workflow Basics explain the concepts underneath the build.
• The related experiments Local AI Lab 001: First n8n to Ollama Test, Local AI Lab 002: First n8n to Ollama Workflow, and Local AI Lab 003: Summarizing Qurio Notes with Ollama and n8n prove the system incrementally.

Next Steps

• Save generated outputs into a more reusable Markdown flow.
• Improve prompt structure so summaries, takeaways, and action items are less generic.
• Explore whether the workflow can produce draft Qurio artifacts instead of only summaries.
• Add a few repeatable workflows that solve small real tasks.
• Document service updates, restores, and maintenance more clearly.