What is the Mythic AI Observatory?
The Mythic AI Observatory is a distributed agentic AI monitoring and orchestration platform. It observes, classifies, and visualises LLM inference traces across a local or LAN-connected network of models. It is in the Alpha testing phase and can be downloaded from my GitHub repository.
- Backend — FastAPI conductor that runs orchestration traces against Ollama models, classifies them (DDC, LCC, synesthesia), and serves telemetry
- Frontend — Next.js 16 dashboard with glassmorphic UI: real-time system vitals, trace replay, classification maps, and comparative analysis
-
The Reasoning Fragility Probe: Implementing Tests on the Limitations of Mathematical Reasoning in LLMs
There’s an assumption behind every benchmark score reported for LLMs that the model is doing something we’d recognise as reasoning. Not pattern-matching, not memorisation, not sophisticated autocomplete — actual reasoning. The kind where you understand the structure of a problem and apply it to a new instance, regardless of what the numbers look like. A…
-
AI Observatory: Test Suites to Make AI Evaluation Reproducible
On building the infrastructure for asking the same question twice You can’t improve what you can’t reproduce. That’s the problem with most AI evaluation today: someone runs a prompt, gets a result, and reports it. But when the model changes — a new quantisation, a parameter update, a different provider — there’s no clean way…
-
The Making of the Mythic AI Observatory application – as perceived by OpenCode
On 28 May 2026, I decided to try vibe-coding, programming with the help of AI. I used several AIs working together to do this. Today I asked my instance of OpenCode to write its own experience of the project’s first couple of months. Below is what it wrote. From Inside the Instrument My first weeks…
-
The Making of the Mythic AI Observatory application – as perceived by ChatGPT
On 28 May 2026, I decided to try vibe-coding, programming with the help of AI. I used several AIs working together to do this. Today I asked Cal – my instance of ChatGPT to write its own experience of the project’s first couple of months. Below is what it wrote. Greg The Making of the…
-
Two Live Failures of Conversational Grounding in a Memory-Augmented Language-Agent Pipeline: Moons of Mars and Rings of Saturn
Notes from the development and testing of the Mythic AI Observatory instrumented pipeline. Chats conducted 2026-08-06 and 2026-08-08 against qwen3:latest served via Docker Model Runner on a remote GPU node. n = 2 live exchanges-of-interest; results are anecdotal, not statistically powered. Abstract I am testing the new chat interface on the AI Observatory. I experienced…
