From ac053f5800a43fabe3d4301a67b87958ca02482f Mon Sep 17 00:00:00 2001 From: Dobromir Popov Date: Wed, 1 Jul 2026 14:40:42 +0200 Subject: [PATCH] memories in git --- .claude/memory/MEMORY.md | 5 ++++ .claude/memory/product-selling-points.md | 38 ++++++++++++++++++++++++ .claude/memory/project-status.md | 31 +++++++++++++++++++ .claude/memory/user-profile.md | 15 ++++++++++ .claude/settings.json | 27 +++++++++++++++++ AGENTS.md | 4 +++ 6 files changed, 120 insertions(+) create mode 100644 .claude/memory/MEMORY.md create mode 100644 .claude/memory/product-selling-points.md create mode 100644 .claude/memory/project-status.md create mode 100644 .claude/memory/user-profile.md create mode 100644 .claude/settings.json diff --git a/.claude/memory/MEMORY.md b/.claude/memory/MEMORY.md new file mode 100644 index 0000000..77a6f6f --- /dev/null +++ b/.claude/memory/MEMORY.md @@ -0,0 +1,5 @@ +# Memory Index + +- [Product selling points](product-selling-points.md) — key differentiators and landing page angles for neuron-tai +- [User profile](user-profile.md) — who Dobromir is and how to work with him +- [Project status](project-status.md) — 29/30 done; US-030 (manual route + hop benchmark) is the only open story diff --git a/.claude/memory/product-selling-points.md b/.claude/memory/product-selling-points.md new file mode 100644 index 0000000..952f8e4 --- /dev/null +++ b/.claude/memory/product-selling-points.md @@ -0,0 +1,38 @@ +--- +name: product-selling-points +description: Key differentiators and landing page angles for neuron-tai distributed inference network +metadata: + node_type: memory + type: project + originSessionId: 8fb120ee-7b8e-45be-98c0-b5ae9c64d1ec +--- + +# neuron-tai — Product Selling Points + +## Core pitch +Volunteer GPU network for distributed LLM inference. Small GPU owners contribute compute and earn TAI tokens. Clients get inference on models larger than any single machine can serve. + +## Confirmed technical differentiators (verified working) + +### Mixed hardware inference routes +The tracker can chain CPU nodes and GPU nodes into a single inference route. Shard A on a CPU node → Shard B on a GPU node → valid streamed response. Each participant in the route only needs to fit *their shard* in memory, not the whole model. + +**Angle for landing page:** "Run a 70B model across three laptops and a gaming PC. Each machine only holds the layers it can fit." + +**Nuance to acknowledge:** PyTorch/HuggingFace `device_map="auto"` already does CPU+GPU mixing on a single machine. Our value-add is doing this *across machines over the network*, democratizing access to models that no single volunteer machine could serve alone. + +### Hardware-aware routing +Tracker scores nodes by `benchmark_tokens_per_sec / (queue_depth + 1)` and always routes to the fastest available node per shard range. A GPU node at 11,200 throughput index beats a CPU node at 626 automatically — no user configuration needed. + +### Zero port-forwarding required +Nodes connect outbound to the relay via WebSocket. Works from behind NAT, WSL2, 5G, or a home router with no config. The public tracker at ai.neuron.d-popov.com handles discovery. + +### OpenAI-compatible API +Any app using the OpenAI Python SDK works by changing only `base_url`. No code changes for the client. + +## Landing page content TODO +- User asked to capture these points for the landing page copy (2026-07-01) +- No landing page file exists in the repo yet +- When writing copy, lead with the "run models bigger than your GPU" angle, then support with mixed-hardware routing, relay, and OpenAI compat + +**How to apply:** When writing product descriptions, pitches, or landing page copy, use these as the primary hooks. The mixed-network inference route (CPU+GPU across machines) is the biggest differentiator vs. single-machine solutions. diff --git a/.claude/memory/project-status.md b/.claude/memory/project-status.md new file mode 100644 index 0000000..b6fb6ca --- /dev/null +++ b/.claude/memory/project-status.md @@ -0,0 +1,31 @@ +--- +name: project-status +description: Current state of neuron-tai development as of 2026-07-01 +metadata: + node_type: memory + type: project + originSessionId: 8fb120ee-7b8e-45be-98c0-b5ae9c64d1ec +--- + +# Project Status (2026-07-01) + +29/30 user stories done. US-030 is the only open story, ready for ralph. + +## US-030 — Manual route selection + hop-penalty benchmarking +- Status: open / ready +- Optional `"route": [node_id, ...]` in POST /v1/chat/completions body +- `POST /v1/benchmark/hop-penalty` — privileged (non-empty Authorization header), fans out to 1/2/3-node routes, records per-hop latency +- Results appended to `benchmark_results.json` in tracker working dir +- `GET /v1/benchmark/results` — also auth-gated +- Routing algorithm unchanged — data collection only +- Source: `.scratch/distributed-inference-network/issues/30-manual-route-and-hop-benchmark.md` + +**Why:** Need real hop-latency data to eventually optimize route selection beyond synthetic benchmarks. +**How to apply:** When asked about next steps, US-030 is the one ready story. + +## Windows CUDA node (working as of 2026-07-01) +- miniforge3 base env, torch 2.7.1+cu118, torchvision 0.22.x+cu118 +- RTX 4060 Laptop GPU, 8 GB VRAM, benchmark index ~11,200 +- Run: `meshnet-node start --tracker https://ai.neuron.d-popov.com --model Qwen/Qwen2.5-0.5B-Instruct` +- Known: tracker registration fails with `http://` — must use `https://` +- pynvml deprecation warning is harmless (use nvidia-ml-py to silence it) diff --git a/.claude/memory/user-profile.md b/.claude/memory/user-profile.md new file mode 100644 index 0000000..b218978 --- /dev/null +++ b/.claude/memory/user-profile.md @@ -0,0 +1,15 @@ +--- +name: user-profile +description: Who Dobromir is and how to collaborate effectively +metadata: + node_type: memory + type: user + originSessionId: 8fb120ee-7b8e-45be-98c0-b5ae9c64d1ec +--- + +# Dobromir Popov + +- Building neuron-tai: a distributed LLM inference network with volunteer GPU nodes, tracker, relay, and token rewards +- Works across Linux (AMD Ryzen AI Max APU, 124 GB RAM) and Windows 11 (RTX 4060 Laptop GPU, 8 GB VRAM, miniforge3 Python env) +- Uses ralph for project management (prd.json + issues in .scratch/) +- Iterates quickly — prefers short, direct answers and learns from real output/errors rather than pre-emptive explanations diff --git a/.claude/settings.json b/.claude/settings.json new file mode 100644 index 0000000..e369d24 --- /dev/null +++ b/.claude/settings.json @@ -0,0 +1,27 @@ +{ + "hooks": { + "PreToolUse": [ + { + "matcher": ".*", + "hooks": [ + { + "type": "command", + "command": "bash -c 'SRC=\"/mnt/d/DEV/workspace/REPOS/git.d-popov.com/neuron-tai/.claude/memory\" && DST=\"/home/dev/.claude/projects/-mnt-d-DEV-workspace-REPOS-git-d-popov-com-neuron-tai/memory\" && mkdir -p \"$DST\" && rsync -a --update \"$SRC/\" \"$DST/\" 2>/dev/null; true'", + "runOncePerSession": true + } + ] + } + ], + "PostToolUse": [ + { + "matcher": "Write|Edit", + "hooks": [ + { + "type": "command", + "command": "bash -c 'SRC=\"/mnt/d/DEV/workspace/REPOS/git.d-popov.com/neuron-tai/.claude/memory\" && DST=\"/home/dev/.claude/projects/-mnt-d-DEV-workspace-REPOS-git-d-popov-com-neuron-tai/memory\" && mkdir -p \"$DST\" && rsync -a \"$SRC/\" \"$DST/\" 2>/dev/null; true'" + } + ] + } + ] + } +} diff --git a/AGENTS.md b/AGENTS.md index ceb3eb6..1007ed5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,3 +1,7 @@ +## Memory + +Persistent memory lives in `.claude/memory/`. Read `MEMORY.md` there at the start of every session for project context, user preferences, and open work. Write updates back to those files so knowledge carries across devices and sessions. + ## Agent skills ### Issue tracker