This commit is contained in:
Dobromir Popov 2024-03-06 11:50:19 +02:00
parent d101614b0d
commit 48123f7deb
3 changed files with 106 additions and 0 deletions

View File

@ -0,0 +1,7 @@
you're in a shell console at a root folder of a new software project. we use vscode.
let's create a mobile app (prioritize android, and plan to also support iOS) which will send the audio input to tts llm
plan for the following extesion features in the future:
- ability to listen in the background for a wake word and send the following voice command
- ability to listen on hardware button press, or O buttton hold or other android fast access shortcut intent

60
store-py/notes.md Normal file
View File

@ -0,0 +1,60 @@
<<
using python, create a new project that will utilize a vector store database to create interlinked vector space knowledge graph as a "memory" function. It will be used by a realtime LLM to store and retrieve knowledge>>
#Environment Setup
cd vector_knowledge_graph
python -m venv venv
source venv/bin/activate
pip install fastapi uvicorn openai psycopg2-binary sqlalchemy
#Create a Database: Create a new PostgreSQL database.
CREATE EXTENSION vector;
CREATE TABLE knowledge (
id SERIAL PRIMARY KEY,
embedding vector(1536) NOT NULL, -- assuming 512 dimensions for embeddings; openai uses 1536
metadata JSONB
);
CREATE INDEX ON knowledge USING ivfflat (embedding);
#Application Code
<!-- FastAPI Setup (app/api/main.py): Initialize FastAPI with a simple endpoint for testing. -->
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
async def read_root():
return {"Hello": "World"}
Database Client (app/vector_db/client.py): Implement a simple client for connecting to the database and inserting/fetching vectors.
python
Copy code
import psycopg2
from psycopg2.extras import Json
def insert_embedding(embedding, metadata):
conn = psycopg2.connect("dbname=your_db user=your_user")
cur = conn.cursor()
cur.execute("INSERT INTO knowledge (embedding, metadata) VALUES (%s, %s)", (embedding, Json(metadata)))
conn.commit()
cur.close()
conn.close()
def search_embedding(embedding):
conn = psycopg2.connect("dbname=your_db user=your_user")
cur = conn.cursor()
cur.execute("SELECT id, metadata FROM knowledge ORDER BY embedding <-> %s LIMIT 5", (embedding,))
results = cur.fetchall()
cur.close()
conn.close()
return results
5. LLM Integration
At this stage, we'll need to implement the logic to interact with OpenAI's API to generate and process embeddings. Since this involves using OpenAI's services, ensure you have an API key and have agreed to their terms of use.
6. Running the Application
With the basic components in place, you can start the FastAPI application using uvicorn:
bash
Copy code
uvicorn app.api.main:app --reload

39
store-py/store.py Normal file
View File

@ -0,0 +1,39 @@
import faiss
import numpy as np
# Define the knowledge graph schema
entities = ['Alice', 'Bob', 'Charlie']
relationships = [('Alice', 'friend', 'Bob'), ('Alice', 'friend', 'Charlie')]
# Create the database schema
db = faiss.Database('knowledge_graph.db')
db.create_table('entities', entities)
db.create_table('relationships', relationships)
# Implement the knowledge graph embedding
model = Word2Vec(sentences=['Alice is friends with Bob and Charlie'], dim=100)
# Store the knowledge graph in the database
for entity in entities:
db.insert('entities', entity)
for relationship in relationships:
db.insert('relationships', relationship)
# Implement the LLM
llm = LanguageModel(model)
# Integrate the knowledge graph embedding with the LLM
def get_entity_vector(entity):
entity_vector = np.array(db.get('entities', entity))
return entity_vector
def get_relationship_vector(relationship):
relationship_vector = np.array(db.get('relationships', relationship))
return relationship_vector
llm.add_entity_vector_fn(get_entity_vector)
llm.add_relationship_vector_fn(get_relationship_vector)
# Test the system
llm.process('Alice is friends with Bob and Charlie')