[SYSTEM DOCS] Project FAQ

1. Project Overview & Motivation

Who am I?

I am César Blanc, a Data Engineer/Scientist passionate about Artificial Intelligence, processing complex datasets and creating models based on real-world use cases. I am currently looking for a job as a Data Scientist. Check out my LinkedIn page with my resume for more information and my Email if you want to contact me

Why choose a video game mod (OP-2.2) for a data project?

S.T.A.L.K.E.R. OP-2.2 is notoriously known as one of the most complex and data-dense modifications in the S.T.A.L.K.E.R. community. It's an esoteric and obscure mod featuring thousands of interlocked quests, items, coordinates, and hidden mechanics spread across unstructured community wikis and raw spreadsheets. Since I love both the base game and the mod, and since it's impossible to play without spending hours on the wiki, I came up with the idea of creating a personal assistant to help me and other players to beat the game. All data used in this projet comes from this community wiki.

Affiliation

I am in no way affiliated with the holder of the S.T.A.L.K.E.R. license GSC Game World, the creators of the mod OP-2.2 or the OP-2 fandom wiki OP-2 fandom wiki. This is a standalone fan project which I use for my portfolio and as a personal assistant.

2. Technical Stack

3. Key Engineering Challenges Solved

Hybrid Search (Vector + Relational + Graph)

The system is not just a standard semantic search engine. The LLM acts as an autonomous agent capable of deciding which tool to use: it queries the Qdrant Vector Database for unstructured narrative data (quest guides, tactics...), queries the PostgreSQL Database for exact numerical statistics (weapon damage, armor carry-weight...) and search in the NetworkX Graph for quest routing and pathfinding. This hybrid approach eliminates hallucinations on critical data points.

Agentic Workflow & State Orchestration (LangGraph)

To overcome the limitations of traditional linear RAG pipelines (such as truncation on massive wiki tables or inaccurate answers because they were unverified and rushed), the agent has been migrated from a simple single-pass execution to a cyclical StateGraph using LangGraph:

• Autonomous Loops & Multi-Pass Retrieval: The agent is no longer constrained to a single tool call. If retrieved data appears incomplete or split across multiple chunks, the graph allows the agent to loop back and perform additional targeted tool queries.
• Self-Reflection & Quality Assurance: A dedicated QA auditor node inspects the generated draft against the raw tool data before final rendering, automatically rejecting truncated responses and forcing the agent to fetch missing fragments.
• State Persistence: Conversation states and revision counts are safely tracked across nodes, combining the deterministic precision of PostgreSQL querries and NetworkX quest/map paths with autonomous LLM reasoning.

LLM Memory Management & Cost Optimization

Initially, preserving conversation history caused memory leaks and `400 Bad Request` crashes due to the LLM returning massive internal metadata blocks. I implemented a strict memory middleware that intercepts the LLM's raw output, extracts only the sanitized user-facing text, and dynamically truncates the context window to a rolling 10-message limit. This guarantees server stability and drastically reduces API token costs.

Automated Data Pipeline & Health Monitoring

A RAG system based on a fan-driven wiki must be up to date. To prevent outdated information and ensure synchronization with the ever-evolving community data, I engineered a fully automated data pipeline:

• Continuous Integration for Data: Once a day, a dedicated script automatically check and scrapes the latest Wiki pages, chunks the new information, generates fresh embeddings via OpenAI, and upserts them into Qdrant, PostgreSQL and update the quest graph.
• Automated Health Checks: Before updating the live database, a validation protocol verifies the integrity of the embeddings and the schema structure, ensuring that a bad scrape doesn't corrupt the assistant's knowledge base. If the data are corrupted, the script automatically rebuilt all the databases.

Hallucination Prevention & Traceability

For a RAG system to be trustworthy, traceability is mandatory. I utilized Few-Shot Prompt Engineering to strictly enforce source citations. The agent is hardcoded to dynamically generate Markdown links pointing exactly to the originating Wiki page URL or the specific Google Sheet / Database table used for its answer. This way, the user can verify the information and understand how the RAG generates its answers.

Security & API Stability

The FastAPI backend is built for production readiness, featuring:

• Concurrency Control: `asyncio.Semaphore` prevents API bottlenecks by limiting simultaneous LLM invocations.
• Rate Limiting: Custom IP-based tracking enforces both requests-per-minute and daily hard caps to prevent API abuse.
• Payload Validation: Enforced maximum body size limits and payload hashing for immediate cached responses on duplicate queries.
• CORS Policies: Strict middleware configuration to safely allow cross-origin requests from the standalone frontend.

Lightweight UX / UI

Instead of relying on heavy modern frameworks (React/Vue), the frontend was deliberately built using vanilla HTML/CSS/JavaScript. It relies on a single external dependency (`marked.js`) to parse Markdown in real-time. This decision was made because I wanted to create a Minimum Viable Product and prioritizing backend complexity and data delivery over frontend overhead. Also, I like the raw look, which reminds me of the interface of S.T.A.L.K.E.R. PDAs.