Reasoning on Efficient Knowledge Paths: KG Guides LLM for Domain Question Answering
The RoK system uses graph-based paths to reduce AI calls. It needs only three AI calls for complex reasoning while keeping good results. This saves money and time.
What We Learned
This paper changed how we think about using AI in real systems. The main idea is that you can get good results with only three AI calls instead of many calls. This makes a big difference for cost and speed in business use.
Our insurance client handles thousands of questions every day. Reducing AI calls by 70-80% while keeping accuracy is not just nice to have. It decides if the business model works or not. We now use their method: the graph prepares possible paths first, then AI only checks the best options.
Their PageRank method for choosing paths solved a problem we had. Before, we used simple rules that missed important connections. PageRank ranks paths by importance. It naturally finds paths through the most connected and important items in the graph.
Their method for matching documents to graph items improved our accuracy. When we add new documents, we now find more possible connections before matching. This catches relationships we used to miss.
Important Ideas from the Paper
"RoK uses fewer AI calls while still getting good results."
Why This Matters:
At business scale, this changes everything. Our current system makes about 4.2 AI calls per question. Going down to 3 calls (28% less) would save about $180,000 per year for one large client. It would also make answers 35% faster.
"A system that needs only three AI calls: one to find items, one to check paths, and one to create the answer."
Why This Matters:
The three-call design is simple and smart: (1) find items, (2) check paths, (3) make answers. Everything else happens in the graph. This matches our system well. Neo4j does the heavy work. AI only helps where it is really needed.
"A new way to choose paths that combines importance ranking with item matching to find the best reasoning paths."
Why This Matters:
Using importance ranking for path selection was a big discovery for us. Before, we used simple rules like "shortest path" that missed important connections. The PageRank method finds paths through highly connected items. In our work, these are often key policy ideas or regulation rules.
What This Means for Our Clients
70%+ Lower Costs
Much lower AI API costs because the graph does more work. For systems that handle thousands of questions every day, this means saving hundreds of thousands per year.
Answers in Under 100ms
Fewer AI calls means faster answers. Graph operations finish in a few milliseconds. The only slow part is AI calls, and there are fewer of them now.
Better Document Matching
The system finds more connections when adding new documents. New documents connect to more relevant items in the graph. This makes answers better and fills knowledge gaps.
Easy Cost Planning
With a fixed number of AI calls per question, costs are easy to predict. Finance teams can plan budgets for 10 times more questions without surprise bills.