Document-based AI chatbot PoC at a KRW 1 trillion+ auto parts maker

You need to find a document or a rule for your work, but the documents are scattered across Google Docs, the ERP and various internal systems, so they are hard to find. Most of us have been there at least once.

The bigger the company, the worse this information-silo problem gets.

With different systems and access rights in each department, and rules that are constantly updated, it is realistically impossible for every employee to keep track of everything in real time.

So can an enterprise AI chatbot solve this problem?

Today we introduce a PoC that a global auto parts manufacturer with over KRW 1 trillion in annual revenue ran with DARVIS, DFINITE's document-based AI solution.

A global company's practical concern

The company is a global auto parts manufacturer. With annual revenue of over KRW 1 trillion, it is a global player that does business with automakers around the world.

The company's IT team was looking for ways to make document management more efficient across the whole company. In particular, it needed to fix the problem of employees spending too much time finding and navigating the information they needed.

The company had been growing steadily on the strength of its technology and global network, but it had reached the point where it needed a more efficient knowledge management system. Its many rules and procedures were spread across Google Docs, the ERP and various internal systems, which made information hard to reach.

Documents scattered across internal systems

The company's technical documents and rules were spread across Google Docs, the ERP and various internal systems. Each system had its own access method, search interface and permission scheme, so employees had to move between several systems to find what they needed.

What made it harder was that document visibility differed by department. Even within the same document, the sections a department could access and its update rights were set differently. This complex permission scheme was necessary for security, but it also dragged down work efficiency.

The direction DARVIS proposed

The key was applying DARVIS's vision of connecting company knowledge with a single question to this company's environment. Employees would not need to understand the complex system structure: they ask in natural language, and the AI finds the right information in the right system.

A test chatbot built in a short PoC

By the end of the PoC, the company was able to see results that beat its expectations. What DFINITE delivered was more than a set of numbers: the results were validated at a level that can be applied to real work.

The goal of the PoC was "to develop a chatbot that generates accurate answers based on the company's documents".

Five key documents were used as training data for this.

Real training data and a structured approach

The data DARVIS learned from were rule documents the company uses in its actual work. Documents of this kind demand high accuracy, which made them a good fit for testing the AI's performance.

Results from a three-stage evaluation

DFINITE did not just measure accuracy; it ran a structured evaluation in three stages. An LLM generated questions from the content of each document, and we evaluated whether DARVIS found the correct source for the document each question came from.

The evaluation criteria were:

• File Search: whether it finds the documents relevant to the question

• Page Search: whether it finds the exact page within the document

• Detail Search: whether it finds the specific detail the question asks about

The results beat expectations.

The evaluation was based on real work scenarios.

Rather than having people judge subjectively, we used an LLM to generate questions automatically and checked the answers. This was a way to keep bias low while measuring performance under conditions close to real use.

What makes this evaluation data important is that it tested DARVIS's ability to go past finding the document and pinpoint the exact page and detail.

What sets manufacturing-specialized AI apart

The core strength of DARVIS confirmed in this PoC was AI built for manufacturing, going beyond simple document search. That is a different approach from a typical enterprise AI chatbot.

Careful multimodal document processing

DARVIS can process both the text and the table data in PDF documents at the same time. This reflects the nature of manufacturing documents.

When complex rules are organized in tables, as in the company's practice guideline documents, simple text extraction cannot get the information right.

The strength of the hybrid search algorithm

Another core technology in DARVIS is hybrid search. A two-stage search that combines BM25 and vector search let us use the strengths of both keyword-based and meaning-based search.

The first pass uses hybrid search to find highly relevant documents; the second pass filters the top 2 documents and searches again. This approach significantly improved chunk-ID matching accuracy.

Understanding the manufacturing domain

The most important difference was a deep understanding of the manufacturing domain. Beyond searching documents, DARVIS could grasp the context of manufacturing terms such as subcontracting, suppliers and contract signing, and how they relate.

For example, for the question "What is the supplier registration procedure?", it did not just find documents containing those keywords. It could work out how the related practice guideline documents connect and give a comprehensive answer. This was possible because it understands the hierarchy and cross-references of the rule documents.

Connecting company knowledge with one question

The most striking part of this PoC was seeing DARVIS's core vision actually put into practice. Connecting complex, scattered company knowledge with one simple question is exactly the vision DARVIS pursues.

Before and after

Previously, when a question like "What were the supplier selection criteria again?" came up, an employee had to go through a complicated process:

1. Search for the relevant document in Google Docs

2. Check the related rules in the ERP system

3. Check the latest updates in internal systems

4. Compare several documents to work out the right answer

5. Time needed: a lengthy search across several systems

With the DARVIS chatbot, it changes like this:

1. Type the question into DARVIS in natural language

2. The AI analyzes the information from all systems together

3. It gives an accurate answer with the source documents and pages

4. Time needed: much shorter, in a single question

This is more than saved time; it changes the quality of the work. Employees spend less time looking for information and can focus on more valuable work.

Enterprise AI chatbot PoCs with DFINITE

This PoC shows that an enterprise AI chatbot PoC is not a complex, difficult process; with a structured approach it can be run quickly and safely.

A tested PoC method

The PoC method DFINITE used with this company is a standard process that other companies can apply too.

Step 1: Define the problem and set goals
We identify the client's real needs and set clear goals. In this case, the specific goal was generating accurate answers based on documents.

Step 2: Select key documents (around 5)
Instead of processing every document at once, we run the proof of concept on the most important documents. This keeps risk low while still showing practical results.

Step 3: A short, focused development period
A short, intensive development period gives fast results. The period is kept short enough to be a light commitment for the client while still allowing proper validation.

Step 4: Structured measurement of results
Results are measured with objective metrics, not subjective impressions.

Start your enterprise AI chatbot with a PoC

When it comes to building an enterprise AI chatbot, trying it first-hand through a PoC leads to a faster decision than deliberating indefinitely.

Take this chance to try it through a PoC with DFINITE.

DARVIS, DFINITE's enterprise AI solution, provides AI tailored to each company. With txt2sql and RAG technology, it brings together data scattered across the company and makes instant analysis possible.

Does building an enterprise AI chatbot still feel vague?

We are always open to PoC inquiries. Take a look at our brochure and get in touch.

The DARVIS brochure is in the Brochures menu at the top of this page, and the "Request a consultation" button below puts you in touch with us.