This article is written from the perspective of GFLOPS Co., Ltd., provider of the enterprise generative AI platform AskDona. Drawing on our firsthand experience deploying AskDona for many organizations, including RIKEN, we candidly discuss the conditions that separate implementations that stall at the PoC stage from those that scale, as well as the design decisions we have made as a solution provider. Rather than presenting general theory, we focus on concrete examples of where projects have actually encountered obstacles.

Key argument The difference between teams that escape the PoC trap and those that stay stuck lies not in the technology, but in how the organization operates.
Three traits of projects that scale Business owners treat the initiative as their own / the initial use case is tightly scoped / lack of adoption is made an early KPI
Three pitfalls that cause projects to stall Perfectionism in data preparation / mistaking effort for outcomes / rebuilding technology in-house
What sets AskDona apart No domain-specific tuning is required; customers follow the same three-step process in every use case.

Three Characteristics of Organizations That Scale Successfully

We will begin with the characteristics shared by customers that have moved from PoC to production and achieved sustained adoption. These are not questions of technology stack or industry; they are questions of how the organization operates.

First, the business owner treats the project as their own responsibility. Projects led independently by IT almost invariably lose momentum midway through. By contrast, when a business-side leader—such as the head of support, a laboratory director, or a sales planning executive—takes ownership and connects the initiative to their own performance goals, the project is far more likely to cross organizational boundaries. In RIKEN’s case, the on-site R-CCS leader set the operational goal of helping users resolve issues on their own and positioned AskDona as the means to achieve it. That alignment became a major driver of the project.

Second, the initial use case is small, but clearly defined. Projects that begin with a vague ambition such as “make all internal knowledge searchable” often become bogged down in data-source preparation. By contrast, when the scope, users, and success criteria are narrow—for example, “help Fugaku users resolve initial inquiries on their own”—a meaningful PoC can be completed within three months, creating the evidence needed to secure funding for the next phase.

Third, lack of adoption is made an early KPI. Every organization goes through a period when usage in the first deployed department is slow to take off. Organizations that can measure non-use can identify missing data sources, improve the user journey, and revise training programs. Those that cannot are left with little more than the vague impression that the project failed.

Three Common Patterns in Projects That Stall

Now for the opposite side. Projects that sink into the PoC trap tend to follow three patterns.

First, too much effort is spent perfecting data before the project begins. The idea of entering production only after building a perfect dataset sounds sensible, but it is a classic pattern that delays the start of a PoC by six months to a year. Organizational documents are living assets and continue to change even while they are being prepared. If perfection is the prerequisite, the project never starts. In practice, it is faster to begin with the documents available today, identify gaps through actual use, and improve them iteratively. That approach ultimately reaches high accuracy sooner.

Second, too much effort goes into system tuning, while validation of business impact is deferred. Some organizations become absorbed in an endless technical improvement loop: “Perhaps we should adjust the chunk size a little more,” or “Maybe we should try a different embedding model.” Technical reports accumulate, documenting how many benchmark points were gained over six months, while the business impact remains unmeasured. At a deeper level, teams may feel compelled to treat the effort itself as an achievement, allowing the ROI discussion to be replaced by a technical one. Executives ask, “How did the work actually change?” and the project team responds, “Here is how much tuning we completed.” Effort and outcomes are not the same.

Third, the organization tries to recreate a less mature version of the technology in-house. LLM and RAG technologies are improved on a weekly basis by foundation-model providers, SaaS vendors, and open-source communities. Attempting to keep pace with limited internal resources often results in a triple burden: being six months behind, falling short on quality, and carrying ongoing operational overhead. The assumption that building in-house will be cheaper rarely holds once engineering time, hiring costs, operations, and maintenance are included. Organizations should retain control over use-case and data selection, while relying on external specialists for the internal mechanics of RAG. When that boundary is drawn incorrectly, resources are continually diverted into technology development that is not the organization’s core business.

Why We Designed Knowledge Setup as a Three-Step Process

AskDona’s RAG knowledge-building workflow was deliberately reduced to three steps—upload, add context that helps the AI understand the content, and automatic data processing. There is a clear reason for this.

One lesson we saw repeatedly during implementations was that the burden of data preparation often determines whether a PoC succeeds. “How should the content be chunked for vectorization?” “Which embedding model should we use?” “Please design the metadata schema.” If customers are required to make these decisions themselves, one or two months can disappear into technical-selection discussions.

At AskDona, we chose to absorb the complexity of technical selection on the provider side. By enabling users to upload documents by drag and drop and complete knowledge setup in only a few clicks, we narrowed the customer’s decision to a business question: Which documents should be included?

AskDona also includes advanced OCR that extracts textual information from documents containing images, charts, and diagrams. In manufacturing manuals and technical documents used by research institutions, critical information is often embedded in visual elements. A RAG system that can reference only body text may be unable to answer more than half of real-world questions. We therefore made the extraction of information from charts and diagrams a standard technical capability.

A design in which the provider absorbs the technical decisions involved in building RAG

It is important to emphasize that conventional RAG systems generally require custom tuning for each customer’s business domain, data characteristics, and expected queries. Adjusting chunk size, selecting embedding models, and optimizing search-algorithm parameters are typically customized for each industry and organization. It is not unusual to see the same solution moved into another industry, fail to perform, and trigger a new retuning project.

What fundamentally distinguishes AskDona from other RAG systems is that this custom tuning is not required. The customer follows the same three-step process regardless of whether the documents are manufacturing specifications, specialist manuals at a research institution, or policy collections in financial services. AskDona has also achieved production-level accuracy on approximately 16,000 pages of specialist documentation for RIKEN’s Fugaku supercomputer without additional tuning (see Chapter 4 of the Agentic RAG Accuracy Benchmark Report, which publishes comparative results against Azure AI Search, Vertex AI Search, Amazon Bedrock, and LangChain).

The foundation for this is our proprietary Agentic RAG architecture. It autonomously decomposes questions into subqueries and integrates information across multiple documents, enabling high accuracy across a wide range of business domains without relying on case-by-case optimization. AskDona’s promise at the outset is not merely that the 90 days of a PoC will not be consumed by technology selection; it is that customers do not need to make those technical selections in the first place.

What the Data Improvement Assistant Makes Visible

One of the most consistent findings across implementations was that gaps in the available data inevitably become visible within the first one to three months of operation. Customers often begin with the goal of answering every possible question, but once the system is used in practice, unexpected questions quickly emerge.

How these unexpected questions are handled determines whether the initiative moves beyond the PoC. AskDona therefore includes a Data Improvement Assistant for administrators. It automatically identifies questions the AI could not answer adequately and suggests what kinds of documents should be added to improve coverage.

This changes how the person responsible for operations interprets the issue:

  • Previous response: “The AI is not smart enough; complain to the vendor.”
  • Improved response: “We have discovered that this document does not exist internally; ask the responsible business team to create it.”

By making the issue visible as a gap in organizational knowledge rather than simply a failure of AI accuracy, the improvement loop can be owned within the organization. Instead of freezing the PoC conclusion as “effective” or “ineffective,” we embedded a mechanism that improves progressively through continued operation.

Principles Behind Our UI and User Journeys

At the heart of the design is a UI built around the human-in-the-loop principle. Every decision in the interface is grounded in a vision of role allocation: work that AI can perform should be handled by AI, while people should focus on work where human judgment creates distinctive value. AI is not intended to replace people; it supports them so they can concentrate on higher-value work. This philosophy is embedded in small details throughout the interface.

Always show the evidence behind an answer. The source documents appear beside each response and can be previewed directly as PDFs. Rather than presenting an AI answer as an unquestionable conclusion, the interface gives people the material they need to make the final decision. Authority remains with the user.

Be honest when the system cannot answer. When the source documents contain no supporting evidence, the system does not manufacture an answer; it preserves a path to a human expert. The design does not aim for 100% automation. Instead, it distinguishes between what AI can handle and what should remain with people.

Build in a feedback loop. Users can mark responses as helpful or unhelpful, and the results are aggregated in the administrator dashboard. Human judgment becomes organizational input for improving the AI.

Deciding at the outset not only what AI should do, but also what it should not do, is essential to maintaining motivation in the workplace. AI supports; people remain at the center. This allocation of roles makes the organization stronger.

What Actually Happened in Fugaku Support

The deployment of AskDona on the support site for RIKEN’s Fugaku supercomputer produced the most comprehensive set of verification data we have been able to publish as a provider. The key figures are as follows.

  • It handles an average of approximately 680 questions per month and has become an established part of day-to-day support.
  • Over the first ten months, the share of complex queries requiring the integration of information from multiple sources rose from 2.3% to 10.0%—an increase of approximately 4.3 times. This suggests that users began treating the system not merely as a search tool, but as a partner in reasoning.
  • In April 2025, the number of inquiry tickets escalated to human support fell by as much as 61% year over year.
  • In one documented case, an inquiry that previously required approximately four hours of staff time was resolved in about five seconds.
  • Following approximately six months of operational validation after the PoC began in July 2024, initial inquiries were fully transitioned to AskDona in February 2025.

The most important lesson from this project was that a full transition is possible when it is approached in stages. We often hear that an immediate full-scale transition to generative AI feels too risky. Yet even in highly specialized domains, a complete transition is achievable when the organization follows a deliberate sequence: PoC, partial operation, evaluation, and full transition. Detailed operational trends and accuracy-validation data are available in AskDona × Fugaku Support: Operational Performance Report.

Conclusion: Success Conditions Seen from the Provider’s Perspective

To close, here is a concise summary of the conditions we have observed in implementations that scale.

  • The business owner treats the project as their own responsibility.
  • The initial use case is small and tightly scoped.
  • Data preparation and the user journey are not treated as secondary concerns.
  • Lack of adoption is made an early KPI.
  • Accuracy evaluation at the PoC stage is not used as a final gate; the system is designed to improve through operation.
  • Provider capabilities—such as the administrator dashboard, Data Improvement Assistant, and source preview—are built into the operational loop.

Building the most technically advanced RAG system and building a RAG system that an organization continues to use are two different challenges. From the provider’s perspective, the important point is not to confuse them. The latter must be addressed through operational design.

The evidence behind the figures in this article, ten months of operational trends, Agentic RAG accuracy comparisons, and details of the phased transition from PoC to full deployment are published in full in AskDona × Fugaku Support: Operational Performance Report. We hope it will serve as a useful reference when preparing your implementation plan.

Interested in AskDona?

Thank you for reading. The design decisions and operational insights discussed in this article are grounded in our experience implementing and operating AskDona, an enterprise generative AI platform. By integrating RAG, GPT, and Deep Research in a practical platform, AskDona is used by organizations to make better use of internal documents, automate customer support, and enhance research and analysis.

For details about the service, its features, customer stories, or how to contact us, visit the AskDona website. We also welcome discussions about how AskDona could address specific operational challenges through our contact form.

References and Related Links