医療機関AI検索ラボMEDICAL AI SEARCH LAB

Does AI Cite FAQs Verbatim? 150 Empirical Trials

We empirically tested the common claim that "if you write an FAQ, AI will cite it verbatim" across 30 questions × 5 AI search environments and 150 responses. Zero verbatim matches. And a more important fact: search itself often does not happen.

AI Search

What this article covers

  1. The empirical results for the common claim that "AI cites FAQs verbatim"
  2. How much the Web search trigger rate differs across AI services
  3. An accurate positioning of the purpose of writing an FAQ

Conclusion

Of the 120 responses that could be checked verbatim among the 150 responses, there were zero verbatim matches with FAQ wording and zero close paraphrases. The search trigger rate ranged from 0% to 83% depending on the service, and the cited sources were centered on primary information such as government agencies and academic societies (FAQ pages accounted for 8 out of 120 responses). Rather than a means of "being copied and exposed," an FAQ is more accurately positioned as a format for organizing information.

"If you write an FAQ on your site, AI will cite it verbatim in its answers"—this is a claim you often see in explanations of AI search optimization. We empirically tested whether that is really true. This is the first installment of our common-belief verification series.

The design is as follows. We prepared 30 general questions about visiting medical institutions—how to choose a primary care physician, the high-cost medical expense benefit system, judging when to seek emergency care, and so on—and submitted them to three AI services (OpenAI, Anthropic, Google) under both a condition with Web search enabled and a condition with it disabled. That totals 150 responses. We mechanically compared the AI's answer text with the body text of the pages the AI cited, and judged them on a four-level scale: "verbatim match / close paraphrase / structure use only / no FAQ reference" (measured on July 13, 2026).

The first finding was that the premise of the common belief had collapsed. Even when set to "Web search on," whether the AI actually searches is left to the AI's own judgment. In the measurement, Anthropic's model (the lightweight version) did not trigger a search for any of the 30 questions. OpenAI searched and returned citations for 12 of the 30 questions, and Google for 25. Even when you throw the same question with the same "search on" setting, one AI looks at today's web and answers while another answers using only its training-time knowledge—before you even get to "whether the FAQ is cited," there are many cases where the search itself does not happen.

Next are the comparison results for responses in which citation did occur. Of the 120 responses for which verbatim comparison was possible, there were zero verbatim matches with FAQ wording and zero close paraphrases. Five responses were judged to have referred only to the structure, and the remaining 115 responses did not reference the FAQ. The fact that an FAQ-format page was included among the citation sources at all was limited to 8 out of 120 responses. The center of the citation sources was government pages such as the Ministry of Health, Labour and Welfare, academic societies, and primary information pages of medical institutions.

In other words, the mechanism in which "if you write an FAQ, it will be copied as is and exposed" was not observed in this measurement. Even when there is a citation source, AI integrates multiple information sources and answers by paraphrasing them into its own words. This is not to say that there is no point in writing an FAQ. The fact that a page with a structure corresponding to questions is easy to read remains as a separate consideration. It is simply that the explanation of the effect of "being cited verbatim" does not match the actual measurement.

We state the limitations clearly. This verification is an empirical measurement with 30 questions, one point in time, and specific models, and the results can change depending on the question domain and model updates. In addition, Google's citations are provided via redirect URLs, and because that relay server restricts automated access, the verbatim comparison was limited to the 120 responses from OpenAI and Anthropic (for the Google portion, only the types of citation sources were tallied, and it is classified as comparison-unverified).

FAQ for this article

Q. Does that mean there is no point in creating an FAQ page?
A. The effect of "being cited verbatim" was not observed, but a structure in which questions and answers correspond to each other is a format that is easy for both people and AI to read. We believe it is reasonable to describe the effect accurately and then position it as a means of organizing information.
Q. Why do results change depending on which AI you use?
A. It is because the criteria for deciding whether to trigger a Web search, and the way sources are selected, differ from service to service. In this measurement as well, the search trigger rate ranged from 0% to 83%.
Q. Can this verification be reproduced?
A. We have preserved the question set and the judgment criteria, so re-measurement under the same conditions is possible. Re-verification at a different point in time is also a verification theme of this site.