まず、記事の埋め込みを計算します。通常、この処理は記事ごとに一度だけ行い、埋め込みとメタデータをデータベースに保存しますが、ここでは簡単にするため、スクリプトを実行するたびに毎回計算しています。
```python lines theme={null}
from openai import OpenAI
import weave
from weave import Model
import numpy as np
import json
import asyncio
articles = [
"Novo Nordisk and Eli Lilly rival soars 32 percent after promising weight loss drug results Shares of Denmarks Zealand Pharma shot 32 percent higher in morning trade, after results showed success in its liver disease treatment survodutide, which is also on trial as a drug to treat obesity. The trial “tells us that the 6mg dose is safe, which is the top dose used in the ongoing [Phase 3] obesity trial too,” one analyst said in a note. The results come amid feverish investor interest in drugs that can be used for weight loss.",
"Berkshire shares jump after big profit gain as Buffetts conglomerate nears $1 trillion valuation Berkshire Hathaway shares rose on Monday after Warren Buffetts conglomerate posted strong earnings for the fourth quarter over the weekend. Berkshires Class A and B shares jumped more than 1.5%, each. Class A shares are higher by more than 17% this year, while Class B has gained more than 18%. Berkshire was last valued at $930.1 billion, up from $905.5 billion where it closed on Friday, according to FactSet. Berkshire on Saturday posted fourth-quarter operating earnings of $8.481 billion, about 28 percent higher than the $6.625 billion from the year-ago period, driven by big gains in its insurance business. Operating earnings refers to profits from businesses across insurance, railroads and utilities. Meanwhile, Berkshires cash levels also swelled to record levels. The conglomerate held $167.6 billion in cash in the fourth quarter, surpassing the $157.2 billion record the conglomerate held in the prior quarter.",
"Highmark Health says its combining tech from Google and Epic to give doctors easier access to information Highmark Health announced it is integrating technology from Google Cloud and the health-care software company Epic Systems. The integration aims to make it easier for both payers and providers to access key information they need, even if its stored across multiple points and formats, the company said. Highmark is the parent company of a health plan with 7 million members, a provider network of 14 hospitals and other entities",
"Rivian and Lucid shares plunge after weak EV earnings reports Shares of electric vehicle makers Rivian and Lucid fell Thursday after the companies reported stagnant production in their fourth-quarter earnings after the bell Wednesday. Rivian shares sank about 25 percent, and Lucids stock dropped around 17 percent. Rivian forecast it will make 57,000 vehicles in 2024, slightly less than the 57,232 vehicles it produced in 2023. Lucid said it expects to make 9,000 vehicles in 2024, more than the 8,428 vehicles it made in 2023.",
"Mauritius blocks Norwegian cruise ship over fears of a potential cholera outbreak Local authorities on Sunday denied permission for the Norwegian Dawn ship, which has 2,184 passengers and 1,026 crew on board, to access the Mauritius capital of Port Louis, citing “potential health risks.” The Mauritius Ports Authority said Sunday that samples were taken from at least 15 passengers on board the cruise ship. A spokesperson for the U.S.-headquartered Norwegian Cruise Line Holdings said Sunday that 'a small number of guests experienced mild symptoms of a stomach-related illness' during Norwegian Dawns South Africa voyage.",
"Intuitive Machines lands on the moon in historic first for a U.S. company Intuitive Machines Nova-C cargo lander, named Odysseus after the mythological Greek hero, is the first U.S. spacecraft to soft land on the lunar surface since 1972. Intuitive Machines is the first company to pull off a moon landing — government agencies have carried out all previously successful missions. The company's stock surged in extended trading Thursday, after falling 11 percent in regular trading.",
"Lunar landing photos: Intuitive Machines Odysseus sends back first images from the moon Intuitive Machines cargo moon lander Odysseus returned its first images from the surface. Company executives believe the lander caught its landing gear sideways on the moon's surface while touching down and tipped over. Despite resting on its side, the company's historic IM-1 mission is still operating on the moon.",
]
def docs_to_embeddings(docs: list) -> list:
openai = OpenAI()
document_embeddings = []
for doc in docs:
response = (
openai.embeddings.create(input=doc, model="text-embedding-3-small")
.data[0]
.embedding
)
document_embeddings.append(response)
return document_embeddings
article_embeddings = docs_to_embeddings(articles) # 注意: 通常はこの処理を一度だけ実行し、埋め込みとメタデータをデータベースに格納します
```
```typescript lines theme={null}
require('dotenv').config();
import { OpenAI } from 'openai';
import * as weave from 'weave';
interface Article {
text: string;
embedding?: number[];
}
const articles: Article[] = [
{
text: `Novo Nordisk and Eli Lilly rival soars 32 percent after promising weight loss drug results Shares of Denmarks Zealand Pharma shot 32 percent higher in morning trade, after results showed success in its liver disease treatment survodutide, which is also on trial as a drug to treat obesity. The trial tells us that the 6mg dose is safe, which is the top dose used in the ongoing [Phase 3] obesity trial too, one analyst said in a note. The results come amid feverish investor interest in drugs that can be used for weight loss.`
},
{
text: `Berkshire shares jump after big profit gain as Buffetts conglomerate nears $1 trillion valuation Berkshire Hathaway shares rose on Monday after Warren Buffetts conglomerate posted strong earnings for the fourth quarter over the weekend. Berkshires Class A and B shares jumped more than 1.5%, each. Class A shares are higher by more than 17% this year, while Class B has gained more than 18%. Berkshire was last valued at $930.1 billion, up from $905.5 billion where it closed on Friday, according to FactSet. Berkshire on Saturday posted fourth-quarter operating earnings of $8.481 billion, about 28 percent higher than the $6.625 billion from the year-ago period, driven by big gains in its insurance business. Operating earnings refers to profits from businesses across insurance, railroads and utilities. Meanwhile, Berkshires cash levels also swelled to record levels. The conglomerate held $167.6 billion in cash in the fourth quarter, surpassing the $157.2 billion record the conglomerate held in the prior quarter.`
},
{
text: `Highmark Health says its combining tech from Google and Epic to give doctors easier access to information Highmark Health announced it is integrating technology from Google Cloud and the health-care software company Epic Systems. The integration aims to make it easier for both payers and providers to access key information they need, even if its stored across multiple points and formats, the company said. Highmark is the parent company of a health plan with 7 million members, a provider network of 14 hospitals and other entities`
}
];
function cosineSimilarity(a: number[], b: number[]): number {
const dotProduct = a.reduce((sum, val, i) => sum + val * b[i], 0);
const magnitudeA = Math.sqrt(a.reduce((sum, val) => sum + val * val, 0));
const magnitudeB = Math.sqrt(b.reduce((sum, val) => sum + val * val, 0));
return dotProduct / (magnitudeA * magnitudeB);
}
const docsToEmbeddings = weave.op(async function(docs: Article[]): Promise {
const openai = new OpenAI();
const enrichedDocs = await Promise.all(docs.map(async (doc) => {
const response = await openai.embeddings.create({
input: doc.text,
model: "text-embedding-3-small"
});
return {
...doc,
embedding: response.data[0].embedding
};
}));
return enrichedDocs;
});
```
## RAG アプリを作成する
次に、取得関数 `get_most_relevant_document` を `weave.op()` デコレータでラップし、`Model` クラスを作成します。`weave.init('/rag-quickstart')` を呼び出して、後で確認できるように関数のすべての入力と出力のトラッキングを開始します。チーム名を指定しない場合、出力は [W\&B のデフォルト team または entity](/ja/platform/app/settings-page/user-settings#default-team) に記録されます。
```python lines theme={null}
from openai import OpenAI
import weave
from weave import Model
import numpy as np
import asyncio
@weave.op()
def get_most_relevant_document(query):
openai = OpenAI()
query_embedding = (
openai.embeddings.create(input=query, model="text-embedding-3-small")
.data[0]
.embedding
)
similarities = [
np.dot(query_embedding, doc_emb)
/ (np.linalg.norm(query_embedding) * np.linalg.norm(doc_emb))
for doc_emb in article_embeddings
]
# 最も類似するドキュメントのインデックスを取得
most_relevant_doc_index = np.argmax(similarities)
return articles[most_relevant_doc_index]
class RAGModel(Model):
system_message: str
model_name: str = "gpt-3.5-turbo-1106"
@weave.op()
def predict(self, question: str) -> dict: # 注: `question` は後で評価行からデータを選択するために使用されます
from openai import OpenAI
context = get_most_relevant_document(question)
client = OpenAI()
query = f"""Use the following information to answer the subsequent question. If the answer cannot be found, write "I don't know."
Context:
\"\"\"
{context}
\"\"\"
Question: {question}"""
response = client.chat.completions.create(
model=self.model_name,
messages=[
{"role": "system", "content": self.system_message},
{"role": "user", "content": query},
],
temperature=0.0,
response_format={"type": "text"},
)
answer = response.choices[0].message.content
return {'answer': answer, 'context': context}
# チーム名とプロジェクト名を設定
weave.init('/rag-quickstart')
model = RAGModel(
system_message="You are an expert in finance and answer questions related to finance, financial services, and financial markets. When responding based on provided information, be sure to cite the source."
)
model.predict("What significant result was reported about Zealand Pharma's obesity trial?")
```
```typescript lines theme={null}
class RAGModel {
private openai: OpenAI;
private systemMessage: string;
private modelName: string;
private articleEmbeddings: Article[];
constructor(config: {
systemMessage: string;
modelName?: string;
articleEmbeddings: Article[];
}) {
this.openai = new OpenAI();
this.systemMessage = config.systemMessage;
this.modelName = config.modelName || "gpt-3.5-turbo-1106";
this.articleEmbeddings = config.articleEmbeddings;
this.predict = weave.op(this, this.predict);
}
async predict(question: string): Promise<{
answer: string;
context: string;
}> {
const context = await this.getMostRelevantDocument(question);
const response = await this.openai.chat.completions.create({
model: this.modelName,
messages: [
{ role: "system", content: this.systemMessage },
{ role: "user", content: `Use the following information to answer the subsequent question. If the answer cannot be found, write "I don't know."
Context:
"""
${context}
"""
Question: ${question}` }
],
temperature: 0
});
return {
answer: response.choices[0].message.content || "",
context
};
}
}
```
RAG アプリで同じ結果を得るには、次のようにします。
* LLM 呼び出しと検索 step 関数を `weave.op()` でラップする
* (任意) `predict` 関数とアプリの詳細を持つ `Model` のサブクラスを作成する
* 評価用の例を収集する
* 1 つの例を採点するスコアリング関数を作成する
* `Evaluation` クラスを使用して、例に対して評価を実行する
**注:** `Evaluation` の非同期実行により、OpenAI や Anthropic などのモデルで rate limit に達することがあります。これを防ぐには、並列ワーカー数を制限する環境変数を設定します。たとえば `WEAVE_PARALLELISM=3` です。
以下にコード全体を示します。
```python lines {34,52-77} theme={null}
from openai import OpenAI
import weave
from weave import Model
import numpy as np
import json
import asyncio
# 評価に使用するサンプル
articles = [
"Novo Nordisk and Eli Lilly rival soars 32 percent after promising weight loss drug results Shares of Denmarks Zealand Pharma shot 32 percent higher in morning trade, after results showed success in its liver disease treatment survodutide, which is also on trial as a drug to treat obesity. The trial “tells us that the 6mg dose is safe, which is the top dose used in the ongoing [Phase 3] obesity trial too,” one analyst said in a note. The results come amid feverish investor interest in drugs that can be used for weight loss.",
"Berkshire shares jump after big profit gain as Buffetts conglomerate nears $1 trillion valuation Berkshire Hathaway shares rose on Monday after Warren Buffetts conglomerate posted strong earnings for the fourth quarter over the weekend. Berkshires Class A and B shares jumped more than 1.5%, each. Class A shares are higher by more than 17% this year, while Class B has gained more than 18%. Berkshire was last valued at $930.1 billion, up from $905.5 billion where it closed on Friday, according to FactSet. Berkshire on Saturday posted fourth-quarter operating earnings of $8.481 billion, about 28 percent higher than the $6.625 billion from the year-ago period, driven by big gains in its insurance business. Operating earnings refers to profits from businesses across insurance, railroads and utilities. Meanwhile, Berkshires cash levels also swelled to record levels. The conglomerate held $167.6 billion in cash in the fourth quarter, surpassing the $157.2 billion record the conglomerate held in the prior quarter.",
"Highmark Health says its combining tech from Google and Epic to give doctors easier access to information Highmark Health announced it is integrating technology from Google Cloud and the health-care software company Epic Systems. The integration aims to make it easier for both payers and providers to access key information they need, even if it's stored across multiple points and formats, the company said. Highmark is the parent company of a health plan with 7 million members, a provider network of 14 hospitals and other entities",
"Rivian and Lucid shares plunge after weak EV earnings reports Shares of electric vehicle makers Rivian and Lucid fell Thursday after the companies reported stagnant production in their fourth-quarter earnings after the bell Wednesday. Rivian shares sank about 25 percent, and Lucids stock dropped around 17 percent. Rivian forecast it will make 57,000 vehicles in 2024, slightly less than the 57,232 vehicles it produced in 2023. Lucid said it expects to make 9,000 vehicles in 2024, more than the 8,428 vehicles it made in 2023.",
"Mauritius blocks Norwegian cruise ship over fears of a potential cholera outbreak Local authorities on Sunday denied permission for the Norwegian Dawn ship, which has 2,184 passengers and 1,026 crew on board, to access the Mauritius capital of Port Louis, citing “potential health risks.” The Mauritius Ports Authority said Sunday that samples were taken from at least 15 passengers on board the cruise ship. A spokesperson for the U.S.-headquartered Norwegian Cruise Line Holdings said Sunday that 'a small number of guests experienced mild symptoms of a stomach-related illness' during Norwegian Dawns South Africa voyage.",
"Intuitive Machines lands on the moon in historic first for a U.S. company Intuitive Machines Nova-C cargo lander, named Odysseus after the mythological Greek hero, is the first U.S. spacecraft to soft land on the lunar surface since 1972. Intuitive Machines is the first company to pull off a moon landing — government agencies have carried out all previously successful missions. The company's stock surged in extended trading Thursday, after falling 11 percent in regular trading.",
"Lunar landing photos: Intuitive Machines Odysseus sends back first images from the moon Intuitive Machines cargo moon lander Odysseus returned its first images from the surface. Company executives believe the lander caught its landing gear sideways on the surface of the moon while touching down and tipped over. Despite resting on its side, the company's historic IM-1 mission is still operating on the moon.",
]
def docs_to_embeddings(docs: list) -> list:
openai = OpenAI()
document_embeddings = []
for doc in docs:
response = (
openai.embeddings.create(input=doc, model="text-embedding-3-small")
.data[0]
.embedding
)
document_embeddings.append(response)
return document_embeddings
article_embeddings = docs_to_embeddings(articles) # 注意: 通常はこの処理を一度だけ実行し、埋め込みとメタデータをデータベースに保存します
# 検索ステップにデコレーターを追加する
@weave.op()
def get_most_relevant_document(query):
openai = OpenAI()
query_embedding = (
openai.embeddings.create(input=query, model="text-embedding-3-small")
.data[0]
.embedding
)
similarities = [
np.dot(query_embedding, doc_emb)
/ (np.linalg.norm(query_embedding) * np.linalg.norm(doc_emb))
for doc_emb in article_embeddings
]
# 最も類似したドキュメントのインデックスを取得する
most_relevant_doc_index = np.argmax(similarities)
return articles[most_relevant_doc_index]
# アプリの詳細を含む Model サブクラスを作成し、レスポンスを生成する predict 関数を定義する
class RAGModel(Model):
system_message: str
model_name: str = "gpt-3.5-turbo-1106"
@weave.op()
def predict(self, question: str) -> dict: # 注意: `question` は後で評価行からデータを選択する際に使用されます
from openai import OpenAI
context = get_most_relevant_document(question)
client = OpenAI()
query = f"""以下の情報をもとに、後続の質問に答えてください。答えが見つからない場合は「わかりません」と書いてください。
コンテキスト:
\"\"\"
{context}
\"\"\"
質問: {question}"""
response = client.chat.completions.create(
model=self.model_name,
messages=[
{"role": "system", "content": self.system_message},
{"role": "user", "content": query},
],
temperature=0.0,
response_format={"type": "text"},
)
answer = response.choices[0].message.content
return {'answer': answer, 'context': context}
# チームとプロジェクトの名前を設定する
weave.init('/rag-quickstart')
model = RAGModel(
system_message="あなたは金融の専門家です。金融、金融サービス、金融市場に関する質問に答えてください。提供された情報に基づいて回答する際は、必ずソースを引用してください。"
)
# 質問と出力を使用してスコアを算出するスコアリング関数
@weave.op()
async def context_precision_score(question, output):
context_precision_prompt = """質問、回答、コンテキストをもとに、そのコンテキストが回答を導き出すのに役立ったかどうかを検証してください。役立った場合は「1」、そうでない場合は「0」をJSON形式で出力してください。
有効なJSON形式のみで出力してください。
question: {question}
context: {context}
answer: {answer}
verdict: """
client = OpenAI()
prompt = context_precision_prompt.format(
question=question,
context=output['context'],
answer=output['answer'],
)
response = client.chat.completions.create(
model="gpt-4-turbo-preview",
messages=[{"role": "user", "content": prompt}],
response_format={ "type": "json_object" }
)
response_message = response.choices[0].message
response = json.loads(response_message.content)
return {
"verdict": int(response["verdict"]) == 1,
}
questions = [
{"question": "Zealand Pharmaの肥満治療薬試験について報告された重要な結果は何ですか？"},
{"question": "Berkshire Hathawayの第4四半期における現金保有額はどれくらい増加しましたか？"},
{"question": "Highmark HealthによるGoogle CloudとEpic Systemsの技術統合の目的は何ですか？"},
{"question": "RivianとLucidの2024年の車両生産予測はどのようなものでしたか？"},
{"question": "Norwegian Dawnクルーズ船がモーリシャスへの入港を拒否された理由は何ですか？"},
{"question": "1972年以来初めて米国の月面着陸を達成した企業はどこですか？"},
{"question": "Intuitive Machinesの月面着陸船が着陸時に遭遇した問題は何ですか？"}
]
# Evaluation オブジェクトを定義し、サンプルの質問とスコアリング関数を渡す
evaluation = weave.Evaluation(dataset=questions, scorers=[context_precision_score])
asyncio.run(evaluation.evaluate(model))
```
```typescript lines theme={null}
require('dotenv').config();
import { OpenAI } from 'openai';
import * as weave from 'weave';
interface Article {
text: string;
embedding?: number[];
}
const articles: Article[] = [
{
text: `ノボ・ノルディスクとイーライ・リリーのライバル企業、有望な減量薬の結果を受けて株価32%急騰 デンマークのZealand Pharmaの株価は、肝疾患治療薬survodutideの試験で成功が示されたことを受け、午前の取引で32%上昇した。同薬は肥満治療薬としても臨床試験中である。「この試験により、6mgの用量が安全であることが確認された。これは現在進行中の[フェーズ3]肥満試験でも使用されている最高用量だ」とあるアナリストはレポートで述べた。この結果は、減量に使用できる薬への投資家の熱狂的な関心が高まる中で発表された。`
},
{
text: `バークシャーの株価、大幅増益を受けて急騰——バフェットのコングロマリットが時価総額1兆ドルに迫る ウォーレン・バフェット率いるBerkshire Hathawayが週末に第4四半期の好決算を発表したことを受け、同社株は月曜日に上昇した。クラスAおよびクラスB株はそれぞれ1.5%超上昇した。クラスA株は今年に入り17%超、クラスB株は18%超の上昇となっている。FactSetによると、バークシャーの時価総額は直近で9,301億ドルとなり、金曜日の終値時点の9,055億ドルから上昇した。バークシャーは土曜日、第4四半期の営業利益が84億8,100万ドルと発表した。これは前年同期の66億2,500万ドルから約28%増加しており、保険事業の大幅な利益増が牽引した。営業利益とは、保険、鉄道、公益事業などの各事業からの利益を指す。一方、バークシャーの現金保有額も過去最高水準に膨らんだ。同コングロマリットの第4四半期末の現金保有額は1,676億ドルとなり、前四半期に記録した1,572億ドルを上回った。`
},
{
text: `Highmark Health、GoogleとEpicの技術を統合して医師の情報アクセスを容易に Highmark Healthは、Google Cloudおよび医療ソフトウェア企業Epic Systemsの技術を統合すると発表した。同社によると、このインテグレーションは、情報が複数の場所や形式にまたがって保存されている場合でも、支払者とプロバイダーの双方が必要な重要情報にアクセスしやすくすることを目的としている。Highmarkは700万人の会員を持つ医療保険プランと、14の病院およびその他の関連組織からなるプロバイダーネットワークの親会社である。`
}
];
function cosineSimilarity(a: number[], b: number[]): number {
const dotProduct = a.reduce((sum, val, i) => sum + val * b[i], 0);
const magnitudeA = Math.sqrt(a.reduce((sum, val) => sum + val * val, 0));
const magnitudeB = Math.sqrt(b.reduce((sum, val) => sum + val * val, 0));
return dotProduct / (magnitudeA * magnitudeB);
}
const docsToEmbeddings = weave.op(async function(docs: Article[]): Promise {
const openai = new OpenAI();
const enrichedDocs = await Promise.all(docs.map(async (doc) => {
const response = await openai.embeddings.create({
input: doc.text,
model: "text-embedding-3-small"
});
return {
...doc,
embedding: response.data[0].embedding
};
}));
return enrichedDocs;
});
class RAGModel {
private openai: OpenAI;
private systemMessage: string;
private modelName: string;
private articleEmbeddings: Article[];
constructor(config: {
systemMessage: string;
modelName?: string;
articleEmbeddings: Article[];
}) {
this.openai = new OpenAI();
this.systemMessage = config.systemMessage;
this.modelName = config.modelName || "gpt-3.5-turbo-1106";
this.articleEmbeddings = config.articleEmbeddings;
this.predict = weave.op(this, this.predict);
}
private async getMostRelevantDocument(query: string): Promise {
const queryEmbedding = await this.openai.embeddings.create({
input: query,
model: "text-embedding-3-small"
});
const similarities = this.articleEmbeddings.map(doc => {
if (!doc.embedding) return 0;
return cosineSimilarity(queryEmbedding.data[0].embedding, doc.embedding);
});
const mostRelevantIndex = similarities.indexOf(Math.max(...similarities));
return this.articleEmbeddings[mostRelevantIndex].text;
}
async predict(question: string): Promise<{
answer: string;
context: string;
}> {
const context = await this.getMostRelevantDocument(question);
const response = await this.openai.chat.completions.create({
model: this.modelName,
messages: [
{ role: "system", content: this.systemMessage },
{
role: "user",
content: `以下の情報を参考に、後続の質問に回答してください。回答が見つからない場合は「わかりません」と記述してください。
コンテキスト:
"""
${context}
"""
質問: ${question}`
}
],
temperature: 0
});
return {
answer: response.choices[0].message.content || "",
context
};
}
}
interface ScorerResult {
verdict: boolean;
}
interface QuestionRow {
question: string;
}
function createQuestionDataset(): weave.Dataset {
return new weave.Dataset({
id: 'rag-questions',
rows: [
{ question: "Zealand Pharmaの肥満試験について、どのような重要な結果が報告されましたか？" },
{ question: "Berkshire Hathawayの第4四半期における現金保有額はどれだけ増加しましたか？" },
{ question: "Highmark HealthによるGoogle CloudとEpic Systemsの技術統合の目的は何ですか？" }
]
});
}
const contextPrecisionScore = weave.op(async function(args: {
datasetRow: QuestionRow;
modelOutput: { answer: string; context: string; }
}): Promise {
const openai = new OpenAI();
const prompt = `質問、回答、およびコンテキストをもとに、そのコンテキストが回答の導出に役立ったかどうかを検証してください。役立った場合は "1"、そうでない場合は "0" をJSON形式で出力してください。
有効なJSON形式のみで出力してください。
question: ${args.datasetRow.question}
context: ${args.modelOutput.context}
answer: ${args.modelOutput.answer}
verdict: `;
const response = await openai.chat.completions.create({
model: "gpt-4-turbo-preview",
messages: [{ role: "user", content: prompt }],
response_format: { type: "json_object" }
});
const result = JSON.parse(response.choices[0].message.content || "{}");
return {
verdict: parseInt(result.verdict) === 1
};
});
async function main() {
# チームとプロジェクト名を設定する
await weave.init('/rag-quickstart');
const articleEmbeddings = await docsToEmbeddings(articles);
const model = new RAGModel({
systemMessage: "あなたは金融の専門家であり、金融、金融サービス、および金融市場に関する質問に回答します。提供された情報に基づいて回答する際は、必ずソースを引用してください。",
articleEmbeddings
});
const evaluation = new weave.Evaluation({
dataset: createQuestionDataset(),
scorers: [contextPrecisionScore]
});
const results = await evaluation.evaluate({
model: weave.op((args: { datasetRow: QuestionRow }) =>
model.predict(args.datasetRow.question)
)
});
console.log('評価結果:', results);
}
if (require.main === module) {
main().catch(console.error);
}
```
