Skip to content
This repository was archived by the owner on Nov 21, 2024. It is now read-only.

Commit 846a752

Browse files
hyper-clovaErick Friisvbarda
authored
community: Add Naver chat model & embeddings (langchain-ai#25162)
Reopened as a personal repo outside the organization. ## Description - Naver HyperCLOVA X community package - Add chat model & embeddings - Add unit test & integration test - Add chat model & embeddings docs - I changed partner package(langchain-ai#24252) to community package on this PR - Could this embeddings(langchain-ai#21890) be deprecated? We are trying to replace it with embedding model(**ClovaXEmbeddings**) in this PR. Twitter handle: None. (if needed, contact with joonha.jeon@navercorp.com) --- you can check our previous discussion below: > one question on namespaces - would it make sense to have these in .clova namespaces instead of .naver? I would like to keep it as is, unless it is essential to unify the package name. (ClovaX is a branding for the model, and I plan to add other models and components. They need to be managed as separate classes.) > also, could you clarify the difference between ClovaEmbeddings and ClovaXEmbeddings? There are 3 models that are being serviced by embedding, and all are supported in the current PR. In addition, all the functionality of CLOVA Studio that serves actual models, such as distinguishing between test apps and service apps, is supported. The existing PR does not support this content because it is hard-coded. --------- Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: Vadym Barda <vadym@langchain.dev>
1 parent 6227396 commit 846a752

File tree

17 files changed

+1871
-1
lines changed

17 files changed

+1871
-1
lines changed

‎docs/docs/integrations/chat/naver.ipynb

Lines changed: 444 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/docs/integrations/providers/naver.mdx

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# NAVER
2+
3+
All functionality related to `Naver`, including HyperCLOVA X models, particularly those accessible through `Naver Cloud` [CLOVA Studio](https://clovastudio.ncloud.com/).
4+
5+
> [Naver](https://navercorp.com/) is a global technology company with cutting-edge technologies and a diverse business portfolio including search, commerce, fintech, content, cloud, and AI.
6+
7+
> [Naver Cloud](https://www.navercloudcorp.com/lang/en/) is the cloud computing arm of Naver, a leading cloud service provider offering a comprehensive suite of cloud services to businesses through its [Naver Cloud Platform (NCP)](https://www.ncloud.com/).
8+
9+
Please refer to [NCP User Guide](https://guide.ncloud-docs.com/docs/clovastudio-overview) for more detailed instructions (also in Korean).
10+
11+
## Installation and Setup
12+
13+
- Get both CLOVA Studio API Key and API Gateway Key by [creating your app](https://guide.ncloud-docs.com/docs/en/clovastudio-playground01#create-test-app) and set them as environment variables respectively (`NCP_CLOVASTUDIO_API_KEY`, `NCP_APIGW_API_KEY`).
14+
- Install the integration Python package with:
15+
16+
```bash
17+
pip install -U langchain-community
18+
```
19+
20+
## Chat models
21+
22+
### ChatClovaX
23+
24+
See a [usage example](/docs/integrations/chat/naver).
25+
26+
```python
27+
from langchain_community.chat_models import ChatClovaX
28+
```
29+
30+
## Embedding models
31+
32+
### ClovaXEmbeddings
33+
34+
See a [usage example](/docs/integrations/text_embedding/naver).
35+
36+
```python
37+
from langchain_community.embeddings import ClovaXEmbeddings
38+
```

‎docs/docs/integrations/text_embedding/naver.ipynb

Lines changed: 318 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,318 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "raw",
5+
"id": "afaf8039",
6+
"metadata": {},
7+
"source": [
8+
"---\n",
9+
"sidebar_label: Naver\n",
10+
"---"
11+
]
12+
},
13+
{
14+
"cell_type": "markdown",
15+
"id": "e49f1e0d",
16+
"metadata": {},
17+
"source": [
18+
"# ClovaXEmbeddings\n",
19+
"\n",
20+
"This notebook covers how to get started with embedding models provided by CLOVA Studio. For detailed documentation on `ClovaXEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/community/embeddings/langchain_community.naver.ClovaXEmbeddings.html).\n",
21+
"\n",
22+
"## Overview\n",
23+
"### Integration details\n",
24+
"\n",
25+
"| Provider | Package |\n",
26+
"|:--------:|:-------:|\n",
27+
"| [Naver](/docs/integrations/providers/naver.mdx) | [langchain-community](https://python.langchain.com/api_reference/community/embeddings/langchain_community.naver.ClovaXEmbeddings.html) |\n",
28+
"\n",
29+
"## Setup\n",
30+
"\n",
31+
"Before using embedding models provided by CLOVA Studio, you must go through the three steps below.\n",
32+
"\n",
33+
"1. Creating [NAVER Cloud Platform](https://www.ncloud.com/) account \n",
34+
"2. Apply to use [CLOVA Studio](https://www.ncloud.com/product/aiService/clovaStudio)\n",
35+
"3. Find API Keys after creating CLOVA Studio Test App or Service App (See [here](https://guide.ncloud-docs.com/docs/en/clovastudio-playground01#테스트앱생성).)\n",
36+
"\n",
37+
"### Credentials\n",
38+
"\n",
39+
"CLOVA Studio requires 3 keys (`NCP_CLOVASTUDIO_API_KEY`, `NCP_APIGW_API_KEY` and `NCP_CLOVASTUDIO_APP_ID`) for embeddings.\n",
40+
"- `NCP_CLOVASTUDIO_API_KEY` and `NCP_CLOVASTUDIO_APP_ID` is issued per serviceApp or testApp\n",
41+
"- `NCP_APIGW_API_KEY` is issued per account\n",
42+
"\n",
43+
"The two API Keys could be found by clicking `App Request Status` > `Service App, Test App List` > `‘Details’ button for each app` in [CLOVA Studio](https://clovastudio.ncloud.com/studio-application/service-app)."
44+
]
45+
},
46+
{
47+
"cell_type": "code",
48+
"execution_count": null,
49+
"id": "c52e8a50-3e67-4272-bc80-3954d98f8dea",
50+
"metadata": {},
51+
"outputs": [],
52+
"source": [
53+
"import getpass\n",
54+
"import os\n",
55+
"\n",
56+
"if not os.getenv(\"NCP_CLOVASTUDIO_API_KEY\"):\n",
57+
" os.environ[\"NCP_CLOVASTUDIO_API_KEY\"] = getpass.getpass(\n",
58+
" \"Enter NCP CLOVA Studio API Key: \"\n",
59+
" )\n",
60+
"if not os.getenv(\"NCP_APIGW_API_KEY\"):\n",
61+
" os.environ[\"NCP_APIGW_API_KEY\"] = getpass.getpass(\"Enter NCP API Gateway API Key: \")"
62+
]
63+
},
64+
{
65+
"cell_type": "code",
66+
"execution_count": null,
67+
"id": "83520d8e-ecf8-4e47-b3bc-1ac205b3a2ab",
68+
"metadata": {},
69+
"outputs": [],
70+
"source": [
71+
"os.environ[\"NCP_CLOVASTUDIO_APP_ID\"] = input(\"Enter NCP CLOVA Studio App ID: \")"
72+
]
73+
},
74+
{
75+
"cell_type": "markdown",
76+
"id": "ff00653e",
77+
"metadata": {},
78+
"source": [
79+
"### Installation\n",
80+
"\n",
81+
"ClovaXEmbeddings integration lives in the `langchain_community` package:"
82+
]
83+
},
84+
{
85+
"cell_type": "code",
86+
"execution_count": null,
87+
"id": "99400c9b",
88+
"metadata": {},
89+
"outputs": [],
90+
"source": [
91+
"# install package\n",
92+
"!pip install -U langchain-community"
93+
]
94+
},
95+
{
96+
"cell_type": "markdown",
97+
"id": "2651e611-9d5b-4315-9bbd-f99f56be4e19",
98+
"metadata": {},
99+
"source": [
100+
"## Instantiation\n",
101+
"\n",
102+
"Now we can instantiate our embeddings object and embed query or document:\n",
103+
"\n",
104+
"- There are several embedding models available in CLOVA Studio. Please refer [here](https://guide.ncloud-docs.com/docs/en/clovastudio-explorer03#임베딩API) for further details.\n",
105+
"- Note that you might need to normalize the embeddings depending on your specific use case."
106+
]
107+
},
108+
{
109+
"cell_type": "code",
110+
"execution_count": 7,
111+
"id": "62e0dbc3",
112+
"metadata": {
113+
"scrolled": true,
114+
"tags": []
115+
},
116+
"outputs": [],
117+
"source": [
118+
"from langchain_community.embeddings import ClovaXEmbeddings\n",
119+
"\n",
120+
"embeddings = ClovaXEmbeddings(\n",
121+
" model=\"clir-emb-dolphin\", # set with the model name of corresponding app id. Default is `clir-emb-dolphin`\n",
122+
" # app_id=\"...\" # set if you prefer to pass app id directly instead of using environment variables\n",
123+
")"
124+
]
125+
},
126+
{
127+
"cell_type": "markdown",
128+
"id": "0493b4a8",
129+
"metadata": {},
130+
"source": [
131+
"## Indexing and Retrieval\n",
132+
"\n",
133+
"Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our RAG tutorials under the [working with external knowledge tutorials](/docs/tutorials/#working-with-external-knowledge).\n",
134+
"\n",
135+
"Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`."
136+
]
137+
},
138+
{
139+
"cell_type": "code",
140+
"execution_count": 8,
141+
"id": "d4d59653",
142+
"metadata": {},
143+
"outputs": [
144+
{
145+
"data": {
146+
"text/plain": [
147+
"'CLOVA Studio is an AI development tool that allows you to customize your own HyperCLOVA X models.'"
148+
]
149+
},
150+
"execution_count": 8,
151+
"metadata": {},
152+
"output_type": "execute_result"
153+
}
154+
],
155+
"source": [
156+
"# Create a vector store with a sample text\n",
157+
"from langchain_core.vectorstores import InMemoryVectorStore\n",
158+
"\n",
159+
"text = \"CLOVA Studio is an AI development tool that allows you to customize your own HyperCLOVA X models.\"\n",
160+
"\n",
161+
"vectorstore = InMemoryVectorStore.from_texts(\n",
162+
" [text],\n",
163+
" embedding=embeddings,\n",
164+
")\n",
165+
"\n",
166+
"# Use the vectorstore as a retriever\n",
167+
"retriever = vectorstore.as_retriever()\n",
168+
"\n",
169+
"# Retrieve the most similar text\n",
170+
"retrieved_documents = retriever.invoke(\"What is CLOVA Studio?\")\n",
171+
"\n",
172+
"# show the retrieved document's content\n",
173+
"retrieved_documents[0].page_content"
174+
]
175+
},
176+
{
177+
"cell_type": "markdown",
178+
"id": "b1a249e1",
179+
"metadata": {},
180+
"source": [
181+
"## Direct Usage\n",
182+
"\n",
183+
"Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n",
184+
"\n",
185+
"You can directly call these methods to get embeddings for your own use cases.\n",
186+
"\n",
187+
"### Embed single texts\n",
188+
"\n",
189+
"You can embed single texts or documents with `embed_query`:"
190+
]
191+
},
192+
{
193+
"cell_type": "code",
194+
"execution_count": 9,
195+
"id": "12fcfb4b",
196+
"metadata": {},
197+
"outputs": [
198+
{
199+
"name": "stdout",
200+
"output_type": "stream",
201+
"text": [
202+
"[-0.094717406, -0.4077411, -0.5513184, 1.6024436, -1.3235079, -1.0720996, -0.44471845, 1.3665184, 0.\n"
203+
]
204+
}
205+
],
206+
"source": [
207+
"single_vector = embeddings.embed_query(text)\n",
208+
"print(str(single_vector)[:100]) # Show the first 100 characters of the vector"
209+
]
210+
},
211+
{
212+
"cell_type": "markdown",
213+
"id": "8b383b53",
214+
"metadata": {},
215+
"source": [
216+
"### Embed multiple texts\n",
217+
"\n",
218+
"You can embed multiple texts with `embed_documents`:"
219+
]
220+
},
221+
{
222+
"cell_type": "code",
223+
"execution_count": 10,
224+
"id": "1f2e6104",
225+
"metadata": {},
226+
"outputs": [
227+
{
228+
"name": "stdout",
229+
"output_type": "stream",
230+
"text": [
231+
"[-0.094717406, -0.4077411, -0.5513184, 1.6024436, -1.3235079, -1.0720996, -0.44471845, 1.3665184, 0.\n",
232+
"[-0.25525448, -0.84877056, -0.6928286, 1.5867524, -1.2930486, -0.8166254, -0.17934391, 1.4236152, 0.\n"
233+
]
234+
}
235+
],
236+
"source": [
237+
"text2 = \"LangChain is the framework for building context-aware reasoning applications\"\n",
238+
"two_vectors = embeddings.embed_documents([text, text2])\n",
239+
"for vector in two_vectors:\n",
240+
" print(str(vector)[:100]) # Show the first 100 characters of the vector"
241+
]
242+
},
243+
{
244+
"cell_type": "markdown",
245+
"id": "eee40d32367cc5c4",
246+
"metadata": {},
247+
"source": [
248+
"## Additional functionalities\n",
249+
"\n",
250+
"### Service App\n",
251+
"\n",
252+
"When going live with production-level application using CLOVA Studio, you should apply for and use Service App. (See [here](https://guide.ncloud-docs.com/docs/en/clovastudio-playground01#서비스앱신청).)\n",
253+
"\n",
254+
"For a Service App, corresponding `NCP_CLOVASTUDIO_API_KEY` and `NCP_CLOVASTUDIO_APP_ID` are issued and can only be called with them."
255+
]
256+
},
257+
{
258+
"cell_type": "code",
259+
"execution_count": null,
260+
"id": "08f9f44e-c6a4-4163-8caf-27a0cda345b7",
261+
"metadata": {},
262+
"outputs": [],
263+
"source": [
264+
"# Update environment variables\n",
265+
"\n",
266+
"os.environ[\"NCP_CLOVASTUDIO_API_KEY\"] = getpass.getpass(\n",
267+
" \"Enter NCP CLOVA Studio API Key for Service App: \"\n",
268+
")\n",
269+
"os.environ[\"NCP_CLOVASTUDIO_APP_ID\"] = input(\"Enter NCP CLOVA Studio Service App ID: \")"
270+
]
271+
},
272+
{
273+
"cell_type": "code",
274+
"execution_count": null,
275+
"id": "86f59698-b3f4-4b19-a9d4-4facfcea304b",
276+
"metadata": {},
277+
"outputs": [],
278+
"source": [
279+
"embeddings = ClovaXEmbeddings(\n",
280+
" service_app=True,\n",
281+
" model=\"clir-emb-dolphin\", # set with the model name of corresponding app id of your Service App\n",
282+
" # app_id=\"...\" # set if you prefer to pass app id directly instead of using environment variables\n",
283+
")"
284+
]
285+
},
286+
{
287+
"cell_type": "markdown",
288+
"id": "1ddeaee9",
289+
"metadata": {},
290+
"source": [
291+
"## API Reference\n",
292+
"\n",
293+
"For detailed documentation on `ClovaXEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/latest/api_reference/community/embeddings/langchain_community.embeddings.naver.ClovaXEmbeddings.html)."
294+
]
295+
}
296+
],
297+
"metadata": {
298+
"kernelspec": {
299+
"display_name": "Python 3 (ipykernel)",
300+
"language": "python",
301+
"name": "python3"
302+
},
303+
"language_info": {
304+
"codemirror_mode": {
305+
"name": "ipython",
306+
"version": 3
307+
},
308+
"file_extension": ".py",
309+
"mimetype": "text/x-python",
310+
"name": "python",
311+
"nbconvert_exporter": "python",
312+
"pygments_lexer": "ipython3",
313+
"version": "3.12.3"
314+
}
315+
},
316+
"nbformat": 4,
317+
"nbformat_minor": 5
318+
}

‎libs/community/langchain_community/chat_models/__init__.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,9 @@
125125
from langchain_community.chat_models.moonshot import (
126126
MoonshotChat,
127127
)
128+
from langchain_community.chat_models.naver import (
129+
ChatClovaX,
130+
)
128131
from langchain_community.chat_models.oci_data_science import (
129132
ChatOCIModelDeployment,
130133
ChatOCIModelDeploymentTGI,
@@ -193,6 +196,7 @@
193196
"ChatAnthropic",
194197
"ChatAnyscale",
195198
"ChatBaichuan",
199+
"ChatClovaX",
196200
"ChatCohere",
197201
"ChatCoze",
198202
"ChatOctoAI",
@@ -257,6 +261,7 @@
257261
"ChatAnthropic": "langchain_community.chat_models.anthropic",
258262
"ChatAnyscale": "langchain_community.chat_models.anyscale",
259263
"ChatBaichuan": "langchain_community.chat_models.baichuan",
264+
"ChatClovaX": "langchain_community.chat_models.naver",
260265
"ChatCohere": "langchain_community.chat_models.cohere",
261266
"ChatCoze": "langchain_community.chat_models.coze",
262267
"ChatDatabricks": "langchain_community.chat_models.databricks",

0 commit comments

Comments
 (0)