Add docs for new Python utils in 0.4.x and for LangServe integration (#127)

Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>

Author: eolivelli

SUMMARY.md

@@ -52,6 +52,7 @@
 
 * [CLI Commands](langstream-cli/langstream-cli-commands.md)
 * [CLI Configuration](langstream-cli/langstream-cli-configuration.md)
+* [Web interface](langstream-cli/langstream-ui.md)
 
 ## Integrations
 
@@ -68,6 +69,7 @@
   * [Solr](configuration-resources/data-storage/solr.md)
   * [JDBC](configuration-resources/data-storage/jdbc.md)
   * [OpenSearch](configuration-resources/data-storage/opensearch.md)
+* [LangServe](configuration-resources/langserve/README.md)
 
 ## Pipeline Agents
 
@@ -110,9 +112,11 @@
     * [Agent Types](pipeline-agents/agent-developer-guide/agent-types.md)
     * [Agent Creation](pipeline-agents/agent-developer-guide/agent-creation.md)
     * [Configuration and Testing](pipeline-agents/agent-developer-guide/configuration-and-testing.md)
+    * [Enviroment variables](pipeline-agents/agent-developer-guide/enviroment.md)
   * [Python sink](pipeline-agents/custom-agents/python-sink.md)
   * [Python source](pipeline-agents/custom-agents/python-source.md)
   * [Python processor](pipeline-agents/custom-agents/python-function.md)
+  * [Python service](pipeline-agents/custom-agents/python-service.md)
 
 ## Messaging
 
@@ -127,4 +131,5 @@
 
 ## Examples
 
+* [LangServe chatbot](configuration-resources/langserve/README.md)
 * [LlamaIndex Cassandra sink](examples/llamaindex-cassandra-sink.md)

configuration-resources/langserve/README.md

@@ -0,0 +1,116 @@
+---
+layout:
+  title:
+    visible: true
+  description:
+    visible: true
+  tableOfContents:
+    visible: true
+  outline:
+    visible: false
+  pagination:
+    visible: false
+---
+
+# Integation with LangServe applications
+
+[LangServe](https://github.com/langchain-ai/langserve) is a popular runtime to execute LangChain applications.
+
+LangStream natively integrates with LangServe and allows you to invoke services exposed by LangServe applications.
+
+Use the built-in `langserve-invoke` agent to implement this integration.
+
+This example invokes a LangServe application that exposes a service at `http://localhost:8000/chain/stream`.
+
+```yaml
+topics:
+  - name: "input-topic"
+    creation-mode: create-if-not-exists
+  - name: "output-topic"
+    creation-mode: create-if-not-exists
+  - name: "streaming-answers-topic"
+    creation-mode: create-if-not-exists
+pipeline:
+  - type: "langserve-invoke"
+    input: input-topic
+    output: output-topic
+    id: step1
+    configuration:
+      output-field: value.answer
+      stream-to-topic: streaming-answers-topic
+      stream-response-field: value
+      min-chunks-per-message: 10
+      debug: false
+      method: POST
+      allow-redirects: true
+      handle-cookies: false
+      url: "http://host.docker.internal:8000/chain/stream"
+      fields:
+        - name: topic
+          expression: "value"
+```
+
+
+When you run the LangStream application in docker the URL is `http://host.docker.internal:8000/chain/stream` due to how docker desktop works.
+
+
+To allow your LangStream application to be accessible from a UI, you have to configure a gateway:
+```yaml
+gateways:
+  - id: chat
+    type: chat
+    chat-options:
+      answers-topic: streaming-answers-topic
+      questions-topic: input-topic
+      headers:
+        - value-from-parameters: session-id
+```
+
+## Starting the LangServe application locally
+
+This is the sample code of the LangServe application:
+
+```python
+from fastapi import FastAPI
+from langchain.prompts import ChatPromptTemplate
+from langchain.chat_models import ChatOpenAI
+from langserve import add_routes
+
+
+app = FastAPI(
+    title="LangChain Server",
+    version="1.0",
+    description="A simple api server using Langchain's Runnable interfaces",
+)
+
+model = ChatOpenAI()
+prompt = ChatPromptTemplate.from_template("tell me a joke about {topic}")
+add_routes(
+    app,
+    prompt | model,
+    path="/chain",
+    )
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app, host="localhost", port=8000)
+```
+
+Start the LangServe application with the following command:
+
+```bash
+export OPENAI_API_KEY=...
+pip install fastapi langserve langchain openai sse_starlette uvicorn
+python example.py
+```
+
+## Starting the LangStream application locally
+
+To run the LangStream application on docker locally:
+
+```bash
+langstream docker run -app /path/to/applicationn 
+```
+
+The LangStream UI will be running at [http://localhost:8092/](http://localhost:8092/) 

examples/llamaindex-cassandra-sink.md

@@ -28,6 +28,66 @@ pipeline:
         table: vs_ll_openai
 ```
 
+This is the code for the python sink:
+
+```python
+
+import base64
+import io
+from typing import Dict, Any
+
+import openai
+from cassandra.auth import PlainTextAuthProvider
+from cassandra.cluster import Cluster
+from langstream import Sink, Record
+from llama_index import VectorStoreIndex, Document
+from llama_index.vector_stores import CassandraVectorStore
+
+
+class LlamaIndexCassandraSink(Sink):
+    def __init__(self):
+        self.config = None
+        self.session = None
+        self.index = None
+
+    def init(self, config: Dict[str, Any]):
+        self.config = config
+        openai.api_key = config["openaiKey"]
+
+    def start(self):
+        secure_bundle = self.config["cassandra"]["secureBundle"]
+        secure_bundle = secure_bundle.removeprefix("base64:")
+        secure_bundle = base64.b64decode(secure_bundle)
+        cluster = Cluster(
+            cloud={
+                "secure_connect_bundle": io.BytesIO(secure_bundle),
+                "use_default_tempdir": True,
+            },
+            auth_provider=PlainTextAuthProvider(
+                self.config["cassandra"]["username"],
+                self.config["cassandra"]["password"],
+            ),
+        )
+        self.session = cluster.connect()
+
+        vector_store = CassandraVectorStore(
+            session=self.session,
+            keyspace=self.config["cassandra"]["keyspace"],
+            table=self.config["cassandra"]["table"],
+            embedding_dimension=1536,
+            insertion_batch_size=15,
+        )
+
+        self.index = VectorStoreIndex.from_vector_store(vector_store)
+
+    def write(self, record: Record):
+        self.index.insert(Document(text=record.value()))
+
+    def close(self):
+        if self.session:
+            self.session.shutdown()
+```
+
 ### Topics
 
 **Input**

installation/docker.md

@@ -104,7 +104,7 @@ If you have built [the CLI locally from the sources](../installation/build-and-i
 You can also override the command used to launch the container, the default value is `docker`, but you can pass the `--docker-command` flag to use a different binary path.
 
 
-### Connect to the docker application
+### Connect to the docker application using the CLI
 The docker container exposes the API gateway on port `8091` and the control plane on port `8090` of your local machine. 
 To connect to the docker container it's highly suggested to use a special profile named `local-docker-run`.
 This profile ensures you will always connect to the right endpoints.
@@ -119,3 +119,12 @@ or for getting the application description:
 langstream -p local-docker-run apps get test -o yaml
 ```
 
+### Connect to the docker application using a Web interface
+
+By default the CLI starts a web interface that you can use to test your application. The web interface is available at [http://localhost:8092/](http://localhost:8092/).
+This interface displays:
+
+* the application logs
+* a chatbot-like interface to interact with the gateways
+* a diagram of the application pipelines
+* the JSON description of the application, both the logical and the physical description (execution plan)

langstream-cli/langstream-ui.md

@@ -0,0 +1,23 @@
+# LangStream Web Interface
+
+The LangStream CLI provides a web interface that you can use to test your application.
+The web interface is available at [http://localhost:8092/](http://localhost:8092/).
+
+This interface displays:
+
+* the application logs
+* a chatbot-like interface to interact with the gateways
+* a diagram of the application pipelines
+* the JSON description of the application, both the logical and the physical description (execution plan)
+
+
+The UI is started automatically when you run the application in docker mode (with `langstream docker run...`), but you can also start it manually using the `langstream app ui` command.
+
+```bash
+langstream apps ui application-id
+```
+
+With this command the CLI starts a local web service bound on local host and proxies the requests to the LangStream services - both the Control Plane and the API gateway.
+
+The connection to the service is defined in the [CLI configuration](./langstream-cli-configuration.md).
+

pipeline-agents/agent-developer-guide/README.md

@@ -10,9 +10,10 @@ Developing your own agent for LangStream is quite simple, using Python best prac
 
 The Agent Developer Guide is broken into three milestones:
 
-1\. [Agent Types](agent-types.md) - Understand the three main Python agents you'll use in your applications.\
-2\. [Deploying Agents](broken-reference) - Create agents to process records. Handle exceptions for each agent type.\
-3\. [Configuration and Testing](broken-reference) - Configure, test, and package your agents for production.
+1\. [Agent Types](./agent-types.md) - Understand the three main Python agents you'll use in your applications.\
+2\. [Deploying Agents](./agent-creation.md) - Create agents to process records. Handle exceptions for each agent type.\
+3\. [Configuration and Testing](./configuration-and-testing.md) - Configure, test, and package your agents for production. \
+4\. [Enviroment variables](./enviroment.md) - Configure, test, and package your agents for production.
 
 Get started with [Part 1: Agent Types.](agent-types.md)
 

pipeline-agents/agent-developer-guide/agent-creation.md

@@ -8,7 +8,7 @@ description: Part 2 of the Agent Developer Guide
 This is Part 2 of the Agent Developer Guide. Start at the beginning [here.](./)
 {% endhint %}
 
-Once you have built, tested, and packaged the agent you will need to include it as a part of the LangStream application deployment. Within the “application” directory create a directory named “python”. Within that directory place all the files included in packaging.
+Within the “application” directory create a directory named “python”. Within that directory place all the files included in packaging.
 
 ```
 |- application
@@ -26,6 +26,29 @@ To include the agent as a step in the pipeline, set the className to match the e
     className: main.MySourceAgent
 ```
 
+### Development enviroment
+
+You can run your application locally in a docker container. This is the recommended way to develop your agent.
+You can use your IDE, like VS Studio Code, to develop your agent. You can also use the [VS Code LangStream Extension](https://github.com/LangStream/vscode-extension) to get started even faster.
+
+Start the application locally using this command:
+    
+```bash
+langstream docker run test -app /path/to/application
+```
+
+This launches a docker container with a docker image containing the same runtime that you are going to use in production.
+The container will run your application and will print the logs to the console.
+If you have a gateways.yaml file, it will also start a local gateway that you can use to test your agent.
+
+The UI runs at http://localhost:8092/
+
+The application runs in a process inside the container.
+
+When you change and save a Python file, the process is automatically reloaded in order to pick up your changes.
+This way you don't need to restart the container or the Python process manually.
+
+
 ### Agent records
 
 When developing a custom agent, your contract with the LangStream runtime will be implementing the correct methods as well as working with the `Record` type.

pipeline-agents/agent-developer-guide/agent-types.md

@@ -26,6 +26,12 @@ Processor agents are typically placed throughout an application’s pipeline.&#x
 
 A processor agent might manipulate data as it flows through the pipeline, or could add in context to help downstream agents make decisions. A processor agent is responsible for accepting a list of `Record`s as input, doing some processing as necessary, and returning a `Record` (or many `Record`s) as a result.
 
+### Service
+
+Service agents are generic applications that usually do not process streaming data in the scope of a pipeline. 
+
+Typically a Service exposes an API service that can be consumed by external applications. For instance you can build your ChatBot UI using a Service 
+
 ### What's next?
 
-Continue on to Part 2 of the Agent Developer Guide, [Agent Creation.](broken-reference)
+Continue on to Part 2 of the Agent Developer Guide, [Agent Creation.](agent-creation.md)

pipeline-agents/agent-developer-guide/configuration-and-testing.md

@@ -51,16 +51,14 @@ When you are ready to package the agent for deployment to LangStream, use the fo
 The command assumes you are running it from the “application” folder and your dependencies are declared in "python/requirements.txt"
 
 ```bash
-docker run --rm \
-    -v $(pwd):/app-code-download \
-    --entrypoint "" \
-    -w /app-code-download/python \
-    ghcr.io/langstream/langstream-runtime:0.1.0 \
-    /bin/bash -c 'pip3 install --target ./lib --upgrade --prefer-binary -r requirements.txt'
+
+langstream python load-pip-requirements -app /path/to/application
+
 ```
 
 {% hint style="info" %}
-Note the version of LangStream was provided as the image’s tag. This should match the version of LangStream you are developing for.
+The command above will create a “python/lib” folder with the dependencies installed. This folder is added to the PYTHONPATH environment variable when the agent is run on LangStream. It uses the same docker image as the runtime, with the same version of Python and of the core libraries that will run in production.
+This is very important, especially if you are using Mac or Windows to develop your agent.
 {% endhint %}
 
 #### **Unit testing**
@@ -70,25 +68,20 @@ Similar to packaging, the below Docker command is a starting suggestion of how t
 Using unittest:
 
 ```bash
-docker run --rm \
-    -v $(pwd):/app-code-download \
-    --entrypoint "" \
-    -w /app-code-download/python \
-    ghcr.io/langstream/langstream-runtime:0.1.0 \
-    /bin/bash -c 'PYTHONPATH=$PYTHONPATH:/app-code-download/python/lib python3 -m unittest'
+langstream python run-tests -app /path/to/application 
 ```
 
 Using tox:
 
 ```bash
-docker run --rm \
-    -v $(pwd):/app-code-download \
-    --entrypoint "" \
-    -w /app-code-download/python \
-    ghcr.io/langstream/langstream-runtime:0.1.0 \
-    /bin/bash -c 'tox'
+langstream python run-tests -app /path/to/application -c tox
 ```
 
-### Multiple Python apps in one LangStream application
+### Multiple Python apps with dependency conflicts
+
+If your LangStream application consists of more than one custom agent and you have dependencies conflicts, it is recommended that you separate them into 2 different applications. They can share input or output topics or be put inline with one another indirectly by topic. Separating by application gives you two clear “python” folders to house your artifact. This will aid in dependency collisions and other effects of two apps trying to share the same folder.
+
+
+### What's next?
 
-If your LangStream application consists of more than one custom agent, it is recommended that you separate them into 2 different applications. They can share input or output topics or be put inline with one another indirectly by topic. Separating by application gives you two clear “python” folders to house your artifact. This will aid in dependency collisions and other effects of two apps trying to share the same folder.
+Continue on to Part 4 of the Agent Developer Guide, [Environment variables.](environment.md)