Skip to content

Add observability to Bedrock Titan Embedding model #3014

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

PSriVarshan
Copy link
Contributor

Description

This PR introduces observability to the Bedrock Titan Embedding Model by adding Micrometer Observation around the embedding API call. This ensures consistent metrics and tracing support in accordance with the Spring AI observability pattern.

  • Observations are added using ObservationRegistry.
  • Keys include model, input_type, and input_length.

Changes

  • Modified: BedrockTitanEmbeddingModel.java to wrap the API call in Observation.createNotStarted().observe(...)

Related Issue

Fixes #2983

Checklist

  • Follows project coding standards
  • Observability uses ObservationRegistry
  • Validated locally with sample embedding call

Do let me know if there are any further modifications required within the PR

@PSriVarshan PSriVarshan force-pushed the add-observability-to-bedrock-titan branch from 0ff770f to 95ba402 Compare May 6, 2025 13:28
@markpollack
Copy link
Member

Thanks. A few things to clean up in the PR.

  1. it dosn't compile - need to a) run ./mvnw spring-javaformat:apply
  2. BedrockTitanEmbeddingModelIT has a compile error since the constructor changed.

adding

@Autowired
TestObservationRegistry observationRegistry;

and changing the bean def to be

		@Bean
		public BedrockTitanEmbeddingModel titanEmbedding(TitanEmbeddingBedrockApi titanEmbeddingApi, TestObservationRegistry observationRegistry) {
			return new BedrockTitanEmbeddingModel(titanEmbeddingApi, observationRegistry);
		}

Also, you can use OpenAiEmbeddingModelObservationIT to copy over other tests related to observability.

@markpollack markpollack added this to the 1.0.0-RC1 milestone May 6, 2025
@PSriVarshan PSriVarshan force-pushed the add-observability-to-bedrock-titan branch 3 times, most recently from 0cd6a52 to ed41e60 Compare May 6, 2025 17:43
Signed-off-by: PSriVarshan <psvvarshan@gmail.com>
Signed-off-by: PSriVarshan <psvvarshan@gmail.com>
Signed-off-by: PSriVarshan <psvvarshan@gmail.com>
Signed-off-by: PSriVarshan <psvvarshan@gmail.com>
Signed-off-by: PSriVarshan <psvvarshan@gmail.com>
Signed-off-by: PSriVarshan <psvvarshan@gmail.com>
@PSriVarshan PSriVarshan force-pushed the add-observability-to-bedrock-titan branch from ed41e60 to b127661 Compare May 6, 2025 17:54
@PSriVarshan
Copy link
Contributor Author

I think now the branch is fully ready and optimal. Thanks a ton for the guidance @markpollack. Waiting for your further guidance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Observability to Bedrock Embedding Models
2 participants