Skip to content

init LOADER_IMAGE with env variable MODEL_LOADER_IMAGE #389

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

googs1025
Copy link
Member

What this PR does / why we need it

Which issue(s) this PR fixes

Fixes part of #350 (comment)

Special notes for your reviewer

Does this PR introduce a user-facing change?

init LOADER_IMAGE with env variable MODEL_LOADER_IMAGE in yaml file

@InftyAI-Agent InftyAI-Agent added needs-triage Indicates an issue or PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. do-not-merge/needs-kind Indicates a PR lacks a label and requires one. labels Apr 30, 2025
@InftyAI-Agent InftyAI-Agent requested a review from kerthcet April 30, 2025 15:04
@googs1025
Copy link
Member Author

/kind cleanup

@InftyAI-Agent InftyAI-Agent added cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. and removed do-not-merge/needs-kind Indicates a PR lacks a label and requires one. labels Apr 30, 2025
@googs1025 googs1025 marked this pull request as draft April 30, 2025 15:08
@googs1025
Copy link
Member Author

Coming back here again, if we put this value in playground.backendConfig.env, it will be rendered into service and lws, and also rendered into pod env 🤔 , so it seems to be a convenient way to get this value. But I'm not sure if it's the right way this time.

like this:

func (p *ModelHubProvider) InjectModelLoader(template *corev1.PodTemplateSpec, index int) {
	initContainerName := MODEL_LOADER_CONTAINER_NAME
	if index != 0 {
		initContainerName += "-" + strconv.Itoa(index)
	}
	var loaderImage string
	for _, container := range template.Spec.Containers {
		for _, env := range container.Env {
			if env.Name == "MODEL_LOADER_IMAGE" {
				loaderImage = env.Value
				break
			}
		}
		if loaderImage != "" {
			break
		}
	}

	if loaderImage == "" {
		loaderImage = pkg.LOADER_IMAGE
	}

	// Handle initContainer.
	initContainer := &corev1.Container{
		Name:  initContainerName,
		Image: loaderImage,
		VolumeMounts: []corev1.VolumeMount{
			{
				Name:      MODEL_VOLUME_NAME,
				MountPath: CONTAINER_MODEL_PATH,
			},
		},
	}
        ....

}

import "os"

var (
LOADER_IMAGE = os.Getenv("MODEL_LOADER_IMAGE")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the way this works is to get it from the controller env. I'm not sure if this is what we want.

@kerthcet
Copy link
Member

kerthcet commented May 1, 2025

Can we use configmap for long-term solution, because we may introduce more configures with ai gateway introduced, I think it doesn't require too much efforts.

@googs1025
Copy link
Member Author

it means we don't need this change?

@kerthcet
Copy link
Member

kerthcet commented May 6, 2025

We need the feature but we need to use configmap to set the value instead.

@googs1025
Copy link
Member Author

/close

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants