[Bugfix][V1] Fix molmo text-only inputs

[jeejeelee]

Reproduce Code

import os
from vllm import LLM, SamplingParams


os.environ["VLLM_USE_V1"] = "1"


MODEL_NAME = "allenai/Molmo-7B-D-0924"
llm = LLM(
    model=MODEL_NAME,
    trust_remote_code=True,
    tensor_parallel_size=1,
    gpu_memory_utilization=0.7,
    enforce_eager=True,
)

stop_token_ids = None
sampling_params = SamplingParams(
    stop_token_ids=stop_token_ids,
    temperature=0,
    max_tokens=128,
)
prompts = [
    "Hello, my name is",
]
# text--only inputs
outputs = llm.generate(
    prompts=prompts,
    sampling_params=sampling_params,
)
print(outputs[0].outputs[0].text)

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[DarkLight1337]

Hmm, maybe we should expand our tests to cover this case...

[ywang96]

Thanks for the fix! Can you verify if this yields the same result as main branch on V0? I was mainly curious if the padded dummy image input ids will matter at all for text-only input.

[ywang96]

I wasn't sure why this was in the code originally when the AI2 team made the PR to support Molmo on vLLM, but I guess it wasn't an issue back then because it didn't matter on V0 since we didn't use the placeholder ranges for these "dummy" image input indices padded to the prompt token ids.

[jeejeelee]

Thanks for the fix! Can you verify if this yields the same result as main branch on V0? I was mainly curious if the padded dummy image input ids will matter at all for text-only input.

I have verified using the reproduce code above, and the generated results completely align with the main V0

[jeejeelee]

Hmm, maybe we should expand our tests to cover this case...

Where should tests be added for this PR?

[DarkLight1337]

You can edit the image_size_factors in test_models.py.

Hmm, actually it seems that empty image is already included there... why does the test still pass? Never mind, molmo tests aren't being included in test_models.py yet.

[jeejeelee]

You can edit the image_size_factors in test_models.py.

Hmm, actually it seems that empty image is already included there... why does the test still pass? Never mind, molmo tests aren't being included in test_models.py yet.

Alright, then I won't add tests.

[DarkLight1337]

It seems that we don't have any tests for Molmo at all.

[ywang96]

It seems that we don't have any tests for Molmo at all.

yea I think we decided that if the model support came from the vendor then the test is not required.

[DarkLight1337]

I think to avoid breaking the code in future PRs (especially with the V1 refactoring that's going on), we should add tests for it.

[jeejeelee]

I think to avoid breaking the code in future PRs (especially with the V1 refactoring that's going on), we should add tests for it.

Okay, l will handle this asap

[DarkLight1337]

Sorry didn't realize you added tests.

[DarkLight1337]

Please fix the lint errors