Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring LLMCompiledModel according to review comments in GenAI static_llm::StatefulLLMPipeline #28267

Merged

Conversation

AsyaPronina
Copy link
Contributor

Details:

Tickets:

  • ticket-id

Copy link
Contributor

@TolyaTalamanov TolyaTalamanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, we should avoid asking to provide batch and seq_len dimension but rather figure out them inside LLMCompiledModel. The same actually for OPTIMIZE_V_TENSORS.

Having them as properties might be a problem to remove them in future? But I guess we have no other choice, do we?

Approving this, as temporary solution, but I'd add some FIXME's to refactor it later on

@AsyaPronina AsyaPronina force-pushed the llm_compiled_model_refactoring branch from 44147cb to 86b154d Compare January 6, 2025 00:03
@AsyaPronina AsyaPronina enabled auto-merge January 6, 2025 00:03
@AsyaPronina AsyaPronina disabled auto-merge January 6, 2025 00:06
@AsyaPronina AsyaPronina force-pushed the llm_compiled_model_refactoring branch from 86b154d to eac1a83 Compare January 6, 2025 00:16
@AsyaPronina AsyaPronina force-pushed the llm_compiled_model_refactoring branch from eac1a83 to 2063109 Compare January 6, 2025 00:17
@AsyaPronina AsyaPronina enabled auto-merge January 6, 2025 00:18
@AsyaPronina AsyaPronina added this pull request to the merge queue Jan 6, 2025
Merged via the queue into openvinotoolkit:master with commit 74041ec Jan 6, 2025
161 checks passed
@AsyaPronina AsyaPronina deleted the llm_compiled_model_refactoring branch January 6, 2025 03:54
github-merge-queue bot pushed a commit to openvinotoolkit/openvino.genai that referenced this pull request Jan 6, 2025
Related PRs:
- OpenVINO: *openvinotoolkit/openvino#27651
- OpenVINO Unroll SPDA:
*openvinotoolkit/openvino#27891
- OpenVINO Prefill/Generate sections:
*openvinotoolkit/openvino#28154
- OpenVINO LLMCompiledModel refactoring:
openvinotoolkit/openvino#28267

---------

Co-authored-by: Anatoliy Talamanov <[email protected]>
Co-authored-by: Ilya Lavrenov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: NPU OpenVINO NPU plugin category: NPUW NPUW plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants