Skip to content

llama-graph : fix text position for mrope#13159

Merged
ggerganov merged 3 commits into
ggml-org:masterfrom
ngxson:xsn/qwen2vl_fix_text_pos
Apr 29, 2025
Merged

llama-graph : fix text position for mrope#13159
ggerganov merged 3 commits into
ggml-org:masterfrom
ngxson:xsn/qwen2vl_fix_text_pos

Conversation

@ngxson

@ngxson ngxson commented Apr 28, 2025

Copy link
Copy Markdown
Collaborator

Cont #13138

I misunderstood the original code:

        // TODO: add mrope pos ids somewhere else
        pos.resize(batch.n_tokens * 4);
        std::fill(pos.begin(), pos.end(), 0);
        for (int j = 0; j < batch.n_tokens * 3; j ++) {
            pos[j] = *st_pos_id + (j % batch.n_tokens);
        }
        batch.pos = pos.data();

Which means 3 first dims are the same, and 4th dim is 0, something like this:

1234...1234...1234...0000...

Thanks @mattjcly for spotting this!

@mattjcly

Copy link
Copy Markdown
Contributor

Can confirm this fixes bugs we observed! Thanks @ngxson :)

@ggerganov ggerganov merged commit b6ce743 into ggml-org:master Apr 29, 2025
timwu pushed a commit to timwu/llama.cpp that referenced this pull request Dec 20, 2025
* llama-graph : fix text position for mrope

* fix typo

* explicitly set 4th dim in the loop
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
* llama-graph : fix text position for mrope

* fix typo

* explicitly set 4th dim in the loop
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
* llama-graph : fix text position for mrope

* fix typo

* explicitly set 4th dim in the loop
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
* llama-graph : fix text position for mrope

* fix typo

* explicitly set 4th dim in the loop
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
* llama-graph : fix text position for mrope

* fix typo

* explicitly set 4th dim in the loop
phibya pushed a commit to ziee-ai/llama.cpp that referenced this pull request May 29, 2026
* llama-graph : fix text position for mrope

* fix typo

* explicitly set 4th dim in the loop
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
* llama-graph : fix text position for mrope

* fix typo

* explicitly set 4th dim in the loop
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
* llama-graph : fix text position for mrope

* fix typo

* explicitly set 4th dim in the loop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants