Merge pull request #3208 from instructlab/mergify/bp/release-v0.23/pr…

…-3157

Make SDG batch size configurable via system profile (backport #3157)

mergify: remove old `ruff.sh` reference (backport #3188) (#3193)

PR #1526 moved this to a `tox` target.<hr>This is an automatic backport of pull request #3188 done by [Mergify](https://mergify.com).


Approved-by: ktdreyer

Approved-by: courtneypacheco

chore: Update vLLM to 0.7.3 (backport #3178) (#3180)

<hr>This is an automatic backport of pull request #3178 done by [Mergify](https://mergify.com).


Approved-by: danmcp

Approved-by: courtneypacheco

Merge pull request #3147 from instructlab/mergify/bp/release-v0.23/pr…

…-3060

fix(RAG): Fix RAG support for taxonomies with compositional skills (backport #3060)

Merge pull request #3134 from instructlab/cap-dependencies-0.24

deps: Cap dependencies for 0.24.0 release

fix: remove duplicate log messages in detached SDG (backport #3031) (#…

…3038)

currently, since we add a log file handler and capture the logs from the runnning process when running -dt, there are duplicate log messages. Only add the log file handler if running in attached mode









**Checklist:**

- [ ] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.
<hr>This is an automatic backport of pull request #3031 done by [Mergify](https://mergify.com).


Approved-by: cdoern

Approved-by: nathan-weinberg

deps: cap dependencies for 0.23.0 release (#3015)

**Checklist:**

- [x] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.



Approved-by: alinaryan

Approved-by: nathan-weinberg

Approved-by: booxter

feat: DK-Bench ilab implementation (#2940)

This commit adds Domain Knowledge bench (DK-Bench) as an evaluation option as part of
`ilab model evaluate`.

This commit adds the cli flags to enable run the
benchmark.









**Checklist:**

- [ ] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.



Approved-by: RobotSail

Approved-by: cdoern

Approved-by: alinaryan

Improvement to get_argument (#2939)

Based on #2927 (comment)
```
$ python test-test.py
prefix:  --foo
args:  ['--foo', '4']
flag:  True
value:  4

$ python test-test.py
prefix:  --foo
args:  ['--foo', '4', '--foo']
flag:  True
value:  None

$ python test-test.py
prefix:  --foo
args:  ['--foo', '--foo']
flag:  True
value:  None

$ python test-test.py
prefix:  --foo
args:  ['--foo', '4', '--foo=2']
flag:  True
value:  2

$ python test-test.py
prefix:  --foo
args:  ['foo']
flag:  False
value:  None
```

**Checklist:**

- [ ] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.



Approved-by: danmcp

Approved-by: nathan-weinberg

feat: support llama-cpp-python v0.3.2 (backport #2825) (#2883)

version 0.3.5 of llama-cpp-python has a known issue abetlen/llama-cpp-python#1861 version 0.3.2 has granite 3.0 support and does not have this issue. Bump to this version

this required some additions to how we handle chat exceptions. As of these newer 0.3.z llama-cpp-python versions,
a bad request causes the server to die. This requires us to know the max_ctx_size of the server before passing a completions request so that
we can maintain the behavior of trimming messages until we can respond to one that fits.

in order to do this, the config now contains a `current_max_ctx_size` field that we will update when spinning up a server.
in the case that a user implicitly starts a llama-cpp-python server when calling `ilab model chat`, we set the max_tokens to the
current `max_ctx_size` in the serve config.









**Checklist:**

- [ ] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.
<hr>This is an automatic backport of pull request #2825 done by [Mergify](https://mergify.com).


Approved-by: cdoern

Approved-by: alinaryan