nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

sycl : Improve SYCL doc (#23025)

Browse Source 
* Tidy up SYCL doc a bit

- Add explicit links to referenced items
- Fix spelling errors

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

* Correct documented default for GGML_SYCL_GRAPH

The default is ON, not OFF:

  $ cmake -LAH -B build | grep GGML_SYCL_GRAPH
  ...
  GGML_SYCL_GRAPH:BOOL=ON

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

* Move docker instructions from SYCL.md to docker.md

This makes them directly accesible from the Quick Start section
of the top-level README.md.

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

* Refer to intel.Dockerfile for ARGs and their defaults

The defaults are always changing; this avoids accuracy errors
from duplicating the information.

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

* Remove mention of Nvidia in SYCL row of backend table

This support was removed in 2026.02 - refer to the SYCL.md News.

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>

---------

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
This commit is contained in:
Todd Malsbary
2026-06-03 22:02:54 -07:00
committed by GitHub
parent 94a220cd67
commit f478f1b6d7
2 changed files with 48 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
docs/docker.md
View File

@@ -140,3 +140,39 @@ docker run -v /path/to/models:/models local/llama.cpp:full-musa --run -m /models
docker run -v /path/to/models:/models local/llama.cpp:light-musa -m /models/7B/ggml-model-q4_0.gguf -p "Building a website can be done in 10 simple steps:" -n 512 --n-gpu-layers 1
docker run -v /path/to/models:/models local/llama.cpp:server-musa -m /models/7B/ggml-model-q4_0.gguf --port 8080 --host 0.0.0.0 -n 512 --n-gpu-layers 1
```
## Docker With SYCL
## Building Docker locally
```bash
docker build -t local/llama.cpp:full-intel --target full -f .devops/intel.Dockerfile .
docker build -t local/llama.cpp:light-intel --target light -f .devops/intel.Dockerfile .
docker build -t local/llama.cpp:server-intel --target server -f .devops/intel.Dockerfile .
```
You may want to pass in some different `ARGS`, depending on the SYCL environment supported by your container host, as well as the GPU architecture.
Refer to [.devops/intel.Dockerfile](../.devops/intel.Dockerfile) for the available `ARGS` and their defaults.
The resulting images, are essentially the same as the non-SYCL images:
1. `local/llama.cpp:full-intel`: This image includes both the `llama-cli` and `llama-completion` executables and the tools to convert LLaMA models into ggml and convert into 4-bit quantization.
2. `local/llama.cpp:light-intel`: This image only includes the `llama-cli` and `llama-completion` executables.
3. `local/llama.cpp:server-intel`: This image only includes the `llama-server` executable.
## Usage
After building locally, usage is similar to the non-SYCL examples, but you'll need to add the `--device` flag.
```bash
# First, find all the DRI cards
ls -la /dev/dri
# Then, pick the card that you want to use (here for e.g. /dev/dri/card0).
docker run --device /dev/dri/renderD128:/dev/dri/renderD128 --device /dev/dri/card0:/dev/dri/card0 -v /path/to/models:/models local/llama.cpp:full-intel -m /models/7B/ggml-model-q4_0.gguf -p "Building a website can be done in 10 simple steps:" -n 512 --n-gpu-layers 99
docker run --device /dev/dri/renderD128:/dev/dri/renderD128 --device /dev/dri/card0:/dev/dri/card0 -v /path/to/models:/models local/llama.cpp:light-intel -m /models/7B/ggml-model-q4_0.gguf -p "Building a website can be done in 10 simple steps:" -n 512 --n-gpu-layers 99
docker run --device /dev/dri/renderD128:/dev/dri/renderD128 --device /dev/dri/card0:/dev/dri/card0 -v /path/to/models:/models local/llama.cpp:server-intel -m /models/7B/ggml-model-q4_0.gguf --port 8080 --host 0.0.0.0 -n 512 --n-gpu-layers 99
```
*Notes:*
- Docker has been tested successfully on native Linux. WSL support has not been verified yet.
- You may need to install Intel GPU driver on the **host** machine *(Please refer to the [Linux configuration](./backend/SYCL.md#linux) for details)*.