TEI CPU inference fails with Intel MKL errors on AMD processors when running Qwen3 embedding models

[randomm]

Environment:

• AMD CPU (t3a.2xlarge on AWS)
• TEI image: cpu-sha-bedb2e5
• Model: Qwen/Qwen3-Embedding-0.6B

Error:

Intel MKL ERROR: Parameter 8 was incorrect on entry to SGEMM
Intel MKL ERROR: Parameter 13 was incorrect on entry to SGEMM

What we've tried:

• Set LD_PRELOAD=/usr/local/libfakeintel.so (fake Intel library is present)
• Set MKL_DEBUG_CPU_TYPE=5 (force AVX2)
• Reduced batch sizes and concurrency
• ONNX version has different compatibility issues

Root Cause:
The SGEMM errors indicate incorrect matrix dimensions/parameters being passed to Intel MKL BLAS routines. This appears to be a bug in the Candle backend's Qwen3 CPU implementation when compiled with Intel MKL, not just an AMD CPU detection issue.

Potential Solutions:

1. Use a TEI build compiled with OpenBLAS instead of Intel MKL
2. Fix the matrix dimension bug in the Candle backend code
3. Use a working custom image without MKL dependencies

[randomm]

Proposed Solution: Add Dockerfile-cpu-amd for AMD CPU Compatibility

I'd like to propose adding a dedicated Dockerfile-cpu-amd to resolve the Intel MKL compatibility issues on AMD processors. This follows the project's existing pattern of specialized Dockerfiles (like Dockerfile-cuda, Dockerfile-intel, etc.).

Why This Approach:

1. Follows Project Conventions: Matches the existing pattern of specialized Dockerfiles for different architectures
2. Clean & Maintainable: No complex conditional logic in the main Dockerfile
3. Backward Compatible: Doesn't affect existing Intel CPU users
4. Clear Documentation: Easy to document in the README's Docker Images table

Key Changes in Dockerfile-cpu-amd:

• ❌ Removes Intel MKL: No intel-oneapi-mkl-devel installation
• ❌ Removes MKL Features: Builds without mkl feature flag (--features ort,candle instead of --features ort,candle,mkl)
• ❌ Removes Intel-specific Environment Variables: No MKL_ENABLE_INSTRUCTIONS, LD_PRELOAD, etc.
• ✅ Uses Generic BLAS: Relies on libomp-dev for OpenMP support
• ✅ Same Structure: Maintains all other aspects of the original Dockerfile

Usage:

# Build AMD-compatible image
docker build . -f Dockerfile-cpu-amd

# Or for the project to publish:
# ghcr.io/huggingface/text-embeddings-inference:cpu-amd-1.7

Documentation Update Needed:

The README's Docker Images table would be updated to include:

Architecture	Image
CPU (Intel)	ghcr.io/huggingface/text-embeddings-inference:cpu-1.7
CPU (AMD)	ghcr.io/huggingface/text-embeddings-inference:cpu-amd-1.7

This approach eliminates the Intel MKL SGEMM errors entirely by using generic BLAS libraries that work correctly on AMD processors, while maintaining the same performance characteristics as the current CPU build but with broader compatibility.

Would the maintainers be open to this approach? I can prepare a PR with the Dockerfile-cpu-amd and corresponding documentation updates.