[CUDA] Entries are not distributed in nvptx binaries

[hiaselhans]

When compiling with -fsycl-targets=nvptx64-nvidia-opencl-sycldevice,spir64-unknown-opencl-sycldevice binaries are created for nvptx and spir64.
The problem is that for nvptx binaries the EntriesTable is not distributed and thus program_manager will not choose the right binary.

How to reproduce

sycl/source/detail/program_manager.cpp:40
static constexpr int DbgProgMgr = 10;

dpc++ -Wno-unknown-cuda-version -fsycl -fsycl-targets=nvptx64-nvidia-opencl-sycldevice,spir64-unknown-opencl-sycldevice -o sycl sycl.cpp

SYCL_BE=PI_CUDA ./sycl

Result

Running on 
nvidia
GeForce RTX 2070 (NVIDIA Corporation/PI 0.0): 10
SYCL host device (/1.2): -1
>>> ProgramManager::getOrCreateKernel(-1, 0x23c51d0, _ZTSZZ4mainENK3$_0clERN2cl4sycl7handlerEE10FillBuffer)
>>> ProgramManager::getDeviceImage(-1, "2", 0x23c51d0)
available device images:
  ++++++ Kernel set: 2
  --- Image 0x40ff70
    Version  : 2
    Kind     : 4
    Format   : 0
    Target   : spir64
    Bin size : 15588
    Compile options : 
    Link options    : 
    Entries  : _ZTSZZ4mainENK3$_0clERN2cl4sycl7handlerEE10FillBuffer 
    Properties [0x40ff30-0x40ff48]:
      Category SYCL/specialization constants [0-0]:
    OSModuleHandle=-1
  ++++++ Kernel set: 1
  --- Image 0x40fe40
    Version  : 2
    Kind     : 4
    Format   : 0
    Target   : nvptx64
    Bin size : 3024
    Compile options : 
    Link options    : 
    Entries  : 
    Properties [0-0]:
    OSModuleHandle=-1
selected device image: 0x40ff70
  --- Image 0x40ff70
    Version  : 2
    Kind     : 4
    Format   : 0
    Target   : spir64
    Bin size : 15588
    Compile options : 
    Link options    : 
    Entries  : _ZTSZZ4mainENK3$_0clERN2cl4sycl7handlerEE10FillBuffer 
    Properties [0x40ff30-0x40ff48]:
      Category SYCL/specialization constants [0-0]:
    OSModuleHandle=-1
>>> ProgramManager::createPIProgram(0x2378130)

[hiaselhans]

Thx @kbobrovs for the input:

i will look into it but not sure where to start

sycl-post-link tool generates the symbol table - use -symbols switch.
As PTX does not support spec constants, post-link should generate 2-column table - first column >lists binaries, second - corresponding symbol files. See Driver.cpp/appendLinkDependences code >after the picture illustrating the action graph. To support symbol table, post-link should generate >TY_Tempfiletable in case of PTX too. Use file-table-tform to process filenames in the table, similar >to non-PTX path

[bader]

When compiling with -fsycl-targets=nvptx64-nvidia-opencl-sycldevice,spir64-unknown-opencl-sycldevice binaries are created for nvptx and spir64.

According to the documentation the target triple should be nvptx64-nvidia-cuda and/or spir64-unknown-unknown. opencl looks suspicious - I don't think it's supported, so this might be the reason why images can't be found.

[hiaselhans]

I tried this long time ago. Will try to reproduce next week with new arguments!

[AerialMantis]

We believe this issue to be resolved, the nvptx64-nvidia-opencl triple is not supported by the CUDA backend, if you use the nvptx64-nvidia-cuda triple as @bader describes this should work correctly, so we are closing this ticket. If you are still seeing the issue please leave a comment and we can re-open it.