Skip to content

[SYCL] Merge USMAllocContext for Sub-Devices and Sub-Sub-Devices #8012

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jan 27, 2023
Merged

[SYCL] Merge USMAllocContext for Sub-Devices and Sub-Sub-Devices #8012

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
4019c59
Merge USM Alloc Context of Sub-device and sub-sub-device
uditagarwal97 Jan 13, 2023
4f88cc7
Fix formatting
uditagarwal97 Jan 13, 2023
20fa520
[SYCL] Use pointer access operator instead of .get().
uditagarwal97 Jan 16, 2023
4416591
[SYCL] Use device handle as index for USMAllocation map
uditagarwal97 Jan 26, 2023
883e9da
Minor formatting
uditagarwal97 Jan 26, 2023
b8536a7
Fix typo
uditagarwal97 Jan 26, 2023
9ad0880
Remove explicit isCCS check
uditagarwal97 Jan 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 16 additions & 10 deletions sycl/plugins/level_zero/pi_level_zero.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -770,17 +770,21 @@ pi_device _pi_context::getRootDevice() const {
pi_result _pi_context::initialize() {

// Helper lambda to create various USM allocators for a device.
// Note that the CCS devices and their respective subdevices share a
// common ze_device_handle and therefore, also share USM allocators.
auto createUSMAllocators = [this](pi_device Device) {
SharedMemAllocContexts.emplace(
std::piecewise_construct, std::make_tuple(Device),
std::piecewise_construct, std::make_tuple(Device->ZeDevice),
std::make_tuple(std::unique_ptr<SystemMemory>(
new USMSharedMemoryAlloc(this, Device))));

SharedReadOnlyMemAllocContexts.emplace(
std::piecewise_construct, std::make_tuple(Device),
std::piecewise_construct, std::make_tuple(Device->ZeDevice),
std::make_tuple(std::unique_ptr<SystemMemory>(
new USMSharedReadOnlyMemoryAlloc(this, Device))));

DeviceMemAllocContexts.emplace(
std::piecewise_construct, std::make_tuple(Device),
std::piecewise_construct, std::make_tuple(Device->ZeDevice),
std::make_tuple(std::unique_ptr<SystemMemory>(
new USMDeviceMemoryAlloc(this, Device))));
};
Expand All @@ -807,8 +811,9 @@ pi_result _pi_context::initialize() {
std::unique_ptr<SystemMemory>(new USMHostMemoryAlloc(this)));

// We may allocate memory to this root device so create allocators.
if (SingleRootDevice && DeviceMemAllocContexts.find(SingleRootDevice) ==
DeviceMemAllocContexts.end()) {
if (SingleRootDevice &&
DeviceMemAllocContexts.find(SingleRootDevice->ZeDevice) ==
DeviceMemAllocContexts.end()) {
createUSMAllocators(SingleRootDevice);
}

Expand Down Expand Up @@ -8191,7 +8196,7 @@ pi_result piextUSMDeviceAlloc(void **ResultPtr, pi_context Context,
}

try {
auto It = Context->DeviceMemAllocContexts.find(Device);
auto It = Context->DeviceMemAllocContexts.find(Device->ZeDevice);
if (It == Context->DeviceMemAllocContexts.end())
return PI_ERROR_INVALID_VALUE;

Expand Down Expand Up @@ -8269,7 +8274,7 @@ pi_result piextUSMSharedAlloc(void **ResultPtr, pi_context Context,
try {
auto &Allocator = (DeviceReadOnly ? Context->SharedReadOnlyMemAllocContexts
: Context->SharedMemAllocContexts);
auto It = Allocator.find(Device);
auto It = Allocator.find(Device->ZeDevice);
if (It == Allocator.end())
return PI_ERROR_INVALID_VALUE;

Expand Down Expand Up @@ -8432,10 +8437,11 @@ static pi_result USMFreeHelper(pi_context Context, void *Ptr,
PI_ASSERT(Device, PI_ERROR_INVALID_DEVICE);

auto DeallocationHelper =
[Context, Device, Ptr, OwnZeMemHandle](
std::unordered_map<pi_device, USMAllocContext> &AllocContextMap) {
[Context, Device, Ptr,
OwnZeMemHandle](std::unordered_map<ze_device_handle_t, USMAllocContext>
&AllocContextMap) {
try {
auto It = AllocContextMap.find(Device);
auto It = AllocContextMap.find(Device->ZeDevice);
if (It == AllocContextMap.end())
return PI_ERROR_INVALID_VALUE;

Expand Down
9 changes: 6 additions & 3 deletions sycl/plugins/level_zero/pi_level_zero.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -462,9 +462,12 @@ struct _pi_context : _pi_object {
// Store USM allocator context(internal allocator structures)
// for USM shared and device allocations. There is 1 allocator context
// per each pair of (context, device) per each memory type.
std::unordered_map<pi_device, USMAllocContext> DeviceMemAllocContexts;
std::unordered_map<pi_device, USMAllocContext> SharedMemAllocContexts;
std::unordered_map<pi_device, USMAllocContext> SharedReadOnlyMemAllocContexts;
std::unordered_map<ze_device_handle_t, USMAllocContext>
DeviceMemAllocContexts;
std::unordered_map<ze_device_handle_t, USMAllocContext>
SharedMemAllocContexts;
std::unordered_map<ze_device_handle_t, USMAllocContext>
SharedReadOnlyMemAllocContexts;

// Since L0 native runtime does not distinguisg "shared device_read_only"
// vs regular "shared" allocations, we have keep track of it to use
Expand Down