[SYCL] Host device & queue removal (internal part, not breaking ABI) #14027

KseniyaTikhomirova · 2024-06-04T10:42:36Z

Host device support is deprecated long time ago. Although our internal host task & host accessor implementation was still using it.
This change eliminates it and remove possibility to create host queue/context and device. This brings the following changes:

Commands & Events could not guarantee queried Context != nullptr and Queue!= nullptr since for host task stuff no device queue/context is involved. For host task we have submitted queue instance stored in event to be able to report exceptions to user and to be able to properly handle dependencies. Submitted queue for host task is guaranteed to be not null.
Connection command for cross context dependencies is now attached to the queue of new command (dependency for which is being analyzed). Previously it was also related to host queue only. No perf impact is expected.
Stream flush command is now submitted to the same queue as corresponding kernel (previously it was submitted to the host queue). This could bring negative perf impact for stream usage with in-order queue but stream is not perf oriented feature.

ABI breaking changes to remove is_host methods and some SYCL_EXTERN stuff will be submitted separately.

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

sycl/source/detail/memory_manager.cpp

sycl/source/detail/scheduler/commands.cpp

sycl/source/detail/scheduler/graph_builder.cpp

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

This PR fixes a GPU accuracy bug by upscaling the error-tolerance to a double type if the GPU supports 64-bit floating point arithmetic.

Bumps the github-actions group with 2 updates: [github/codeql-action](https://github.com/github/codeql-action) and [softprops/action-gh-release](https://github.com/softprops/action-gh-release). Updates `github/codeql-action` from 3.25.7 to 3.25.11 <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/github/codeql-action/blob/main/CHANGELOG.md">github/codeql-action's changelog</a>.</em></p> <blockquote> <h1>CodeQL Action Changelog</h1> <p>See the <a href="https://github.com/github/codeql-action/releases">releases page</a> for the relevant changes to the CodeQL CLI and language packs.</p> <p>Note that the only difference between <code>v2</code> and <code>v3</code> of the CodeQL Action is the node version they support, with <code>v3</code> running on node 20 while we continue to release <code>v2</code> to support running on node 16. For example <code>3.22.11</code> was the first <code>v3</code> release and is functionally identical to <code>2.22.11</code>. This approach ensures an easy way to track exactly which features are included in different versions, indicated by the minor and patch version numbers.</p> <h2>[UNRELEASED]</h2> <p>No user facing changes.</p> <h2>3.25.11 - 28 Jun 2024</h2> <ul> <li>Avoid failing the workflow run if there is an error while uploading debug artifacts. <a href="https://redirect.github.com/github/codeql-action/pull/2349">#2349</a></li> <li>Update default CodeQL bundle version to 2.17.6. <a href="https://redirect.github.com/github/codeql-action/pull/2352">#2352</a></li> </ul> <h2>3.25.10 - 13 Jun 2024</h2> <ul> <li>Update default CodeQL bundle version to 2.17.5. <a href="https://redirect.github.com/github/codeql-action/pull/2327">#2327</a></li> </ul> <h2>3.25.9 - 12 Jun 2024</h2> <ul> <li>Avoid failing database creation if the database folder already exists and contains some unexpected files. Requires CodeQL 2.18.0 or higher. <a href="https://redirect.github.com/github/codeql-action/pull/2330">#2330</a></li> <li>The init Action will attempt to clean up the database cluster directory before creating a new database and at the end of the job. This will help to avoid issues where the database cluster directory is left in an inconsistent state. <a href="https://redirect.github.com/github/codeql-action/pull/2332">#2332</a></li> </ul> <h2>3.25.8 - 04 Jun 2024</h2> <ul> <li>Update default CodeQL bundle version to 2.17.4. <a href="https://redirect.github.com/github/codeql-action/pull/2321">#2321</a></li> </ul> <h2>3.25.7 - 31 May 2024</h2> <ul> <li>We are rolling out a feature in May/June 2024 that will reduce the Actions cache usage of the Action by keeping only the newest TRAP cache for each language. <a href="https://redirect.github.com/github/codeql-action/pull/2306">#2306</a></li> </ul> <h2>3.25.6 - 20 May 2024</h2> <ul> <li>Update default CodeQL bundle version to 2.17.3. <a href="https://redirect.github.com/github/codeql-action/pull/2295">#2295</a></li> </ul> <h2>3.25.5 - 13 May 2024</h2> <ul> <li>Add a compatibility matrix of supported CodeQL Action, CodeQL CLI, and GitHub Enterprise Server versions to the <a href="https://github.com/github/codeql-action/blob/main/README.md">https://github.com/github/codeql-action/blob/main/README.md</a>. <a href="https://redirect.github.com/github/codeql-action/pull/2273">#2273</a></li> <li>Avoid printing out a warning for a missing <code>on.push</code> trigger when the CodeQL Action is triggered via a <code>workflow_call</code> event. <a href="https://redirect.github.com/github/codeql-action/pull/2274">#2274</a></li> <li>The <code>tools: latest</code> input to the <code>init</code> Action has been renamed to <code>tools: linked</code>. This option specifies that the Action should use the tools shipped at the same time as the Action. The old name will continue to work for backwards compatibility, but we recommend that new workflows use the new name. <a href="https://redirect.github.com/github/codeql-action/pull/2281">#2281</a></li> </ul> <h2>3.25.4 - 08 May 2024</h2> <ul> <li>Update default CodeQL bundle version to 2.17.2. <a href="https://redirect.github.com/github/codeql-action/pull/2270">#2270</a></li> </ul> <h2>3.25.3 - 25 Apr 2024</h2> <ul> <li>Update default CodeQL bundle version to 2.17.1. <a href="https://redirect.github.com/github/codeql-action/pull/2247">#2247</a></li> <li>Workflows running on <code>macos-latest</code> using CodeQL CLI versions before v2.15.1 will need to either upgrade their CLI version to v2.15.1 or newer, or change the platform to an Intel MacOS runner, such as <code>macos-12</code>. ARM machines with SIP disabled, including the newest <code>macos-latest</code> image, are unsupported for CLI versions before 2.15.1. <a href="https://redirect.github.com/github/codeql-action/pull/2261">#2261</a></li> </ul>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/github/codeql-action/commit/b611370bb5703a7efb587f9d136a52ea24c5c38c"><code>b611370</code></a> Merge pull request <a href="https://redirect.github.com/github/codeql-action/issues/2357">#2357</a> from github/update-v3.25.11-de945755c</li> <li><a href="https://github.com/github/codeql-action/commit/3e6431f3accd84bb42779fc3c9d9f447caa3a6d3"><code>3e6431f</code></a> Update changelog for v3.25.11</li> <li><a href="https://github.com/github/codeql-action/commit/de945755c9edd3a4e5d160a71f1482ece6a3c271"><code>de94575</code></a> Merge pull request <a href="https://redirect.github.com/github/codeql-action/issues/2352">#2352</a> from github/update-bundle/codeql-bundle-v2.17.6</li> <li><a href="https://github.com/github/codeql-action/commit/a32d3058b827f5d2ba08dc2570887f14b164a794"><code>a32d305</code></a> Add changelog note</li> <li><a href="https://github.com/github/codeql-action/commit/9ccc99508a819cb9c340028d7711b129f96c8a2e"><code>9ccc995</code></a> Update default bundle to codeql-bundle-v2.17.6</li> <li><a href="https://github.com/github/codeql-action/commit/9b7c22c3b39078582fa6d0d8f3841e944ec54582"><code>9b7c22c</code></a> Merge pull request <a href="https://redirect.github.com/github/codeql-action/issues/2351">#2351</a> from github/dependabot/npm_and_yarn/npm-6791eaa26c</li> <li><a href="https://github.com/github/codeql-action/commit/9cf3243b0be3a9e0efff20a7fabd5a11246168e9"><code>9cf3243</code></a> Rebuild</li> <li><a href="https://github.com/github/codeql-action/commit/1895b29ac8e2046ddb708ac1eca53e6d5e143337"><code>1895b29</code></a> Update checked-in dependencies</li> <li><a href="https://github.com/github/codeql-action/commit/9dcfde966d641c9a59ee02a83f18329a2b2caace"><code>9dcfde9</code></a> Bump the npm group with 2 updates</li> <li><a href="https://github.com/github/codeql-action/commit/8723b5be41df185b62efd22191bb83fc24539ca0"><code>8723b5b</code></a> Merge pull request <a href="https://redirect.github.com/github/codeql-action/issues/2350">#2350</a> from github/angelapwen/add-exclude-pr-check-param</li> <li>Additional commits viewable in <a href="https://github.com/github/codeql-action/compare/f079b8493333aace61c81488f8bd40919487bd9f...b611370bb5703a7efb587f9d136a52ea24c5c38c">compare view</a></li> </ul> </details> <br /> Updates `softprops/action-gh-release` from 2.0.5 to 2.0.6 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/softprops/action-gh-release/releases">softprops/action-gh-release's releases</a>.</em></p> <blockquote> <h2>v2.0.6</h2> <p>maintenance release with updated dependencies</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md">softprops/action-gh-release's changelog</a>.</em></p> <blockquote> <h2>2.0.6</h2> <ul> <li>maintenance release with updated dependencies</li> </ul> <h2>2.0.5</h2> <ul> <li>Factor in file names with spaces when upserting files <a href="https://redirect.github.com/softprops/action-gh-release/pull/446">#446</a> via <a href="https://github.com/MystiPanda"><code>@MystiPanda</code></a></li> <li>Improvements to error handling <a href="https://redirect.github.com/softprops/action-gh-release/pull/449">#449</a> via <a href="https://github.com/till"><code>@till</code></a></li> </ul> <h2>2.0.4</h2> <ul> <li>Minor follow up to <a href="https://redirect.github.com/softprops/action-gh-release/pull/417">#417</a>. <a href="https://redirect.github.com/softprops/action-gh-release/pull/425">#425</a></li> </ul> <h2>2.0.3</h2> <ul> <li>Declare <code>make_latest</code> as an input field in <code>action.yml</code> <a href="https://redirect.github.com/softprops/action-gh-release/pull/419">#419</a></li> </ul> <h2>2.0.2</h2> <ul> <li>Revisit approach to <a href="https://redirect.github.com/softprops/action-gh-release/pull/384">#384</a> making unresolved pattern failures opt-in <a href="https://redirect.github.com/softprops/action-gh-release/pull/417">#417</a></li> </ul> <h2>2.0.1</h2> <ul> <li>Add support for make_latest property <a href="https://redirect.github.com/softprops/action-gh-release/pull/304">#304</a> via <a href="https://github.com/samueljseay"><code>@samueljseay</code></a></li> <li>Fail run if files setting contains invalid patterns <a href="https://redirect.github.com/softprops/action-gh-release/pull/384">#384</a> via <a href="https://github.com/rpdelaney"><code>@rpdelaney</code></a></li> <li>Add support for proxy env variables (don't use node-fetch) <a href="https://redirect.github.com/softprops/action-gh-release/pull/386/">#386</a> via <a href="https://github.com/timor-raiman"><code>@timor-raiman</code></a></li> <li>Suppress confusing warning when input_files is empty <a href="https://redirect.github.com/softprops/action-gh-release/pull/389">#389</a> via <a href="https://github.com/Drowze"><code>@Drowze</code></a></li> </ul> <h2>2.0.0</h2> <ul> <li><code>2.0.0</code>!? this release corrects a disjunction between git tag versions used in the marketplace and versions list this file. Previous versions should have really been 1.*. Going forward this should be better aligned.</li> <li>Upgrade action.yml declaration to node20 to address deprecations</li> </ul> <h2>0.1.15</h2> <ul> <li>Upgrade to action.yml declaration to node16 to address deprecations</li> <li>Upgrade dependencies</li> <li>Add <code>asset</code> output as a JSON array containing information about the uploaded assets</li> </ul> <h2>0.1.14</h2> <ul> <li>provides an new workflow input option <code>generate_release_notes</code> which when set to true will automatically generate release notes for you based on GitHub activity <a href="https://redirect.github.com/softprops/action-gh-release/pull/179">#179</a>. Please see the <a href="https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes">GitHub docs for this feature</a> for more information</li> </ul> <h2>0.1.13</h2> <ul> <li>fix issue with multiple runs concatenating release bodies <a href="https://redirect.github.com/softprops/action-gh-release/pull/145">#145</a></li> </ul> <h2>0.1.12</h2> <ul> <li>fix bug leading to empty strings subsituted for inputs users don't provide breaking api calls <a href="https://redirect.github.com/softprops/action-gh-release/pull/144">#144</a></li> </ul>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/softprops/action-gh-release/commit/a74c6b72af54cfa997e81df42d94703d6313a2d0"><code>a74c6b7</code></a> update changelog</li> <li><a href="https://github.com/softprops/action-gh-release/commit/b909f761f0367defeee0c2804db31a81c41dd81e"><code>b909f76</code></a> update dist/index.js</li> <li><a href="https://github.com/softprops/action-gh-release/commit/e49d08fa3245e75166ef8eb8bc9d7521d34658eb"><code>e49d08f</code></a> chore(deps): bump glob from 8.0.3 to 10.4.2</li> <li><a href="https://github.com/softprops/action-gh-release/commit/f12ad255e1c9d5cfd57d1ea9dca3e3001392be35"><code>f12ad25</code></a> chore(deps): bump <code>@octokit/plugin-throttling</code> from 4.3.2 to 9.3.0</li> <li><a href="https://github.com/softprops/action-gh-release/commit/7039a825a7b0413e21c08b679d39690ada00386f"><code>7039a82</code></a> chore: release 2.0.6</li> <li><a href="https://github.com/softprops/action-gh-release/commit/f9c2b6ca3738015453adef1caa20347e8ac0fa6d"><code>f9c2b6c</code></a> chore: update deps and run build</li> <li><a href="https://github.com/softprops/action-gh-release/commit/73738a629386c54e67bc26a1e14437a5d0b686a0"><code>73738a6</code></a> chore(deps): bump node dep and <code>@types/node</code></li> <li><a href="https://github.com/softprops/action-gh-release/commit/a500a35279e4ba77e64279527fb2a91bfc538dbe"><code>a500a35</code></a> Bump ts-jest from 29.0.3 to 29.1.4 (<a href="https://redirect.github.com/softprops/action-gh-release/issues/459">#459</a>)</li> <li>See full diff in <a href="https://github.com/softprops/action-gh-release/compare/69320dbe05506a9a39fc8ae11030b214ec2d1f87...a74c6b72af54cfa997e81df42d94703d6313a2d0">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…el#14352) intel#14350 --------- Signed-off-by: Sarnie, Nick <[email protected]>

1) It isn't right 2) We need this change to get rid of deprecated `sycl::exception::get_cl_code`

…tion (intel#8954) This commit adds the sycl_ext_oneapi_virtual_mem experimental extension for reserving and mapping virtual address ranges. Accompanying it is the implementation in the SYCL runtime, together with CUDA and Level Zero backend support for the corresponding features. --------- Signed-off-by: Larsen, Steffen <[email protected]>

Destructors are implicitly noexcept, so we must ensure they don't actually throw exceptions. No change to API or ABI with this PR.

KseniyaTikhomirova added 13 commits May 22, 2024 04:12

not buildable: remove host device from device_impl.*

6e98e3f

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

not-buildable: remove getHostPlatformImpl

abe4586

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

not buildable: remove get_device_info_host

6a0a250

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

not-buildable: remove is_host from context_impl.*

35b6822

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

not-buildable: remove is_host from event_impl.*

77c749c

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

not-buildable: update is_host for API objects to be easily removed

6e71420

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

not-buildable: update most obvious places

7e5abe9

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

not-buildable: remove is_host from obvious places, part2

31a702c

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

non-buildable: remove is_host from obvious places

fa08c2b

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

not-buildable: remove is_host in simple places

d021de9

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

draft

5b60b90

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

non-buildable: eliminate getDefaultHostQueue usage

21ed380

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

non-buildable: cleanup queue usages

c533af7

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

KseniyaTikhomirova commented Jun 4, 2024

View reviewed changes

KseniyaTikhomirova added 16 commits June 5, 2024 04:31

handle nullptr Queue in commands.*

f0868f5

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

non-buildable: handle nullptr queue in memory_manager.cpp

3d044e8

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

non-buildable: build enabling

b3161e8

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

not-buildable: build enabling 2

2258a1c

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

almost buildable: build enabling 3

df27615

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

almost almost buildable: enable build 4

eebc519

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

buildable

c6fe5c8

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

RT-buildable: enabling UT build

24669e2

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

RT-buildable: restore incorrectly deleted code

fcc7748

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

RT buildable: check-sycl-AccessorTests passed

7aa76d9

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

RT-buildable: enable unittests 2

dc4a94e

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

RT-buildable: unittests enabling 3

8c57888

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

tiny cleanup

abfc5bf

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

Merge branch 'sycl' into host_device_removal

6cfa3b7

move stream_impl flush

75f6eab

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

test fix

be12c01

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

KseniyaTikhomirova had a problem deploying to WindowsCILock July 1, 2024 12:23 — with GitHub Actions Error

KseniyaTikhomirova added 2 commits July 1, 2024 05:35

fix format

3fb26e0

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

fix format 2

67a5462

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

KseniyaTikhomirova had a problem deploying to WindowsCILock July 1, 2024 12:45 — with GitHub Actions Failure

KseniyaTikhomirova requested a review from sergey-semenov July 1, 2024 12:49

sergey-semenov approved these changes Jul 1, 2024

View reviewed changes

KseniyaTikhomirova had a problem deploying to WindowsCILock July 1, 2024 14:35 — with GitHub Actions Failure

KseniyaTikhomirova and others added 7 commits July 1, 2024 10:09

update win symbols

76a073c

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

[SYCL] Fix assertion failure in E2E marray test (intel#14234)

741795d

This PR fixes a GPU accuracy bug by upscaling the error-tolerance to a double type if the GPU supports 64-bit floating point arithmetic.

[SYCL][E2E] Disable flaky test host_task_last.cpp on Gen12 Linux (int…

4c4f1b6

…el#14352) intel#14350 --------- Signed-off-by: Sarnie, Nick <[email protected]>

[SYCL] Don't throw in device_impl::has (intel#14355)

3d90aba

1) It isn't right 2) We need this change to get rid of deprecated `sycl::exception::get_cl_code`

[SYCL] no exceptions leaking from destructors (intel#14273)

719e8ef

Destructors are implicitly noexcept, so we must ensure they don't actually throw exceptions. No change to API or ABI with this PR.

KseniyaTikhomirova requested review from a team as code owners July 1, 2024 17:16

KseniyaTikhomirova requested a review from hdelan July 1, 2024 17:17

KseniyaTikhomirova closed this Jul 1, 2024

KseniyaTikhomirova mentioned this pull request Jul 1, 2024

[SYCL] Host device & queue removal (internal part, not breaking ABI) #14370

Merged

KseniyaTikhomirova requested review from bader and a team and removed request for bader, hdelan and a team July 2, 2024 14:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SYCL] Host device & queue removal (internal part, not breaking ABI) #14027

[SYCL] Host device & queue removal (internal part, not breaking ABI) #14027

Uh oh!

KseniyaTikhomirova commented Jun 4, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

[SYCL] Host device & queue removal (internal part, not breaking ABI) #14027

[SYCL] Host device & queue removal (internal part, not breaking ABI) #14027

Uh oh!

Conversation

KseniyaTikhomirova commented Jun 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KseniyaTikhomirova commented Jun 4, 2024 •

edited

Loading