Compare commits

...

36 Commits

Author SHA1 Message Date
nmzik 73615c31ff graphics: preserve logical alpha when blending reordered targets (PPSA02721)
Co-authored-by: EmK530 <admin@emk530.net>
2026-09-29 19:52:40 +04:00
nmzik 6799ecbc5b shader: add native 64-bit LDS OR with shared typed storage views 2026-09-29 19:13:33 +04:00
nmzik 35c4dca2ce shader: add V_CMPX_NLE_F16 opcode 2026-09-29 19:13:33 +04:00
nmzik 6f24b031f5 kernel: block short sleep requests instead of busy waiting 2026-09-29 16:49:09 +04:00
nmzik 0c844e4bc2 profiler: use native Tracy scopes and remove unused manual lifetime tracking 2026-09-29 13:57:32 +04:00
nmzik e57c37fa9c graphics: consume vertex semantics directly from shader metadata 2026-09-29 13:57:32 +04:00
nmzik 6d6666819a graphics: serialize pipeline cache through its renderer owner 2026-09-29 13:57:32 +04:00
nmzik 0fbeac6d3f graphics: compact shader keys and hash packed pipeline state 2026-09-29 13:57:32 +04:00
nmzik 5e4f6afc30 graphics: build color target descriptors in place 2026-09-29 13:57:32 +04:00
nmzik 37501b5a78 graphics: stream clean guest buffers without backing-store lookups 2026-09-29 13:57:32 +04:00
nmzik b43bf9f7fb graphics: store vertex attribute bindings without duplicated arrays 2026-09-29 13:57:32 +04:00
nmzik 8e42e99788 graphics: batch release labels without redundant submissions 2026-09-29 13:57:32 +04:00
João Lobo 539c0f7bf8 graphics: restore block image storage usage where the device supports it (#894) 2026-09-29 01:57:25 +04:00
GodsWill Dev 62a0222e24 tests: cover 1D-array color render targets (#377)
* tests: cover 1D-array color render targets

* tests: build shader regressions on Windows CI

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-29 01:48:56 +04:00
Saif Malik 6f241bef49 Fix macOS aggregate test build (#536)
* tests: add Darwin memory test support

* tests: fix x86_64 aggregate test build

* tests: verify macOS memory and shader tests

* tests: validate macOS death-test setup before fork

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-29 01:43:28 +04:00
Matthew Hurley 21d7e8e1f0 tests: run the saveexec mask cases in a populated subgroup (#876)
tests: run saveexec masks in a populated subgroup

Co-authored-by: Matthew Hurley <171001439+theantipopau@users.noreply.github.com>
2026-09-29 01:42:11 +04:00
Matthew Hurley 49db3ea54c tests: skip rasterization cases instead of aborting the suite on limited GPUs (#875)
tests: skip rasterization cases on limited GPUs

Co-authored-by: Matthew Hurley <171001439+theantipopau@users.noreply.github.com>
2026-09-29 01:41:37 +04:00
Aspen 4b0de1b672 Zarchive Loading (#724)
* Simplify archive loading and harden filesystem integration

* Restore directory entry name length assertion

* Keep graphics-dependent filesystem tests out of CI

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-29 00:06:44 +04:00
nmzik 8e61798bfd graphics: honor CPU ownership in image dirty-region queries 2026-09-28 22:17:43 +04:00
nmzik 6ee1bf9db0 graphics: trust image-local readback validity 2026-09-28 22:17:43 +04:00
nmzik 0471c9991c audio: implement AudioOut2 user capability queries 2026-09-28 22:17:43 +04:00
nmzik b10bd53dd3 audio: prevent ATRAC9 vibration configuration heap corruption 2026-09-28 22:17:43 +04:00
nmzik 4cf456fb77 graphics: retain buffer names across draw bindings 2026-09-28 22:17:43 +04:00
Devran Cosmo Uenal 22ff4693c5 audio: play pad speaker ports on the DualSense speaker (#868)
* audio: play pad speaker ports on the DualSense speaker

Route the right channel to the speaker with the audio-control values the Linux hid-playstation
driver uses without headphones, and fall back to the main output while no DualSense audio device
is active. Also stop the previous DualSense's rumble when another DualSense becomes active.

* audio: harden DualSense speaker routing and fallback

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-28 20:00:58 +04:00
nmzik a107d1e333 filesystem: keep unmapped guest paths out of the host namespace 2026-09-28 19:17:03 +04:00
Emil 05f92b6422 libCes CP932 fix for Windows/macOS, fix SaveDataDialog lockups (#878)
* libs: fix CP932 iconv conversion on Windows and macOS

SDL3's SDL_SYSTEM_ICONV build option defaults off on Windows and macOS,
so SDL falls back to its own bundled iconv there, which only implements
ASCII/Latin-1/UTF/UCS. SDL_iconv_open("UTF-8", "CP932") always failed as
a result, crashing games that convert Shift-JIS strings via
sceCesMbcsStrToUtf8Str (e.g. save-data dialogs).

Add a small CesIconv shim selected at compile time: Windows converts one
character at a time via the Win32 codepage-932 APIs, replicating
SDL_iconv's chunked semantics and error codes; macOS calls real POSIX
iconv directly, since libiconv is always present there and does support
CP932, unlike SDL's fallback. Linux is untouched, since SDL already
links real system iconv by default there.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* libs: fix SaveDataDialog infinite loop on save

SaveDataDialogOpen() jumped straight from SAVE_STATUS_INITIALIZED (1) to
SAVE_STATUS_FINISHED (3), skipping SAVE_STATUS_RUNNING (2) entirely. At
least one title's save-write dialog stage only accepts a FINISHED status
after observing a RUNNING poll first; without it, the title assumed the
Open() request never took effect and re-issued it forever, hanging the
save flow (this only became visible once the CP932 crash above stopped
masking it).

Open() now starts the dialog as RUNNING. UpdateStatus() advances the
state machine, reporting RUNNING once before flipping to FINISHED for
subsequent polls; GetStatus() is untouched, staying a pure peek with no
side effects, matching the real SDK's pump-vs-peek split between the two
calls.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Add iconv linking to macOS

* Fix missing macOS iconv link for tests

* Fix CP932 edge cases and save dialog status consistency

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-28 18:42:22 +04:00
nmzik a892097ff0 tests: fix macOS extended-memory alias regression check (#889)
tests: use a macOS-owned address for extended memory alias checks
2026-09-28 18:33:46 +04:00
nmzik c8baa7b909 shader: derive immutable resource policy once per plan 2026-09-28 18:01:49 +04:00
nmzik 34a3eb6b58 memory: recycle automatic backing through physical ownership 2026-09-28 18:01:49 +04:00
nmzik a20b64d3e5 audio: rewind compressed sampler blocks at loop boundaries 2026-09-28 18:01:49 +04:00
nmzik 2016f8e2fe gpu: address extended guest memory without widening dense cache tables 2026-09-28 18:01:49 +04:00
nmzik c91fe80d64 memory: separate extended allocations from ordinary guest mappings 2026-09-28 18:01:49 +04:00
nmzik ab2c1c4c03 Update README screenshot to UFC 5 2026-09-28 18:01:49 +04:00
gohansharma79-svg 0a5774be7b shader: support FP64 conversions used by FighterZ (#884)
shader: support float and unsigned widening to FP64

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-28 17:39:31 +04:00
nmzik 0791f921f6 UFC 5 in-game (#883)
* shader: release descriptor dependencies

* shader: simplify shared-exit control flow

* shader: fix scalar memory loads

* shader: fix LDS counter bounds

* renderer: skip inactive pixel shaders

* shader: respect FP32 mode in comparisons

* renderer: fix sampler border colors

* shader: resolve descriptors from control flow

* shader: resolve descriptors before resource planning

* shader: handle scalar guards for image bounds

* shader: handle reduced guards for image loops

* shader: fix wave guards and material indices

* shader: preserve scalar buffer offset alignment and carry

* shader: prove masked loop bounds across iteration transitions

* shader: preserve source and IR before resource tracking failures

* shader: track material keys selected from active lanes

* shader: remove obsolete post-dominator analysis

* shader: follow bounds guards through control-flow joins
2026-09-28 16:03:05 +04:00
nmzik 421684e75f docs: add weekly updates and Discord links to README (#866)
* docs: link weekly updates from README

* docs: add Discord development channel

* docs: use Discord server invite link

* docs: refine weekly updates wording
2026-09-27 18:06:26 +04:00
126 changed files with 8314 additions and 3400 deletions
+11 -9
View File
@@ -113,14 +113,15 @@ jobs:
shell: cmd
run: |
cmake --build _Build/windows --target launcher ^
audio_out2_port_tests ime_dialog_tests lru_cache_tests virtual_memory_allocation_tests ^
audio_out2_port_tests pad_haptics_tests ime_dialog_tests lru_cache_tests virtual_memory_allocation_tests ^
shader_recompiler_compute_tests ^
resource_tracking_tests scalar_provenance_tests ^
resource_materialization_tests shader_vertex_metadata_tests --parallel
resource_materialization_tests shader_vertex_metadata_tests archive_file_tests avplayer_file_tests --parallel
- name: Test
shell: cmd
run: |
ctest --test-dir _Build/windows --output-on-failure -R "^(audio_out2_port|ime_dialog|lru_cache|resource_tracking|scalar_provenance|resource_materialization|shader_vertex_metadata|virtual_memory_allocation)$"
ctest --test-dir _Build/windows --output-on-failure -R "^(audio_out2_port|pad_haptics|ime_dialog|lru_cache|resource_tracking|scalar_provenance|resource_materialization|shader_vertex_metadata|archive_file|avplayer_file|virtual_memory_allocation)$"
- name: Install
shell: cmd
@@ -201,16 +202,17 @@ jobs:
shell: bash
run: |
cmake --build _Build/macos \
--target launcher audio_out2_port_tests ime_dialog_tests lru_cache_tests virtual_memory_allocation_tests \
--target launcher audio_out2_port_tests pad_haptics_tests ime_dialog_tests lru_cache_tests virtual_memory_allocation_tests \
memory_tracker_tests page_manager_tests shader_recompiler_compute_tests \
resource_tracking_tests scalar_provenance_tests \
resource_materialization_tests shader_vertex_metadata_tests \
resource_materialization_tests shader_vertex_metadata_tests archive_file_tests avplayer_file_tests \
--parallel
- name: Test
shell: bash
run: |
ctest --test-dir _Build/macos --output-on-failure \
-R '^(audio_out2_port|ime_dialog|lru_cache|resource_tracking|scalar_provenance|resource_materialization|shader_vertex_metadata|virtual_memory_allocation)$'
-R '^(audio_out2_port|pad_haptics|ime_dialog|lru_cache|memory_tracker|page_manager|resource_tracking|scalar_provenance|resource_materialization|shader_vertex_metadata|archive_file|avplayer_file|virtual_memory_allocation)$'
- name: Install
shell: bash
@@ -384,16 +386,16 @@ jobs:
run: |
cmake --build _Build/linux \
--target launcher page_manager_tests memory_tracker_tests \
audio_out2_port_tests ime_dialog_tests lru_cache_tests virtual_memory_allocation_tests \
audio_out2_port_tests pad_haptics_tests ime_dialog_tests lru_cache_tests virtual_memory_allocation_tests \
resource_tracking_tests scalar_provenance_tests \
resource_materialization_tests shader_vertex_metadata_tests \
resource_materialization_tests shader_vertex_metadata_tests archive_file_tests avplayer_file_tests \
--parallel
- name: Test
shell: bash
run: |
ctest --test-dir _Build/linux --output-on-failure \
-R '^(audio_out2_port|ime_dialog|lru_cache|memory_tracker|page_manager|resource_tracking|scalar_provenance|resource_materialization|shader_vertex_metadata|virtual_memory_allocation)$'
-R '^(audio_out2_port|pad_haptics|ime_dialog|lru_cache|memory_tracker|page_manager|resource_tracking|scalar_provenance|resource_materialization|shader_vertex_metadata|archive_file|avplayer_file|virtual_memory_allocation)$'
- name: Install
shell: bash
+47
View File
@@ -28,7 +28,54 @@ FetchContent_Declare(
FetchContent_MakeAvailable(xbyak zydis)
# ZArchive provides random-access reads for compressed game dumps. Build only
# the reader/writer library and its zstd dependency; the command-line tool is
# not part of KytyPS5's runtime distribution.
set(ZSTD_BUILD_PROGRAMS OFF CACHE BOOL "" FORCE)
set(ZSTD_BUILD_SHARED OFF CACHE BOOL "" FORCE)
set(ZSTD_BUILD_STATIC ON CACHE BOOL "" FORCE)
set(ZSTD_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(ZSTD_LEGACY_SUPPORT OFF CACHE BOOL "" FORCE)
set(ZSTD_MULTITHREAD_SUPPORT OFF CACHE BOOL "" FORCE)
FetchContent_Declare(
zstd
GIT_REPOSITORY https://github.com/facebook/zstd.git
GIT_TAG f8745da6ff1ad1e7bab384bd1f9d742439278e99
GIT_SHALLOW FALSE
SOURCE_SUBDIR build/cmake
)
FetchContent_Declare(
zarchive_source
GIT_REPOSITORY https://github.com/Exzap/ZArchive.git
GIT_TAG d2c717730092c7bf8cbb033b12fd4001b7c4d932
GIT_SHALLOW FALSE
PATCH_COMMAND "${GIT_EXECUTABLE}" apply --ignore-whitespace
"${CMAKE_CURRENT_SOURCE_DIR}/patches/zarchive-reader.patch"
SOURCE_SUBDIR kyty-library-only
)
FetchContent_MakeAvailable(zstd zarchive_source)
add_library(zarchive STATIC
"${zarchive_source_SOURCE_DIR}/src/zarchivereader.cpp"
"${zarchive_source_SOURCE_DIR}/src/zarchivewriter.cpp"
"${zarchive_source_SOURCE_DIR}/src/sha_256.c"
)
add_library(ZArchive::zarchive ALIAS zarchive)
target_include_directories(zarchive SYSTEM PUBLIC "${zarchive_source_SOURCE_DIR}/include")
target_include_directories(zarchive PRIVATE "${zstd_SOURCE_DIR}/lib")
target_link_libraries(zarchive PRIVATE libzstd_static)
target_compile_features(zarchive PRIVATE cxx_std_20)
if(MSVC)
target_compile_options(zarchive PRIVATE /W0)
else()
target_compile_options(zarchive PRIVATE -w)
endif()
set(SDL_SHARED OFF CACHE BOOL "" FORCE)
if(APPLE)
# SDL's built-in iconv fallback does not support CP932.
set(SDL_SYSTEM_ICONV ON CACHE BOOL "Use system iconv for CP932 conversion" FORCE)
endif()
add_subdirectory(SDL3)
add_library(Vulkan-Headers INTERFACE)
+100
View File
@@ -0,0 +1,100 @@
--- a/include/zarchive/zarchivecommon.h
+++ b/include/zarchive/zarchivecommon.h
@@ -256,7 +256,7 @@
bool IsWithinValidRange(uint64_t fileSize) const
{
- return (offset + size) <= fileSize;
+ return offset <= fileSize && size <= fileSize - offset;
}
};
--- a/src/zarchivereader.cpp
+++ b/src/zarchivereader.cpp
@@ -14,6 +14,7 @@
static bool _ifstream_readBytes(std::ifstream& file, uint64_t offset, void* buffer, uint32_t size)
{
+ file.clear();
file.seekg(offset, std::ios_base::beg);
file.read((char*)buffer, size);
return file.gcount() == size;
@@ -80,9 +81,27 @@
// verify file tree
if (fileTree[0].IsFile())
return nullptr; // first entry must be root directory
- auto rootName = GetName(nameTable, fileTree[0].GetNameOffset());
- if (!rootName.empty())
+ if (fileTree[0].GetNameOffset() != 0x7FFFFFFF)
return nullptr; // root node must not have a name
+ const uint64_t dataCapacity = (uint64_t)offsetRecords.size() * _ZARCHIVE::ENTRIES_PER_OFFSETRECORD * _ZARCHIVE::COMPRESSED_BLOCK_SIZE;
+ for (size_t i = 0; i < fileTree.size(); ++i)
+ {
+ auto& entry = fileTree[i];
+ if (i != 0)
+ {
+ auto name = GetName(nameTable, entry.GetNameOffset());
+ if (name.empty() || name == "." || name == ".." || name.find_first_of("/\\") != std::string_view::npos || name.find('\0') != std::string_view::npos)
+ return nullptr;
+ }
+ if (entry.IsFile())
+ {
+ if (entry.GetFileOffset() > dataCapacity || entry.GetFileSize() > dataCapacity - entry.GetFileOffset())
+ return nullptr;
+ }
+ else if (entry.directoryRecord.nodeStartIndex > fileTree.size() ||
+ entry.directoryRecord.count > fileTree.size() - entry.directoryRecord.nodeStartIndex)
+ return nullptr;
+ }
// read meta data
// todo
@@ -238,7 +257,7 @@
uint32_t stepSize = std::min<uint32_t>(remainingBytes, _ZARCHIVE::COMPRESSED_BLOCK_SIZE - blockOffset);
CacheBlock* block = GetCachedBlock(blockIdx);
if (!block)
- return 0;
+ return bytesToRead - remainingBytes;
std::memcpy(bufferU8, block->data + blockOffset, stepSize);
rawReadOffset += stepSize;
remainingBytes -= stepSize;
@@ -325,14 +344,18 @@
// determine offset and size of compressed block
auto& record = m_offsetRecords[recordIndex];
uint64_t offset = record.baseOffset;
+ if (offset > m_compressedDataSize)
+ return false;
for (uint32_t i = 0; i < recordSubIndex; i++)
{
- offset += (uint64_t)record.size[i];
- offset++;
+ const uint64_t size = (uint64_t)record.size[i] + 1;
+ if (size > m_compressedDataSize - offset)
+ return false;
+ offset += size;
}
uint32_t compressedSize = (uint32_t)record.size[recordSubIndex] + 1;
// load file data
- if ((offset + compressedSize) > m_compressedDataSize)
+ if (compressedSize > m_compressedDataSize - offset)
return false;
offset += m_compressedDataOffset;
if (compressedSize == _ZARCHIVE::COMPRESSED_BLOCK_SIZE)
@@ -350,7 +373,7 @@
// returns empty view on failure
std::string_view ZArchiveReader::GetName(const std::vector<uint8_t>& nameTable, uint32_t nameOffset)
{
- if (nameOffset == 0x7FFFFFFF || nameOffset > nameTable.size())
+ if (nameOffset == 0x7FFFFFFF || nameOffset >= nameTable.size())
return "";
// parse header
uint16_t nameLength = nameTable[nameOffset] & 0x7F;
@@ -359,7 +382,7 @@
// extended 2-byte length
if (nameOffset + 1 >= nameTable.size())
return "";
- nameLength |= ((uint16_t)nameTable[nameOffset] << 7);
+ nameLength |= ((uint16_t)nameTable[nameOffset + 1] << 7);
nameOffset += 2;
}
else
+33 -1
View File
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.16...4.3)
cmake_minimum_required(VERSION 3.18...4.3)
if(CMAKE_VERSION VERSION_GREATER_EQUAL 4.0)
set(CMAKE_POLICY_VERSION_MINIMUM 3.5 CACHE STRING "Minimum policy version for bundled dependencies")
@@ -407,6 +407,16 @@ add_executable(page_manager_tests EXCLUDE_FROM_ALL
target_link_libraries(page_manager_tests common)
target_include_directories(page_manager_tests PRIVATE ${inc_headers})
add_executable(archive_file_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/ArchiveFileTests.cpp"
)
target_link_libraries(archive_file_tests common ZArchive::zarchive)
target_include_directories(archive_file_tests PRIVATE ${inc_headers})
add_executable(avplayer_file_tests EXCLUDE_FROM_ALL "${KYTY_TESTS_DIR}/AvPlayerFileTests.cpp")
target_link_libraries(avplayer_file_tests common FFmpeg::ffmpeg ZArchive::zarchive)
target_include_directories(avplayer_file_tests PRIVATE ${inc_headers})
add_executable(bit_array_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/BitArrayTests.cpp"
)
@@ -490,6 +500,17 @@ add_executable(save_data_memory_tests EXCLUDE_FROM_ALL
target_link_libraries(save_data_memory_tests common fmt::fmt)
target_include_directories(save_data_memory_tests PRIVATE ${inc_headers})
add_executable(ces_tests EXCLUDE_FROM_ALL "${KYTY_TESTS_DIR}/CesTests.cpp")
target_link_libraries(ces_tests common fmt::fmt SDL3::SDL3-static)
target_include_directories(ces_tests PRIVATE ${inc_headers})
add_executable(save_data_dialog_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/SaveDataDialogTests.cpp"
"${KYTY_SOURCE_DIR}/libs/dialog.cpp"
)
target_link_libraries(save_data_dialog_tests common fmt::fmt)
target_include_directories(save_data_dialog_tests PRIVATE ${inc_headers})
add_kyty_full_emulator_test(http_uri_parse_tests
"${KYTY_TESTS_DIR}/HttpUriParseTests.cpp")
@@ -503,6 +524,7 @@ target_include_directories(event_queue_lifetime_tests PRIVATE ${inc_headers})
add_kyty_full_emulator_test(kernel_file_system_tests
"${KYTY_TESTS_DIR}/KernelFileSystemTests.cpp")
target_link_libraries(kernel_file_system_tests ZArchive::zarchive)
add_executable(sync_on_address_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/SyncOnAddressTests.cpp"
@@ -569,6 +591,8 @@ if(BUILD_TESTING)
add_test(NAME image_page_table COMMAND $<TARGET_FILE:image_page_table_tests>)
add_test(NAME memory_tracker COMMAND $<TARGET_FILE:memory_tracker_tests>)
add_test(NAME page_manager COMMAND $<TARGET_FILE:page_manager_tests>)
add_test(NAME archive_file COMMAND $<TARGET_FILE:archive_file_tests>)
add_test(NAME avplayer_file COMMAND $<TARGET_FILE:avplayer_file_tests>)
add_test(NAME bit_array COMMAND $<TARGET_FILE:bit_array_tests>)
add_test(NAME lru_cache COMMAND $<TARGET_FILE:lru_cache_tests>)
add_test(NAME shader_vertex_metadata COMMAND $<TARGET_FILE:shader_vertex_metadata_tests>)
@@ -581,6 +605,8 @@ if(BUILD_TESTING)
add_test(NAME ngs2_sampler COMMAND $<TARGET_FILE:ngs2_sampler_tests>)
add_test(NAME pad_haptics COMMAND $<TARGET_FILE:pad_haptics_tests>)
add_test(NAME save_data_memory COMMAND $<TARGET_FILE:save_data_memory_tests>)
add_test(NAME ces COMMAND $<TARGET_FILE:ces_tests>)
add_test(NAME save_data_dialog COMMAND $<TARGET_FILE:save_data_dialog_tests>)
add_test(NAME http_uri_parse COMMAND $<TARGET_FILE:http_uri_parse_tests>)
add_test(NAME shader_recompiler_compute COMMAND $<TARGET_FILE:shader_recompiler_compute_tests>)
add_test(NAME shader_recompiler_alignbyte
@@ -618,6 +644,8 @@ if(BUILD_TESTING)
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --compute-meta-clear-only)
add_test(NAME buffer_cache_dirty_gc
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --buffer-cache-gc-only)
add_test(NAME color_render_target_1d
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --rt1d-only)
if(WIN32)
# These tests still depend on the Windows multisample-depth path.
@@ -636,6 +664,8 @@ if(BUILD_TESTING)
image_page_table_tests
memory_tracker_tests
page_manager_tests
archive_file_tests
avplayer_file_tests
bit_array_tests
lru_cache_tests
shader_vertex_metadata_tests
@@ -650,6 +680,8 @@ if(BUILD_TESTING)
ngs2_sampler_tests
pad_haptics_tests
save_data_memory_tests
ces_tests
save_data_dialog_tests
http_uri_parse_tests
)
endif()
+17 -8
View File
@@ -7,6 +7,10 @@
[![Status](https://img.shields.io/badge/status-active%20development-orange.svg)](#current-status)
[![License](https://img.shields.io/badge/license-GPL--2.0-blue.svg)](LICENSE)
**[Weekly updates](https://github.com/KytyPS5/KytyPS5/discussions/862)** — game progress, recent fixes and ongoing development.
**[Development on Discord](https://discord.gg/UNrkMqGaBg)** — KytyPS5 development.
KytyPS5 is a free and open-source PlayStation 5 emulator written in C++ for Windows and Linux,
with experimental macOS support. It is based on a heavily modified version of
[Kyty](https://github.com/InoriRus/Kyty). The project is in active development, and behavior
@@ -68,8 +72,8 @@ or graphical glitches, so please include the version you tested when reporting a
<img src="docs/screenshots/ps5-02.png" width="300" alt="Demon's Souls running in KytyPS5">
</td>
<td align="center">
<strong>Hellboy</strong><br>
<img src="docs/screenshots/ps5-06.png" width="300" alt="Hellboy running in KytyPS5">
<strong>UFC 5</strong><br>
<img src="docs/screenshots/ps5-06.png" width="300" alt="UFC 5 running in KytyPS5">
</td>
</tr>
</table>
@@ -232,9 +236,10 @@ cmake --build _Build/linux --target launcher --parallel
cmake --install _Build/linux --prefix _Build/linux/install
```
The configure step downloads the FFmpeg prebuilts and the `xbyak`/`zydis` sources, so it needs
network access; a fully sandboxed `nix build` would require vendoring those inputs. A Vulkan 1.3
driver must be available at runtime (on NixOS, `hardware.graphics.enable = true`).
The configure step downloads the FFmpeg prebuilts and the `xbyak`, `zydis`, `zstd`, and ZArchive
sources, so it needs network access; a fully sandboxed `nix build` would require vendoring those
inputs. A Vulkan 1.3 driver must be available at runtime (on NixOS,
`hardware.graphics.enable = true`).
### Building on macOS
@@ -328,17 +333,21 @@ open _Build/macos/install/KytyPS5.app # or double-click in Finder
```
On first launch, add one or more game folders in the global settings. The launcher searches those
folders recursively for game directories containing `eboot.bin`. Select a detected game and run it
from the game list.
folders recursively for game directories containing `eboot.bin` and ZArchive (`.zar`) game dumps
whose archive root contains `eboot.bin`. Select a detected game and run it from the game list.
ZArchive dumps are mounted read-only and streamed directly; they do not need to be extracted first.
The emulator can also be started directly with a legally obtained game directory or ELF file:
The emulator can also be started directly with a legally obtained game directory, ELF file, or
ZArchive dump:
```powershell
.\_Build\windows\install\kyty_emulator.exe --game "D:\Games\ExampleGame"
.\_Build\windows\install\kyty_emulator.exe --game "D:\Games\ExampleGame.zar"
```
```bash
./_Build/linux/install/kyty_emulator --game "/games/ExampleGame"
./_Build/linux/install/kyty_emulator --game "/games/ExampleGame.zar"
```
On macOS, the adjacent flat or app-bundled `libMoltenVK.dylib` is found automatically; no
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 MiB

After

Width:  |  Height:  |  Size: 1.7 MiB

+1
View File
@@ -26,6 +26,7 @@ target_link_libraries(common
cpuinfo
kyty_git_version
spdlog::spdlog
ZArchive::zarchive
)
get_property(inc_headers TARGET common PROPERTY INCLUDE_DIRECTORIES)
+397
View File
@@ -0,0 +1,397 @@
#include "common/archive.h"
#include "common/stringUtils.h"
#include <algorithm>
#include <condition_variable>
#include <cstring>
#include <mutex>
#include <optional>
#include <string_view>
#include <thread>
#include <unordered_map>
#include <utility>
#include <zarchive/zarchivereader.h>
// POSIX hosts and MinGW's pthread-backed libstdc++ have cancellable condition waits.
// MSVC/ClangCL and MinGW's native Win32 threading backend do not use pthread waits.
#if !defined(_WIN32) || defined(_GLIBCXX_GCC_GTHR_POSIX_H)
#include <pthread.h>
#define KYTY_ARCHIVE_PTHREAD_CANCELLATION
#endif
namespace Common {
// Backends expose immutable metadata and offset-based reads. Cursor management, path routing,
// reader lifetime and the host-stack worker are shared by every archive format.
class ArchiveReader {
public:
struct Entry {
uint64_t id;
uint64_t size;
bool is_file;
};
virtual ~ArchiveReader() = default;
virtual std::optional<Entry> Find(std::string_view member) = 0;
virtual std::vector<File::DirEntry> List(std::string_view member) = 0;
virtual uint64_t Read(uint64_t id, uint64_t offset, uint32_t size, void* data) = 0;
};
namespace {
class ZArchiveReaderBackend final: public ArchiveReader {
public:
explicit ZArchiveReaderBackend(std::unique_ptr<ZArchiveReader> reader)
: m_reader(std::move(reader)) {}
std::optional<Entry> Find(std::string_view member) override {
const auto node = m_reader->LookUp(member);
if (node == ZARCHIVE_INVALID_NODE) {
return {};
}
return Entry {node, m_reader->GetFileSize(node), m_reader->IsFile(node)};
}
std::vector<File::DirEntry> List(std::string_view member) override {
const auto node = m_reader->LookUp(member);
if (!m_reader->IsDirectory(node)) {
return {};
}
const auto count = m_reader->GetDirEntryCount(node);
if (count > (1u << 20u)) {
return {};
}
std::vector<File::DirEntry> result;
result.reserve(count);
for (uint32_t index = 0; index < count; ++index) {
ZArchiveReader::DirEntry entry {};
if (m_reader->GetDirEntry(node, index, entry)) {
result.push_back({std::string(entry.name), entry.isFile});
}
}
return result;
}
uint64_t Read(uint64_t id, uint64_t offset, uint32_t size, void* data) override {
// ZArchiveReader already serializes access to its stream and decompression cache.
return m_reader->ReadFromFile(static_cast<ZArchiveNodeHandle>(id), offset, size, data);
}
private:
std::unique_ptr<ZArchiveReader> m_reader;
};
std::shared_ptr<ArchiveReader> OpenZArchive(const std::filesystem::path& path) {
auto reader = std::unique_ptr<ZArchiveReader>(ZArchiveReader::OpenFromFile(path));
return reader ? std::make_shared<ZArchiveReaderBackend>(std::move(reader)) : nullptr;
}
struct ArchiveFormat {
std::string_view extension;
std::shared_ptr<ArchiveReader> (*open)(const std::filesystem::path&);
};
constexpr ArchiveFormat Formats[] = {{".zar", OpenZArchive}};
using NativeView = std::basic_string_view<std::filesystem::path::value_type>;
const ArchiveFormat* FindFormat(NativeView path) {
for (const auto& format: Formats) {
if (path.size() < format.extension.size()) {
continue;
}
const auto extension = path.substr(path.size() - format.extension.size());
if (std::equal(
extension.begin(), extension.end(), format.extension.begin(),
[](auto a, char b) { return (a >= 'A' && a <= 'Z' ? a + ('a' - 'A') : a) == b; })) {
return &format;
}
}
return nullptr;
}
size_t FindMarker(NativeView path) {
// Native files use this check on every operation. The usual path has no marker and allocates
// nothing.
for (auto marker = path.find('!'); marker != NativeView::npos;
marker = path.find('!', marker + 1)) {
if ((marker + 1 == path.size() || path[marker + 1] == '/' || path[marker + 1] == '\\') &&
FindFormat(path.substr(0, marker)) != nullptr) {
return marker;
}
}
return NativeView::npos;
}
struct ParsedPath {
std::filesystem::path archive;
std::string member;
};
std::optional<ParsedPath> ParsePath(const std::filesystem::path& path) {
const auto& native = path.native();
const auto marker = FindMarker(native);
if (marker == NativeView::npos) {
return {};
}
ParsedPath parsed {.archive = std::filesystem::path(native.substr(0, marker))};
auto member = PathToGenericString(std::filesystem::path(native.substr(marker + 1)));
std::replace(member.begin(), member.end(), '\\', '/');
std::string_view remaining = member;
while (!remaining.empty()) {
const auto separator = remaining.find('/');
const auto component = remaining.substr(0, separator);
remaining.remove_prefix(separator == std::string_view::npos ? remaining.size() : separator + 1);
if (component.empty() || component == ".") {
continue;
}
if (component == "..") {
if (parsed.member.empty()) {
return {};
}
const auto separator = parsed.member.rfind('/');
parsed.member.resize(separator == std::string::npos ? 0 : separator);
} else {
if (!parsed.member.empty()) {
parsed.member += '/';
}
parsed.member += component;
}
}
return parsed;
}
std::mutex g_readers_mutex;
std::unordered_map<std::filesystem::path, std::weak_ptr<ArchiveReader>> g_readers;
// Guest stacks can be only 64 KiB. Decompress on an ordinary host stack, then copy into guest
// memory on the caller so GPU-tracked writes use its fault handler. A single worker bounds host
// threads and avoids duplicating the archive's own block cache. Requests live on waiting callers.
class ArchiveIoThread {
public:
static ArchiveIoThread& Instance() {
// Guest exit can run static destructors while other guest threads are still reading.
// Keep the worker alive until process termination, as with the guest threads themselves.
static auto* instance = new ArchiveIoThread;
return *instance;
}
uint64_t Read(ArchiveReader& reader, uint64_t id, uint64_t offset, uint32_t size, void* data) {
Request request {reader, id, offset, size, data};
std::unique_lock lock(m_mutex);
if (m_last != nullptr) {
m_last->next = &request;
} else {
m_first = &request;
}
m_last = &request;
m_wake.notify_one();
request.ready.wait(lock, [&] { return request.done; });
return request.result;
}
KYTY_CLASS_NO_COPY(ArchiveIoThread);
private:
struct Request {
ArchiveReader& reader;
uint64_t id;
uint64_t offset;
uint32_t size;
void* data;
Request* next = nullptr;
uint64_t result = 0;
bool done = false;
std::condition_variable ready {};
};
ArchiveIoThread() {
std::thread([this] { Loop(); }).detach();
}
void Loop() {
for (;;) {
Request* request;
{
std::unique_lock lock(m_mutex);
m_wake.wait(lock, [this] { return m_first != nullptr; });
request = m_first;
m_first = request->next;
if (m_first == nullptr) {
m_last = nullptr;
}
}
const auto result =
request->reader.Read(request->id, request->offset, request->size, request->data);
{
std::lock_guard lock(m_mutex);
request->result = result;
request->done = true;
// Notify before releasing the mutex: the waiting caller owns the condition
// variable.
request->ready.notify_one();
}
}
}
std::mutex m_mutex;
std::condition_variable m_wake;
Request* m_first = nullptr;
Request* m_last = nullptr;
};
} // namespace
struct ArchiveFile::Private {
std::shared_ptr<ArchiveReader> reader;
ArchiveReader::Entry entry;
uint64_t position = 0;
};
ArchiveFile::ArchiveFile(std::unique_ptr<Private> p): m_p(std::move(p)) {}
ArchiveFile::~ArchiveFile() = default;
uint64_t ArchiveFile::Size() const {
return m_p->entry.size;
}
uint64_t ArchiveFile::Tell() const {
return m_p->position;
}
bool ArchiveFile::Seek(uint64_t offset) {
m_p->position = offset;
return true;
}
void ArchiveFile::Read(void* data, uint32_t size, uint32_t* bytes_read) {
#if defined(KYTY_ARCHIVE_PTHREAD_CANCELLATION)
// A pending cancellation must not destroy the waiting request or its TLS buffer while the
// worker still uses them. Restore the caller's state only after the copy is complete.
int cancellation_state = PTHREAD_CANCEL_ENABLE;
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &cancellation_state);
#endif
constexpr uint32_t chunk_size = 1u << 20u;
thread_local std::vector<uint8_t> chunk;
uint32_t read = 0;
if (data != nullptr && size != 0 && m_p->position < Size()) {
const auto requested =
static_cast<uint32_t>(std::min<uint64_t>(size, Size() - m_p->position));
if (chunk.size() < std::min(chunk_size, requested)) {
chunk.resize(std::min(chunk_size, requested));
}
while (read < requested) {
const auto step = std::min(chunk_size, requested - read);
const auto got = ArchiveIoThread::Instance().Read(*m_p->reader, m_p->entry.id,
m_p->position, step, chunk.data());
if (got == 0 || got > step) {
break;
}
std::memcpy(static_cast<uint8_t*>(data) + read, chunk.data(), got);
read += static_cast<uint32_t>(got);
m_p->position += got;
if (got < step) {
break;
}
}
}
if (bytes_read != nullptr) {
*bytes_read = read;
}
#if defined(KYTY_ARCHIVE_PTHREAD_CANCELLATION)
pthread_setcancelstate(cancellation_state, nullptr);
#endif
}
std::filesystem::path MakeArchivePath(const std::filesystem::path& archive,
const std::filesystem::path& member) {
auto root = archive;
root += "!";
return member.empty() ? root : root / member.relative_path();
}
bool IsSupportedArchive(const std::filesystem::path& path) {
return FindFormat(path.native()) != nullptr;
}
bool IsArchivePath(const std::filesystem::path& path) {
return FindMarker(path.native()) != NativeView::npos;
}
std::filesystem::path GetArchiveHostPath(const std::filesystem::path& path) {
const auto marker = FindMarker(path.native());
return marker == NativeView::npos ? std::filesystem::path {}
: std::filesystem::path(path.native().substr(0, marker));
}
std::shared_ptr<ArchiveReader> OpenArchive(const std::filesystem::path& path) {
const auto archive = IsArchivePath(path) ? GetArchiveHostPath(path) : path;
const auto* format = FindFormat(archive.native());
if (format == nullptr) {
return {};
}
std::error_code error;
auto key = std::filesystem::absolute(archive, error);
if (error) {
return {};
}
{
std::lock_guard lock(g_readers_mutex);
if (auto it = g_readers.find(key); it != g_readers.end()) {
if (auto reader = it->second.lock()) {
return reader;
}
}
}
// Do not hold the cache mutex while opening an unrelated archive from disk.
auto reader = format->open(archive);
if (reader == nullptr) {
return {};
}
std::lock_guard lock(g_readers_mutex);
std::erase_if(g_readers, [](const auto& entry) { return entry.second.expired(); });
auto& cached = g_readers[key];
if (auto existing = cached.lock()) {
return existing;
}
cached = reader;
return reader;
}
std::optional<File::Info> GetArchiveInfo(const std::filesystem::path& path) {
const auto parsed = ParsePath(path);
if (parsed) {
if (auto reader = OpenArchive(parsed->archive)) {
if (auto entry = reader->Find(parsed->member)) {
return File::Info {entry->is_file, entry->size};
}
}
}
return {};
}
std::vector<File::DirEntry> GetArchiveDirEntries(const std::filesystem::path& path) {
const auto parsed = ParsePath(path);
if (parsed) {
if (auto reader = OpenArchive(parsed->archive)) {
return reader->List(parsed->member);
}
}
return {};
}
std::unique_ptr<ArchiveFile> OpenArchiveFile(const std::filesystem::path& path) {
auto parsed = ParsePath(path);
if (!parsed) {
return {};
}
auto reader = OpenArchive(parsed->archive);
const auto entry = reader ? reader->Find(parsed->member) : std::nullopt;
if (!entry || !entry->is_file) {
return {};
}
auto p = std::make_unique<ArchiveFile::Private>();
p->reader = std::move(reader);
p->entry = *entry;
return std::unique_ptr<ArchiveFile>(new ArchiveFile(std::move(p)));
}
} // namespace Common
+50
View File
@@ -0,0 +1,50 @@
#ifndef KYTY_COMMON_ARCHIVE_H_
#define KYTY_COMMON_ARCHIVE_H_
#include "common/file.h"
#include <cstdint>
#include <filesystem>
#include <memory>
#include <vector>
namespace Common {
class ArchiveReader;
class ArchiveFile {
public:
~ArchiveFile();
[[nodiscard]] uint64_t Size() const;
[[nodiscard]] uint64_t Tell() const;
bool Seek(uint64_t offset);
void Read(void* data, uint32_t size, uint32_t* bytes_read);
KYTY_CLASS_NO_COPY(ArchiveFile);
private:
struct Private;
explicit ArchiveFile(std::unique_ptr<Private> p);
std::unique_ptr<Private> m_p;
friend std::unique_ptr<ArchiveFile> OpenArchiveFile(const std::filesystem::path& path);
};
// Virtual paths use "archive.ext!/member" and retain native host-path encoding.
[[nodiscard]] std::filesystem::path MakeArchivePath(const std::filesystem::path& archive,
const std::filesystem::path& member = {});
[[nodiscard]] bool IsSupportedArchive(const std::filesystem::path& path);
[[nodiscard]] bool IsArchivePath(const std::filesystem::path& path);
[[nodiscard]] std::optional<File::Info> GetArchiveInfo(const std::filesystem::path& path);
[[nodiscard]] std::vector<File::DirEntry> GetArchiveDirEntries(const std::filesystem::path& path);
[[nodiscard]] std::filesystem::path GetArchiveHostPath(const std::filesystem::path& path);
[[nodiscard]] std::unique_ptr<ArchiveFile> OpenArchiveFile(const std::filesystem::path& path);
// Retain this handle while a mount or metadata scan is active to reuse the index and block cache.
// Accepts either a host archive filename or a virtual path inside the archive.
[[nodiscard]] std::shared_ptr<ArchiveReader> OpenArchive(const std::filesystem::path& path);
} // namespace Common
#endif /* KYTY_COMMON_ARCHIVE_H_ */
+140 -17
View File
@@ -1,5 +1,6 @@
#include "common/file.h"
#include "common/archive.h"
#include "common/assert.h"
#include "common/common.h"
#include "common/platform/sysFileIO.h"
@@ -61,7 +62,8 @@ SysFileTimeStruct DateTimeToFileTimeUtc(const DateTime& date_time) {
} // namespace
struct File::FilePrivate {
sys_file_t* f;
sys_file_t* f;
std::unique_ptr<ArchiveFile> archive;
};
File::File(): m_p(std::make_unique<FilePrivate>()) {
@@ -85,11 +87,15 @@ File::File(const std::filesystem::path& name, Mode mode): m_p(std::make_unique<F
}
bool File::IsInvalid() const {
return m_p->f == nullptr;
return m_p->f == nullptr && m_p->archive == nullptr;
}
bool File::Create(const std::filesystem::path& name) {
EXIT_IF(m_p->f != nullptr);
EXIT_IF(!IsInvalid());
if (IsArchivePath(name)) {
return false;
}
m_file_name = name;
@@ -104,9 +110,16 @@ bool File::Create(const std::filesystem::path& name) {
}
bool File::Open(const std::filesystem::path& name, Mode mode) {
EXIT_IF(m_p->f != nullptr);
EXIT_IF(!IsInvalid());
m_file_name = name;
if (IsArchivePath(name)) {
if (mode != Mode::Read) {
return false;
}
m_p->archive = OpenArchiveFile(name);
return m_p->archive != nullptr;
}
switch (mode) {
case Mode::Read: m_p->f = SysFileOpenR(name); break;
@@ -124,7 +137,7 @@ bool File::Open(const std::filesystem::path& name, Mode mode) {
}
bool File::OpenInMem(void* buf, uint32_t buf_size) {
EXIT_IF(m_p->f != nullptr);
EXIT_IF(!IsInvalid());
m_p->f = SysFileOpen(static_cast<uint8_t*>(buf), buf_size);
@@ -137,7 +150,7 @@ bool File::OpenInMem(void* buf, uint32_t buf_size) {
}
bool File::CreateInMem() {
EXIT_IF(m_p->f != nullptr);
EXIT_IF(!IsInvalid());
m_p->f = SysFileCreate();
@@ -150,6 +163,7 @@ bool File::CreateInMem() {
}
void File::Close() {
m_p->archive.reset();
if (m_p->f != nullptr) {
SysFileClose(m_p->f);
m_p->f = nullptr;
@@ -157,7 +171,11 @@ void File::Close() {
}
uint64_t File::Size() const {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return m_p->archive->Size();
}
if (m_p->f == nullptr) {
return 0;
@@ -167,41 +185,76 @@ uint64_t File::Size() const {
}
uint64_t File::Remaining() const {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
return Size() - Tell();
const auto size = Size();
const auto position = Tell();
return position < size ? size - position : 0;
}
uint64_t File::Size(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
const auto info = GetArchiveInfo(name);
return info ? info->size : 0;
}
return SysFileSize(name);
}
std::optional<File::Info> File::GetInfo(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
return GetArchiveInfo(name);
}
Info info {};
return SysFileGetInfo(name, &info.is_file, &info.size) ? std::optional(info) : std::nullopt;
}
bool File::Seek(uint64_t offset) {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return m_p->archive->Seek(offset);
}
return SysFileSeek(*m_p->f, offset);
}
bool File::Truncate(uint64_t size) {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return false;
}
return SysFileTruncate(*m_p->f, size);
}
bool File::Unlink() {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return false;
}
return SysFileUnlink(*m_p->f, m_file_name);
}
uint64_t File::Tell() const {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return m_p->archive->Tell();
}
return SysFileTell(*m_p->f);
}
void File::Read(void* data, uint32_t size, uint32_t* bytes_read) {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
m_p->archive->Read(data, size, bytes_read);
return;
}
if (m_p->f != nullptr) {
SysFileRead(data, size, *m_p->f, bytes_read);
@@ -209,7 +262,14 @@ void File::Read(void* data, uint32_t size, uint32_t* bytes_read) {
}
void File::Write(const void* data, uint32_t size, uint32_t* bytes_written) {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
if (bytes_written != nullptr) {
*bytes_written = 0;
}
return;
}
SysFileWrite(data, size, *m_p->f, bytes_written);
}
@@ -257,24 +317,41 @@ static std::filesystem::path WithoutTrailingSeparator(
bool File::IsDirectoryExisting(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
const auto info = GetArchiveInfo(path);
return info && !info->is_file;
}
return SysFileIsDirectoryExisting(WithoutTrailingSeparator(path));
}
bool File::IsFileExisting(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
const auto info = GetArchiveInfo(name);
return info && info->is_file;
}
return SysFileIsFileExisting(name);
}
bool File::CreateDirectory(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
return false;
}
return SysFileCreateDirectory(
WithoutTrailingSeparator(path));
}
bool File::DeleteDirectory(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
return false;
}
return SysFileDeleteDirectory(
WithoutTrailingSeparator(path));
}
bool File::CreateDirectories(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
return false;
}
const auto normalized_path =
WithoutTrailingSeparator(path);
@@ -297,6 +374,9 @@ bool File::CreateDirectories(const std::filesystem::path& path) {
}
bool File::DeleteDirectories(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
return false;
}
const auto normalized_path =
WithoutTrailingSeparator(path);
@@ -322,11 +402,18 @@ bool File::DeleteDirectories(const std::filesystem::path& path) {
bool File::DeleteFile(
const std::filesystem::path& name) // @suppress("Member declaration not found")
{
if (IsArchivePath(name)) {
return false;
}
return SysFileDeleteFile(name);
}
bool File::Flush() {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return true;
}
return SysFileFlush(*m_p->f);
}
@@ -348,6 +435,9 @@ std::vector<std::byte> File::ReadWholeBuffer() {
}
DateTime File::GetLastAccessTimeUTC(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
return GetLastAccessTimeUTC(GetArchiveHostPath(name));
}
SysTimeStruct t {};
SysFileToSystemTimeUtc(SysFileGetLastAccessTimeUtc(name), t);
@@ -359,6 +449,9 @@ DateTime File::GetLastAccessTimeUTC(const std::filesystem::path& name) {
}
DateTime File::GetLastWriteTimeUTC(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
return GetLastWriteTimeUTC(GetArchiveHostPath(name));
}
SysTimeStruct t {};
SysFileToSystemTimeUtc(SysFileGetLastWriteTimeUtc(name), t);
@@ -373,6 +466,10 @@ void File::GetLastAccessAndWriteTimeUTC(const std::filesystem::path& name, DateT
DateTime* write) {
EXIT_IF(access == nullptr);
EXIT_IF(write == nullptr);
if (IsArchivePath(name)) {
GetLastAccessAndWriteTimeUTC(GetArchiveHostPath(name), access, write);
return;
}
SysTimeStruct at {};
SysTimeStruct wt {};
@@ -399,7 +496,12 @@ void File::GetLastAccessAndWriteTimeUTC(DateTime* access, DateTime* write) {
EXIT_IF(access == nullptr);
EXIT_IF(write == nullptr);
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
GetLastAccessAndWriteTimeUTC(m_file_name, access, write);
return;
}
if (m_p->f == nullptr) {
*access = DateTime();
@@ -429,23 +531,35 @@ void File::GetLastAccessAndWriteTimeUTC(DateTime* access, DateTime* write) {
}
bool File::SetLastAccessTimeUTC(const std::filesystem::path& name, const DateTime& dt) {
if (IsArchivePath(name)) {
return false;
}
auto f = DateTimeToFileTimeUtc(dt);
return SysFileSetLastAccessTimeUtc(name, f);
}
bool File::SetLastWriteTimeUTC(const std::filesystem::path& name, const DateTime& dt) {
if (IsArchivePath(name)) {
return false;
}
auto f = DateTimeToFileTimeUtc(dt);
return SysFileSetLastWriteTimeUtc(name, f);
}
bool File::SetLastAccessAndWriteTimeUTC(const std::filesystem::path& name, const DateTime& access,
const DateTime& write) {
if (IsArchivePath(name)) {
return false;
}
auto af = DateTimeToFileTimeUtc(access);
auto wf = DateTimeToFileTimeUtc(write);
return SysFileSetLastAccessAndWriteTimeUtc(name, af, wf);
}
std::vector<File::DirEntry> File::GetDirEntries(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
return GetArchiveDirEntries(path);
}
std::vector<sys_dir_entry_t> files;
SysFileGetDents(path, files);
@@ -463,12 +577,18 @@ std::vector<File::DirEntry> File::GetDirEntries(const std::filesystem::path& pat
bool File::CopyFile(const std::filesystem::path& src,
const std::filesystem::path& dst) // @suppress("Member declaration not found")
{
if (IsArchivePath(src) || IsArchivePath(dst)) {
return false;
}
return SysFileCopyFile(src, dst);
}
bool File::RenameFile(const std::filesystem::path& src,
const std::filesystem::path& dst) // @suppress("Member declaration not found")
{
if (IsArchivePath(src) || IsArchivePath(dst)) {
return false;
}
if (IsFileExisting(dst)) {
DeleteFile(dst); // @suppress("Invalid arguments")
}
@@ -477,6 +597,9 @@ bool File::RenameFile(const std::filesystem::path& src,
}
void File::RemoveReadonly(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
return;
}
SysFileRemoveReadonly(name);
}
+6
View File
@@ -7,6 +7,7 @@
#include <cstddef>
#include <filesystem>
#include <memory>
#include <optional>
#include <string>
#include <vector>
@@ -34,6 +35,10 @@ public:
std::string name;
bool is_file;
};
struct Info {
bool is_file;
uint64_t size;
};
File();
explicit File(const std::filesystem::path& name);
@@ -72,6 +77,7 @@ public:
std::vector<std::byte> ReadWholeBuffer();
static uint64_t Size(const std::filesystem::path& name);
static std::optional<Info> GetInfo(const std::filesystem::path& name);
static bool IsDirectoryExisting(const std::filesystem::path& path);
static bool IsFileExisting(const std::filesystem::path& name);
+1
View File
@@ -45,6 +45,7 @@ bool SysFileTruncate(sys_file_t& f, uint64_t size); // NOLINT(google-runt
bool SysFileUnlink(sys_file_t& f,
const std::filesystem::path& name); // NOLINT(google-runtime-references)
uint64_t SysFileSize(const std::filesystem::path& file_name);
bool SysFileGetInfo(const std::filesystem::path& name, bool* is_file, uint64_t* size);
bool SysFileIsError(sys_file_t& f); // NOLINT(google-runtime-references)
bool SysFileIsDirectoryExisting(const std::filesystem::path& path);
bool SysFileIsFileExisting(const std::filesystem::path& name);
+21 -24
View File
@@ -54,7 +54,7 @@ struct sys_file_t {
#endif
static std::filesystem::path get_internal_name(const std::filesystem::path& name) {
return name.is_absolute() ? name : (std::filesystem::path(".") / name);
return name.empty() || name.is_absolute() ? name : (std::filesystem::path(".") / name);
}
// Pass access-pattern hints to the host.
@@ -297,10 +297,21 @@ uint64_t SysFileSize(sys_file_t& f) {
return 0;
}
bool SysFileGetInfo(const std::filesystem::path& name, bool* is_file, uint64_t* size) {
const auto path = get_internal_name(name);
struct stat info {};
if (stat(path.c_str(), &info) != 0) {
return false;
}
*is_file = !S_ISDIR(info.st_mode);
*size = *is_file ? static_cast<uint64_t>(info.st_size) : 0;
return true;
}
uint64_t SysFileSize(const std::filesystem::path& file_name) {
sys_file_t* f = SysFileOpenR(file_name);
uint64_t size = SysFileSize(*f);
SysFileClose(f);
bool is_file;
uint64_t size = 0;
SysFileGetInfo(file_name, &is_file, &size);
return size;
}
@@ -353,29 +364,15 @@ bool SysFileIsError(sys_file_t& f) {
}
bool SysFileIsDirectoryExisting(const std::filesystem::path& path) {
auto real_name = get_internal_name(path);
auto real_name_str = real_name.string();
struct stat s {};
if (0 != stat(real_name_str.c_str(), &s)) {
return false;
}
return S_ISDIR(s.st_mode); // NOLINT
bool is_file;
uint64_t size;
return SysFileGetInfo(path, &is_file, &size) && !is_file;
}
bool SysFileIsFileExisting(const std::filesystem::path& name) {
auto real_name = get_internal_name(name);
auto real_name_str = real_name.string();
struct stat s {};
if (0 != stat(real_name_str.c_str(), &s)) {
return false;
}
return !S_ISDIR(s.st_mode); // NOLINT
bool is_file;
uint64_t size;
return SysFileGetInfo(name, &is_file, &size) && is_file;
}
bool SysFileCreateDirectory(const std::filesystem::path& path) {
+25 -19
View File
@@ -266,19 +266,27 @@ uint64_t SysFileSize(sys_file_t& f) {
return 0;
}
uint64_t SysFileSize(const std::filesystem::path& file_name) {
LARGE_INTEGER s;
WIN32_FILE_ATTRIBUTE_DATA a;
auto wide = file_name.wstring();
if (GetFileAttributesExW(wide.c_str(), GetFileExInfoStandard, &a) == 0) {
return 0;
bool SysFileGetInfo(const std::filesystem::path& name, bool* is_file, uint64_t* size) {
const bool directory_path = !name.empty() && !name.has_filename() && name != name.root_path();
WIN32_FILE_ATTRIBUTE_DATA info {};
if (GetFileAttributesExW(directory_path ? name.parent_path().c_str() : name.c_str(),
GetFileExInfoStandard, &info) == 0) {
return false;
}
const bool file = (info.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0;
if (directory_path && file) {
return false;
}
*is_file = file;
*size = *is_file ? ((static_cast<uint64_t>(info.nFileSizeHigh) << 32u) | info.nFileSizeLow) : 0;
return true;
}
s.HighPart = static_cast<LONG>(a.nFileSizeHigh);
s.LowPart = a.nFileSizeLow;
return s.QuadPart;
uint64_t SysFileSize(const std::filesystem::path& file_name) {
bool is_file;
uint64_t size = 0;
SysFileGetInfo(file_name, &is_file, &size);
return size;
}
bool SysFileTruncate(sys_file_t& f, uint64_t size) {
@@ -346,17 +354,15 @@ bool SysFileIsError(sys_file_t& f) {
}
bool SysFileIsDirectoryExisting(const std::filesystem::path& path) {
auto wide = path.wstring();
DWORD a = GetFileAttributesW(wide.c_str());
return a != INVALID_FILE_ATTRIBUTES &&
((a & static_cast<DWORD>(FILE_ATTRIBUTE_DIRECTORY)) != 0u);
bool is_file;
uint64_t size;
return SysFileGetInfo(path, &is_file, &size) && !is_file;
}
bool SysFileIsFileExisting(const std::filesystem::path& name) {
auto wide = name.wstring();
DWORD a = GetFileAttributesW(wide.c_str());
return a != INVALID_FILE_ATTRIBUTES &&
((a & static_cast<DWORD>(FILE_ATTRIBUTE_DIRECTORY)) == 0u);
bool is_file;
uint64_t size;
return SysFileGetInfo(name, &is_file, &size) && is_file;
}
bool SysFileCreateDirectory(const std::filesystem::path& path) {
-39
View File
@@ -2,52 +2,13 @@
#include "common/emulatorConfig.h"
#include <algorithm>
#include <common/TracyProtocol.hpp>
#include <common/TracyVersion.hpp>
#include <cstdio>
#include <tracy/Tracy.hpp>
#include <vector>
namespace {
thread_local std::vector<Profiler::ScopedBlock*> g_block_stack;
void RemoveBlock(Profiler::ScopedBlock* block) {
auto block_it = std::find(g_block_stack.rbegin(), g_block_stack.rend(), block);
if (block_it != g_block_stack.rend()) {
g_block_stack.erase(std::next(block_it).base());
}
}
} // namespace
namespace Profiler {
ScopedBlock::ScopedBlock(const tracy::SourceLocationData* source_location) {
if (tracy::ProfilerAvailable()) {
m_zone.emplace(source_location, TRACY_CALLSTACK, true);
g_block_stack.push_back(this);
}
}
ScopedBlock::~ScopedBlock() {
End();
}
void ScopedBlock::End() {
if (m_zone.has_value()) {
m_zone.reset();
RemoveBlock(this);
}
}
void EndBlock() {
if (!g_block_stack.empty()) {
g_block_stack.back()->End();
}
}
void SetThreadName(const char* name) {
if (tracy::ProfilerAvailable() && name != nullptr) {
tracy::SetThreadName(name);
+4 -29
View File
@@ -4,7 +4,6 @@
#include "common/common.h"
#include <cstdint>
#include <optional>
#include <tracy/Tracy.hpp> // IWYU pragma: export
namespace profiler::colors {
@@ -23,22 +22,6 @@ inline constexpr uint32_t DeepOrangeA200 = 0xff6e40;
namespace Profiler {
class ScopedBlock {
public:
explicit ScopedBlock(const tracy::SourceLocationData* source_location);
ScopedBlock(const ScopedBlock&) = delete;
ScopedBlock& operator=(const ScopedBlock&) = delete;
ScopedBlock(ScopedBlock&&) = delete;
ScopedBlock& operator=(ScopedBlock&&) = delete;
~ScopedBlock();
void End();
private:
std::optional<tracy::ScopedZone> m_zone;
};
void EndBlock();
void SetThreadName(const char* name);
void Initialize();
@@ -66,19 +49,11 @@ struct Lifecycle {
kyty_profiler_source_location_, line) {name, TracyFunction, TracyFile, \
static_cast<uint32_t>(line), \
KYTY_PROFILER_COLOR_OR_DEFAULT(0, __VA_ARGS__)}; \
Profiler::ScopedBlock KYTY_PROFILER_CONCAT(kyty_profiler_block_, line)( \
&KYTY_PROFILER_CONCAT(kyty_profiler_source_location_, line))
tracy::ScopedZone KYTY_PROFILER_CONCAT(kyty_profiler_block_, line)( \
&KYTY_PROFILER_CONCAT(kyty_profiler_source_location_, line), \
TRACY_CALLSTACK, tracy::ProfilerAvailable())
#define KYTY_PROFILER_FUNCTION(...) KYTY_PROFILER_FUNCTION_IMPL(__LINE__ __VA_OPT__(, ) __VA_ARGS__)
#define KYTY_PROFILER_FUNCTION_IMPL(line, ...) \
static constexpr tracy::SourceLocationData KYTY_PROFILER_CONCAT( \
kyty_profiler_source_location_, line) {nullptr, TracyFunction, TracyFile, \
static_cast<uint32_t>(line), \
KYTY_PROFILER_COLOR_OR_DEFAULT(0, __VA_ARGS__)}; \
Profiler::ScopedBlock KYTY_PROFILER_CONCAT(kyty_profiler_block_, line)( \
&KYTY_PROFILER_CONCAT(kyty_profiler_source_location_, line))
#define KYTY_PROFILER_END_BLOCK Profiler::EndBlock()
#define KYTY_PROFILER_FUNCTION(...) KYTY_PROFILER_BLOCK(nullptr __VA_OPT__(, ) __VA_ARGS__)
#define KYTY_PROFILER_THREAD(name) Profiler::SetThreadName(name)
+2 -40
View File
@@ -2,7 +2,6 @@
#include "common/assert.h"
#include <algorithm>
#include <atomic>
#include <cerrno>
#include <chrono> // IWYU pragma: keep
@@ -26,8 +25,7 @@
#ifdef KYTY_WIN_CS
#include <windows.h> // IWYU pragma: keep
// IWYU pragma: no_include <winbase.h>
constexpr DWORD KYTY_CS_SPIN_COUNT = 4000;
constexpr uint64_t KYTY_SLEEP_SPIN_LIMIT_100NS = 500; // 50 us
constexpr DWORD KYTY_CS_SPIN_COUNT = 4000;
#ifndef CREATE_WAITABLE_TIMER_HIGH_RESOLUTION
#define CREATE_WAITABLE_TIMER_HIGH_RESOLUTION 0x00000002
@@ -38,30 +36,6 @@ static void SleepHighResolution100ns(uint64_t units_100ns) {
return;
}
// Keep spinning only where a kernel transition is
// likely to cost more than the requested delay; ordinary millisecond sleeps use the
// per-thread high-resolution waitable timer below.
if (units_100ns <= KYTY_SLEEP_SPIN_LIMIT_100NS) {
LARGE_INTEGER frequency {};
LARGE_INTEGER start {};
if (QueryPerformanceFrequency(&frequency) != 0 && QueryPerformanceCounter(&start) != 0 &&
frequency.QuadPart > 0) {
const auto wait_ticks =
static_cast<LONGLONG>((static_cast<long double>(units_100ns) *
static_cast<long double>(frequency.QuadPart)) /
10000000.0L);
const auto deadline = start.QuadPart + std::max<LONGLONG>(wait_ticks, 1);
LARGE_INTEGER now {};
do {
if (QueryPerformanceCounter(&now) == 0) {
break;
}
YieldProcessor();
} while (now.QuadPart < deadline);
return;
}
}
thread_local HANDLE timer = CreateWaitableTimerExW(
nullptr, nullptr, CREATE_WAITABLE_TIMER_HIGH_RESOLUTION, TIMER_ALL_ACCESS);
if (timer == nullptr) {
@@ -131,14 +105,13 @@ static SleepConditionVariableCS_func_t ResolveSleepConditionVariableCS() {
#endif
#ifdef KYTY_POSIX_HIGH_RES_SLEEP
// Spin for very short waits; use an absolute deadline for longer waits.
// An absolute deadline preserves the requested sleep across signal interruptions.
static void SleepHighResolutionNanos(uint64_t nanos) {
if (nanos == 0) {
return;
}
constexpr uint64_t NANOS_PER_SEC = 1000000000;
constexpr uint64_t SPIN_LIMIT_NS = 50000; // below this a context switch dominates
timespec deadline {};
if (clock_gettime(CLOCK_MONOTONIC, &deadline) != 0) {
@@ -150,17 +123,6 @@ static void SleepHighResolutionNanos(uint64_t nanos) {
deadline.tv_sec += static_cast<time_t>(target_nsec / NANOS_PER_SEC);
deadline.tv_nsec = static_cast<long>(target_nsec % NANOS_PER_SEC);
if (nanos <= SPIN_LIMIT_NS) {
timespec now {};
do {
if (clock_gettime(CLOCK_MONOTONIC, &now) != 0) {
return;
}
} while (now.tv_sec < deadline.tv_sec ||
(now.tv_sec == deadline.tv_sec && now.tv_nsec < deadline.tv_nsec));
return;
}
while (clock_nanosleep(CLOCK_MONOTONIC, TIMER_ABSTIME, &deadline, nullptr) == EINTR) {
}
}
@@ -2266,7 +2266,6 @@ KYTY_CP_OP_PARSER(CpOpReleaseMem) {
const bool gl2_writeback = ((gcr_cntl & GcrGl2Writeback) != 0);
auto trigger_interrupt = [&]() {
bool queued = false;
switch (interrupt_selector) {
case 0x00:
case 0x03: break;
@@ -2274,13 +2273,10 @@ KYTY_CP_OP_PARSER(CpOpReleaseMem) {
case 0x02:
case 0x04:
cp.TriggerEopEventAtEndOfPipe(interrupt_context_id);
queued = true;
cp.BufferFlush();
break;
default: EXIT("unknown release_mem interrupt selector\n");
}
if (queued) {
cp.BufferFlush();
}
};
if (data_sel == 0 || interrupt_selector == 4) {
@@ -2318,7 +2314,9 @@ KYTY_CP_OP_PARSER(CpOpReleaseMem) {
cp.WriteAtEndOfPipe32(cache_policy, event_write_dest, eop_event_type, cache_action,
event_index, event_source, dst_gpu_addr, static_cast<uint32_t>(value),
interrupt_selector, interrupt_context_id);
cp.BufferFlush();
if (interrupt_selector == 0x01 || interrupt_selector == 0x02) {
cp.BufferFlush();
}
return 7;
}
+8 -3
View File
@@ -3,6 +3,7 @@
#include "common/bitArray.h"
#include "common/common.h"
#include "kernel/memory.h"
#include <compare>
@@ -10,7 +11,8 @@ namespace Libs::Graphics {
constexpr uint64_t TRACKER_PAGE_SIZE = 4ull * 1024ull;
constexpr uint64_t TRACKER_REGION_SIZE = 4ull * 1024ull * 1024ull;
constexpr uint64_t TRACKER_ADDRESS_SIZE = 1ull << 40u;
constexpr uint64_t TRACKER_ADDRESS_SIZE = 1ull << 44u;
constexpr uint64_t LOWER_ADDRESS_SIZE = 1ull << 40u;
constexpr size_t TRACKER_REGION_PAGES = TRACKER_REGION_SIZE / TRACKER_PAGE_SIZE;
struct GuestRange {
@@ -19,8 +21,11 @@ struct GuestRange {
[[nodiscard]] constexpr bool Empty() const noexcept { return address == 0 && size == 0; }
[[nodiscard]] constexpr bool Valid() const noexcept {
return address != 0 && size != 0 && address < TRACKER_ADDRESS_SIZE &&
size <= TRACKER_ADDRESS_SIZE - address;
if (address == 0 || size == 0) return false;
if (address < LOWER_ADDRESS_SIZE) return size <= LOWER_ADDRESS_SIZE - address;
constexpr auto base = LibKernel::Memory::kExtendedMemoryBase;
constexpr auto length = LibKernel::Memory::kExtendedMemorySize;
return address >= base && address - base < length && size <= length - (address - base);
}
[[nodiscard]] constexpr bool ValidOrEmpty() const noexcept { return Empty() || Valid(); }
[[nodiscard]] constexpr uint64_t End() const noexcept { return address + size; }
+13 -6
View File
@@ -55,7 +55,8 @@ template <bool insert>
void BufferCache::ChangeRegister(BufferId id) {
auto& buffer = m_slot_buffers[id];
PageTable::PageRange pages {};
EXIT_IF(!PageTable::TryGetPageRange(buffer.CpuAddress(), buffer.Size(), pages));
EXIT_IF(!(GuestRange {buffer.CpuAddress(), buffer.Size()}.Valid()) ||
!PageTable::TryGetPageRange(buffer.CpuAddress(), buffer.Size(), pages));
for (size_t page = pages.first; page < pages.last_exclusive; ++page) {
if constexpr (insert) {
m_page_table[page] = id;
@@ -64,6 +65,7 @@ void BufferCache::ChangeRegister(BufferId id) {
}
}
const auto size_pages = pages.last_exclusive - pages.first;
const auto table_offset = PageIndex(buffer.CpuAddress()) * sizeof(vk::DeviceAddress);
if constexpr (insert) {
const auto [it, inserted] = m_buffers.emplace(buffer.CpuAddress(), id);
(void)it;
@@ -75,7 +77,7 @@ void BufferCache::ChangeRegister(BufferId id) {
for (uint64_t i = 0; i < size_pages; ++i) {
addresses.push_back(buffer.BufferDeviceAddress() + (i << CACHING_PAGEBITS));
}
WriteDataBuffer(m_bda_pagetable_buffer, pages.first * sizeof(vk::DeviceAddress),
WriteDataBuffer(m_bda_pagetable_buffer, table_offset,
addresses.data(), addresses.size() * sizeof(vk::DeviceAddress));
} else {
const auto found = m_buffers.find(buffer.CpuAddress());
@@ -84,7 +86,7 @@ void BufferCache::ChangeRegister(BufferId id) {
EXIT_IF(buffer.Size() > m_total_used_memory);
m_total_used_memory -= buffer.Size();
m_lru_cache.Free(buffer.lru_id);
m_bda_pagetable_buffer.Fill(pages.first * sizeof(vk::DeviceAddress),
m_bda_pagetable_buffer.Fill(table_offset,
size_pages * sizeof(vk::DeviceAddress), 0);
buffer.is_deleted = true;
}
@@ -312,10 +314,14 @@ BufferCache::OverlapResult BufferCache::ResolveOverlaps(uint64_t vaddr, uint64_t
// Reserve space in the incoming stream's direction of growth.
// The old buffer extending left of the request predicts growth to the right, and vice versa.
if (expands_left) {
end += std::min(StreamLeapSize, PageTable::kAddressSpaceSize - end);
end += std::min(StreamLeapSize, (vaddr < LOWER_ADDRESS_SIZE ? LOWER_ADDRESS_SIZE
: LibKernel::Memory::kExtendedMemoryBase +
LibKernel::Memory::kExtendedMemorySize) - end);
}
if (expands_right) {
const auto minimum = CACHING_PAGESIZE * 2;
const auto minimum = vaddr < LOWER_ADDRESS_SIZE
? CACHING_PAGESIZE * 2
: LibKernel::Memory::kExtendedMemoryBase;
if (begin > minimum) {
begin -= std::min(StreamLeapSize, begin - minimum);
}
@@ -448,7 +454,8 @@ std::pair<Buffer*, uint64_t> BufferCache::ObtainBuffer(uint64_t vaddr, uint64_t
const auto alignment = std::max<uint64_t>(
m_graphics.physical_device_properties.limits.minUniformBufferOffsetAlignment, 1);
auto [mapped, offset] = m_stream_buffer.Map(size, alignment, false);
if (mapped != nullptr && Libs::LibKernel::Memory::TryReadBacking(vaddr, mapped, size)) {
if (mapped != nullptr) {
std::memcpy(mapped, reinterpret_cast<const void*>(vaddr), size);
m_stream_buffer.Commit();
return {&m_stream_buffer, offset};
}
+14 -2
View File
@@ -29,10 +29,22 @@ class BufferCache {
public:
static constexpr uint32_t CACHING_PAGEBITS = 14;
static constexpr uint64_t CACHING_PAGESIZE = uint64_t {1} << CACHING_PAGEBITS;
static constexpr uint64_t CACHING_NUMPAGES = uint64_t {1} << (40 - CACHING_PAGEBITS);
static constexpr uint64_t CACHING_NUMPAGES = (LOWER_ADDRESS_SIZE + LibKernel::Memory::kExtendedMemorySize) >> CACHING_PAGEBITS;
static constexpr uint64_t BDA_PAGETABLE_SIZE =
CACHING_NUMPAGES * sizeof(vk::DeviceAddress);
static constexpr uint64_t PageIndex(uint64_t address) {
return (address < LOWER_ADDRESS_SIZE
? address
: address - LibKernel::Memory::kExtendedMemoryBase + LOWER_ADDRESS_SIZE) >>
CACHING_PAGEBITS;
}
static constexpr uint64_t GuestAddress(uint64_t offset) {
return offset < LOWER_ADDRESS_SIZE
? offset
: offset - LOWER_ADDRESS_SIZE + LibKernel::Memory::kExtendedMemoryBase;
}
BufferCache(GraphicContext& graphics, CommandScheduler& scheduler, PageManager& page_manager,
TextureCache& texture_cache);
~BufferCache();
@@ -88,7 +100,7 @@ private:
bool has_stream_leap;
};
using PageTable = MultiLevelPageTable<BufferId, CACHING_PAGEBITS, 40, 16>;
using PageTable = MultiLevelPageTable<BufferId, CACHING_PAGEBITS, 44, 20>;
static_assert(CACHING_PAGESIZE == (uint64_t {1} << PageTable::kPageBits));
void WriteDataBuffer(Buffer& buffer, uint64_t address, const void* source, uint64_t size);
void TouchBuffer(const Buffer& buffer);
+3 -2
View File
@@ -134,8 +134,9 @@ void FaultManager::ProcessFaultBuffer() {
const auto* faults = std::bit_cast<const uint64_t*>(mapped);
const auto count = static_cast<uint32_t>(faults[0]);
for (uint32_t index = 1; index <= count; ++index) {
fault_ranges.Add(faults[index], BufferCache::CACHING_PAGESIZE);
LOGF("Accessed non-GPU cached memory at 0x%016" PRIx64 "\n", faults[index]);
const auto address = BufferCache::GuestAddress(faults[index]);
fault_ranges.Add(address, BufferCache::CACHING_PAGESIZE);
LOGF("Accessed non-GPU cached memory at 0x%016" PRIx64 "\n", address);
}
fault_ranges.ForEach([this](uint64_t start, uint64_t end) {
EXIT_IF(end - start > std::numeric_limits<uint32_t>::max());
+9 -9
View File
@@ -14,10 +14,10 @@ SamplerCache::~SamplerCache() {
}
}
vk::Sampler SamplerCache::GetSampler(const ShaderSamplerResource& r) {
vk::Sampler SamplerCache::GetSampler(const ShaderSamplerResource& r, bool integer_border) {
Common::LockGuard lock(m_mutex);
const SamplerKey key {r.fields[0], r.fields[1], r.fields[2], r.fields[3]};
const SamplerKey key {r.fields[0], r.fields[1], r.fields[2], r.fields[3], integer_border};
if (auto iter = m_samplers.find(key); iter != m_samplers.end()) {
return iter->second;
}
@@ -93,23 +93,23 @@ vk::Sampler SamplerCache::GetSampler(const ShaderSamplerResource& r) {
return vk::SamplerAddressMode::eClampToBorder;
};
vk::BorderColor border = vk::BorderColor::eIntTransparentBlack;
vk::BorderColor border = integer_border ? vk::BorderColor::eIntTransparentBlack
: vk::BorderColor::eFloatTransparentBlack;
switch (static_cast<Prospero::SamplerBorderColor>(r.BorderColorType())) {
case Prospero::SamplerBorderColor::kTransBlack:
border = vk::BorderColor::eIntTransparentBlack;
break;
case Prospero::SamplerBorderColor::kTransBlack: break;
case Prospero::SamplerBorderColor::kOpaqueBlack:
border = vk::BorderColor::eIntOpaqueBlack;
border = integer_border ? vk::BorderColor::eIntOpaqueBlack
: vk::BorderColor::eFloatOpaqueBlack;
break;
case Prospero::SamplerBorderColor::kOpaqueWhite:
border = vk::BorderColor::eIntOpaqueWhite;
border = integer_border ? vk::BorderColor::eIntOpaqueWhite
: vk::BorderColor::eFloatOpaqueWhite;
break;
case Prospero::SamplerBorderColor::kFromTable:
LOGF(
"temporary: approximating table border color as transparent black, index = %" PRIu16
"\n",
r.BorderColorPtr());
border = vk::BorderColor::eIntTransparentBlack;
break;
default: EXIT("unknown border color: %d", static_cast<int>(r.BorderColorType()));
}
+2 -2
View File
@@ -24,10 +24,10 @@ public:
~SamplerCache();
KYTY_CLASS_NO_COPY(SamplerCache);
vk::Sampler GetSampler(const ShaderSamplerResource& r);
vk::Sampler GetSampler(const ShaderSamplerResource& r, bool integer_border);
private:
using SamplerKey = std::array<uint32_t, 4>;
using SamplerKey = std::array<uint32_t, 5>;
struct SamplerKeyHash {
std::size_t operator()(const SamplerKey& key) const {
+8 -13
View File
@@ -222,14 +222,6 @@ TextureCache::BindingType TextureCache::UploadBinding(const Image& image) {
return image.usage.storage ? BindingType::Storage : BindingType::Texture;
}
bool TextureCache::SafeToDownload(const Image& image) {
if (!image.SafeToDownload()) {
return false;
}
const auto range = image.info.data;
return !m_buffer_cache.HasGpuDirtyBytes(range.address, range.size);
}
ImageId TextureCache::InsertImage(const ImageInfo& info) {
const auto id = m_slot_images.insert(m_graphics, m_scheduler, info);
if (!info.data.Empty()) {
@@ -1360,7 +1352,7 @@ ImageId TextureCache::FindImageFromRange(uint64_t address, uint64_t size, bool e
if (ensure_valid && owner->depth_id) {
owner = m_slot_images.try_get(owner->depth_id);
}
if (owner == nullptr || (ensure_valid && !SafeToDownload(*owner))) {
if (owner == nullptr || (ensure_valid && !owner->SafeToDownload())) {
continue;
}
matches.push_back(id);
@@ -1751,7 +1743,7 @@ bool BufferCache::SynchronizeBufferFromImage(Buffer& buffer, uint64_t vaddr, uin
std::scoped_lock lock {m_texture_cache.m_lock};
auto& image = m_texture_cache.m_slot_images[selected];
// The GPU thread owns image retirement; CPU invalidation can dirty this image after lookup.
if (!m_texture_cache.SafeToDownload(image)) {
if (!image.SafeToDownload()) {
return false;
}
if (!buffer.IsInBounds(image.info.data.address, 1)) {
@@ -1814,7 +1806,7 @@ bool TextureCache::DownloadImageMemory(ImageId id) {
return false;
}
auto transfer = BuildDownload(image);
if (!transfer.valid || !SafeToDownload(image)) {
if (!transfer.valid || !image.SafeToDownload()) {
return false;
}
const auto range = image.info.data;
@@ -1875,7 +1867,10 @@ bool TextureCache::IsRegionGpuModified(uint64_t address, uint64_t size) {
std::scoped_lock lock {m_lock};
for (const auto id: FindImagesInRegion(address, size, false)) {
const auto& image = m_slot_images[id];
if (!image.depth_id && image.IsGpuModified()) {
// PPSA17168: S_LOAD_DWORD reads shader data at an address overlapping an old
// render target whose memory the CPU has reused. The cached image still retains
// its earlier GPU-modified flag.
if (!image.depth_id && image.IsGpuModified() && !image.IsDefinitelyCpuDirty()) {
return true;
}
}
@@ -2001,7 +1996,7 @@ void TextureCache::RunGarbageCollector() {
continue;
}
if (owner->IsGpuModified()) {
const bool safe = SafeToDownload(*owner);
const bool safe = owner->SafeToDownload();
if (safe && owner->info.IsTiled()) {
continue;
}
+2 -3
View File
@@ -91,9 +91,9 @@ private:
};
using ImageIds = InlinePageOwnerList<ImageId, 16>;
using ImagePageTable = MultiLevelPageTable<ImageIds, 20, 40, 10>;
using ImagePageTable = MultiLevelPageTable<ImageIds, 20, 44, 14>;
// Callers have validated the nonempty 40-bit range with TryGetPageRange.
// Callers have validated the nonempty 44-bit range with TryGetPageRange.
template <typename Func>
static void ForEachPage(uint64_t address, size_t size, Func&& func) {
using FuncReturn = typename std::invoke_result<Func, uint64_t>::type;
@@ -128,7 +128,6 @@ private:
[[nodiscard]] static bool SameBacking(const ImageInfo& cached, const ImageInfo& requested,
bool exact_format);
[[nodiscard]] static BindingType UploadBinding(const Image& image);
[[nodiscard]] bool SafeToDownload(const Image& image);
// Caller holds m_lock; it also serializes the per-image query epoch.
[[nodiscard]] ImageIds FindImagesInRegion(uint64_t address, uint64_t size,
@@ -21,8 +21,6 @@
namespace Libs::Graphics {
static std::atomic<uint32_t> g_render_color_log_count = 0;
static bool DccAlphaOnMsb(const HW::ColorInfo& info) {
switch (info.format) {
case Prospero::ChannelLayout::k10_10_10_2:
@@ -262,21 +260,7 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
EXIT("render-target backing range is invalid\n");
}
const vk::Extent2D view_extent = {std::max(width >> rt.view.current_mip_level, 1u),
std::max(height >> rt.view.current_mip_level, 1u)};
auto decision_log_id = g_render_color_log_count.fetch_add(1);
if (decision_log_id < 128) {
LOGF("RenderColorTarget: slot=%" PRIu32 " addr=0x%010" PRIx64 " size=0x%016" PRIx64
" extent=%ux%ux%u view_mip=%u view_extent=%ux%u levels=%u pitch=%u"
" fmt=0x%08" PRIx32 " nfmt=0x%08" PRIx32 " order=0x%08" PRIx32 " samples=%u tile=%s\n",
rt_slot, rt.base.addr, backing_size, width, height, depth, rt.view.current_mip_level,
view_extent.width, view_extent.height, levels, pitch,
static_cast<uint32_t>(rt.info.format), static_cast<uint32_t>(rt.info.channel_type),
static_cast<uint32_t>(rt.info.channel_order), samples, tile ? "tiled" : "linear");
}
TextureCache::ImageDesc desc {};
auto& desc = r.desc;
desc.type = TextureCache::BindingType::RenderTarget;
desc.info.data = {rt.base.addr, backing_size};
desc.info.pixel_format = target_format.format;
@@ -348,7 +332,6 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
desc.view_info.layer_count = view.layer_count;
desc.view_info.usage = vk::ImageUsageFlagBits::eColorAttachment;
auto& texture_cache = m_context.GetTextureCache();
r.desc = std::move(desc);
r.guest_mip_level = rt.view.current_mip_level;
r.guest_array_layer = view.base_layer;
r.image_id = texture_cache.FindImage(r.desc, exact_format);
+25 -7
View File
@@ -1,6 +1,7 @@
#include "graphics/host_gpu/renderer/image/image.h"
#include "common/assert.h"
#include "common/logging/log.h"
#include "common/profiler.h"
#include "graphics/host_gpu/renderer/cache/streamBuffer.h"
#include "graphics/host_gpu/renderer/commandScheduler.h"
@@ -10,7 +11,9 @@
#include <algorithm>
#include <array>
#include <atomic>
#include <cstdint>
#include <fmt/format.h>
#include <xxhash.h>
namespace Libs::Graphics {
@@ -48,12 +51,29 @@ namespace {
}
[[nodiscard]] vk::ImageUsageFlags ImageUsageFlags(GraphicContext& graphics, const ImageInfo& info) {
auto usage = vk::ImageUsageFlagBits::eTransferSrc | vk::ImageUsageFlagBits::eTransferDst;
if (info.IsBlock()) {
return vk::ImageUsageFlagBits::eTransferSrc | vk::ImageUsageFlagBits::eTransferDst |
vk::ImageUsageFlagBits::eSampled;
usage |= vk::ImageUsageFlagBits::eSampled;
if (graphics.supports_block_texel_view) {
const auto storage = usage | vk::ImageUsageFlagBits::eStorage;
if (graphics.GetImageFormatProperties(info.pixel_format, HostImageType(info.type),
vk::ImageTiling::eOptimal, storage,
ImageCreateFlags(graphics, info),
nullptr) == vk::Result::eSuccess) {
usage = storage;
} else {
static std::atomic_flag warned = ATOMIC_FLAG_INIT;
if (!warned.test_and_set(std::memory_order_relaxed)) {
Log::WriteToConsoleAndLog(fmt::format(
"Warning: format {} does not support storage access; block-compressed "
"textures written by the guest will not render.\n",
vk::to_string(info.pixel_format)));
}
}
}
return usage;
}
const auto properties = graphics.GetFormatProperties(info.pixel_format);
auto usage = vk::ImageUsageFlagBits::eTransferSrc | vk::ImageUsageFlagBits::eTransferDst;
if (HasFormatFeature(properties, vk::FormatFeatureFlagBits::eSampledImage)) {
usage |= vk::ImageUsageFlagBits::eSampled;
}
@@ -628,10 +648,8 @@ void Validate(const ImageInfo& info) {
break;
case ImageMetadataKind::Dcc:
case ImageMetadataKind::Cmask:
if (info.metadata.range.address == 0 ||
info.metadata.range.address >= TRACKER_ADDRESS_SIZE ||
(info.metadata.range.size != 0 &&
info.metadata.range.size > TRACKER_ADDRESS_SIZE - info.metadata.range.address) ||
if (!GuestRange {info.metadata.range.address,
std::max<uint64_t>(info.metadata.range.size, 1)}.Valid() ||
info.metadata.compression == VideoOutCompression::Unsupported) {
EXIT("invalid color metadata\n");
}
@@ -0,0 +1,58 @@
#include "graphics/host_gpu/renderer/pipeline/blendMapping.h"
#include "graphics/guest_gpu/hardwareContext.h"
#include <initializer_list>
namespace Libs::Graphics {
bool BlendFactorIsDualSource(uint8_t factor) {
return factor >= static_cast<uint8_t>(Prospero::BlendFactor::kSrc1Color) &&
factor <= static_cast<uint8_t>(Prospero::BlendFactor::kOneMinusSrc1Alpha);
}
namespace {
bool BlendFactorIsConstantColor(uint8_t factor) {
return factor == static_cast<uint8_t>(Prospero::BlendFactor::kConstantColor) ||
factor == static_cast<uint8_t>(Prospero::BlendFactor::kOneMinusConstantColor);
}
} // namespace
BlendMappingSupport ClassifyBlendMapping(const HW::BlendControl& blend,
const Prospero::ColorComponentMapping& mapping) {
// Color constants are not swizzled with the exports; scalar constant alpha is unaffected.
if (!mapping.IsIdentity() && (BlendFactorIsConstantColor(blend.color_srcblend) ||
BlendFactorIsConstantColor(blend.color_destblend))) {
return BlendMappingSupport::Unsupported;
}
if (mapping.Map(3) == 3u) {
return BlendMappingSupport::Direct;
}
// Moving alpha requires the same equation for all channels.
if (blend.separate_alpha_blend && (blend.alpha_srcblend != blend.color_srcblend ||
blend.alpha_destblend != blend.color_destblend ||
blend.alpha_comb_fcn != blend.color_comb_fcn)) {
return BlendMappingSupport::Unsupported;
}
auto support = BlendMappingSupport::Direct;
for (const auto factor: {blend.color_srcblend, blend.color_destblend}) {
if (BlendFactorIsDualSource(factor)) {
return BlendMappingSupport::Unsupported;
}
switch (static_cast<Prospero::BlendFactor>(factor)) {
case Prospero::BlendFactor::kSrcAlpha:
case Prospero::BlendFactor::kOneMinusSrcAlpha:
support = BlendMappingSupport::SourceAlpha;
break;
case Prospero::BlendFactor::kDstAlpha:
case Prospero::BlendFactor::kOneMinusDstAlpha:
case Prospero::BlendFactor::kSrcAlphaSaturate: return BlendMappingSupport::Unsupported;
default: break;
}
}
return support;
}
} // namespace Libs::Graphics
@@ -0,0 +1,27 @@
#ifndef EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_PIPELINE_BLENDMAPPING_H_
#define EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_PIPELINE_BLENDMAPPING_H_
#include <cstdint>
namespace Libs::Graphics {
namespace HW {
struct BlendControl;
}
namespace Prospero {
struct ColorComponentMapping;
}
enum class BlendMappingSupport {
Direct,
SourceAlpha, // Requires logical alpha in the second blend source.
Unsupported,
};
bool BlendFactorIsDualSource(uint8_t factor);
BlendMappingSupport ClassifyBlendMapping(const HW::BlendControl& blend,
const Prospero::ColorComponentMapping& mapping);
} // namespace Libs::Graphics
#endif // EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_PIPELINE_BLENDMAPPING_H_
@@ -91,19 +91,6 @@ vk::DescriptorImageInfo MakeImageInfo(const TextureBinding& texture, uint32_t el
return {nullptr, view, texture.layout};
}
static const char* ShaderStageResourceName(ShaderType stage) {
switch (stage) {
case ShaderType::Vertex: return "Vertex";
case ShaderType::Mesh: return "Mesh";
case ShaderType::Local: return "Local";
case ShaderType::TessellationControl: return "Hull";
case ShaderType::TessellationEvaluation: return "Domain";
case ShaderType::Pixel: return "Pixel";
case ShaderType::Compute: return "Compute";
default: return "Unknown";
}
}
static Prospero::ImageType TextureType(const ShaderTextureResource& descriptor) {
const auto type = descriptor.Type();
return type == Prospero::ImageType::kCube ? Prospero::ImageType::kColor2DArray : type;
@@ -126,8 +113,7 @@ static bool IsMultisampledTexture(Prospero::ImageType type) {
static vk::DescriptorBufferInfo
NativeStorageBuffer(RenderContext& context, const PreparedBindings::BufferSource& source,
const ShaderRecompiler::IR::BufferResource& resource, ShaderType stage,
uint32_t slot, uint32_t& buffer_offset) {
const ShaderRecompiler::IR::BufferResource& resource, uint32_t& buffer_offset) {
buffer_offset = 0;
const auto& [address, size, id] = source;
@@ -152,16 +138,6 @@ NativeStorageBuffer(RenderContext& context, const PreparedBindings::BufferSource
if (resource.written) {
context.GetTextureCache().InvalidateMemoryFromGPU(address, size);
}
const char* access = "Read";
if (resource.written && resource.read) {
access = "ReadWrite";
} else if (resource.written) {
access = "Write";
}
SetVulkanObjectNameF(
graphics.device, result.buffer,
"Kyty.{}.StorageBuffer[slot={} guest=0x{:016x} size=0x{:x} access={} formatted={}]",
ShaderStageResourceName(stage), slot, address, size, access, resource.formatted);
return result;
}
@@ -196,8 +172,8 @@ bool IsSupportedDepthTextureEncoding(const ShaderTextureResource& descriptor, bo
constexpr uint32_t htile_control = 0x00280000u;
const uint32_t expected_control = htile_control | (descriptor.MsaaDepth() ? (1u << 10u) : 0u);
const auto metadata_addr = descriptor.MetaAddr() << 8u;
return metadata_control == expected_control && metadata_addr != 0 &&
metadata_addr < TRACKER_ADDRESS_SIZE && (metadata_addr & 0x7fffu) == 0 &&
return metadata_control == expected_control && GuestRange {metadata_addr, 1}.Valid() &&
(metadata_addr & 0x7fffu) == 0 &&
descriptor.TileMode() == Prospero::TileMode::kDepth;
}
@@ -738,14 +714,15 @@ static vk::Sampler NativeSampler(RenderContext& context,
const ShaderRecompiler::IR::CompiledShaderInfo& program,
uint32_t index,
const ShaderRecompiler::IR::DescriptorValue& value) {
auto descriptor = DecodeNativeDescriptor<ShaderSamplerResource>(value);
if (!program.info.samplers[index].depth_compare) {
auto descriptor = DecodeNativeDescriptor<ShaderSamplerResource>(value);
const auto& sampler = program.info.samplers[index];
if (!sampler.depth_compare) {
descriptor.fields[0] &= ~(0x7u << 12u);
}
if (program.info.samplers[index].force_point_filtering) {
if (sampler.force_point_filtering) {
descriptor.SetPointFiltering();
}
return context.GetSamplerCache().GetSampler(descriptor);
return context.GetSamplerCache().GetSampler(descriptor, sampler.integer_border);
}
static vk::DescriptorBufferInfo NativeUpload(RenderContext& context,
@@ -873,7 +850,7 @@ void RenderExecutor::RebindBuffers(PreparedBindings& prepared) {
uint32_t buffer_offset = 0;
prepared.buffers.push_back(NativeStorageBuffer(m_context, prepared.buffer_sources[i],
program.info.buffers[resource],
program.stage, resource, buffer_offset));
buffer_offset));
pack_memory_offset(i, buffer_offset);
}
if (ShaderRecompiler::IR::FindBinding(
@@ -5,11 +5,13 @@
#include "common/file.h"
#include "common/logging/log.h"
#include "common/profiler.h"
#include "common/threads.h"
#include "graphics/guest_gpu/hardwareContext.h"
#include "graphics/host_gpu/renderer/colorRenderTarget.h"
#include "graphics/host_gpu/renderer/debug.h"
#include "graphics/host_gpu/renderer/depthRenderTarget.h"
#include "graphics/host_gpu/renderer/image/imageView.h"
#include "graphics/host_gpu/renderer/pipeline/blendMapping.h"
#include "graphics/host_gpu/renderer/render.h"
#include "graphics/host_gpu/renderer/renderContext.h"
#include "graphics/shader/recompiler/ShaderRecompiler.h"
@@ -29,7 +31,6 @@
#include <span>
#include <spirv-tools/libspirv.hpp>
#include <string_view>
#include <tuple>
#include <utility>
#include <vector>
#include <xxhash.h>
@@ -116,33 +117,22 @@ void DumpShaderSpirv(const char* stage_name, uint64_t shader_hash,
}
void DumpShaderOriginal(const char* stage_name, uint64_t shader_hash,
std::span<const uint32_t> code, const std::string& decoded_dump) {
std::span<const uint32_t> code) {
if (!Config::GraphicsDebugDumpEnabled()) {
return;
}
EXIT_IF(code.empty());
static std::atomic_int id = 0;
const auto base = Config::GetShaderLogFolder() / "original" /
fmt::format("{:04d}_new_shader_{}_{:016x}", id++, stage_name, shader_hash);
Common::File::CreateDirectories(base.parent_path());
for (const auto& [suffix, data, size]: {
std::tuple {".bin", static_cast<const void*>(code.data()), code.size_bytes()},
std::tuple {".rdna2", static_cast<const void*>(decoded_dump.data()),
decoded_dump.size()},
}) {
if (size == 0) {
continue;
}
auto path = base;
path += suffix;
Common::File file(path);
if (file.IsInvalid()) {
const auto path_text = Common::PathToString(path);
LOGF_COLOR(Log::Color::BrightRed, "Can't create file: %s\n", path_text.c_str());
} else {
file.Write(data, size);
}
const auto path = Config::GetShaderLogFolder() / "original" /
fmt::format("{:04d}_new_shader_{}_{:016x}.bin", id++, stage_name, shader_hash);
Common::File::CreateDirectories(path.parent_path());
Common::File file(path);
if (file.IsInvalid()) {
const auto path_text = Common::PathToString(path);
LOGF_COLOR(Log::Color::BrightRed, "Can't create file: %s\n", path_text.c_str());
return;
}
file.Write(code.data(), code.size_bytes());
}
bool ValidateShaderSpirv(const char* label, uint64_t shader_hash,
@@ -177,6 +167,32 @@ bool ValidateShaderSpirv(const char* label, uint64_t shader_hash,
} // namespace
std::size_t PipelineCache::GraphicsPipelineKeyHash::operator()(const GraphicsPipelineKey& key) const {
std::size_t hash = 0;
PipelineKeyHash::Mix(hash, key.rendering.color_count);
for (uint32_t i = 0; i < key.rendering.color_count; i++) {
PipelineKeyHash::Mix(hash, static_cast<uint32_t>(key.rendering.color_formats[i]));
}
PipelineKeyHash::Mix(hash, static_cast<uint32_t>(key.rendering.depth_format));
PipelineKeyHash::Mix(hash, static_cast<uint32_t>(key.rendering.stencil_format));
for (const auto id: key.vertex_shader_ids) {
PipelineKeyHash::Mix(hash, id);
}
PipelineKeyHash::Mix(hash, key.ps_shader_id);
PipelineKeyHash::Mix(hash, key.vertex_input.binding_count);
for (uint32_t i = 0; i < key.vertex_input.binding_count; i++) {
PipelineKeyHash::Mix(hash, key.vertex_input.bindings[i].stride);
PipelineKeyHash::Mix(hash, key.vertex_input.bindings[i].instance);
}
PipelineKeyHash::Mix(hash, key.vertex_input.attribute_count);
for (uint32_t i = 0; i < key.vertex_input.attribute_count; i++) {
PipelineKeyHash::Mix(hash, key.vertex_input.attributes[i].offset);
PipelineKeyHash::Mix(hash, key.vertex_input.attributes[i].binding);
}
PipelineKeyHash::Mix(hash, XXH3_64bits(&key.static_params, sizeof(key.static_params)));
return hash;
}
struct PipelineCache::ProgramCache {
struct ProgramKey {
ShaderType stage = ShaderType::Unknown;
@@ -218,32 +234,20 @@ struct PipelineCache::ProgramCache {
PipelineKeyHash::Mix(hash, key.code_size);
PipelineKeyHash::Mix(hash, key.static_state.size());
// Bucket same-shape static variants by source. ProgramKey equality performs the one
// exact state comparison needed on a stable hit without hashing up to 429 words first.
// exact state comparison needed on a stable hit without hashing the full state first.
return hash;
}
};
static constexpr std::size_t MaxStaticKeyWords = 13 + ShaderVertexInputInfo::RES_MAX * 13;
static constexpr std::size_t MaxStaticKeyWords = 32 + ShaderVertexInputInfo::RES_MAX * 6;
Permutation CompilePermutation(const ShaderParams& params,
Permutation CompilePermutation(const char* stage_name,
const ShaderRecompiler::CompileOptions& options,
ShaderRecompiler::TranslateResult translated,
ShaderRecompiler::IR::ResourceSpecialization specialization,
uint32_t push_data_start_dword) {
const char* stage_name = nullptr;
switch (options.stage) {
case ShaderType::Vertex: stage_name = "vs"; break;
case ShaderType::Mesh: stage_name = "ms"; break;
case ShaderType::Local: stage_name = "ls"; break;
case ShaderType::TessellationControl: stage_name = "hs"; break;
case ShaderType::TessellationEvaluation: stage_name = "ds"; break;
case ShaderType::Pixel: stage_name = "ps"; break;
case ShaderType::Compute: stage_name = "cs"; break;
default: EXIT("invalid pipeline shader stage\n");
}
auto result = ShaderRecompiler::CompileProgram(std::move(translated), options,
specialization, push_data_start_dword);
DumpShaderOriginal(stage_name, options.shader_hash, params.code, result.decoded_dump);
if (!ValidateShaderSpirv(options.dump_label, options.shader_hash, result.spirv)) {
DumpShaderSpirv(stage_name, options.shader_hash, result.spirv);
EXIT("%s failed hash=0x%016" PRIx64 ": SPIR-V validation failed\n", options.dump_label,
@@ -321,14 +325,15 @@ struct PipelineCache::ProgramCache {
stage_input.compute = &input_info;
}
const char* label = nullptr;
const char* stage_name = nullptr;
switch (stage) {
case ShaderType::Vertex: label = "ShaderRecompiler VS"; break;
case ShaderType::Mesh: label = "ShaderRecompiler MS"; break;
case ShaderType::Local: label = "ShaderRecompiler LS"; break;
case ShaderType::TessellationControl: label = "ShaderRecompiler HS"; break;
case ShaderType::TessellationEvaluation: label = "ShaderRecompiler DS"; break;
case ShaderType::Pixel: label = "ShaderRecompiler PS"; break;
case ShaderType::Compute: label = "ShaderRecompiler CS"; break;
case ShaderType::Vertex: label = "ShaderRecompiler VS"; stage_name = "vs"; break;
case ShaderType::Mesh: label = "ShaderRecompiler MS"; stage_name = "ms"; break;
case ShaderType::Local: label = "ShaderRecompiler LS"; stage_name = "ls"; break;
case ShaderType::TessellationControl: label = "ShaderRecompiler HS"; stage_name = "hs"; break;
case ShaderType::TessellationEvaluation: label = "ShaderRecompiler DS"; stage_name = "ds"; break;
case ShaderType::Pixel: label = "ShaderRecompiler PS"; stage_name = "ps"; break;
case ShaderType::Compute: label = "ShaderRecompiler CS"; stage_name = "cs"; break;
default: EXIT("invalid pipeline shader stage\n");
}
ShaderRecompiler::CompileOptions options;
@@ -351,6 +356,7 @@ struct PipelineCache::ProgramCache {
} else {
options.wave_size = input_info.wave_size;
}
DumpShaderOriginal(stage_name, options.shader_hash, params.code);
auto translated = ShaderRecompiler::TranslateProgram(params.code, options);
if (translated.skip_dispatch) {
entry = programs.try_emplace(lookup_key, ShaderRecompiler::IR::ResourcePlan {}).first;
@@ -365,7 +371,7 @@ struct PipelineCache::ProgramCache {
entry->second.specialization));
}
entry->second.permutations.push_back(CompilePermutation(
params, options, std::move(translated), entry->second.specialization, push_data_cursor));
stage_name, options, std::move(translated), entry->second.specialization, push_data_cursor));
const auto& permutation = entry->second.permutations.back();
input_info.stage = {.program = &permutation.program, .resources = &entry->second.resources};
permutation.program.bindings.AdvancePushData(push_data_cursor);
@@ -514,7 +520,6 @@ void PipelineCache::InitializeDriverCache() {
}
void PipelineCache::Save() {
Common::LockGuard lock(m_mutex);
if (m_driver_cache == nullptr) {
return;
}
@@ -605,21 +610,30 @@ PipelineCache::GraphicsPrograms PipelineCache::GetGraphicsPrograms(
}
ShaderParams pixel_params;
if (pixel_active) {
pixel_params = PrepareProgram(pixel_regs, sh, target_export_mapping, pixel_info);
const auto& blend = context.GetBlendControl(0);
const auto is_dual_source = [](uint8_t factor) {
return factor >= static_cast<uint8_t>(Prospero::BlendFactor::kSrc1Color) &&
factor <= static_cast<uint8_t>(Prospero::BlendFactor::kOneMinusSrc1Alpha);
};
pixel_params = PrepareProgram(pixel_regs, sh, target_export_mapping, pixel_info);
const auto& blend = context.GetBlendControl(0);
pixel_info.dual_source_blending =
blend.enable && !context.GetRenderTarget(0).info.blend_bypass &&
(is_dual_source(blend.color_srcblend) || is_dual_source(blend.color_destblend) ||
(blend.separate_alpha_blend &&
(is_dual_source(blend.alpha_srcblend) || is_dual_source(blend.alpha_destblend))));
(BlendFactorIsDualSource(blend.color_srcblend) ||
BlendFactorIsDualSource(blend.color_destblend) ||
(blend.separate_alpha_blend && (BlendFactorIsDualSource(blend.alpha_srcblend) ||
BlendFactorIsDualSource(blend.alpha_destblend))));
if (pixel_info.dual_source_blending) {
// MRT1 supplies a second blend source for the same render target as MRT0.
// MRT1 supplies the second blend source for target 0.
pixel_info.target_output_mode[1] = pixel_info.target_output_mode[0];
pixel_info.target_export_mapping[1] = pixel_info.target_export_mapping[0];
} else if (blend.enable && !context.GetRenderTarget(0).info.blend_bypass &&
pixel_info.target_output_mode[0] != 0 && pixel_info.target_output_mode[0] != 7 &&
std::all_of(std::begin(pixel_info.target_output_mode) + 1,
std::end(pixel_info.target_output_mode),
[](uint8_t mode) { return mode == 0; }) &&
ClassifyBlendMapping(blend, pixel_info.target_export_mapping[0]) ==
BlendMappingSupport::SourceAlpha) {
// Preserve logical alpha when the export mapping moves it.
pixel_info.alpha_blend_source_remap = true;
pixel_info.dual_source_blending = true;
pixel_info.target_output_mode[1] = pixel_info.target_output_mode[0];
pixel_info.target_export_mapping[1] = {};
}
}
if (context.GetClipControl().clip_disable) {
@@ -636,7 +650,6 @@ PipelineCache::GraphicsPrograms PipelineCache::GetGraphicsPrograms(
static_cast<float>(std::min(limits.maxViewportDimensions[1], 16384u)) * 0.5f;
clip.enabled = true;
}
Common::LockGuard lock(m_mutex);
uint32_t push_data_cursor =
mesh_active ? ShaderRecompiler::IR::PushData::MeshDrawDwordCount : 0;
GraphicsPrograms result;
@@ -654,7 +667,6 @@ ShaderProgram PipelineCache::GetComputeProgram(const HW::ComputeShaderInfo& regs
ShaderComputeInputInfo& input_info) {
input_info.host_subgroup_size = m_graphics.SupportsComputeWave64() ? 64u : 32u;
const auto params = PrepareProgram(regs, sh, input_info);
Common::LockGuard lock(m_mutex);
uint32_t push_data_cursor = 0;
return m_program_cache->Get(params, input_info, push_data_cursor);
}
@@ -679,7 +691,6 @@ PipelineCache::Pipeline& PipelineCache::GetGraphicsPipeline(
EXIT_IF(ps_active && !pixel_program);
const auto color_count = static_cast<uint32_t>(colors.size());
Common::LockGuard lock(m_mutex);
auto& ctx = command.GetRegisters();
const HW::ModeControl& mc = ctx.GetModeControl();
@@ -719,7 +730,24 @@ PipelineCache::Pipeline& PipelineCache::GetGraphicsPipeline(
static_params.alpha_comb_fcn[slot] = bc.alpha_comb_fcn;
static_params.alpha_destblend[slot] = bc.alpha_destblend;
static_params.separate_alpha_blend[slot] = bc.separate_alpha_blend;
static_params.blend_enable[slot] = bc.enable && !rt.info.blend_bypass;
const bool alpha_remap =
slot == 0 && ps_input_info != nullptr && ps_input_info->alpha_blend_source_remap;
static_params.blend_enable[slot] = bc.enable && !rt.info.blend_bypass;
if (static_params.blend_enable[slot] && !alpha_remap &&
ClassifyBlendMapping(bc, colors[i].export_mapping) != BlendMappingSupport::Direct) {
static_params.blend_enable[slot] = false;
static std::atomic_bool warned = false;
if (!warned.exchange(true, std::memory_order_relaxed)) {
Log::WriteToConsoleAndLog(fmt::format(
"Warning: blending disabled for unsupported color mapping "
"(slot={} mapping=0x{:02x} color={}/{} alpha={}/{} separate={}).\n",
slot, colors[i].export_mapping.packed, bc.color_srcblend, bc.color_destblend,
bc.alpha_srcblend, bc.alpha_destblend, bc.separate_alpha_blend ? 1 : 0));
}
}
if (alpha_remap) {
static_params.blend_alpha_source_remap = true;
}
}
const bool with_depth =
depth.desc.view_info.format != vk::Format::eUndefined && static_cast<bool>(depth.image_id);
@@ -783,24 +811,20 @@ PipelineCache::Pipeline& PipelineCache::GetGraphicsPipeline(
vs_input_info.resources_num > ShaderVertexInputInfo::RES_MAX);
key.vertex_input.binding_count = static_cast<uint8_t>(vs_input_info.buffers_num);
key.vertex_input.attribute_count = static_cast<uint8_t>(vs_input_info.resources_num);
uint32_t attributes_num = 0;
for (int binding = 0; binding < vs_input_info.buffers_num; binding++) {
const auto& buffer = vs_input_info.buffers[binding];
EXIT_IF(buffer.attr_num < 0 || buffer.attr_num > ShaderVertexInputBuffer::ATTR_MAX);
attributes_num += static_cast<uint32_t>(buffer.attr_num);
EXIT_IF(attributes_num > static_cast<uint32_t>(vs_input_info.resources_num));
key.vertex_input.bindings[binding] = {.stride = buffer.stride,
.instance = buffer.fetch_index != 0};
for (int attribute = 0; attribute < buffer.attr_num; attribute++) {
const auto index = buffer.attr_indices[attribute];
EXIT_IF(index < 0 || index >= vs_input_info.resources_num);
key.vertex_input.attributes[index] = {
.offset = buffer.attr_offsets[attribute],
.binding = static_cast<uint8_t>(binding),
};
}
}
EXIT_IF(attributes_num != static_cast<uint32_t>(vs_input_info.resources_num));
for (int attribute = 0; attribute < vs_input_info.resources_num; attribute++) {
const auto binding = vs_input_info.resources_dst[attribute].buffer_index;
EXIT_IF(binding < 0 || binding >= vs_input_info.buffers_num);
key.vertex_input.attributes[attribute] = {
.offset = static_cast<uint32_t>(vs_input_info.resources[attribute].Base48() -
vs_input_info.buffers[binding].addr),
.binding = static_cast<uint8_t>(binding),
};
}
}
if (auto iter = m_graphics_pipelines.find(key); iter != m_graphics_pipelines.end()) {
@@ -839,8 +863,6 @@ PipelineCache::GetComputePipeline(const ShaderComputeInputInfo& input_info,
EXIT_IF(!compute_program);
Common::LockGuard lock(m_mutex);
if (auto iter = m_compute_pipelines.find(compute_program.id);
iter != m_compute_pipelines.end()) {
return *iter->second;
@@ -4,7 +4,6 @@
#include "common/abi.h"
#include "common/assert.h"
#include "common/common.h"
#include "common/threads.h"
#include "graphics/host_gpu/renderer/renderTarget.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include "graphics/shader/shader.h"
@@ -56,6 +55,7 @@ struct PipelineStaticParameters {
uint8_t alpha_destblend[RENDER_COLOR_ATTACHMENTS_MAX] = {};
bool separate_alpha_blend[RENDER_COLOR_ATTACHMENTS_MAX] = {};
bool blend_enable[RENDER_COLOR_ATTACHMENTS_MAX] = {};
bool blend_alpha_source_remap = false;
bool operator==(const PipelineStaticParameters& other) const noexcept;
};
@@ -65,7 +65,7 @@ struct PipelineStaticParameters {
static_assert(std::is_trivially_copyable_v<PipelineStaticParameters>);
static_assert(std::is_standard_layout_v<PipelineStaticParameters>);
static_assert(alignof(PipelineStaticParameters) == 1);
static_assert(sizeof(PipelineStaticParameters) == 125);
static_assert(sizeof(PipelineStaticParameters) == 126);
struct PipelineRenderingState {
std::array<vk::Format, RENDER_COLOR_ATTACHMENTS_MAX> color_formats {};
@@ -103,6 +103,7 @@ struct ShaderProgram {
explicit operator bool() const { return id != 0 && module != nullptr; }
};
// The owning renderer serializes access, including saves while the GPU is running.
class PipelineCache {
public:
explicit PipelineCache(GraphicContext& graphics);
@@ -166,45 +167,10 @@ private:
hash ^= value + static_cast<std::size_t>(0x9e3779b97f4a7c15ull) + (hash << 6u) +
(hash >> 2u);
}
static void MixStaticParams(std::size_t& hash, const PipelineStaticParameters& params) {
const auto* bytes = reinterpret_cast<const uint8_t*>(&params);
for (std::size_t i = 0; i < sizeof(params); i++) {
Mix(hash, bytes[i]);
}
}
static void MixRendering(std::size_t& hash, const PipelineRenderingState& rendering) {
Mix(hash, rendering.color_count);
for (uint32_t i = 0; i < rendering.color_count; i++) {
Mix(hash, static_cast<uint32_t>(rendering.color_formats[i]));
}
Mix(hash, static_cast<uint32_t>(rendering.depth_format));
Mix(hash, static_cast<uint32_t>(rendering.stencil_format));
}
};
struct GraphicsPipelineKeyHash {
std::size_t operator()(const GraphicsPipelineKey& key) const {
std::size_t hash = 0;
PipelineKeyHash::MixRendering(hash, key.rendering);
for (const auto id: key.vertex_shader_ids) {
PipelineKeyHash::Mix(hash, id);
}
PipelineKeyHash::Mix(hash, key.ps_shader_id);
PipelineKeyHash::Mix(hash, key.vertex_input.binding_count);
for (uint32_t i = 0; i < key.vertex_input.binding_count; i++) {
PipelineKeyHash::Mix(hash, key.vertex_input.bindings[i].stride);
PipelineKeyHash::Mix(hash, key.vertex_input.bindings[i].instance);
}
PipelineKeyHash::Mix(hash, key.vertex_input.attribute_count);
for (uint32_t i = 0; i < key.vertex_input.attribute_count; i++) {
PipelineKeyHash::Mix(hash, key.vertex_input.attributes[i].offset);
PipelineKeyHash::Mix(hash, key.vertex_input.attributes[i].binding);
}
PipelineKeyHash::MixStaticParams(hash, key.static_params);
return hash;
}
std::size_t operator()(const GraphicsPipelineKey& key) const;
};
GraphicContext& m_graphics;
@@ -214,7 +180,6 @@ private:
std::unordered_map<GraphicsPipelineKey, std::unique_ptr<Pipeline>, GraphicsPipelineKeyHash>
m_graphics_pipelines;
std::unordered_map<uint64_t, std::unique_ptr<Pipeline>> m_compute_pipelines;
Common::Mutex m_mutex;
void InitializeDriverCache();
};
@@ -137,14 +137,17 @@ static void GetInputFormat(const ShaderBufferResource& res, vk::Format& format,
}
}
static vk::BlendFactor GetBlendFactor(uint32_t factor) {
static vk::BlendFactor GetBlendFactor(uint32_t factor, bool remap_source_alpha) {
switch (static_cast<Prospero::BlendFactor>(factor)) {
case Prospero::BlendFactor::kZero: return vk::BlendFactor::eZero;
case Prospero::BlendFactor::kOne: return vk::BlendFactor::eOne;
case Prospero::BlendFactor::kSrcColor: return vk::BlendFactor::eSrcColor;
case Prospero::BlendFactor::kOneMinusSrcColor: return vk::BlendFactor::eOneMinusSrcColor;
case Prospero::BlendFactor::kSrcAlpha: return vk::BlendFactor::eSrcAlpha;
case Prospero::BlendFactor::kOneMinusSrcAlpha: return vk::BlendFactor::eOneMinusSrcAlpha;
case Prospero::BlendFactor::kSrcAlpha:
return remap_source_alpha ? vk::BlendFactor::eSrc1Color : vk::BlendFactor::eSrcAlpha;
case Prospero::BlendFactor::kOneMinusSrcAlpha:
return remap_source_alpha ? vk::BlendFactor::eOneMinusSrc1Color
: vk::BlendFactor::eOneMinusSrcAlpha;
case Prospero::BlendFactor::kDstAlpha: return vk::BlendFactor::eDstAlpha;
case Prospero::BlendFactor::kOneMinusDstAlpha: return vk::BlendFactor::eOneMinusDstAlpha;
case Prospero::BlendFactor::kDstColor: return vk::BlendFactor::eDstColor;
@@ -397,17 +400,20 @@ void CreatePipelineInternal(GraphicContext& graphics, PipelineCache::Pipeline& p
color_blend_attachment[i].colorWriteMask =
vk::ColorComponentFlags {static_params.color_mask[i]};
color_blend_attachment[i].blendEnable = static_params.blend_enable[i] ? VK_TRUE : VK_FALSE;
// Only target 0 can use the second blend source carrying logical alpha.
const bool remap_source_alpha = i == 0 && static_params.blend_alpha_source_remap;
color_blend_attachment[i].srcColorBlendFactor =
GetBlendFactor(static_params.color_srcblend[i]);
GetBlendFactor(static_params.color_srcblend[i], remap_source_alpha);
color_blend_attachment[i].dstColorBlendFactor =
GetBlendFactor(static_params.color_destblend[i]);
GetBlendFactor(static_params.color_destblend[i], remap_source_alpha);
color_blend_attachment[i].colorBlendOp = GetBlendOp(static_params.color_comb_fcn[i]);
color_blend_attachment[i].srcAlphaBlendFactor =
(static_params.separate_alpha_blend[i] ? GetBlendFactor(static_params.alpha_srcblend[i])
: color_blend_attachment[i].srcColorBlendFactor);
(static_params.separate_alpha_blend[i]
? GetBlendFactor(static_params.alpha_srcblend[i], remap_source_alpha)
: color_blend_attachment[i].srcColorBlendFactor);
color_blend_attachment[i].dstAlphaBlendFactor =
(static_params.separate_alpha_blend[i]
? GetBlendFactor(static_params.alpha_destblend[i])
? GetBlendFactor(static_params.alpha_destblend[i], remap_source_alpha)
: color_blend_attachment[i].dstColorBlendFactor);
color_blend_attachment[i].alphaBlendOp =
(static_params.separate_alpha_blend[i] ? GetBlendOp(static_params.alpha_comb_fcn[i])
+47 -43
View File
@@ -249,8 +249,8 @@ static void LogDrawInputState(const CommandBuffer& buffer, const RenderColorInfo
for (int bi = 0; bi < vs_input_info.buffers_num; bi++) {
const auto& b = vs_input_info.buffers[bi];
LOGF("DrawInputState[%u]: vb[%d] addr=0x%010" PRIx64
" stride=%u records=%u fetch_index=%u attr_num=%d\n",
log_id, bi, b.addr, b.stride, b.num_records, b.fetch_index, b.attr_num);
" stride=%u records=%u fetch_index=%u\n",
log_id, bi, b.addr, b.stride, b.num_records, b.fetch_index);
const auto* bytes = reinterpret_cast<const uint8_t*>(b.addr);
if (bytes != nullptr && b.stride != 0) {
@@ -272,11 +272,13 @@ static void LogDrawInputState(const CommandBuffer& buffer, const RenderColorInfo
raw[5], raw[6], raw[7], raw[8], flt[0], flt[1], flt[2], flt[3], flt[4], flt[5],
flt[6], flt[7], flt[8]);
for (int ai = 0; ai < b.attr_num; ai++) {
const auto res_index = b.attr_indices[ai];
const auto& r = vs_input_info.resources[res_index];
const auto& rd = vs_input_info.resources_dst[res_index];
const auto offset = b.attr_offsets[ai];
for (int ai = 0; ai < vs_input_info.resources_num; ai++) {
const auto& r = vs_input_info.resources[ai];
const auto& rd = vs_input_info.resources_dst[ai];
if (rd.buffer_index != bi) {
continue;
}
const auto offset = static_cast<uint32_t>(r.Base48() - b.addr);
if (offset + 4u <= b.stride &&
r.Format() == Prospero::BufferFormat::k8_8_8_8UNorm) {
uint32_t packed = 0;
@@ -296,14 +298,16 @@ static void LogDrawInputState(const CommandBuffer& buffer, const RenderColorInfo
}
}
for (int ai = 0; ai < b.attr_num; ai++) {
const auto res_index = b.attr_indices[ai];
const auto& r = vs_input_info.resources[res_index];
const auto& rd = vs_input_info.resources_dst[res_index];
LOGF("DrawInputState[%u]: attr[%d] res=%d offset=%u dst=v%d regs=%d fetch_index=%u "
for (int ai = 0; ai < vs_input_info.resources_num; ai++) {
const auto& r = vs_input_info.resources[ai];
const auto& rd = vs_input_info.resources_dst[ai];
if (rd.buffer_index != bi) {
continue;
}
LOGF("DrawInputState[%u]: attr[%d] offset=%u dst=v%d regs=%d fetch_index=%u "
"sharp=%08" PRIx32 " %08" PRIx32 " %08" PRIx32 " %08" PRIx32 "\n",
log_id, ai, res_index, b.attr_offsets[ai], rd.register_start, rd.registers_num,
rd.fetch_index, r.fields[0], r.fields[1], r.fields[2], r.fields[3]);
log_id, ai, static_cast<uint32_t>(r.Base48() - b.addr), rd.register_start,
rd.registers_num, rd.fetch_index, r.fields[0], r.fields[1], r.fields[2], r.fields[3]);
}
}
}
@@ -575,12 +579,17 @@ RenderState RenderExecutor::AcquireRenderTargets(CommandBuffer& buffer, RenderCo
return state;
}
static bool DrawHasActivePixelShader(const CommandBuffer& buffer) {
const auto& ctx = buffer.GetRegisters();
const auto& sh_regs = ctx.GetShaderRegisters();
const bool has_color_output = (ctx.GetRenderTargetMask() & sh_regs.m_cbShaderMask) != 0;
return buffer.GetShaders().GetPs().ps_regs.data_addr != 0 &&
(has_color_output || PixelShaderHasDepthOrCoverageSideEffects(sh_regs));
static uint32_t DrawColorOutputMask(const HW::Context& ctx) {
const auto& sh_regs = ctx.GetShaderRegisters();
const auto write_mask = ctx.GetRenderTargetMask() & sh_regs.m_cbShaderMask;
uint32_t output_mask = 0;
for (uint32_t slot = 0; slot < RENDER_COLOR_ATTACHMENTS_MAX; slot++) {
if (sh_regs.target_output_mode[slot] != 0 &&
render_target_mask_slot(write_mask, slot) != 0) {
output_mask |= 1u << slot;
}
}
return output_mask;
}
enum class CbColorMode : uint8_t {
@@ -625,19 +634,22 @@ struct PreparedIndexBuffer {
vk::IndexType type = vk::IndexType::eUint16;
};
static uint64_t VertexBufferDescriptorSize(const ShaderVertexInputBuffer& buffer,
const ShaderVertexInputInfo& info) {
static uint64_t VertexBufferDescriptorSize(int binding, const ShaderVertexInputInfo& info) {
const auto& buffer = info.buffers[binding];
if (buffer.stride != 0 || buffer.num_records == 0) {
return static_cast<uint64_t>(buffer.stride) * buffer.num_records;
}
uint64_t size = 0;
for (int i = 0; i < buffer.attr_num; i++) {
const auto& resource = info.resources[buffer.attr_indices[i]];
for (int i = 0; i < info.resources_num; i++) {
if (info.resources_dst[i].buffer_index != binding) {
continue;
}
const auto& resource = info.resources[i];
// RDNA2 OOB_SELECT=2 only checks NumRecords != 0. A constant attribute still
// fetches its entire format; NumRecords is not a byte count in this mode.
const uint64_t extent = resource.OutOfBounds() == 2
? static_cast<uint64_t>(buffer.attr_offsets[i]) +
? resource.Base48() - buffer.addr +
ShaderRecompiler::Format::GetFormatInfo(resource.Format()).byte_size
: buffer.num_records;
size = std::max(size, extent);
@@ -674,7 +686,7 @@ static PreparedVertexBuffers AcquireVertexBuffers(CommandBuffer& b
uint32_t range_count = 0;
for (int i = 0; i < vs_input_info.buffers_num; i++) {
const auto& vertex = vs_input_info.buffers[i];
const auto size = VertexBufferDescriptorSize(vertex, vs_input_info);
const auto size = VertexBufferDescriptorSize(i, vs_input_info);
sizes[i] = size;
if (size == 0) {
continue;
@@ -713,9 +725,6 @@ static PreparedVertexBuffers AcquireVertexBuffers(CommandBuffer& b
Libs::LibKernel::Memory::ClampRangeSize(range.base_address, range.RequestedSize());
range.acquired_end = range.base_address + size;
range.binding = cache.ObtainBuffer(range.base_address, size, false);
SetVulkanObjectNameF(
buffer.GetContext().GetGraphics().device, range.binding.first->Handle(),
"Kyty.VertexBufferRange[guest=0x{:016x} size=0x{:x}]", range.base_address, size);
}
// Rebuild slot bindings, offsetting non-empty slots into their acquired merged range.
@@ -747,10 +756,6 @@ static PreparedVertexBuffers AcquireVertexBuffers(CommandBuffer& b
prepared.buffers[i] = range->binding.first->Handle();
prepared.offsets[i] = range->binding.second + vertex.addr - range->base_address;
prepared.sizes[i] = std::min(size, range->acquired_end - vertex.addr);
SetVulkanObjectNameF(
buffer.GetContext().GetGraphics().device, prepared.buffers[i],
"Kyty.VertexBuffer[slot={} guest=0x{:016x} size=0x{:x} stride={} records={}]", i,
vertex.addr, size, vertex.stride, vertex.num_records);
}
return prepared;
@@ -847,7 +852,7 @@ static bool ResolvePrimitiveRestart(const CommandBuffer& buffer,
}
static void RefreshShaders(CommandBuffer& buffer, const DrawCallInfo& draw,
DrawRenderState& state) {
uint32_t color_output_mask, DrawRenderState& state) {
auto& ctx = buffer.GetRegisters();
auto& sh_ctx = buffer.GetShaders();
@@ -861,7 +866,7 @@ static void RefreshShaders(CommandBuffer& buffer, const DrawCallInfo& draw,
target_export_mapping {};
for (uint32_t slot = 0; slot < RENDER_COLOR_ATTACHMENTS_MAX; slot++) {
const auto& rt = ctx.GetRenderTarget(slot);
if (rt.base.addr != 0 && render_target_mask_slot(ctx.GetRenderTargetMask(), slot) != 0) {
if ((color_output_mask & (1u << slot)) != 0 && rt.base.addr != 0) {
target_export_mapping[slot] =
TextureGetRenderTargetFormat(rt.info.format, rt.info.channel_type,
rt.info.channel_order)
@@ -880,8 +885,12 @@ static void RefreshShaders(CommandBuffer& buffer, const DrawCallInfo& draw,
bool RenderExecutor::PrepareDrawRenderState(CommandBuffer& buffer, const DrawCallInfo& draw,
uint32_t render_target_slice_offset,
DrawRenderState& state) {
state.ps_active = DrawHasActivePixelShader(buffer);
RefreshShaders(buffer, draw, state);
const auto& shader_regs = buffer.GetRegisters().GetShaderRegisters();
const auto color_output_mask = DrawColorOutputMask(buffer.GetRegisters());
state.ps_active = buffer.GetShaders().GetPs().ps_regs.data_addr != 0 &&
(color_output_mask != 0 ||
PixelShaderHasDepthOrCoverageSideEffects(shader_regs));
RefreshShaders(buffer, draw, color_output_mask, state);
uint32_t mrt_mask = 0;
if (state.ps_active) {
for (const auto& output: state.ps_input_info.stage.program->info.outputs) {
@@ -890,6 +899,7 @@ bool RenderExecutor::PrepareDrawRenderState(CommandBuffer& buffer, const DrawCal
}
}
}
mrt_mask &= color_output_mask;
if (draw.IsIndexed()) {
LogDrawPhase(draw.Name(), "ResolveRenderColorTarget");
}
@@ -927,17 +937,11 @@ static PreparedIndexBuffer PrepareIndexBuffer(CommandBuffer& buffe
auto& stream = buffer.GetContext().GetBufferCache().GetUtilityBuffer(MemoryUsage::Stream);
prepared.offset = stream.Copy(source.host_data, source.size, 16);
prepared.buffer = stream.Handle();
SetVulkanObjectNameF(buffer.GetContext().GetGraphics().device, prepared.buffer,
"Kyty.IndexBuffer[guest=transient size=0x{:x} type={}]", source.size,
static_cast<uint32_t>(source.type));
} else {
auto [buffer_ptr, offset] =
buffer.GetContext().GetBufferCache().ObtainBuffer(source.address, source.size, false);
prepared.buffer = buffer_ptr->Handle();
prepared.offset = offset;
SetVulkanObjectNameF(buffer.GetContext().GetGraphics().device, prepared.buffer,
"Kyty.IndexBuffer[guest=0x{:016x} size=0x{:x} type={}]",
source.address, source.size, static_cast<uint32_t>(source.type));
}
return prepared;
}
@@ -520,16 +520,20 @@ static vk::Device VulkanCreateDevice(GraphicContext& graphics,
#endif
depth_clip_control.depthClipControl = VK_TRUE;
auto features12 = WindowContext::RequiredVulkan12Features();
features12.pNext = &depth_clip_control;
const bool workgroup_layout_extension =
HasExtension(device_extensions, VK_KHR_WORKGROUP_MEMORY_EXPLICIT_LAYOUT_EXTENSION_NAME);
vk::PhysicalDeviceWorkgroupMemoryExplicitLayoutFeaturesKHR supported_workgroup_layout {};
vk::PhysicalDeviceVulkan12Features supported_features12 {};
supported_features12.pNext = workgroup_layout_extension ? &supported_workgroup_layout : nullptr;
vk::PhysicalDeviceVulkan13Features supported_features13 {};
supported_features13.pNext = &supported_features12;
const auto robustness2_ext_enabled =
HasExtension(device_extensions, VK_EXT_ROBUSTNESS_2_EXTENSION_NAME);
vk::PhysicalDeviceRobustness2FeaturesEXT supported_robustness2 {};
if (robustness2_ext_enabled) {
supported_robustness2.pNext = supported_features13.pNext;
supported_features13.pNext = &supported_robustness2;
}
@@ -557,6 +561,22 @@ static vk::Device VulkanCreateDevice(GraphicContext& graphics,
supported_features2.pNext = &provoking_vertex;
}
physical_device.getFeatures2(&supported_features2);
auto features12 = WindowContext::RequiredVulkan12Features();
features12.shaderSharedInt64Atomics = supported_features12.shaderSharedInt64Atomics;
vk::PhysicalDeviceWorkgroupMemoryExplicitLayoutFeaturesKHR workgroup_layout {};
workgroup_layout.workgroupMemoryExplicitLayout =
supported_workgroup_layout.workgroupMemoryExplicitLayout;
workgroup_layout.pNext = &depth_clip_control;
features12.pNext = workgroup_layout_extension ? static_cast<void*>(&workgroup_layout)
: static_cast<void*>(&depth_clip_control);
if (!features12.shaderSharedInt64Atomics || !workgroup_layout.workgroupMemoryExplicitLayout) {
Log::WriteToConsoleAndLog(fmt::format(
"WARNING: Native 64-bit LDS atomics are unavailable: shaderSharedInt64Atomics={}, "
"workgroupMemoryExplicitLayout={}. Continuing with the selected Vulkan device.\n",
features12.shaderSharedInt64Atomics != VK_FALSE,
workgroup_layout.workgroupMemoryExplicitLayout != VK_FALSE));
}
graphics.mesh_shader_enabled = mesh_extension && supported_mesh.meshShader;
vk::PhysicalDeviceSubgroupSizeControlProperties subgroup_size_control {};
@@ -1049,6 +1069,7 @@ void WindowContext::CreateVulkan() {
for (const auto* extension: {VK_EXT_ROBUSTNESS_2_EXTENSION_NAME,
VK_EXT_PROVOKING_VERTEX_EXTENSION_NAME,
VK_EXT_MESH_SHADER_EXTENSION_NAME,
VK_KHR_WORKGROUP_MEMORY_EXPLICIT_LAYOUT_EXTENSION_NAME,
VK_EXT_DEPTH_RANGE_UNRESTRICTED_EXTENSION_NAME}) {
if (HasExtension(available_extensions, extension)) {
device_extensions.push_back(extension);
@@ -817,6 +817,7 @@ void WindowRun() {
EXIT_IF(g_window == nullptr);
g_window->Run();
Common::LockGuard lock(g_window->render_context->GetMutex());
g_window->render_context->GetPipelineCache().Save();
}
@@ -15,7 +15,6 @@
#include "graphics/shader/recompiler/ir/passes/ResourceMaterialization.h"
#include "graphics/shader/recompiler/ir/passes/ResourceTracking.h"
#include "graphics/shader/recompiler/ir/passes/ShaderInfoCollection.h"
#include "graphics/shader/recompiler/ir/passes/SrtWalker.h"
#include "graphics/shader/recompiler/ir/passes/SsaRewrite.h"
#include <algorithm>
@@ -548,30 +547,40 @@ TranslateResult TranslateProgram(std::span<const uint32_t> code, const CompileOp
LOGF("%s phase begin: stage=%s hash=0x%016" PRIx64 " CFG BuildGraph\n", GetDumpLabel(options),
StageName(options.stage), options.shader_hash);
auto cfg = CFG::BuildGraph(decoded);
auto native_cfg = CFG::BuildGraph(decoded);
CFG::Graph structured_cfg;
auto* selected_cfg = &native_cfg;
LOGF("%s phase end: stage=%s hash=0x%016" PRIx64 " CFG BuildGraph blocks=%" PRIu64
" loops=%" PRIu64 " back_edges=%" PRIu64 " elapsed_ms=%" PRIu64 "\n",
GetDumpLabel(options), StageName(options.stage), options.shader_hash,
static_cast<uint64_t>(cfg.blocks.size()), static_cast<uint64_t>(cfg.natural_loops.size()),
static_cast<uint64_t>(cfg.back_edges.size()), phase_ms());
if (cfg.irreducible) {
LogDispatcherFallback(options, cfg, "build");
static_cast<uint64_t>(native_cfg.blocks.size()),
static_cast<uint64_t>(native_cfg.natural_loops.size()),
static_cast<uint64_t>(native_cfg.back_edges.size()), phase_ms());
if (native_cfg.irreducible) {
LogDispatcherFallback(options, native_cfg, "build");
} else {
LOGF("%s phase begin: stage=%s hash=0x%016" PRIx64 " CFG Structurize\n",
GetDumpLabel(options), StageName(options.stage), options.shader_hash);
if (!CFG::Structurize(cfg)) {
LogDispatcherFallback(options, cfg, "structurize");
structured_cfg = CFG::Structurize(native_cfg);
if (structured_cfg.unsupported) {
native_cfg.unsupported = true;
native_cfg.failure_kind = structured_cfg.failure_kind;
native_cfg.failure_block = structured_cfg.failure_block;
native_cfg.unsupported_reason = structured_cfg.unsupported_reason;
LogDispatcherFallback(options, native_cfg, "structurize");
} else {
selected_cfg = &structured_cfg;
LOGF("%s structured CFG success: blocks=%" PRIu64 "\n", GetDumpLabel(options),
static_cast<uint64_t>(cfg.blocks.size()));
static_cast<uint64_t>(selected_cfg->blocks.size()));
}
LOGF("%s phase end: stage=%s hash=0x%016" PRIx64 " CFG Structurize blocks=%" PRIu64
" loops=%" PRIu64 " elapsed_ms=%" PRIu64 "\n",
GetDumpLabel(options), StageName(options.stage), options.shader_hash,
static_cast<uint64_t>(cfg.blocks.size()),
static_cast<uint64_t>(cfg.natural_loops.size()), phase_ms());
static_cast<uint64_t>(selected_cfg->blocks.size()),
static_cast<uint64_t>(selected_cfg->natural_loops.size()), phase_ms());
}
const auto& cfg = *selected_cfg;
Frontend::EmbeddedFetchPlan embedded_fetch;
if ((options.stage == ShaderType::Vertex || options.stage == ShaderType::Local) &&
options.input_info.vertex != nullptr && options.input_info.vertex->fetch_embedded) {
@@ -614,15 +623,21 @@ TranslateResult TranslateProgram(std::span<const uint32_t> code, const CompileOp
IR::EliminateDeadCode(ir.blocks);
}
LowerTessellationMemory(ir, options);
IR::BuildSrtPlan(ir);
IR::EliminateDeadCode(ir.blocks);
IR::TrackResources(ir);
std::string cfg_dump;
if (options.dump_ir) {
cfg_dump = CFG::GraphToString(cfg);
if (options.early_dump) {
LOGF("%s native IR before resource tracking:\n%s", GetDumpLabel(options),
MakeIrDump(cfg_dump, ir).c_str());
}
}
IR::TrackResources(ir, decoded, native_cfg);
IR::EliminateDeadCode(ir.blocks);
TranslateResult result;
result.program = std::move(ir);
if (options.dump_ir) {
result.decoded_dump = std::move(decoded_dump);
result.cfg_dump = CFG::GraphToString(cfg);
result.cfg_dump = std::move(cfg_dump);
}
return result;
}
@@ -634,6 +649,35 @@ CompileResult CompileProgram(TranslateResult translated, const CompileOptions& o
const auto emit_begin = std::chrono::steady_clock::now();
auto& ir = translated.program;
IR::ApplyResourceSpecialization(ir, specialization);
// The resource plan owns host descriptor evaluation now. Keep only dependencies consumed
// by GPU memory operations; bound descriptor dwords must not retain shader instructions.
for (auto& inst: ir.value_storage) {
inst.Invalidate();
}
for (auto* block: ir.blocks) {
for (auto& inst: *block) {
const auto op = inst.GetOpcode();
uint32_t first = 0;
if (op == IR::ValueOpcode::GetBufferResource) {
if (std::ranges::any_of(inst.Uses(), [&](const IR::Use& use) {
return ir.memory_info[use.user->Flags<IR::MemoryFlags>().index].kind ==
IR::ResourceKind::IndirectBuffer;
})) {
continue;
}
} else if (op == IR::ValueOpcode::GetImageResource) {
const auto resource = inst.Flags<uint32_t>();
first = resource < ir.info.images.size() &&
ir.info.images[resource].indirect_root == resource ? 1u : 0u;
} else if (op != IR::ValueOpcode::GetSamplerResource) {
continue;
}
for (size_t index = first; index < inst.NumArgs(); index++) {
inst.SetArg(index, IR::Value(0u));
}
}
}
ir.value_storage.clear();
IR::RemoveIdentities(ir.blocks);
IR::EliminateDeadCode(ir.blocks);
@@ -62,12 +62,7 @@ void ValidateNativeProgram(const IR::Program& program) {
Expect(Kind::BdaPagetable);
Expect(Kind::FaultBuffer);
}
const bool uses_flattened_runtime =
!program.srt_reads.empty() ||
std::ranges::any_of(program.info.images, [](const IR::ImageResource& image) {
return image.indirect_search_iterations != 0u;
});
if (uses_flattened_runtime) {
if (IR::UsesFlattenedSrt(program)) {
Expect(Kind::FlattenedSrt);
}
if (program.bindings.ShaderDataDwords() != 0 && !program.bindings.UsesPushData()) {
@@ -234,6 +229,12 @@ Emitter::SpirvRequirements Emitter::AnalyzeProgramRequirements(const IR::Program
if (kind != IR::ResourceKind::Lds && kind != IR::ResourceKind::Gds) {
Fail(program, "shared operation has invalid resource kind");
}
if (inst.GetOpcode() == IR::ValueOpcode::SharedAtomicOr64) {
if (kind != IR::ResourceKind::Lds || program.stage != ShaderType::Compute) {
Fail(program, "64-bit shared atomics require compute LDS");
}
requirements.shared_int64_atomics = true;
}
if (program.stage != ShaderType::Compute && program.stage != ShaderType::Mesh &&
kind == IR::ResourceKind::Lds) {
requirements.function_lds = true;
@@ -307,6 +307,11 @@ uint32_t EmitConvertF32F64(EmitterState& state, uint32_t arg0) {
Unary(state, spv::OpBitcast, TypeF32(state), clamped), converted));
}
uint32_t EmitConvertF64F32(EmitterState& state, uint32_t arg0) {
return EmitNative<spv::OpFConvert, IR::Type::F64>(state,
EmitFlushF32DenormToSignedZero(state, arg0));
}
uint32_t EmitCompositeExtractU64(EmitterState& state, uint32_t arg0, IR::Value arg1) {
return EmitNative<spv::OpCompositeExtract, IR::Type::U32>(state, arg0, arg1.U32());
}
@@ -203,9 +203,7 @@ uint32_t EmitFlushF32DenormToSignedZero(EmitterState& state, uint32_t value) {
const auto bits = state.builder.AllocateId();
const auto abs_bits = state.builder.AllocateId();
const auto sign_bits = state.builder.AllocateId();
const auto non_zero = state.builder.AllocateId();
const auto subnormal = state.builder.AllocateId();
const auto flush = state.builder.AllocateId();
const auto selected = state.builder.AllocateId();
const auto ret = state.builder.AllocateId();
state.builder.AddFunction(spv::OpBitcast, TypeU32(state), bits, value);
@@ -213,12 +211,9 @@ uint32_t EmitFlushF32DenormToSignedZero(EmitterState& state, uint32_t value) {
ConstantU32(state, 0x7fffffffu));
state.builder.AddFunction(spv::OpBitwiseAnd, TypeU32(state), sign_bits, bits,
ConstantU32(state, 0x80000000u));
state.builder.AddFunction(spv::OpINotEqual, TypeBool(state), non_zero, abs_bits,
ConstantU32(state, 0));
state.builder.AddFunction(spv::OpULessThan, TypeBool(state), subnormal, abs_bits,
ConstantU32(state, 0x00800000u));
state.builder.AddFunction(spv::OpLogicalAnd, TypeBool(state), flush, non_zero, subnormal);
state.builder.AddFunction(spv::OpSelect, TypeU32(state), selected, flush, sign_bits, bits);
state.builder.AddFunction(spv::OpSelect, TypeU32(state), selected, subnormal, sign_bits, bits);
state.builder.AddFunction(spv::OpBitcast, TypeF32(state), ret, selected);
return ret;
}
@@ -452,6 +452,11 @@ void EmitSetAttribute(ValueEmitContext& ctx, const IR::Inst& inst) {
if (exp.kind == IR::ExportTargetKind::Null || exp.en == 0u) {
return;
}
// Skip dormant color exports after their valid mask; MRT1 is reserved for logical alpha.
if (state.program.stage == ShaderType::Pixel && exp.kind == IR::ExportTargetKind::Mrt &&
exp.index != 0 && state.input_info.pixel->alpha_blend_source_remap) {
return;
}
EmitIfCondition(state, exec, [&]() {
const auto data = ctx.Arg(inst, 0);
if (exp.kind == IR::ExportTargetKind::Primitive) {
@@ -492,6 +497,18 @@ void EmitSetAttribute(ValueEmitContext& ctx, const IR::Inst& inst) {
const bool uint_output = MrtOutputMode(state, exp) == 7u;
const auto vector_type = uint_output ? TypeU32Vector(state, 4) : TypeF32Vector(state, 4);
auto value = ExportVector(ctx, data, exp, uint_output);
if (state.program.stage == ShaderType::Pixel && exp.kind == IR::ExportTargetKind::Mrt &&
exp.index == 0 && !uint_output && state.input_info.pixel->alpha_blend_source_remap) {
// Broadcast logical alpha before swizzling the primary output.
const auto blend_output =
OutputVariableForExport(state, {.kind = IR::ExportTargetKind::Mrt, .index = 1});
if (blend_output != 0) {
const auto alpha = state.builder.AllocateId();
state.builder.AddFunction(spv::OpVectorShuffle, vector_type, alpha, value, value,
3u, 3u, 3u, 3u);
state.builder.AddFunction(spv::OpStore, blend_output, alpha);
}
}
if (state.program.stage == ShaderType::Pixel && exp.kind == IR::ExportTargetKind::Mrt &&
exp.index < state.input_info.pixel->target_export_mapping.size()) {
const auto mapping = state.input_info.pixel->target_export_mapping[exp.index];
@@ -636,6 +653,29 @@ uint32_t EmitDppUpdateU32(ValueEmitContext& ctx, const IR::Inst& inst) {
ctx.Arg(inst, 1));
}
uint32_t EmitConditionRef(ValueEmitContext& ctx, const IR::Inst& inst) {
if (ctx.other_half == nullptr) return ctx.Arg(inst, 0);
// A native scalar branch makes one decision for both emulated wave halves.
if (ctx.half != 0) return ctx.other_half->Def(IR::Value(&inst));
const auto kind = inst.Flags<CFG::BranchCondition>();
if (kind == CFG::BranchCondition::ScalarInstruction) return ctx.Arg(inst, 0);
const auto ballot = ctx.Ballot(inst.Arg(0));
const auto low = ctx.state.builder.AllocateId();
const auto high = ctx.state.builder.AllocateId();
const auto combined = ctx.state.builder.AllocateId();
const auto result = ctx.state.builder.AllocateId();
ctx.state.builder.AddFunction(spv::OpCompositeExtract, TypeU32(ctx.state), low, ballot, 0);
ctx.state.builder.AddFunction(spv::OpCompositeExtract, TypeU32(ctx.state), high, ballot, 1);
const bool zero = kind == CFG::BranchCondition::ExecZero ||
kind == CFG::BranchCondition::VccZero || kind == CFG::BranchCondition::SccZero;
ctx.state.builder.AddFunction(zero ? spv::OpBitwiseAnd : spv::OpBitwiseOr,
TypeU32(ctx.state), combined, low, high);
ctx.state.builder.AddFunction(zero ? spv::OpIEqual : spv::OpINotEqual,
TypeBool(ctx.state), result, combined,
ConstantU32(ctx.state, zero ? ~0u : 0u));
return result;
}
uint32_t EmitBallot(ValueEmitContext& ctx, IR::Value predicate) {
return ctx.Ballot(predicate);
}
@@ -2,6 +2,8 @@
#include "graphics/shader/recompiler/backend/spirv/spirvEmitterInternal.h"
#include <bit>
namespace Libs::Graphics::ShaderRecompiler::Spirv::Emitter {
// Operand signatures distinguish SPIR-V IDs (uint32_t), raw literals/values
// (IR::Value), and instructions whose metadata or operand loads must stay lazy.
@@ -29,7 +31,9 @@ uint32_t EmitConvertSigned32ToFloat(EmitterState& state, uint32_t arg0) {
inline constexpr auto EmitConvertF32S32 = EmitConvertSigned32ToFloat<IR::Type::F32>;
inline constexpr auto EmitConvertF64S32 = EmitConvertSigned32ToFloat<IR::Type::F64>;
uint32_t EmitConvertF32F64(EmitterState& state, uint32_t arg0);
uint32_t EmitConvertF64F32(EmitterState& state, uint32_t arg0);
EMIT_NATIVE(ConvertF32U32, OpConvertUToF, F32, uint32_t)
EMIT_NATIVE(ConvertF64U32, OpConvertUToF, F64, uint32_t)
EMIT_NATIVE(CompositeConstructU64, OpCompositeConstruct, U64, uint32_t, uint32_t)
EMIT_NATIVE(CompositeConstructU32x2, OpCompositeConstruct, U32x2, uint32_t, uint32_t)
EMIT_NATIVE(CompositeConstructU32x3, OpCompositeConstruct, U32x3, uint32_t, uint32_t, uint32_t)
@@ -112,18 +116,34 @@ EMIT_NATIVE(LogicalOr, OpLogicalOr, U1, uint32_t, uint32_t)
EMIT_NATIVE(LogicalAnd, OpLogicalAnd, U1, uint32_t, uint32_t)
EMIT_NATIVE(LogicalXor, OpLogicalNotEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(LogicalNot, OpLogicalNot, U1, uint32_t)
EMIT_NATIVE(FPOrdEqual32, OpFOrdEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordEqual32, OpFUnordEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdNotEqual32, OpFOrdNotEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordNotEqual32, OpFUnordNotEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdLessThan32, OpFOrdLessThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordLessThan32, OpFUnordLessThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdGreaterThan32, OpFOrdGreaterThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordGreaterThan32, OpFUnordGreaterThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdLessThanEqual32, OpFOrdLessThanEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordLessThanEqual32, OpFUnordLessThanEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdGreaterThanEqual32, OpFOrdGreaterThanEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordGreaterThanEqual32, OpFUnordGreaterThanEqual, U1, uint32_t, uint32_t)
template <spv::Op opcode>
uint32_t EmitFloatCompare32(ValueEmitContext& ctx, const IR::Inst& inst) {
// SPIR-V's DenormFlushToZero mode does not require comparison operands to flush.
const bool flush = inst.Flags<IR::FPCompareFlags>().flush_input_denorms;
const auto operand = [&](size_t index) {
const auto value = inst.Arg(index);
if (flush && value.IsImmediate()) {
const auto bits = std::bit_cast<uint32_t>(value.F32Value());
return ConstantF32(ctx.state, (bits & 0x7fffffffu) < 0x00800000u
? bits & 0x80000000u : bits);
}
const auto id = ctx.Arg(inst, index);
return flush ? EmitFlushF32DenormToSignedZero(ctx.state, id) : id;
};
return EmitNative<opcode, IR::Type::U1>(ctx.state, operand(0), operand(1));
}
inline constexpr auto EmitFPOrdEqual32 = EmitFloatCompare32<spv::OpFOrdEqual>;
inline constexpr auto EmitFPUnordEqual32 = EmitFloatCompare32<spv::OpFUnordEqual>;
inline constexpr auto EmitFPOrdNotEqual32 = EmitFloatCompare32<spv::OpFOrdNotEqual>;
inline constexpr auto EmitFPUnordNotEqual32 = EmitFloatCompare32<spv::OpFUnordNotEqual>;
inline constexpr auto EmitFPOrdLessThan32 = EmitFloatCompare32<spv::OpFOrdLessThan>;
inline constexpr auto EmitFPUnordLessThan32 = EmitFloatCompare32<spv::OpFUnordLessThan>;
inline constexpr auto EmitFPOrdGreaterThan32 = EmitFloatCompare32<spv::OpFOrdGreaterThan>;
inline constexpr auto EmitFPUnordGreaterThan32 = EmitFloatCompare32<spv::OpFUnordGreaterThan>;
inline constexpr auto EmitFPOrdLessThanEqual32 = EmitFloatCompare32<spv::OpFOrdLessThanEqual>;
inline constexpr auto EmitFPUnordLessThanEqual32 = EmitFloatCompare32<spv::OpFUnordLessThanEqual>;
inline constexpr auto EmitFPOrdGreaterThanEqual32 = EmitFloatCompare32<spv::OpFOrdGreaterThanEqual>;
inline constexpr auto EmitFPUnordGreaterThanEqual32 = EmitFloatCompare32<spv::OpFUnordGreaterThanEqual>;
uint32_t EmitFPIsNan32(EmitterState& state, uint32_t arg0);
inline constexpr auto EmitFPCmpClass32 = EmitClassMaskF32;
EMIT_NATIVE(FPAdd32, OpFAdd, F32, uint32_t, uint32_t)
@@ -182,6 +202,7 @@ uint32_t EmitDppUpdateU32(ValueEmitContext& ctx, const IR::Inst& in
uint32_t EmitWqmU64(EmitterState& state, uint32_t value);
uint32_t EmitLaneId(EmitterState& state);
uint32_t EmitBallot(ValueEmitContext& ctx, IR::Value predicate);
uint32_t EmitConditionRef(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitReadFirstLane(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitReadLane(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitWriteLane(ValueEmitContext& ctx, const IR::Inst& inst);
@@ -201,6 +222,7 @@ void EmitLoadMemory(ValueEmitContext& ctx, const IR::Inst& inst
void EmitStoreMemory(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitAtomic32(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitBufferAtomic64(ValueEmitContext& ctx, const IR::Inst& inst);
void EmitSharedAtomicOr64(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitBufferFloatAtomic(ValueEmitContext& ctx, const IR::Inst& inst);
void EmitSharedFloatAtomic(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitSharedIncDec(ValueEmitContext& ctx, const IR::Inst& inst);
@@ -67,6 +67,7 @@ struct SpirvRequirements {
bool function_scratch = false;
bool pixel_valid_mask = false;
bool buffer_int64_atomics = false;
bool shared_int64_atomics = false;
bool coherent_buffers = false;
bool float64 = false;
};
@@ -101,6 +102,7 @@ struct EmitterState {
uint32_t shader_data_storage_variable = 0;
uint32_t flattened_srt_variable = 0;
uint32_t lds_variable = 0;
uint32_t lds_u64_variable = 0;
std::array<uint32_t, 2> scratch_variable {};
std::array<uint32_t, IR::ImageBindingCount> image_variables {};
uint32_t sampler_variable = 0;
@@ -376,6 +378,7 @@ Prospero::BufferFormat StorageBufferFormat(const EmitterState& state, const IR::
void EmitMemoryOffsets(EmitterState& state);
uint32_t LdsDwordCount(const EmitterState& state);
void EnsureLdsStorage(EmitterState& state);
struct MemoryResourceAccess {
IR::ResourceKind kind = IR::ResourceKind::None;
@@ -146,7 +146,12 @@ uint32_t GuestAddress(ValueEmitContext& ctx, const IR::Inst& inst, const IR::Mem
ctx.Fail(inst, "has no address base pair");
return ConstantDeviceAddress(state, 0);
}
const auto base = DeviceAddressFromWords(state, ctx.Arg(*handle, 0), ctx.Arg(*handle, 1));
auto base_low = ctx.Arg(*handle, 0);
if (mem.kind == IR::ResourceKind::ScalarAddress) {
base_low = Binary(state, spv::OpBitwiseAnd, TypeU32(state), base_low,
ConstantU32(state, ~3u));
}
const auto base = DeviceAddressFromWords(state, base_low, ctx.Arg(*handle, 1));
address = Binary(state, spv::OpIAdd, TypeScalarU64(state), base,
Unary(state, spv::OpUConvert, TypeScalarU64(state), low));
}
@@ -543,6 +548,7 @@ spv::Op SpirvAtomicOpcode(IR::ValueOpcode opcode) {
case IR::ValueOpcode::SharedAtomicAnd32: return spv::OpAtomicAnd;
case IR::ValueOpcode::BufferAtomicOr32:
case IR::ValueOpcode::BufferAtomicOr64:
case IR::ValueOpcode::SharedAtomicOr64:
case IR::ValueOpcode::SharedAtomicOr32: return spv::OpAtomicOr;
case IR::ValueOpcode::BufferAtomicXor32:
case IR::ValueOpcode::SharedAtomicXor32: return spv::OpAtomicXor;
@@ -970,7 +976,14 @@ void DefineGetBdaPointer(EmitterState& state) {
state.builder.AddFunction(spv::OpFunctionParameter, type, address);
EmitLabel(state, entry_label);
const auto page64 = Binary(state, spv::OpShiftRightLogical, type, address,
const auto extended = Binary(state, spv::OpUGreaterThanEqual, TypeBool(state), address,
ConstantDeviceAddress(state, LibKernel::Memory::kExtendedMemoryBase));
const auto packed = Select(state, type, extended,
Binary(state, spv::OpISub, type, address,
ConstantDeviceAddress(state,
LibKernel::Memory::kExtendedMemoryBase - LOWER_ADDRESS_SIZE)),
address);
const auto page64 = Binary(state, spv::OpShiftRightLogical, type, packed,
ConstantDeviceAddress(state, BufferCache::CACHING_PAGEBITS));
const auto page = Unary(state, spv::OpUConvert, TypeU32(state), page64);
const auto entry_pointer = state.builder.AllocateId();
@@ -1052,6 +1065,32 @@ uint32_t EmitBufferAtomic64(ValueEmitContext& ctx, const IR::Inst& inst) {
});
}
void EmitSharedAtomicOr64(ValueEmitContext& ctx, const IR::Inst& inst) {
auto& state = ctx.state;
const auto& mem = ctx.Memory(inst);
EnsureLdsStorage(state);
EmitIfCondition(state, ctx.Arg(inst, 2), [&]() {
const auto address = Binary(state, spv::OpBitwiseAnd, TypeU32(state),
ByteAddress(ctx, inst, mem), ConstantU32(state, 0xfff8u));
const auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), address,
ConstantU32(state, 3u));
const auto in_bounds = Binary(state, spv::OpULessThan, TypeBool(state), index,
ConstantU32(state, LdsDwordCount(state) / 2u));
EmitIfCondition(state, in_bounds, [&]() {
const auto pointer = state.builder.AllocateId();
state.builder.AddFunction(spv::OpAccessChain,
TypePointer(state, spv::StorageClassWorkgroup, TypeScalarU64(state)),
pointer, state.lds_u64_variable, ConstantU32(state, 0), index);
const auto value = Unary(state, spv::OpBitcast, TypeScalarU64(state), ctx.Arg(inst, 1));
state.builder.AddFunction(
SpirvAtomicOpcode(inst.GetOpcode()), TypeScalarU64(state), state.builder.AllocateId(),
pointer, ConstantU32(state, spv::ScopeWorkgroup),
ConstantU32(state, spv::MemorySemanticsAcquireReleaseMask |
spv::MemorySemanticsWorkgroupMemoryMask), value);
});
});
}
uint32_t EmitBufferFloatAtomic(ValueEmitContext& ctx, const IR::Inst& inst) {
const auto& mem = ctx.Memory(inst);
const bool max_value = inst.GetOpcode() == IR::ValueOpcode::BufferAtomicFMax32;
@@ -1099,16 +1138,27 @@ uint32_t EmitAppendConsume(ValueEmitContext& ctx, const IR::Inst& inst) {
if (ctx.half == 1) {
return ctx.other_half->Def(IR::Value(const_cast<IR::Inst*>(&inst)));
}
const auto m0 = ctx.Arg(inst, 0);
const auto base =
Binary(state, spv::OpShiftRightLogical, TypeU32(state), m0, ConstantU32(state, 16));
const auto size =
Binary(state, spv::OpBitwiseAnd, TypeU32(state), m0, ConstantU32(state, 0xffffu));
const auto address = Binary(state, spv::OpIAdd, TypeU32(state), base,
ConstantU32(state, ctx.Memory(inst).offset));
const auto mem = ctx.Memory(inst);
const auto offset = mem.offset & 0xfffcu;
auto address = ConstantU32(state, offset);
uint32_t region_bounds = 0;
if (mem.kind == IR::ResourceKind::Gds) {
const auto m0 = ctx.Arg(inst, 0);
const auto base =
Binary(state, spv::OpShiftRightLogical, TypeU32(state), m0, ConstantU32(state, 16));
const auto size =
Binary(state, spv::OpBitwiseAnd, TypeU32(state), m0, ConstantU32(state, 0xffffu));
address = Binary(state, spv::OpIAdd, TypeU32(state), base, address);
// The entire M0 region must fit the PS5's 48 KiB GDS partition.
region_bounds = AndCondition(
state, Binary(state, spv::OpULessThan, TypeBool(state),
ConstantU32(state, offset + 3u), size),
Binary(state, spv::OpULessThanEqual, TypeBool(state),
Binary(state, spv::OpIAdd, TypeU32(state), base, size),
ConstantU32(state, 0xc000u)));
}
const auto raw_index =
Binary(state, spv::OpShiftRightLogical, TypeU32(state), address, ConstantU32(state, 2));
const auto mem = ctx.Memory(inst);
const auto access = PrepareMemoryResourceAccess(state, mem);
const auto index = EmitMemoryElementIndex(state, access, raw_index);
const auto exec = ctx.Arg(inst, 1);
@@ -1127,19 +1177,17 @@ uint32_t EmitAppendConsume(ValueEmitContext& ctx, const IR::Inst& inst) {
: first;
const auto is_first = Binary(state, spv::OpIEqual, TypeBool(state),
EmitSubgroupLocalInvocationId(state), source_lane);
const auto storage_bounds = EmitMemoryElementInBounds(state, access, index);
const auto m0_bounds =
mem.kind == IR::ResourceKind::Gds
? Binary(state, spv::OpINotEqual, TypeBool(state), size, ConstantU32(state, 0))
: Binary(state, spv::OpULessThan, TypeBool(state),
ConstantU32(state, ctx.Memory(inst).offset + 3u), size);
auto bounds = EmitMemoryElementInBounds(state, access, index);
if (mem.kind == IR::ResourceKind::Gds) {
bounds = AndCondition(state, bounds, region_bounds);
}
const auto condition = AndCondition(
state, is_first,
AndCondition(state,
state.lane_count == 2 ? Binary(state, spv::OpINotEqual, TypeBool(state), count,
ConstantU32(state, 0))
: exec,
AndCondition(state, storage_bounds, m0_bounds)));
bounds));
const auto atomic = EmitValueOrZeroIfCondition(state, condition, [&]() {
const auto value = state.builder.AllocateId();
state.builder.AddFunction(append ? spv::OpAtomicIAdd : spv::OpAtomicISub, TypeU32(state),
@@ -1173,10 +1221,11 @@ void EmitReadConstBuffer(ValueEmitContext& ctx, const IR::Inst& inst) {
if (mem.planning_only) return;
auto& state = ctx.state;
mem.kind = IR::ResourceKind::ScalarBuffer;
const auto address = Binary(state, spv::OpIAdd, TypeU32(state), ctx.Arg(inst, 1),
ConstantU32(state, mem.offset));
const auto index =
Binary(state, spv::OpShiftRightLogical, TypeU32(state), address, ConstantU32(state, 2));
auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), ctx.Arg(inst, 1),
ConstantU32(state, 2));
if (mem.offset >= 4u) {
index = Binary(state, spv::OpIAdd, TypeU32(state), index, ConstantU32(state, mem.offset >> 2u));
}
const auto access = PrepareMemoryResourceAccess(state, mem);
const auto element = EmitMemoryElementIndex(state, access, index);
const auto condition = EmitMemoryElementInBounds(state, access, element);
@@ -1198,6 +1247,8 @@ void EmitLoadMemory(ValueEmitContext& ctx, const IR::Inst& inst) {
value = LoadWideBuffer(ctx, inst, buffer_components);
else if (shared_components > 1u)
value = LoadWideShared(ctx, inst, shared_components);
else if (mem.kind == IR::ResourceKind::ScalarAddress)
value = LoadBdaDword(ctx, GuestAddress(ctx, inst, mem));
else if (address_info.access == IR::AddressAccess::Read &&
mem.kind != IR::ResourceKind::Scratch)
value = LoadBda(ctx, GuestAddress(ctx, inst, mem), ctx.Arg(inst, inst.NumArgs() - 1),
@@ -63,16 +63,34 @@ uint32_t LdsDwordCount(const EmitterState& state) {
return workgroup != nullptr ? workgroup->lds_size_dwords : 8192u;
}
static void EnsureLdsStorage(EmitterState& state) {
void EnsureLdsStorage(EmitterState& state) {
if (state.lds_variable != 0) {
return;
}
if (ShaderWorkgroupInput(state.program.stage, state.input_info) == nullptr) {
EXIT("function LDS was not prepared before SPIR-V function emission\n");
}
state.lds_variable = state.builder.DefineGlobalVariable(
TypeU32ArrayPointer(state, spv::StorageClassWorkgroup, LdsDwordCount(state)),
spv::StorageClassWorkgroup);
const auto define = [&](uint32_t type, uint32_t bytes) {
const auto array = state.builder.DecoratedType(
spv::OpTypeArray, {{spv::OpDecorate, {spv::DecorationArrayStride, bytes}}}, type,
ConstantU32(state, std::max(LdsDwordCount(state) * 4u / bytes, 1u)));
const auto block = state.builder.DecoratedType(
spv::OpTypeStruct, {{spv::OpMemberDecorate, {0, spv::DecorationOffset, 0}},
{spv::OpDecorate, {spv::DecorationBlock}}}, array);
const auto variable = state.builder.DefineGlobalVariable(
TypePointer(state, spv::StorageClassWorkgroup, block), spv::StorageClassWorkgroup);
state.builder.AddAnnotation(spv::OpDecorate, variable, spv::DecorationAliased);
return variable;
};
if (state.requirements.shared_int64_atomics) {
state.lds_variable = define(TypeU32(state), 4u);
state.lds_u64_variable = define(TypeScalarU64(state), 8u);
state.builder.AddName(state.lds_u64_variable, "lds_qwords");
} else {
state.lds_variable = state.builder.DefineGlobalVariable(
TypeU32ArrayPointer(state, spv::StorageClassWorkgroup, LdsDwordCount(state)),
spv::StorageClassWorkgroup);
}
state.builder.AddName(state.lds_variable, "lds_dwords");
}
@@ -167,6 +185,11 @@ uint32_t EmitMemoryElementPointer(EmitterState& state, const MemoryResourceAcces
: ShaderWorkgroupInput(state.program.stage, state.input_info) != nullptr
? spv::StorageClassWorkgroup
: spv::StorageClassFunction;
if (access.kind == IR::ResourceKind::Lds && state.requirements.shared_int64_atomics) {
state.builder.AddFunction(spv::OpAccessChain, TypeU32ElementPointer(state, storage_class),
pointer, access.object_pointer, ConstantU32(state, 0), index);
return pointer;
}
state.builder.AddFunction(spv::OpAccessChain, TypeU32ElementPointer(state, storage_class),
pointer, access.object_pointer, index);
return pointer;
@@ -626,10 +626,15 @@ void DefineModule(EmitterState& state) {
state.builder.RequireCapability(spv::CapabilityPhysicalStorageBufferAddresses);
state.builder.RequireExtension("SPV_KHR_physical_storage_buffer");
}
if (state.requirements.buffer_int64_atomics) {
if (state.requirements.buffer_int64_atomics || state.requirements.shared_int64_atomics) {
state.builder.RequireCapability(spv::CapabilityInt64);
state.builder.RequireCapability(spv::CapabilityInt64Atomics);
}
if (state.requirements.shared_int64_atomics) {
state.builder.RequireVersion(0x00010400u);
state.builder.RequireExtension("SPV_KHR_workgroup_memory_explicit_layout");
state.builder.RequireCapability(spv::CapabilityWorkgroupMemoryExplicitLayoutKHR);
}
if (state.clip_distance_variable != 0) {
state.builder.RequireCapability(spv::CapabilityClipDistance);
}
@@ -684,8 +689,8 @@ void DefineModule(EmitterState& state) {
state.builder.AddExecutionMode(state.main_func, spv::ExecutionModeSignedZeroInfNanPreserve,
32u);
if (state.requirements.float64) {
EXIT_NOT_IMPLEMENTED(state.program.stage != ShaderType::Compute);
EXIT_NOT_IMPLEMENTED(state.input_info.compute->float_mode != 0xc0);
EXIT_NOT_IMPLEMENTED(state.program.stage == ShaderType::Compute &&
state.input_info.compute->float_mode != 0xc0);
// MODE=0xc0 uses round-to-nearest-even and preserves FP64 input/output denormals.
state.builder.RequireCapability(spv::CapabilityFloat64);
state.builder.RequireCapability(spv::CapabilityRoundingModeRTE);
@@ -97,30 +97,6 @@ void EmitReturn(ValueEmitContext& ctx) {
ctx.state.builder.AddFunction(spv::OpReturn);
}
uint32_t BranchCondition(ValueEmitContext& ctx, const IR::BlockInfo& info) {
// Scalar-instruction conditions already test the full wave's raw register values.
if (ctx.other_half == nullptr ||
info.terminator.condition == CFG::BranchCondition::ScalarInstruction ||
info.terminator.condition == CFG::BranchCondition::GotoVariable) {
return ctx.Def(info.condition);
}
const auto ballot = ctx.Ballot(info.condition);
const auto low = ctx.state.builder.AllocateId();
const auto high = ctx.state.builder.AllocateId();
const auto result = ctx.state.builder.AllocateId();
ctx.state.builder.AddFunction(spv::OpCompositeExtract, TypeU32(ctx.state), low, ballot, 0);
ctx.state.builder.AddFunction(spv::OpCompositeExtract, TypeU32(ctx.state), high, ballot, 1);
const auto kind = info.terminator.condition;
const bool zero = kind == CFG::BranchCondition::ExecZero ||
kind == CFG::BranchCondition::VccZero ||
kind == CFG::BranchCondition::SccZero;
const auto combined =
EmitBinaryU32(ctx.state, zero ? spv::OpBitwiseAnd : spv::OpBitwiseOr, low, high);
ctx.state.builder.AddFunction(zero ? spv::OpIEqual : spv::OpINotEqual, TypeBool(ctx.state),
result, combined, ConstantU32(ctx.state, zero ? ~0u : 0u));
return result;
}
void EmitStructuredTerminator(ValueEmitContext& ctx, const IR::Block* block,
const IR::BlockInfo& info) {
const auto& program = ctx.state.program;
@@ -160,7 +136,7 @@ void EmitStructuredTerminator(ValueEmitContext& ctx, const IR::Block* block,
EmitReturn(ctx);
return;
}
const auto condition = BranchCondition(ctx, info);
const auto condition = ctx.Def(info.condition);
emit_merge();
ctx.state.builder.AddFunction(spv::OpBranchConditional, condition,
ctx.Label(true_block), ctx.Label(false_block));
@@ -193,7 +169,7 @@ uint32_t EmitDispatcherNextPc(ValueEmitContext& ctx, const DispatcherFunctionSta
EmitDispatcherTarget(ctx, dispatcher, block, term.false_block);
const auto selected = ctx.state.builder.AllocateId();
ctx.state.builder.AddFunction(
spv::OpSelect, TypeU32(ctx.state), selected, BranchCondition(ctx, info),
spv::OpSelect, TypeU32(ctx.state), selected, ctx.Def(info.condition),
ConstantU32(ctx.state, term.true_block), ConstantU32(ctx.state, term.false_block));
return selected;
}
File diff suppressed because it is too large Load Diff
@@ -18,10 +18,17 @@ enum class BranchCondition {
ExecZero,
ExecNonZero,
ScalarInstruction,
GotoVariable,
Expression,
Unknown
};
struct ConditionExpression {
enum class Op { Constant, Variable, Native, Not, Or };
Op op;
uint32_t lhs;
uint32_t rhs = UINT32_MAX;
};
enum class TerminatorKind { Branch, ConditionalBranch, IndirectBranch, Return, Unsupported };
enum class FailureKind {
@@ -48,8 +55,7 @@ struct Terminator {
std::vector<uint32_t> indirect_targets;
std::vector<uint32_t> indirect_selector_values;
std::vector<uint32_t> indirect_selector_targets;
uint32_t goto_variable = UINT32_MAX;
int32_t goto_value = -1;
uint32_t expression = UINT32_MAX;
bool loop_header = false;
};
@@ -59,10 +65,11 @@ struct BasicBlock {
uint32_t end_pc = 0;
uint32_t inst_begin = 0;
uint32_t inst_end = 0;
struct Assignment { uint32_t variable; uint32_t expression; };
std::vector<Assignment> assignments;
std::vector<uint32_t> predecessors;
std::vector<uint32_t> successors;
std::vector<uint32_t> dominators;
std::vector<uint32_t> post_dominators;
Terminator terminator;
};
@@ -73,12 +80,9 @@ struct BackEdge {
};
struct NaturalLoop {
uint32_t header = UINT32_MAX;
uint32_t latch = UINT32_MAX;
uint32_t merge = UINT32_MAX;
uint32_t continue_block = UINT32_MAX;
uint32_t header = UINT32_MAX;
uint32_t latch = UINT32_MAX;
std::vector<uint32_t> body_blocks;
std::vector<uint32_t> exit_blocks;
};
struct StronglyConnectedComponent {
@@ -89,6 +93,7 @@ struct StronglyConnectedComponent {
struct Graph {
std::vector<BasicBlock> blocks;
std::vector<ConditionExpression> expressions;
std::vector<BackEdge> back_edges;
std::vector<NaturalLoop> natural_loops;
std::vector<StronglyConnectedComponent> components;
@@ -105,14 +110,12 @@ struct Graph {
const BasicBlock* FindBlockByPc(uint32_t pc) const;
BasicBlock* FindBlockByPc(uint32_t pc);
bool Dominates(uint32_t dominator, uint32_t block) const;
bool PostDominates(uint32_t post_dominator, uint32_t block) const;
uint32_t FindNearestCommonPostDominator(uint32_t block_a, uint32_t block_b) const;
};
Graph BuildGraph(const Decoder::Program& program);
// Commits structured control flow on success; preserves the original graph with
// failure diagnostics on failure. failure_block is an original block ID or UINT32_MAX.
bool Structurize(Graph& graph);
// Returns structured control flow or failure diagnostics without changing the native graph.
// On failure, failure_block is an original block ID or UINT32_MAX.
Graph Structurize(const Graph& graph);
std::string BranchConditionToString(BranchCondition condition);
std::string FailureKindToString(FailureKind kind);
std::string GraphToString(const Graph& graph);
@@ -106,6 +106,7 @@ constexpr MemoryOpcodeInfo DS_OPCODE_LIST[] = {
{0x39u, Opcode::DS_READ_I8, 1, 8, true}, {0x3au, Opcode::DS_READ_U8, 1, 8},
{0x3bu, Opcode::DS_READ_I16, 1, 16, true}, {0x3cu, Opcode::DS_READ_U16, 1, 16},
{0x3du, Opcode::DS_CONSUME, 1, 32}, {0x3eu, Opcode::DS_APPEND, 1, 32},
{0x4au, Opcode::DS_OR_B64, 2, 32},
{0x4du, Opcode::DS_WRITE_B64, 2, 32}, {0x4eu, Opcode::DS_WRITE2_B64, 4, 32},
{0x4fu, Opcode::DS_WRITE2ST64_B64, 4, 32}, {0x76u, Opcode::DS_READ_B64, 2, 32},
{0x77u, Opcode::DS_READ2_B64, 4, 32}, {0x78u, Opcode::DS_READ2ST64_B64, 4, 32},
@@ -185,6 +186,7 @@ bool IsDsAtomicOpcode(Opcode opcode) {
case Opcode::DS_AND_B32:
case Opcode::DS_AND_RTN_B32:
case Opcode::DS_OR_B32:
case Opcode::DS_OR_B64:
case Opcode::DS_OR_RTN_B32:
case Opcode::DS_XOR_B32:
case Opcode::DS_XOR_RTN_B32:
@@ -681,6 +681,7 @@ std::string InstructionToString(const Instruction& inst) {
case Opcode::DS_AND_B32:
case Opcode::DS_AND_RTN_B32:
case Opcode::DS_OR_B32:
case Opcode::DS_OR_B64:
case Opcode::DS_OR_RTN_B32:
case Opcode::DS_XOR_B32:
case Opcode::DS_XOR_RTN_B32:
@@ -156,6 +156,8 @@ enum class Opcode {
V_DOT2C_F32_F16,
V_CVT_F64_I32,
V_CVT_F32_F64,
V_CVT_F64_F32,
V_CVT_F64_U32,
V_CVT_F32_I32,
V_CVT_F32_U32,
V_CVT_U32_F32,
@@ -397,6 +399,7 @@ enum class Opcode {
V_CMPX_GT_F16,
V_CMPX_GE_F16,
V_CMPX_NGT_F16,
V_CMPX_NLE_F16,
V_CMPX_NEQ_F16,
V_CMPX_NLT_F16,
V_CMPX_LT_I32,
@@ -525,6 +528,7 @@ enum class Opcode {
DS_AND_B32,
DS_AND_RTN_B32,
DS_OR_B32,
DS_OR_B64,
DS_OR_RTN_B32,
DS_XOR_B32,
DS_XOR_RTN_B32,
@@ -89,6 +89,8 @@ constexpr OpcodeMap VOP1_OPCODE_LIST[] = {
{0x02u, Opcode::V_READFIRSTLANE_B32},
{0x04u, Opcode::V_CVT_F64_I32},
{0x0fu, Opcode::V_CVT_F32_F64},
{0x10u, Opcode::V_CVT_F64_F32},
{0x16u, Opcode::V_CVT_F64_U32},
{0x2fu, Opcode::V_RCP_F64},
{0x05u, Opcode::V_CVT_F32_I32},
{0x06u, Opcode::V_CVT_F32_U32},
@@ -151,6 +153,8 @@ constexpr OpcodeMap VOP3_ENCODED_VOP1_OPCODE_LIST[] = {
{0x02u, Opcode::V_READFIRSTLANE_B32},
{0x04u, Opcode::V_CVT_F64_I32},
{0x0fu, Opcode::V_CVT_F32_F64},
{0x10u, Opcode::V_CVT_F64_F32},
{0x16u, Opcode::V_CVT_F64_U32},
{0x2fu, Opcode::V_RCP_F64},
{0x05u, Opcode::V_CVT_F32_I32},
{0x06u, Opcode::V_CVT_F32_U32},
@@ -258,7 +262,8 @@ constexpr VopcOpcodeInfo VOPC_OPCODE_LIST[] = {
{0xd9u, Opcode::V_CMPX_LT_F16}, {0xdau, Opcode::V_CMPX_EQ_F16},
{0xdbu, Opcode::V_CMPX_LE_F16}, {0xdcu, Opcode::V_CMPX_GT_F16},
{0xdeu, Opcode::V_CMPX_GE_F16}, {0xfbu, Opcode::V_CMPX_NGT_F16},
{0xfdu, Opcode::V_CMPX_NEQ_F16}, {0xfeu, Opcode::V_CMPX_NLT_F16},
{0xfcu, Opcode::V_CMPX_NLE_F16}, {0xfdu, Opcode::V_CMPX_NEQ_F16},
{0xfeu, Opcode::V_CMPX_NLT_F16},
};
constexpr auto VOPC_OPS = Detail::MakeOpcodeTable<0x100>(VOPC_OPCODE_LIST);
@@ -461,6 +466,7 @@ bool IsVopcCompareExec(Opcode opcode);
bool IsVop1FloatSourceOpcode(Opcode opcode) {
switch (opcode) {
case Opcode::V_CVT_F32_F64:
case Opcode::V_CVT_F64_F32:
case Opcode::V_RCP_F64:
case Opcode::V_MOV_B32:
case Opcode::V_CVT_F32_F16:
@@ -721,6 +727,7 @@ void ApplyDppModifier(Operand& operand, uint32_t modifier, uint32_t encoding) {
void DecodeVop1Dpp(uint32_t pc, std::span<const uint32_t> code, uint32_t word_index,
uint32_t opcode, uint32_t vdst, Instruction& inst) {
if (inst.opcode == Opcode::V_CVT_F64_I32 || inst.opcode == Opcode::V_CVT_F32_F64 ||
inst.opcode == Opcode::V_CVT_F64_F32 || inst.opcode == Opcode::V_CVT_F64_U32 ||
inst.opcode == Opcode::V_RCP_F64) {
SetUnsupported(inst, Family::VOP1, opcode, "FP64 instructions do not support DPP");
return;
@@ -862,6 +869,7 @@ bool IsVopcFloatCompareOpcode(Opcode opcode) {
case Opcode::V_CMPX_GT_F16:
case Opcode::V_CMPX_GE_F16:
case Opcode::V_CMPX_NGT_F16:
case Opcode::V_CMPX_NLE_F16:
case Opcode::V_CMPX_NEQ_F16:
case Opcode::V_CMPX_NLT_F16:
case Opcode::V_CMP_CLASS_F32:
@@ -1488,6 +1496,7 @@ bool IsVopcCompareExec(Opcode opcode) {
case Opcode::V_CMPX_GT_F16:
case Opcode::V_CMPX_GE_F16:
case Opcode::V_CMPX_NGT_F16:
case Opcode::V_CMPX_NLE_F16:
case Opcode::V_CMPX_NEQ_F16:
case Opcode::V_CMPX_NLT_F16: return true;
default: return false;
@@ -44,7 +44,8 @@ void Translator::EmitFloatCompare(const Decoder::Instruction& inst, IR::ValueOpc
half ? IR::Value(ReadF16AsF32(inst.src0)) : ReadOperand(inst.src0, IR::Type::F32);
const auto rhs =
half ? IR::Value(ReadF16AsF32(inst.src1)) : ReadOperand(inst.src1, IR::Type::F32);
EmitCompareResult(inst, IR::U1(ir.Emit(opcode, {lhs, rhs})), false, cmpx);
const IR::FPCompareFlags flags{.flush_input_denorms = !half && flush_f32_inputs};
EmitCompareResult(inst, IR::U1(ir.Emit(opcode, {lhs, rhs}, flags)), false, cmpx);
}
void Translator::EmitFloatOrderedCompare(const Decoder::Instruction& inst, bool ordered, bool cmpx) {
@@ -235,6 +235,7 @@ Decoder::Operand MemorySourceAt(const Decoder::Instruction& decoded, uint32_t in
return index == 0u ? decoded.src0 : MakeImmediate(decoded.offset & 0xffffu);
case Decoder::Opcode::DS_CONSUME:
case Decoder::Opcode::DS_APPEND:
return decoded.gds ? MakeM0Operand() : MakeImmediate(0);
case Decoder::Opcode::DS_READ_ADDTID_B32: return MakeM0Operand();
case Decoder::Opcode::DS_WRITE_ADDTID_B32:
return index == 0u ? decoded.src1 : MakeM0Operand();
@@ -569,7 +570,10 @@ void Translator::DS_ATOMIC(const Decoder::Instruction& inst, IR::ValueOpcode opc
bool returns_value) {
const auto memory = MemoryInfoFromDecoded(inst);
const auto address = ReadU32(MemorySourceAt(inst, 1));
const auto result = ir.Emit(opcode, {address, ReadU32(MemorySourceAt(inst, 0)), ir.GetExec()},
const auto data_src = MemorySourceAt(inst, 0);
const IR::Value value =
memory.data_dwords == 2u ? IR::Value(ReadU64(data_src)) : IR::Value(ReadU32(data_src));
const auto result = ir.Emit(opcode, {address, value, ir.GetExec()},
AddMemoryInfo(memory, inst.pc));
if (returns_value) {
WriteOperand(inst.dst, result);
@@ -845,8 +849,7 @@ void Translator::DS_MINMAX_F32(const Decoder::Instruction& inst, IR::ValueOpcode
void Translator::DS_APPEND_CONSUME(const Decoder::Instruction& inst, IR::ValueOpcode opcode) {
const auto memory = MemoryInfoFromDecoded(inst);
WriteOperand(inst.dst, ir.Emit(opcode,
{ReadU32(MemorySourceAt(inst, 0)), ir.GetExec(), ir.GetExecLo(),
ir.GetExecHi()},
{ReadU32(MemorySourceAt(inst, 0)), ir.GetExec()},
AddMemoryInfo(memory, inst.pc)));
}
@@ -1003,6 +1006,8 @@ void Translator::EmitMemory(const Decoder::Instruction& inst) {
return DS_ATOMIC(inst, IR::ValueOpcode::SharedAtomicAnd32, true);
case Decoder::Opcode::DS_OR_B32:
return DS_ATOMIC(inst, IR::ValueOpcode::SharedAtomicOr32, false);
case Decoder::Opcode::DS_OR_B64:
return DS_ATOMIC(inst, IR::ValueOpcode::SharedAtomicOr64, false);
case Decoder::Opcode::DS_OR_RTN_B32:
return DS_ATOMIC(inst, IR::ValueOpcode::SharedAtomicOr32, true);
case Decoder::Opcode::DS_XOR_B32:
@@ -317,6 +317,7 @@ void Translator::WriteRawU32(const Decoder::Operand& operand, IR::U32 value) {
case Decoder::OperandKind::Sgpr: {
const auto reg = static_cast<IR::ScalarReg>(operand.reg);
ir.SetScalarReg(reg, value);
program.scalar_writes.push_back({current_pc, reg});
ir.SetScalarMaskTag(reg, IR::U1(IR::Value(false)));
if (IR::RegIndex(reg) > 0u) {
ir.SetScalarMaskTag(static_cast<IR::ScalarReg>(IR::RegIndex(reg) - 1u),
@@ -699,11 +700,13 @@ std::array<IR::U32, 2> Translator::WriteMask(const Decoder::Operand& operand, IR
IR::U1(IR::Value(false)));
}
ir.SetScalarReg(reg, mask[0]);
program.scalar_writes.push_back({current_pc, reg});
// A wave32 VALU mask destination must not overwrite the neighboring SGPR.
if ((write_64 || program.wave_size == 64u) &&
IR::RegIndex(reg) + 1u < IR::NumScalarRegs) {
const auto high = static_cast<IR::ScalarReg>(IR::RegIndex(reg) + 1u);
ir.SetScalarReg(high, mask[1]);
program.scalar_writes.push_back({current_pc, high});
ir.SetThreadBitScalarReg(high, IR::U1(IR::Value(false)));
ir.SetScalarMaskTag(high, IR::U1(IR::Value(false)));
}
@@ -746,55 +749,61 @@ void Translator::WriteCompareResult(const Decoder::Operand& operand, IR::U1 valu
WriteMask(operand, ir.LogicalAnd(ir.GetExec(), value));
}
void Translator::AddBranchCondition(const CFG::BasicBlock& source, IR::BlockInfo& info) {
if (source.terminator.goto_value >= 0) {
if (source.terminator.goto_variable == UINT32_MAX) {
EXIT("block %u sets an invalid goto variable", source.id);
void Translator::AddBranchCondition(const CFG::Graph& graph, const CFG::BasicBlock& source,
IR::BlockInfo& info) {
const auto native_condition = [&](CFG::BranchCondition kind) -> IR::U1 {
IR::U1 condition;
switch (kind) {
case CFG::BranchCondition::Always: condition = IR::U1(IR::Value(true)); break;
case CFG::BranchCondition::SccZero: condition = ir.LogicalNot(ir.GetScc()); break;
case CFG::BranchCondition::SccNonZero: condition = ir.GetScc(); break;
case CFG::BranchCondition::VccZero: condition = ir.LogicalNot(ir.GetVcc()); break;
case CFG::BranchCondition::VccNonZero: condition = ir.GetVcc(); break;
case CFG::BranchCondition::ExecZero: condition = ir.LogicalNot(ir.GetExec()); break;
case CFG::BranchCondition::ExecNonZero: condition = ir.GetExec(); break;
case CFG::BranchCondition::ScalarInstruction:
EXIT_IF(instruction_branch_condition.IsEmpty());
condition = instruction_branch_condition;
break;
default: EXIT("block %u has an invalid native branch condition", source.id);
}
ir.SetGotoVariable(source.terminator.goto_variable,
IR::U1(IR::Value(source.terminator.goto_value != 0)));
return IR::U1(ir.Emit(IR::ValueOpcode::ConditionRef, {condition}, kind));
};
const auto expression = [&](auto&& self, uint32_t index) -> IR::U1 {
const auto& value = graph.expressions.at(index);
switch (value.op) {
case CFG::ConditionExpression::Op::Constant: return IR::U1(IR::Value(value.lhs != 0));
case CFG::ConditionExpression::Op::Variable: return ir.GetGotoVariable(value.lhs);
case CFG::ConditionExpression::Op::Native:
return native_condition(static_cast<CFG::BranchCondition>(value.lhs));
case CFG::ConditionExpression::Op::Not: return ir.LogicalNot(self(self, value.lhs));
case CFG::ConditionExpression::Op::Or:
return ir.LogicalOr(self(self, value.lhs), self(self, value.rhs));
}
EXIT("invalid CFG condition expression");
};
for (const auto& assignment: source.assignments) {
ir.SetGotoVariable(assignment.variable, expression(expression, assignment.expression));
}
if (source.terminator.kind == CFG::TerminatorKind::IndirectBranch) {
if (source.terminator.indirect_selector_code != UINT32_MAX) {
info.indirect_target = ReadScalarCode(source.terminator.indirect_selector_code);
} else if (source.terminator.indirect_pc_sgpr != UINT32_MAX) {
info.indirect_target =
ir.GetScalarReg(static_cast<IR::ScalarReg>(source.terminator.indirect_pc_sgpr));
const auto& term = source.terminator;
if (term.kind == CFG::TerminatorKind::IndirectBranch) {
if (term.indirect_selector_code != UINT32_MAX) {
info.indirect_target = ReadScalarCode(term.indirect_selector_code);
} else if (term.indirect_pc_sgpr != UINT32_MAX) {
info.indirect_target = ir.GetScalarReg(static_cast<IR::ScalarReg>(term.indirect_pc_sgpr));
} else {
EXIT("block %u has no indirect branch selector", source.id);
}
ir.Emit(IR::ValueOpcode::ReferenceU32, {info.indirect_target});
return;
}
if (source.terminator.kind != CFG::TerminatorKind::ConditionalBranch) {
return;
if (term.kind == CFG::TerminatorKind::ConditionalBranch) {
const auto condition = term.expression != UINT32_MAX
? expression(expression, term.expression)
: native_condition(term.condition);
info.condition = condition;
ir.Emit(IR::ValueOpcode::Reference, {condition});
}
// EXEC and VCC are invocation-local Boolean masks. Branching on that Boolean lets inactive
// invocations leave the region without reconstructing a host-subgroup mask.
IR::U1 condition;
switch (source.terminator.condition) {
case CFG::BranchCondition::Always: condition = IR::U1(IR::Value(true)); break;
case CFG::BranchCondition::SccZero: condition = ir.LogicalNot(ir.GetScc()); break;
case CFG::BranchCondition::SccNonZero: condition = ir.GetScc(); break;
case CFG::BranchCondition::VccZero: condition = ir.LogicalNot(ir.GetVcc()); break;
case CFG::BranchCondition::VccNonZero: condition = ir.GetVcc(); break;
case CFG::BranchCondition::ExecZero: condition = ir.LogicalNot(ir.GetExec()); break;
case CFG::BranchCondition::ExecNonZero: condition = ir.GetExec(); break;
case CFG::BranchCondition::ScalarInstruction:
EXIT_IF(instruction_branch_condition.IsEmpty());
condition = instruction_branch_condition;
break;
case CFG::BranchCondition::GotoVariable:
if (source.terminator.goto_variable == UINT32_MAX) {
EXIT("block %u reads an invalid goto variable", source.id);
}
condition = ir.GetGotoVariable(source.terminator.goto_variable);
break;
case CFG::BranchCondition::Unknown:
EXIT("block %u has an unknown branch condition", source.id);
}
info.condition = condition;
ir.Emit(IR::ValueOpcode::Reference, {condition});
}
namespace {
@@ -867,7 +876,9 @@ void IncludeInstructionVectorRegisters(const Decoder::Instruction& inst, uint32_
include_vector(inst.src2);
include_vector(inst.src3);
switch (inst.opcode) {
case Decoder::Opcode::V_CVT_F64_I32: include_vector(inst.dst, 2u); break;
case Decoder::Opcode::V_CVT_F64_I32:
case Decoder::Opcode::V_CVT_F64_F32:
case Decoder::Opcode::V_CVT_F64_U32: include_vector(inst.dst, 2u); break;
case Decoder::Opcode::V_FMA_F64: include_vector(inst.src2, 2u); [[fallthrough]];
case Decoder::Opcode::V_MUL_F64: include_vector(inst.src1, 2u); [[fallthrough]];
case Decoder::Opcode::V_RCP_F64: include_vector(inst.dst, 2u); [[fallthrough]];
@@ -1272,9 +1283,11 @@ IR::Program TranslateProgram(const Decoder::Program& decoded, const CFG::Graph&
builtin(IR::StageInputKind::InstanceIndex));
}
}
const bool flush_f32_inputs = options.stage == ShaderType::Compute &&
(options.input_info.compute->float_mode & 0x10u) == 0;
for (const auto& cfg_block: cfg.blocks) {
const auto typed_index = block_indices.at(cfg_block.id);
Translator translator(result, result.blocks[typed_index], vector_limit);
Translator translator(result, result.blocks[typed_index], vector_limit, flush_f32_inputs);
for (uint32_t index = cfg_block.inst_begin; index < cfg_block.inst_end; index++) {
const auto& instruction = decoded.instructions[index];
if (IsCodeTableLoad(cfg, instruction.pc)) {
@@ -1297,7 +1310,7 @@ IR::Program TranslateProgram(const Decoder::Program& decoded, const CFG::Graph&
}
translator.TranslateInstruction(instruction);
}
translator.AddBranchCondition(cfg_block, result.block_info[typed_index]);
translator.AddBranchCondition(cfg, cfg_block, result.block_info[typed_index]);
}
IR::ValidateProgram(result, false);
return result;
@@ -9,13 +9,15 @@ namespace Libs::Graphics::ShaderRecompiler::Frontend {
class Translator {
public:
Translator(IR::Program& program, IR::Block* block, uint32_t vector_limit)
: program(program), ir(block), current_vector_limit(vector_limit) {}
Translator(IR::Program& program, IR::Block* block, uint32_t vector_limit,
bool flush_f32_inputs)
: program(program), ir(block), current_vector_limit(vector_limit),
flush_f32_inputs(flush_f32_inputs) {}
void TranslateInstruction(const Decoder::Instruction& inst);
void TranslateEmbeddedFetch(const Decoder::Instruction& inst, uint32_t attribute,
uint32_t component_count, const ShaderBufferResource& resource);
void AddBranchCondition(const CFG::BasicBlock& source, IR::BlockInfo& info);
void AddBranchCondition(const CFG::Graph& graph, const CFG::BasicBlock& source, IR::BlockInfo& info);
private:
const Decoder::Operand& SourceAt(const Decoder::Instruction& inst, uint32_t index);
@@ -261,6 +263,7 @@ private:
Decoder::Opcode current_opcode = Decoder::Opcode::UNKNOWN;
uint32_t current_pc = 0;
uint32_t current_vector_limit = 1;
bool flush_f32_inputs;
};
} // namespace Libs::Graphics::ShaderRecompiler::Frontend
@@ -273,6 +273,9 @@ void Translator::EmitVector(const Decoder::Instruction& inst) {
case O::V_CMPX_NGT_F16:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordLessThanEqual32, true, true);
return;
case O::V_CMPX_NLE_F16:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordGreaterThan32, true, true);
return;
case O::V_CMP_NEQ_F16:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordNotEqual32, true, false);
return;
@@ -299,6 +302,8 @@ void Translator::EmitVector(const Decoder::Instruction& inst) {
case O::V_CVT_F32_I32: V_CVT_F32_I32(inst); return;
case O::V_CVT_F64_I32: return FloatUnary(inst, IR::ValueOpcode::ConvertF64S32);
case O::V_CVT_F32_F64: return FloatUnary(inst, IR::ValueOpcode::ConvertF32F64);
case O::V_CVT_F64_F32: return FloatUnary(inst, IR::ValueOpcode::ConvertF64F32);
case O::V_CVT_F64_U32: return FloatUnary(inst, IR::ValueOpcode::ConvertF64U32);
case O::V_CVT_U32_F32: V_CVT_U32_F32(inst); return;
case O::V_CVT_I32_F32: V_CVT_I32_F32(inst); return;
case O::V_CVT_F16_F32: V_CVT_F16_F32(inst); return;
@@ -81,7 +81,8 @@ bool EquivalentValue(const ResourcePlan& program, Value left, Value right,
program.memory_info[li] != program.memory_info[ri]) {
return false;
}
} else if (lhs->Flags<uint64_t>() != rhs->Flags<uint64_t>()) {
} else if (lhs->GetOpcode() != ValueOpcode::ReadConst &&
lhs->Flags<uint64_t>() != rhs->Flags<uint64_t>()) {
return false;
}
for (size_t index = 0; index < lhs->NumArgs(); index++) {
+5 -2
View File
@@ -147,6 +147,7 @@ struct SamplerResource {
uint32_t first_use_pc = 0;
bool force_point_filtering = false;
bool depth_compare = false;
bool integer_border = false;
bool operator==(const SamplerResource& other) const = default;
};
@@ -548,7 +549,7 @@ struct ResourcePlan {
std::vector<SrtRead> srt_reads;
std::vector<uint8_t> clean_flat_slots;
bool requires_specialization_memory = false;
bool has_address_writes = false;
bool capture_specialization_reads = false;
bool srt_plan_complete = false;
bool resource_tracking_complete = false;
ShaderInfo info;
@@ -582,10 +583,12 @@ struct Program: ResourcePlan {
CFG::FailureKind cfg_failure_kind = CFG::FailureKind::None;
std::string fallback_reason;
std::vector<BlockInfo> block_info;
struct ScalarWrite { uint32_t pc; ScalarReg reg; };
std::vector<ScalarWrite> scalar_writes;
// Typed memory and export instructions reference shader-local metadata by dense index.
// Decoder-only details (such as NSA register numbers) have already become IR operands.
std::vector<ExportInfo> export_info;
std::vector<Value> dynamic_reads;
bool has_address_writes = false;
bool shader_info_complete = false;
BindingLayout bindings;
bool binding_layout_complete = false;
@@ -173,6 +173,7 @@ SharedAccess SharedAccessOf(ValueOpcode opcode) {
case ValueOpcode::SharedAtomicUMax32:
case ValueOpcode::SharedAtomicAnd32:
case ValueOpcode::SharedAtomicOr32:
case ValueOpcode::SharedAtomicOr64:
case ValueOpcode::SharedAtomicXor32: return SharedAccess::Atomic;
case ValueOpcode::DataAppend: return SharedAccess::Append;
case ValueOpcode::DataConsume: return SharedAccess::Consume;
@@ -182,6 +183,7 @@ SharedAccess SharedAccessOf(ValueOpcode opcode) {
uint32_t SharedComponentCount(ValueOpcode opcode) {
switch (opcode) {
case ValueOpcode::SharedAtomicOr64:
case ValueOpcode::LoadSharedU32x2:
case ValueOpcode::WriteSharedU32x2: return 2u;
case ValueOpcode::LoadSharedU32x3:
@@ -52,6 +52,10 @@ struct PermlaneFlags {
static_assert(sizeof(PermlaneFlags) <= sizeof(uint64_t));
static_assert(std::is_trivially_copyable_v<PermlaneFlags>);
struct FPCompareFlags {
bool flush_input_denorms = false;
};
struct MemoryFlags {
uint32_t index = 0;
uint32_t pc = 0;
@@ -4,6 +4,7 @@ VALUE_OPCODE(Identity, Opaque, Opaque)
VALUE_OPCODE(Void, Void)
VALUE_OPCODE(Reference, Void, U1)
VALUE_OPCODE(ReferenceU32, Void, U32)
VALUE_OPCODE(ConditionRef, U1, U1)
// Translation and register-state pseudo operations.
VALUE_OPCODE(GetUserData, U32, ScalarReg)
@@ -65,7 +66,9 @@ VALUE_OPCODE(ConvertU32F32, U32, F32)
VALUE_OPCODE(ConvertF32S32, F32, U32)
VALUE_OPCODE(ConvertF64S32, F64, U32)
VALUE_OPCODE(ConvertF32F64, F32, F64)
VALUE_OPCODE(ConvertF64F32, F64, F32)
VALUE_OPCODE(ConvertF32U32, F32, U32)
VALUE_OPCODE(ConvertF64U32, F64, U32)
VALUE_OPCODE(CompositeConstructU64, U64, U32, U32)
VALUE_OPCODE(CompositeConstructU32x2, U32x2, U32, U32)
VALUE_OPCODE(CompositeConstructU32x3, U32x3, U32, U32, U32)
@@ -273,9 +276,10 @@ VALUE_OPCODE(SharedAtomicSMax32, U32, U32, U32, U1)
VALUE_OPCODE(SharedAtomicUMax32, U32, U32, U32, U1)
VALUE_OPCODE(SharedAtomicAnd32, U32, U32, U32, U1)
VALUE_OPCODE(SharedAtomicOr32, U32, U32, U32, U1)
VALUE_OPCODE(SharedAtomicOr64, Void, U32, U64, U1)
VALUE_OPCODE(SharedAtomicXor32, U32, U32, U32, U1)
VALUE_OPCODE(DataAppend, U32, U32, U1, U32, U32)
VALUE_OPCODE(DataConsume, U32, U32, U1, U32, U32)
VALUE_OPCODE(DataAppend, U32, U32, U1)
VALUE_OPCODE(DataConsume, U32, U32, U1)
VALUE_OPCODE(SwizzleU32, U32, U32, U32, U1)
VALUE_OPCODE(ImageQueryDimensions, U32x4, ImageResource, ImageAddress)
@@ -85,6 +85,16 @@ bool CollectMemoryResources(const Program& program, std::vector<uint32_t>& buffe
return uses_gds;
}
bool UsesFlattenedSrt(const Program& program) {
return std::ranges::any_of(program.blocks, [](const Block* block) {
return std::ranges::any_of(*block, [](const Inst& inst) {
return inst.GetOpcode() == ValueOpcode::ReadConst;
});
}) || std::ranges::any_of(program.info.images, [](const ImageResource& image) {
return image.indirect_search_iterations != 0u;
});
}
void AllocateBindings(Program& program, uint32_t push_data_start_dword) {
if (!program.shader_info_complete || program.binding_layout_complete) {
EXIT("shader binding layout failed: %s", !program.shader_info_complete
@@ -145,12 +155,7 @@ void AllocateBindings(Program& program, uint32_t push_data_start_dword) {
AddBinding(next, DescriptorBindingKind::BdaPagetable);
AddBinding(next, DescriptorBindingKind::FaultBuffer);
}
const bool uses_flattened_runtime =
!program.srt_reads.empty() ||
std::ranges::any_of(program.info.images, [](const ImageResource& image) {
return image.indirect_search_iterations != 0u;
});
if (uses_flattened_runtime) {
if (UsesFlattenedSrt(program)) {
AddBinding(next, DescriptorBindingKind::FlattenedSrt);
}
@@ -9,6 +9,7 @@ void AllocateBindings(Program& program, uint32_t push_data_start_dword = 0);
// Collect live direct buffer indices and report whether the shader accesses GDS.
bool CollectMemoryResources(const Program& program, std::vector<uint32_t>& buffers);
bool UsesFlattenedSrt(const Program& program);
const DescriptorBinding* FindBinding(const BindingLayout& layout, DescriptorBindingKind kind);
@@ -630,6 +630,11 @@ void FoldInstruction(Block& block, Block::iterator instruction,
}
}
return;
case ValueOpcode::ConditionRef: {
const auto value = Arg(inst, 0);
if (IsImmediate(value, Type::U1)) Replace(inst, value);
return;
}
case ValueOpcode::LogicalNot: {
const auto value = Arg(inst, 0);
if (IsImmediate(value, Type::U1)) {
@@ -107,14 +107,16 @@ Prospero::BufferFormat ImageConversionFormat(Prospero::BufferFormat format) {
: Prospero::BufferFormat::kInvalid;
}
bool RequiresPointSampler(const ImageResource& image) {
return image.numeric_class == Prospero::TextureNumericClass::Sint ||
image.conversion_format != Prospero::BufferFormat::kInvalid;
}
enum class SamplerClass : uint8_t { Float, Integer, PointInteger };
bool RequiresPointSampler(const ResourceSpecialization::Image& image) {
return image.numeric_class == Prospero::TextureNumericClass::Sint ||
image.conversion_format != Prospero::BufferFormat::kInvalid;
template <typename Image>
SamplerClass ClassifySampler(const Image& image) {
if (image.numeric_class == Prospero::TextureNumericClass::Sint ||
image.conversion_format != Prospero::BufferFormat::kInvalid) {
return SamplerClass::PointInteger;
}
return image.numeric_class == Prospero::TextureNumericClass::Uint ? SamplerClass::Integer
: SamplerClass::Float;
}
bool DescriptorIsCube(const DescriptorValue& descriptor) {
@@ -223,9 +225,9 @@ void MarkCleanFlatSlots(const ResourcePlan& program, const DescriptorSource* sou
}
}
bool ReadScalarTable(uint64_t base, uint64_t size, uint32_t dynamic_offset,
bool ReadScalarTable(uint64_t base, uint64_t size, uint64_t dynamic_offset,
const SrtRuntime& runtime, std::span<uint32_t> words) {
const auto offset = static_cast<uint64_t>(dynamic_offset) & ~uint64_t {3};
const auto offset = dynamic_offset & ~uint64_t {3};
const auto count = std::min<uint64_t>(words.size(), offset < size ? (size - offset) / 4u : 0u);
std::ranges::fill(words.subspan(count), 0u);
if (count == 0u) {
@@ -304,26 +306,24 @@ bool MaterializeIndirectImage(const ResourcePlan& program,
material.Stride() != indirect.selector_stride) {
return false;
}
// Enumerate every wrapped scalar-buffer offset that can pass the descriptor bounds.
const auto step = std::gcd<uint64_t>(indirect.selector_stride, uint64_t {1} << 32u);
const auto residue = static_cast<uint64_t>(indirect.selector_offset) % step;
const auto limit = std::min<uint64_t>(UINT32_MAX, material.GetSize() + 3u);
const auto probe_count = residue <= limit ? (limit - residue) / step + 1u : 0u;
// The first aligned offset includes the immediate added after shader U32 arithmetic.
const auto step = std::max<uint64_t>(4u,
std::gcd<uint64_t>(indirect.selector_stride, uint64_t {1} << 32u));
const uint64_t first = indirect.selector_offset;
const auto size = material.GetSize();
const auto limit = std::min(first + (uint64_t {1} << 32u) - step, size >= 4u ? size - 4u : 0u);
const auto probe_count = size >= 4u && first <= limit ? (limit - first) / step + 1u : 0u;
if (probe_count > MaxIndirectImageProbes) {
return false;
}
keys.reserve(static_cast<size_t>(probe_count) + 1u);
keys.push_back(0u);
for (uint64_t offset = residue; offset <= limit && probe_count != 0u; offset += step) {
for (uint64_t probe = 0, offset = first; probe < probe_count; ++probe, offset += step) {
uint32_t key = 0;
if (!ReadScalarTable(material.Base48(), material.GetSize(),
static_cast<uint32_t>(offset), runtime, {&key, 1})) {
if (!ReadScalarTable(material.Base48(), size, offset, runtime, {&key, 1})) {
return false;
}
keys.push_back(key);
if (limit - offset < step) {
break;
}
}
std::ranges::sort(keys);
keys.erase(std::unique(keys.begin(), keys.end()), keys.end());
@@ -380,8 +380,13 @@ bool MaterializeIndirectImage(const ResourcePlan& program,
} // namespace
struct SamplerPlan {
std::array<uint32_t, ShaderInfo::MaxSamplers> point_sampler {};
uint32_t sampler_count = 0;
struct Binding {
uint32_t source;
SamplerClass type;
};
std::array<std::array<uint32_t, 3>, ShaderInfo::MaxSamplers> mapping;
std::array<Binding, ShaderInfo::MaxSamplers> bindings;
uint32_t sampler_count = 0;
};
struct ImageRemap {
@@ -595,11 +600,9 @@ static bool BuildResourceSpecialization(const ResourcePlan& program, ResourceSna
if (!BuildSamplerPlan(program.info, specialization.images, sampler_plan)) {
return SpecializationFail("specialized sampler layout exceeds its resource limit");
}
for (uint32_t index = 0; index < program.info.samplers.size(); index++) {
const auto target = sampler_plan.point_sampler[index];
if (target != UINT32_MAX && target >= program.info.samplers.size()) {
snapshot.samplers.push_back(snapshot.samplers[index]);
}
for (uint32_t index = static_cast<uint32_t>(program.info.samplers.size());
index < sampler_plan.sampler_count; index++) {
snapshot.samplers.push_back(snapshot.samplers[sampler_plan.bindings[index].source]);
}
ImageRemap(specialization).Apply(snapshot.images);
return true;
@@ -607,29 +610,29 @@ static bool BuildResourceSpecialization(const ResourcePlan& program, ResourceSna
template <typename Images>
bool BuildSamplerPlan(const ShaderInfo& base, const Images& images, SamplerPlan& plan) {
if (base.samplers.size() > plan.point_sampler.size()) {
if (base.samplers.size() > plan.mapping.size()) {
return false;
}
std::array<uint8_t, ShaderInfo::MaxSamplers> usage {};
plan.point_sampler.fill(UINT32_MAX);
plan.sampler_count = static_cast<uint32_t>(base.samplers.size());
for (const auto& pair: base.sampled_pairs) {
if (pair.image >= images.size() || pair.sampler >= base.samplers.size()) {
return false;
}
usage[pair.sampler] |= RequiresPointSampler(images[pair.image]) ? 2u : 1u;
usage[pair.sampler] |= 1u << static_cast<uint32_t>(ClassifySampler(images[pair.image]));
}
for (uint32_t index = 0; index < base.samplers.size(); index++) {
if ((usage[index] & 2u) == 0u) {
continue;
}
if ((usage[index] & 1u) == 0u) {
plan.point_sampler[index] = index;
} else {
if (plan.sampler_count >= ShaderInfo::MaxSamplers) {
return false;
}
plan.point_sampler[index] = plan.sampler_count++;
auto& mapping = plan.mapping[index];
mapping.fill(UINT32_MAX);
const auto classes = usage[index] == 0u ? 1u : usage[index];
bool first = true;
for (uint32_t type = 0; type < mapping.size(); type++) {
if ((classes & (1u << type)) == 0u) continue;
const auto target = first ? index : plan.sampler_count++;
if (target >= ShaderInfo::MaxSamplers) return false;
mapping[type] = target;
plan.bindings[target] = {index, static_cast<SamplerClass>(type)};
first = false;
}
}
return true;
@@ -862,7 +865,6 @@ ResourcePlan ExtractResourcePlan(const Program& program) {
plan.memory_info = program.memory_info;
plan.srt_plan_complete = program.srt_plan_complete;
plan.resource_tracking_complete = program.resource_tracking_complete;
plan.has_address_writes = program.has_address_writes;
std::unordered_map<const Inst*, Inst*> cloned;
std::function<Value(Value)> Clone = [&](Value value) -> Value {
@@ -921,17 +923,25 @@ ResourcePlan ExtractResourcePlan(const Program& program) {
plan.uniform_fill.values[i] = Clone(plan.uniform_fill.values[i]);
}
plan.clean_flat_slots.resize(plan.srt_reads.size());
bool masked_image = false;
for (const auto& image: plan.info.images) {
const auto* source = Source(plan, image.source);
if (source == nullptr || !source->indirect_image.has_value()) {
continue;
}
plan.requires_specialization_memory = true;
masked_image |= !source->indirect_image->selector_mask.IsEmpty();
MarkCleanFlatSlots(plan, Source(plan, source->indirect_image->material_source),
plan.clean_flat_slots, source->indirect_image->selector_mask);
MarkCleanFlatSlots(plan, Source(plan, source->indirect_image->table_source),
plan.clean_flat_slots);
}
if (masked_image) {
plan.resource_tracking_complete &= !program.has_address_writes &&
!std::ranges::any_of(plan.info.images, &ImageResource::written);
plan.capture_specialization_reads =
std::ranges::any_of(plan.info.buffers, &BufferResource::written);
}
return plan;
}
@@ -941,18 +951,7 @@ bool MaterializeResources(const ResourcePlan& program, const SrtRuntime& runtime
(program.requires_specialization_memory && runtime.read_specialization_memory == nullptr)) {
return false;
}
const bool masked_image = std::ranges::any_of(program.info.images, [&](const auto& image) {
const auto* source = Source(program, image.source);
return source != nullptr && source->indirect_image.has_value() &&
!source->indirect_image->selector_mask.IsEmpty();
});
if (masked_image &&
(program.has_address_writes ||
std::ranges::any_of(program.info.images, &ImageResource::written))) {
return false;
}
const bool capture_reads = masked_image &&
std::ranges::any_of(program.info.buffers, &BufferResource::written);
const bool capture_reads = program.capture_specialization_reads;
auto& reads = program.specialization_reads;
ReadCapture capture {runtime, reads};
SrtRuntime observed = runtime;
@@ -1105,25 +1104,18 @@ void ApplyResourceSpecialization(Program& program, const ResourceSpecialization&
auto samplers = program.info.samplers;
auto sampled_pairs = program.info.sampled_pairs;
samplers.reserve(sampler_plan.sampler_count);
for (uint32_t index = 0; index < program.info.samplers.size(); index++) {
const auto target = sampler_plan.point_sampler[index];
if (target == UINT32_MAX) {
continue;
}
if (target == index) {
samplers[index].force_point_filtering = true;
} else {
EXIT_IF(target != samplers.size());
auto sampler = samplers[index];
sampler.force_point_filtering = true;
samplers.push_back(std::move(sampler));
for (uint32_t index = 0; index < sampler_plan.sampler_count; index++) {
const auto& binding = sampler_plan.bindings[index];
if (index >= program.info.samplers.size()) {
samplers.push_back(program.info.samplers[binding.source]);
}
samplers[index].force_point_filtering = binding.type == SamplerClass::PointInteger;
samplers[index].integer_border = binding.type != SamplerClass::Float;
}
for (auto& pair: sampled_pairs) {
if (RequiresPointSampler(images[pair.image])) {
EXIT_IF(sampler_plan.point_sampler[pair.sampler] == UINT32_MAX);
pair.sampler = sampler_plan.point_sampler[pair.sampler];
}
const auto type = static_cast<uint32_t>(ClassifySampler(images[pair.image]));
pair.sampler = sampler_plan.mapping[pair.sampler][type];
EXIT_IF(pair.sampler == UINT32_MAX);
samplers[pair.sampler].depth_compare |= images[pair.image].depth_compare;
}
@@ -1201,10 +1193,11 @@ void ApplyResourceSpecialization(Program& program, const ResourceSpecialization&
inst.ReplaceUsesWith(Value(&*result));
continue;
}
if (image_opcode.needs_sampler && RequiresPointSampler(image) &&
if (image_opcode.needs_sampler &&
memory.sampler < program.info.samplers.size()) {
EXIT_IF(sampler_plan.point_sampler[memory.sampler] == UINT32_MAX);
memory.sampler = sampler_plan.point_sampler[memory.sampler];
const auto type = static_cast<uint32_t>(ClassifySampler(image));
memory.sampler = sampler_plan.mapping[memory.sampler][type];
EXIT_IF(memory.sampler == UINT32_MAX);
}
EXIT_IF(image.indirect_root == memory.resource &&
inst.GetOpcode() != ValueOpcode::ImageSampleRaw);
File diff suppressed because it is too large Load Diff
@@ -5,9 +5,8 @@
namespace Libs::Graphics::ShaderRecompiler::IR {
// Collects immutable resource topology from typed SSA handles, interns their resolved dwords in
// descriptor_sources, then writes dense indices to handle flags and MemoryInfo.
void TrackResources(Program& program);
// Resolves native descriptor sources, plans their scalar reads, and assigns dense resource bindings.
void TrackResources(Program& program, const Decoder::Program& decoded, const CFG::Graph& native_cfg);
} // namespace Libs::Graphics::ShaderRecompiler::IR
@@ -326,6 +326,8 @@ void CollectBuiltinInputs(const Program& program, ShaderInfo& info) {
}
void CollectOutputs(const Program& program, ShaderStageInputInfo input_info, ShaderInfo& info) {
const bool alpha_remap = program.stage == ShaderType::Pixel && input_info.pixel != nullptr &&
input_info.pixel->alpha_blend_source_remap;
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
if (inst.GetOpcode() != ValueOpcode::SetAttribute) {
@@ -385,8 +387,14 @@ void CollectOutputs(const Program& program, ShaderStageInputInfo input_info, Sha
export_info.index, fmt::format("out_param_{}", export_info.index));
break;
case ExportTargetKind::Mrt:
if (alpha_remap && export_info.index != 0) {
break;
}
AddOutput(info, StageOutputKind::Mrt, export_info.index, export_info.index,
fmt::format("out_mrt_{}", export_info.index));
if (alpha_remap) {
AddOutput(info, StageOutputKind::Mrt, 1, 1, "out_mrt_1");
}
break;
default: break;
}
@@ -7,7 +7,6 @@
#include <bit>
#include <cmath>
#include <cstring>
#include <fmt/format.h>
#include <unordered_set>
namespace Libs::Graphics::ShaderRecompiler::IR {
@@ -23,21 +22,6 @@ namespace {
constexpr uint64_t AddressMask = 0x0000ffffffffffffull;
const char* StageName(ShaderType stage) {
switch (stage) {
case ShaderType::Vertex: return "vertex";
case ShaderType::Pixel: return "pixel";
case ShaderType::Fetch: return "fetch";
case ShaderType::Compute: return "compute";
default: return "unknown";
}
}
std::string Diagnostic(const ResourcePlan& program, uint32_t pc, const std::string& message) {
return fmt::format("shader SRT: hash=0x{:016x} stage={} pc=0x{:08x} {}", program.shader_hash,
StageName(program.stage), pc, message);
}
bool AddSignedAddress(uint64_t base, int64_t offset, uint64_t& result) {
if (base > AddressMask) {
return false;
@@ -89,6 +73,7 @@ bool IsRuntimeSelect(ValueOpcode op) {
bool IsRuntimeUniformOp(ValueOpcode op) {
switch (op) {
case ValueOpcode::ConditionRef:
case ValueOpcode::BitCastU32F32:
case ValueOpcode::BitCastF32U32:
case ValueOpcode::ConvertU32F32:
@@ -312,151 +297,6 @@ private:
std::unordered_set<const Inst*> m_validated_dependencies;
};
class PlanBuilder {
public:
explicit PlanBuilder(Program& program): m_program(program) {}
void Run() {
m_program.srt_reads.clear();
m_program.dynamic_reads.clear();
for (auto* block: m_program.blocks) {
for (auto& inst: *block) {
const auto op = inst.GetOpcode();
if (op == ValueOpcode::LoadAddressU32 || op == ValueOpcode::ReadConstBuffer) {
const auto flags = inst.Flags<MemoryFlags>();
if (flags.index < m_program.memory_info.size()) {
const auto kind = m_program.memory_info[flags.index].kind;
const bool crosswired = (op == ValueOpcode::LoadAddressU32 &&
kind == ResourceKind::ScalarBuffer) ||
(op == ValueOpcode::ReadConstBuffer &&
kind == ResourceKind::ScalarAddress);
if (crosswired) {
Fail(flags.pc,
fmt::format("{} has incompatible scalar memory metadata",
ValueOpcodeName(op)));
}
}
}
if (IsDescriptorHandle(inst.GetOpcode())) {
for (size_t index = 0; index < inst.NumArgs(); index++) {
Collect(inst.Arg(index), 0);
}
}
}
}
for (auto* block: m_program.blocks) {
for (auto& inst: *block) {
if (inst.GetOpcode() == ValueOpcode::LoadAddressU32 && IsRawRead(m_program, inst) &&
inst.Arg(1).Resolve().IsImmediate() &&
ValidateRuntimeValue(m_program, Value(&inst))) {
Collect(Value(&inst), inst.Flags<MemoryFlags>().pc);
}
}
}
PatchReads();
}
private:
struct Patch {
Inst* inst = nullptr;
uint32_t slot = 0;
bool keep = false;
};
[[noreturn]] void Fail(uint32_t pc, const std::string& message) const {
const auto diagnostic = Diagnostic(m_program, pc, message);
EXIT("shader SRT planning failed: %s", diagnostic.c_str());
std::abort();
}
void Collect(Value value, uint32_t use_pc) {
value = value.Resolve();
if (value.IsImmediate()) {
return;
}
auto* inst = value.TryInstruction();
if (inst == nullptr) {
Fail(use_pc, "invalid typed planning value");
}
const auto cycle = std::ranges::find(m_visiting, inst);
if (cycle != m_visiting.end()) {
const auto contains_phi = std::any_of(cycle, m_visiting.end(), [](const Inst* value) {
return value->GetOpcode() == ValueOpcode::Phi;
});
if (contains_phi) {
return;
}
Fail(use_pc, fmt::format("cyclic typed planning value {} without a phi",
ValueOpcodeName(inst->GetOpcode())));
}
if (std::ranges::find(m_visited, inst) != m_visited.end()) {
return;
}
m_visiting.push_back(inst);
for (size_t index = 0; index < inst->NumArgs(); index++) {
Collect(inst->Arg(index), use_pc);
}
m_visiting.pop_back();
m_visited.push_back(inst);
if (!IsRawRead(m_program, *inst)) {
return;
}
const auto offset = inst->Arg(1).Resolve();
if (!offset.IsImmediate() || offset.GetType() != Type::U32) {
if (std::ranges::find(m_program.dynamic_reads, value) ==
m_program.dynamic_reads.end()) {
m_program.dynamic_reads.push_back(value);
}
return;
}
for (uint32_t slot = 0; slot < m_program.srt_reads.size(); slot++) {
if (EquivalentValue(m_program, value, m_program.srt_reads[slot].value)) {
m_patches.push_back({inst, slot, false});
return;
}
}
const auto slot = static_cast<uint32_t>(m_program.srt_reads.size());
m_program.srt_reads.push_back({value, slot});
m_patches.push_back({inst, slot, true});
}
void PatchReads() {
for (const auto& patch: m_patches) {
auto* block = patch.inst->Parent();
auto& list = block->Instructions();
auto where =
std::ranges::find_if(list, [&](const Inst& inst) { return &inst == patch.inst; });
const auto resource =
Value(&*block->PrependNewInst(where, ValueOpcode::GetSrtResource));
const auto flat = Value(&*block->PrependNewInst(where, ValueOpcode::ReadConst,
{resource, Value(patch.slot)}));
const auto uses = patch.inst->Uses();
for (const auto& use: uses) {
use.user->SetArg(use.operand, flat);
}
for (auto& info: m_program.block_info) {
if (info.condition.Resolve() == Value(patch.inst)) {
info.condition = flat;
}
if (info.indirect_target.Resolve() == Value(patch.inst)) {
info.indirect_target = flat;
}
}
if (patch.keep) {
const auto memory = patch.inst->Flags<MemoryFlags>().index;
if (memory < m_program.memory_info.size()) {
m_program.memory_info[memory].planning_only = true;
}
block->AppendNewInst(ValueOpcode::ReferenceU32, {Value(patch.inst)});
}
}
}
Program& m_program;
std::vector<Inst*> m_visiting;
std::vector<Inst*> m_visited;
std::vector<Patch> m_patches;
};
} // namespace
@@ -616,16 +456,15 @@ bool SrtWalker::EvaluateRawRead(const Inst& inst, uint64_t& result) {
return false;
}
const auto byte_offset =
static_cast<uint64_t>(immediate) + static_cast<uint32_t>(offset);
const auto aligned = byte_offset & ~uint64_t {3};
(static_cast<uint64_t>(immediate) & ~uint64_t {3}) + (static_cast<uint32_t>(offset) & ~3u);
const auto stride = (static_cast<uint32_t>(high) >> 16u) & 0x3fffu;
const auto size = stride == 0u
? static_cast<uint64_t>(static_cast<uint32_t>(records))
: static_cast<uint64_t>(stride) * static_cast<uint32_t>(records);
if (aligned > size || size - aligned < sizeof(uint32_t)) {
if (byte_offset > size || size - byte_offset < sizeof(uint32_t)) {
return false;
}
address = ((base & ~uint64_t {3}) + byte_offset) & ~uint64_t {3};
address = (base & ~uint64_t {3}) + byte_offset;
} else {
const auto relative = (immediate & ~int64_t {3}) +
static_cast<int64_t>(static_cast<uint32_t>(offset) & ~3u);
@@ -780,14 +619,18 @@ bool SrtWalker::EvaluateInst(const Inst& inst, uint64_t& result) {
}
return false;
case ValueOpcode::FPOrdLessThanEqual32:
if (binary()) {
result = Float32(a) <= Float32(b);
return true;
}
return false;
case ValueOpcode::FPOrdGreaterThanEqual32:
if (binary()) {
result = Float32(a) >= Float32(b);
const auto operand = [&](uint64_t bits) {
if (inst.Flags<FPCompareFlags>().flush_input_denorms &&
(bits & 0x7fffffffu) < 0x00800000u) {
bits &= 0x80000000u;
}
return Float32(bits);
};
result = inst.GetOpcode() == ValueOpcode::FPOrdLessThanEqual32
? operand(a) <= operand(b)
: operand(a) >= operand(b);
return true;
}
return false;
@@ -970,6 +813,7 @@ bool SrtWalker::EvaluateInst(const Inst& inst, uint64_t& result) {
return true;
}
return false;
case ValueOpcode::ConditionRef: return Arg(inst, 0, result);
case ValueOpcode::LogicalNot:
if (Arg(inst, 0, a)) {
result = a == 0u;
@@ -1061,13 +905,5 @@ bool ValidateRuntimeValue(const ResourcePlan& program, Value value, RuntimeValue
return RuntimeValidator(program, type).Run(value);
}
void BuildSrtPlan(Program& program) {
if (program.resource_tracking_complete) {
EXIT("shader SRT planning failed: cannot rebuild SRT after resource tracking");
}
program.srt_plan_complete = false;
PlanBuilder(program).Run();
program.srt_plan_complete = true;
}
} // namespace Libs::Graphics::ShaderRecompiler::IR
@@ -21,9 +21,6 @@ struct SrtRuntime {
enum class RuntimeValueType { Any, Integer };
// Collects reachable ReadConst values. Immediate offsets receive compact flat-buffer slots;
// dynamic offsets remain explicit and are never assigned a fake slot.
void BuildSrtPlan(Program& program);
bool ValidateRuntimeValue(const ResourcePlan& program, Value value,
RuntimeValueType type = RuntimeValueType::Any);
// Uses the strict reader for values that affect shader specialization.
+23 -46
View File
@@ -336,14 +336,15 @@ static void ShaderDetectBuffers(ShaderVertexInputInfo& info) {
for (int ri = 0; ri < info.resources_num; ri++) {
const auto& r = info.resources[ri];
bool merged = false;
for (int bi = 0; bi < info.buffers_num; bi++) {
auto& destination = info.resources_dst[ri];
int bi = 0;
for (; bi < info.buffers_num; bi++) {
auto& b = info.buffers[bi];
uint64_t stride = b.stride;
if (stride == r.Stride() &&
b.fetch_index == static_cast<uint32_t>(info.resources_dst[ri].fetch_index)) {
b.fetch_index == destination.fetch_index) {
uint64_t rbase = r.Base48();
uint64_t base = std::min(rbase, b.addr);
uint64_t offset1 = rbase - base;
@@ -352,45 +353,33 @@ static void ShaderDetectBuffers(ShaderVertexInputInfo& info) {
if (offset1 < stride && offset2 < stride) {
EXIT_NOT_IMPLEMENTED(b.num_records != r.NumRecords());
b.addr = base;
EXIT_NOT_IMPLEMENTED(b.attr_num >= ShaderVertexInputBuffer::ATTR_MAX);
b.attr_indices[b.attr_num++] = ri;
merged = true;
break;
}
}
}
if (!merged) {
if (bi == info.buffers_num) {
EXIT_NOT_IMPLEMENTED(info.buffers_num >= ShaderVertexInputInfo::RES_MAX);
int bi = info.buffers_num++;
info.buffers[bi].addr = r.Base48();
info.buffers[bi].stride = r.Stride();
info.buffers[bi].num_records = r.NumRecords();
info.buffers[bi].fetch_index = info.resources_dst[ri].fetch_index;
info.buffers[bi].attr_num = 1;
info.buffers[bi].attr_indices[0] = ri;
}
}
for (int bi = 0; bi < info.buffers_num; bi++) {
auto& b = info.buffers[bi];
for (int ri = 0; ri < b.attr_num; ri++) {
b.attr_offsets[ri] = info.resources[b.attr_indices[ri]].Base48() - b.addr;
info.buffers[bi] = {.addr = r.Base48(),
.stride = r.Stride(),
.num_records = r.NumRecords(),
.fetch_index = destination.fetch_index};
info.buffers_num++;
}
destination.buffer_index = bi;
}
}
static void ShaderApplyAttribSemantics(ShaderVertexInputInfo& info,
const ShaderSemantic* input_semantics,
uint32_t num_input_semantics, const uint32_t* attrib,
const uint32_t* buffer) {
std::span<const ShaderSemantic> input_semantics,
const uint32_t* attrib, const uint32_t* buffer) {
KYTY_PROFILER_FUNCTION();
EXIT_IF(attrib == nullptr || buffer == nullptr);
const bool debug_dump = Config::GraphicsDebugDumpEnabled();
for (uint32_t i = 0; i < num_input_semantics; i++) {
for (uint32_t i = 0; i < input_semantics.size(); i++) {
const auto& in = input_semantics[i];
EXIT_NOT_IMPLEMENTED(in.static_vb_index == 1 || in.static_attribute == 1);
@@ -544,8 +533,7 @@ static bool ShaderGetStaticVertexInputInfo(uint64_t shader_addr, const HW::UserS
shader_addr);
return false;
}
ShaderApplyAttribSemantics(info, metadata.input_semantics.data(),
metadata.input_semantics_count, attrib, buffer);
ShaderApplyAttribSemantics(info, metadata.input_semantics, attrib, buffer);
ShaderDetectBuffers(info);
}
return true;
@@ -678,15 +666,8 @@ void BuildStageStaticKey(const ShaderVertexInputInfo& info, std::vector<uint32_t
key.push_back(destination.registers_num);
key.push_back(destination.fetch_index);
key.push_back(static_cast<uint32_t>(destination.attr_id));
key.push_back(resource.Stride());
key.push_back(static_cast<uint32_t>(resource.SwizzleEnabled()));
key.push_back(resource.DstSelX());
key.push_back(resource.DstSelY());
key.push_back(resource.DstSelZ());
key.push_back(resource.DstSelW());
key.push_back(resource.RawFormat());
key.push_back(resource.OutOfBounds());
key.push_back(static_cast<uint32_t>(resource.AddTid()));
key.push_back(resource.fields[1] & 0xbfff0000u); // Stride and swizzle enable.
key.push_back(resource.fields[3] & 0x3087ffffu); // Channels, format, OOB, and add TID.
}
}
@@ -712,6 +693,7 @@ void BuildStageStaticKey(const ShaderPixelInputInfo& info, std::vector<uint32_t>
key.push_back(static_cast<uint32_t>(info.ps_sample_mask_export_enable));
key.push_back(static_cast<uint32_t>(info.ps_early_z));
key.push_back(static_cast<uint32_t>(info.dual_source_blending));
key.push_back(static_cast<uint32_t>(info.alpha_blend_source_remap));
key.insert(key.end(), std::begin(info.target_output_mode), std::end(info.target_output_mode));
for (uint32_t base = 0; base < info.target_export_mapping.size(); base += 4u) {
uint32_t packed = 0;
@@ -907,8 +889,9 @@ void ShaderDbgDumpInputInfo(const ShaderVertexInputInfo& info) {
LOGF("\t\t register_start = %d\n"
"\t\t registers_num = %d\n"
"\t\t fetch_index = %" PRIu32 "\n",
rd.register_start, rd.registers_num, rd.fetch_index);
"\t\t fetch_index = %" PRIu32 "\n"
"\t\t buffer_index = %d\n",
rd.register_start, rd.registers_num, rd.fetch_index, rd.buffer_index);
LOGF("\t\t fields = %08" PRIx32 "%08" PRIx32 "%08" PRIx32 "%08" PRIx32 "\n",
r.fields[3], r.fields[2], r.fields[1], r.fields[0]);
LOGF("\t\t Base() = %" PRIx64 "\n"
@@ -934,14 +917,8 @@ void ShaderDbgDumpInputInfo(const ShaderVertexInputInfo& info) {
LOGF("\t\t addr = %" PRIx64 "\n"
"\t\t stride = %" PRIu32 "\n"
"\t\t num_records = %" PRIu32 "\n"
"\t\t fetch_index = %" PRIu32 "\n"
"\t\t attr_num = %" PRId32 "\n",
r.addr, r.stride, r.num_records, r.fetch_index, r.attr_num);
for (int j = 0; j < r.attr_num; j++) {
LOGF("\t\t attr_indices[%d] = %d\n"
"\t\t attr_offsets[%d] = %u\n",
j, r.attr_indices[j], j, r.attr_offsets[j]);
}
"\t\t fetch_index = %" PRIu32 "\n",
r.addr, r.stride, r.num_records, r.fetch_index);
}
}
+2
View File
@@ -178,6 +178,8 @@ struct ShaderPixelInputInfo {
bool ps_sample_mask_export_enable = false;
bool ps_sample_shading = false;
bool dual_source_blending = false;
// Export logical alpha through MRT1 for blending after channel swizzling.
bool alpha_blend_source_remap = false;
bool ps_early_z = false;
bool ps_execute_on_noop = false;
ShaderStageRuntime stage;
+5 -9
View File
@@ -174,15 +174,10 @@ struct ShaderSamplerResource {
};
struct ShaderVertexInputBuffer {
static constexpr int ATTR_MAX = 32;
uint64_t addr = 0;
uint32_t stride = 0;
uint32_t num_records = 0;
uint32_t fetch_index = 0;
int attr_num = 0;
int attr_indices[ATTR_MAX] = {0};
uint32_t attr_offsets[ATTR_MAX] = {0};
uint64_t addr = 0;
uint32_t stride = 0;
uint32_t num_records = 0;
uint32_t fetch_index = 0;
};
struct ShaderVertexDestination {
@@ -190,6 +185,7 @@ struct ShaderVertexDestination {
int registers_num = 0;
int attr_id = -1;
uint32_t fetch_index = 0;
int buffer_index = 0;
};
enum class ShaderStorageUsage {
+1 -4
View File
@@ -68,13 +68,10 @@ bool ShaderReadVertexMetadata(const ShaderMappedData& data, uint32_t max_user_sg
return ShaderError::Fail(error, "vertex semantic count is outside the supported domain");
}
const auto semantic_size =
static_cast<uint64_t>(data.num_input_semantics) * sizeof(ShaderSemantic);
if (data.input_semantics == nullptr) {
return ShaderError::Fail(error, "missing vertex input semantics");
}
std::memcpy(next.input_semantics.data(), data.input_semantics, semantic_size);
next.input_semantics_count = data.num_input_semantics;
next.input_semantics = {data.input_semantics, data.num_input_semantics};
metadata = next;
return true;
}
+5 -6
View File
@@ -3,19 +3,18 @@
#include "graphics/shader/shader.h"
#include <array>
#include <span>
#include <string>
namespace Libs::Graphics {
struct ShaderVertexMetadata {
int vertex_buffer_reg = -1;
int vertex_attrib_reg = -1;
std::array<ShaderSemantic, ShaderVertexInputInfo::RES_MAX> input_semantics {};
uint32_t input_semantics_count = 0;
int vertex_buffer_reg = -1;
int vertex_attrib_reg = -1;
std::span<const ShaderSemantic> input_semantics;
};
// Copies the small AGC metadata subset used by the vertex path after validating every guest range.
// Validates the vertex metadata and returns a view into the shader header.
bool ShaderReadVertexMetadata(const ShaderMappedData& data, uint32_t max_user_sgprs,
ShaderVertexMetadata& metadata, std::string* error);
+43 -14
View File
@@ -1,5 +1,6 @@
#include "kernel/fileSystem.h"
#include "common/archive.h"
#include "common/assert.h"
#include "common/common.h"
#include "common/dateTime.h"
@@ -41,6 +42,7 @@ public:
struct MountPair {
std::filesystem::path dir;
std::string point;
std::shared_ptr<Common::ArchiveReader> archive;
};
MountPoints() { EXIT_NOT_IMPLEMENTED(!Common::Thread::IsMainThread()); }
@@ -260,6 +262,7 @@ void MountPoints::Mount(const std::filesystem::path& folder, const std::string&
MountPair p;
p.dir = folder;
p.point = point_str;
p.archive = Common::OpenArchive(folder);
m_mount_pairs.push_back(std::move(p));
}
@@ -346,11 +349,13 @@ std::filesystem::path MountPoints::ResolvePath(const std::string& mounted_name)
}
const auto native_rel_path = Common::PathFromUtf8(rel_path);
if (Common::IsArchivePath(p.dir)) {
return p.dir / native_rel_path;
}
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
if (HasWindowsForbiddenFilenameCharacter(rel_path)) {
::printf("FileSystem: Windows-incompatible guest filename: %s\n",
mounted_name.c_str());
::printf("FileSystem: Windows-incompatible guest filename: %s\n", mounted_name.c_str());
}
return p.dir / native_rel_path;
#else
@@ -358,7 +363,8 @@ std::filesystem::path MountPoints::ResolvePath(const std::string& mounted_name)
#endif
}
return mounted_name;
// Unmounted guest paths must not fall through to the host filesystem.
return {};
}
void Initialize() {
@@ -461,14 +467,24 @@ int KYTY_SYSV_ABI KernelOpen(const char* path, int flags, uint16_t mode) {
}
file->real_name = g_mount_points->ResolvePath(file->name);
if (file->real_name.empty()) {
g_files->DeleteDescriptor(descriptor);
return KERNEL_ERROR_ENOENT;
}
if (Common::IsArchivePath(file->real_name) &&
(rw_mode != Common::File::Mode::Read || trunc || creat)) {
g_files->DeleteDescriptor(descriptor);
return KERNEL_ERROR_EROFS;
}
if (trunc && rw_mode == Common::File::Mode::Read) {
g_files->DeleteDescriptor(descriptor);
return KERNEL_ERROR_EACCES;
}
bool dir_exist = Common::File::IsDirectoryExisting(file->real_name);
bool file_exist = Common::File::IsFileExisting(file->real_name);
const auto info = Common::File::GetInfo(file->real_name);
const bool dir_exist = info && !info->is_file;
const bool file_exist = info && info->is_file;
// A missing path opened without O_CREAT is ENOENT
if (!creat && !dir_exist && !file_exist) {
@@ -666,7 +682,7 @@ int64_t KYTY_SYSV_ABI KernelWrite(int d, const void* buf, size_t nbytes) {
auto* file = g_files->GetFile(d);
if (file == nullptr) {
if (file == nullptr || !file->opened || !file->writable) {
return KERNEL_ERROR_EBADF;
}
@@ -881,7 +897,7 @@ int64_t KYTY_SYSV_ABI KernelPwrite(int d, const void* buf, size_t nbytes, int64_
auto* file = g_files->GetFile(d);
if (file == nullptr) {
if (file == nullptr || !file->opened || !file->writable) {
return KERNEL_ERROR_EBADF;
}
@@ -1058,15 +1074,13 @@ int KYTY_SYSV_ABI KernelStat(const char* path, FileStat* sb) {
auto real_file_name = g_mount_points->ResolvePath(path);
bool is_dir = Common::File::IsDirectoryExisting(real_file_name);
bool is_file = Common::File::IsFileExisting(real_file_name);
if (!is_dir && !is_file) {
const auto info = Common::File::GetInfo(real_file_name);
if (!info) {
LOGF("\t file not found\n");
return KERNEL_ERROR_ENOENT;
}
EXIT_NOT_IMPLEMENTED(is_dir && is_file);
const bool is_dir = !info->is_file;
FileStat stat {};
stat.st_mode = 0000777u | (is_dir ? 0040000u : 0100000u);
@@ -1080,7 +1094,7 @@ int KYTY_SYSV_ABI KernelStat(const char* path, FileStat* sb) {
stat.st_blksize = 512;
stat.st_blocks = stat.st_size / 512;
} else {
stat.st_size = static_cast<int64_t>(Common::File::Size(real_file_name));
stat.st_size = static_cast<int64_t>(info->size);
stat.st_blksize = 512;
stat.st_blocks = (stat.st_size + 511) / 512;
@@ -1217,6 +1231,9 @@ int KYTY_SYSV_ABI KernelUnlink(const char* path) {
}
auto real_file_name = g_mount_points->ResolvePath(path);
if (Common::IsArchivePath(real_file_name)) {
return KERNEL_ERROR_EROFS;
}
bool is_dir = Common::File::IsDirectoryExisting(real_file_name);
bool is_file = Common::File::IsFileExisting(real_file_name);
@@ -1254,8 +1271,11 @@ int KYTY_SYSV_ABI KernelRename(const char* from, const char* to) {
auto to_path = std::string(to);
auto real_from = g_mount_points->ResolvePath(from_path);
auto real_to = g_mount_points->ResolvePath(to_path);
if (Common::IsArchivePath(real_from) || Common::IsArchivePath(real_to)) {
return KERNEL_ERROR_EROFS;
}
if (!Common::File::IsFileExisting(real_from)) {
if (real_to.empty() || !Common::File::IsFileExisting(real_from)) {
return KERNEL_ERROR_ENOENT;
}
@@ -1335,6 +1355,12 @@ int KYTY_SYSV_ABI KernelMkdir(const char* path, uint16_t mode) {
path, mode);
auto real_name = g_mount_points->ResolvePath(std::string(path));
if (real_name.empty()) {
return KERNEL_ERROR_ENOENT;
}
if (Common::IsArchivePath(real_name)) {
return KERNEL_ERROR_EROFS;
}
if (Common::File::IsDirectoryExisting(real_name)) {
return KERNEL_ERROR_EEXIST;
@@ -1361,6 +1387,9 @@ int KYTY_SYSV_ABI KernelRmdir(const char* path) {
LOGF("\t path = %s\n", path);
auto real_name = g_mount_points->ResolvePath(std::string(path));
if (Common::IsArchivePath(real_name)) {
return KERNEL_ERROR_EROFS;
}
if (!Common::File::IsDirectoryExisting(real_name)) {
return KERNEL_ERROR_ENOENT;
+1
View File
@@ -54,6 +54,7 @@ struct Lifecycle {
void Mount(const std::filesystem::path& folder, const std::string& point);
void Umount(const std::string& folder_or_point);
// Returns an empty path when the guest path has no mounted filesystem.
std::filesystem::path GetRealFilename(const std::string& mounted_file_name);
int KYTY_SYSV_ABI KernelOpen(const char* path, int flags, uint16_t mode);
+182 -69
View File
@@ -78,8 +78,7 @@ static Graphics::RenderContext& GetGpuResources() {
}
static bool IsGpuAddressRange(uint64_t vaddr, uint64_t size) {
constexpr uint64_t GPU_ADDRESS_LIMIT = 1ull << 40u;
return vaddr != 0 && size != 0 && vaddr < GPU_ADDRESS_LIMIT && size < GPU_ADDRESS_LIMIT - vaddr;
return Graphics::GuestRange {vaddr, size}.Valid();
}
static void MapGpuRange(uint64_t vaddr, uint64_t size) {
@@ -660,6 +659,9 @@ static bool g_test_fail_next_fixed_reserve_range_add = false;
class PhysicalMemory {
public:
enum class AllocationKind { Direct, Pooled, Automatic };
using PhysicalRanges = std::vector<std::pair<uint64_t, uint64_t>>;
struct AllocatedBlock {
uint64_t start_addr;
uint64_t size;
@@ -671,7 +673,7 @@ public:
VirtualMemory::Mode mode;
GpuAccessMode gpu_mode;
int memory_type;
bool pool_expansion;
AllocationKind kind;
char name[KERNEL_MAXIMUM_NAME_LENGTH];
};
@@ -690,7 +692,10 @@ public:
}
bool Alloc(uint64_t search_start, uint64_t search_end, size_t len, size_t alignment,
uint64_t* phys_addr_out, int memory_type, bool pool_expansion = false);
uint64_t* phys_addr_out, int memory_type,
AllocationKind kind = AllocationKind::Direct);
bool ReserveAutomatic(uint64_t size, PhysicalRanges* ranges);
void RestoreAutomatic(const PhysicalRanges& ranges);
bool Available(uint64_t search_start, uint64_t search_end, size_t alignment,
uint64_t* phys_addr_out, uint64_t* size_out);
bool Release(uint64_t start, size_t len, uint64_t* vaddr, uint64_t* size,
@@ -718,12 +723,14 @@ public:
[[nodiscard]] const std::vector<AllocatedBlock>& GetMappings() const { return m_mappings; }
private:
void ConsumeFreeRange(std::map<uint64_t, uint64_t>::iterator range, uint64_t start,
uint64_t size);
void AddFreeRange(uint64_t start, uint64_t size);
static void RemoveFreeRange(std::map<uint64_t, uint64_t>& ranges, uint64_t start,
uint64_t size);
static void AddFreeRange(std::map<uint64_t, uint64_t>& ranges, uint64_t start, uint64_t size);
void ReclaimAutomatic(uint64_t start, uint64_t size);
std::map<uint64_t, AllocatedBlock> m_physical;
std::map<uint64_t, uint64_t> m_free;
std::map<uint64_t, uint64_t> m_automatic_free;
std::vector<AllocatedBlock> m_mappings;
Common::Mutex m_mutex;
};
@@ -1093,7 +1100,7 @@ void SetFlexibleMemorySize(uint64_t size) {
}
bool PhysicalMemory::Alloc(uint64_t search_start, uint64_t search_end, size_t len, size_t alignment,
uint64_t* phys_addr_out, int memory_type, bool pool_expansion) {
uint64_t* phys_addr_out, int memory_type, AllocationKind kind) {
if (phys_addr_out == nullptr) {
return false;
}
@@ -1128,10 +1135,13 @@ bool PhysicalMemory::Alloc(uint64_t search_start, uint64_t search_end, size_t le
b.prot = 0;
b.mode = VirtualMemory::Mode::NoAccess;
b.memory_type = memory_type;
b.pool_expansion = pool_expansion;
b.kind = kind;
ConsumeFreeRange(range, free_pos, len);
RemoveFreeRange(m_free, free_pos, len);
EXIT_IF(!m_physical.emplace(b.start_addr, b).second);
if (kind == AllocationKind::Automatic) {
AddFreeRange(m_automatic_free, free_pos, len);
}
*phys_addr_out = free_pos;
return true;
@@ -1180,35 +1190,99 @@ bool PhysicalMemory::Available(uint64_t search_start, uint64_t search_end, size_
return true;
}
void PhysicalMemory::ConsumeFreeRange(std::map<uint64_t, uint64_t>::iterator range, uint64_t start,
uint64_t size) {
const auto range_start = range->first;
const auto range_end = range->first + range->second;
m_free.erase(range);
if (range_start < start) {
m_free.emplace(range_start, start - range_start);
void PhysicalMemory::RemoveFreeRange(std::map<uint64_t, uint64_t>& ranges, uint64_t start,
uint64_t size) {
const auto end = start + size;
auto range = ranges.upper_bound(start);
if (range != ranges.begin()) {
--range;
}
if (start + size < range_end) {
m_free.emplace(start + size, range_end - start - size);
while (range != ranges.end() && range->first < end) {
const auto range_start = range->first;
const auto range_end = range_start + range->second;
if (range_end <= start) {
++range;
continue;
}
range = ranges.erase(range);
if (range_start < start) {
ranges.emplace(range_start, start - range_start);
}
if (end < range_end) {
ranges.emplace(end, range_end - end);
}
}
}
void PhysicalMemory::AddFreeRange(uint64_t start, uint64_t size) {
void PhysicalMemory::AddFreeRange(std::map<uint64_t, uint64_t>& ranges, uint64_t start,
uint64_t size) {
auto end = start + size;
auto next = m_free.lower_bound(start);
if (next != m_free.begin()) {
auto next = ranges.lower_bound(start);
if (next != ranges.begin()) {
auto previous = std::prev(next);
if (previous->first + previous->second >= start) {
start = previous->first;
end = std::max(end, previous->first + previous->second);
next = m_free.erase(previous);
next = ranges.erase(previous);
}
}
while (next != m_free.end() && next->first <= end) {
while (next != ranges.end() && next->first <= end) {
end = std::max(end, next->first + next->second);
next = m_free.erase(next);
next = ranges.erase(next);
}
ranges.emplace(start, end - start);
}
bool PhysicalMemory::ReserveAutomatic(uint64_t size, PhysicalRanges* ranges) {
Common::LockGuard lock(m_mutex);
for (const auto& [start, available]: m_automatic_free) {
const auto length = std::min(size, available);
ranges->emplace_back(start, length);
size -= length;
if (size == 0) {
for (const auto& [offset, length]: *ranges) {
RemoveFreeRange(m_automatic_free, offset, length);
}
return true;
}
}
ranges->clear();
return false;
}
void PhysicalMemory::RestoreAutomatic(const PhysicalRanges& ranges) {
Common::LockGuard lock(m_mutex);
for (const auto& [start, size]: ranges) {
AddFreeRange(m_automatic_free, start, size);
}
}
void PhysicalMemory::ReclaimAutomatic(uint64_t start, uint64_t size) {
const auto end = start + size;
auto owner = m_physical.upper_bound(start);
if (owner != m_physical.begin()) {
--owner;
}
bool automatic = false;
for (; owner != m_physical.end() && owner->first < end; ++owner) {
const auto& block = owner->second;
if (block.kind != AllocationKind::Automatic || block.start_addr + block.size <= start) {
continue;
}
const auto first = std::max(start, block.start_addr);
const auto last = std::min(end, block.start_addr + block.size);
AddFreeRange(m_automatic_free, first, last - first);
automatic = true;
}
if (automatic) {
for (const auto& mapping: m_mappings) {
const auto first = std::max(start, mapping.start_addr);
const auto last = std::min(end, mapping.start_addr + mapping.size);
if (first < last) {
RemoveFreeRange(m_automatic_free, first, last - first);
}
}
}
m_free.emplace(start, end - start);
}
bool PhysicalMemory::Release(uint64_t start, size_t len, uint64_t* vaddr, uint64_t* size,
@@ -1225,18 +1299,22 @@ bool PhysicalMemory::Release(uint64_t start, size_t len, uint64_t* vaddr, uint64
}
auto it = std::prev(next);
auto& b = it->second;
if (b.pool_expansion || start < b.start_addr || start >= b.start_addr + b.size ||
len > b.start_addr + b.size - start) {
if (b.kind == AllocationKind::Pooled || start < b.start_addr ||
start >= b.start_addr + b.size || len > b.start_addr + b.size - start) {
return false;
}
if (b.kind == AllocationKind::Automatic) {
RemoveFreeRange(m_automatic_free, start, len);
}
if (start == b.start_addr && len == b.size) {
*vaddr = b.map_vaddr;
*size = b.map_size;
*gpu_mode = b.gpu_mode;
m_physical.erase(it);
AddFreeRange(start, len);
AddFreeRange(m_free, start, len);
return true;
}
if (start > b.start_addr && start + len < b.start_addr + b.size) {
@@ -1261,7 +1339,7 @@ bool PhysicalMemory::Release(uint64_t start, size_t len, uint64_t* vaddr, uint64
}
m_physical.emplace(right.start_addr, right);
AddFreeRange(start, len);
AddFreeRange(m_free, start, len);
return true;
}
if (start == b.start_addr && len < b.size) {
@@ -1278,7 +1356,7 @@ bool PhysicalMemory::Release(uint64_t start, size_t len, uint64_t* vaddr, uint64
remaining.map_size -= len;
}
m_physical.emplace(remaining.start_addr, remaining);
AddFreeRange(start, len);
AddFreeRange(m_free, start, len);
return true;
}
if (start > b.start_addr && start + len == b.start_addr + b.size) {
@@ -1290,7 +1368,7 @@ bool PhysicalMemory::Release(uint64_t start, size_t len, uint64_t* vaddr, uint64
if (b.map_vaddr != 0) {
b.map_size = b.size;
}
AddFreeRange(start, len);
AddFreeRange(m_free, start, len);
return true;
}
@@ -1311,6 +1389,7 @@ bool PhysicalMemory::Map(uint64_t vaddr, uint64_t phys_addr, size_t len, int pro
return false;
}
auto first = std::prev(next);
bool has_automatic = false;
while (current < phys_addr + len) {
auto block = m_physical.upper_bound(current);
if (block == m_physical.begin()) {
@@ -1318,10 +1397,11 @@ bool PhysicalMemory::Map(uint64_t vaddr, uint64_t phys_addr, size_t len, int pro
}
--block;
const auto block_end = block->second.start_addr + block->second.size;
if (block->second.pool_expansion || current < block->second.start_addr ||
if (block->second.kind == AllocationKind::Pooled || current < block->second.start_addr ||
current >= block_end) {
return false;
}
has_automatic |= block->second.kind == AllocationKind::Automatic;
current = std::min<uint64_t>(phys_addr + len, block_end);
if (current < phys_addr + len) {
const auto following = std::next(block);
@@ -1342,6 +1422,9 @@ bool PhysicalMemory::Map(uint64_t vaddr, uint64_t phys_addr, size_t len, int pro
mapping.mode = mode;
mapping.gpu_mode = gpu_mode;
m_mappings.push_back(mapping);
if (has_automatic) {
RemoveFreeRange(m_automatic_free, phys_addr, len);
}
return true;
}
@@ -1362,7 +1445,7 @@ bool PhysicalMemory::CanMapDirect(uint64_t phys_addr, size_t len) {
}
--block;
const auto block_end = block->second.start_addr + block->second.size;
if (block->second.pool_expansion || current < block->second.start_addr ||
if (block->second.kind == AllocationKind::Pooled || current < block->second.start_addr ||
current >= block_end) {
return false;
}
@@ -1380,11 +1463,12 @@ bool PhysicalMemory::CanMapDirect(uint64_t phys_addr, size_t len) {
bool PhysicalMemory::ReleasePoolExpansion(uint64_t phys_addr, size_t len) {
Common::LockGuard lock(m_mutex);
const auto it = m_physical.find(phys_addr);
if (it == m_physical.end() || !it->second.pool_expansion || it->second.size != len) {
if (it == m_physical.end() || it->second.kind != AllocationKind::Pooled ||
it->second.size != len) {
return false;
}
m_physical.erase(it);
AddFreeRange(phys_addr, len);
AddFreeRange(m_free, phys_addr, len);
return true;
}
@@ -1407,7 +1491,7 @@ bool PhysicalMemory::GetAllocatedSpan(uint64_t phys_addr, size_t len,
}
--block;
const auto block_end = block->second.start_addr + block->second.size;
if (block->second.pool_expansion || current < block->second.start_addr ||
if (block->second.kind == AllocationKind::Pooled || current < block->second.start_addr ||
current >= block_end) {
blocks->clear();
return false;
@@ -1463,49 +1547,39 @@ bool PhysicalMemory::Unmap(uint64_t vaddr, uint64_t size, GpuAccessMode* gpu_mod
}
};
size_t index = 0;
for (auto& b: m_mappings) {
if (b.map_vaddr == vaddr && b.map_size == size) {
*gpu_mode = b.gpu_mode;
const auto host_vaddr = b.host_vaddr;
const auto host_size = b.host_size;
for (size_t index = 0; index < m_mappings.size(); ++index) {
auto& b = m_mappings[index];
if (vaddr < b.map_vaddr || vaddr >= b.map_vaddr + b.map_size ||
size > b.map_vaddr + b.map_size - vaddr) {
continue;
}
*gpu_mode = b.gpu_mode;
const auto phys_addr = b.start_addr + vaddr - b.map_vaddr;
const auto host_vaddr = b.host_vaddr;
const auto host_size = b.host_size;
if (vaddr == b.map_vaddr && size == b.map_size) {
m_mappings.erase(m_mappings.begin() + static_cast<std::ptrdiff_t>(index));
set_host_release_if_last(host_vaddr, host_size);
return true;
}
if (vaddr > b.map_vaddr && vaddr + size < b.map_vaddr + b.map_size) {
*gpu_mode = b.gpu_mode;
} else if (vaddr > b.map_vaddr && vaddr + size < b.map_vaddr + b.map_size) {
AllocatedBlock right = b;
right.start_addr += (vaddr + size) - b.map_vaddr;
right.size = b.map_vaddr + b.map_size - (vaddr + size);
right.map_size = right.size;
right.map_vaddr = vaddr + size;
b.size = vaddr - b.map_vaddr;
b.map_size = b.size;
m_mappings.push_back(right);
return true;
}
if (vaddr == b.map_vaddr && size < b.map_size) {
*gpu_mode = b.gpu_mode;
} else if (vaddr == b.map_vaddr) {
b.start_addr += size;
b.size -= size;
b.map_vaddr += size;
b.map_size -= size;
return true;
}
if (vaddr > b.map_vaddr && vaddr + size == b.map_vaddr + b.map_size) {
*gpu_mode = b.gpu_mode;
} else {
b.size = vaddr - b.map_vaddr;
b.map_size = b.size;
return true;
}
index++;
ReclaimAutomatic(phys_addr, size);
return true;
}
return false;
@@ -2726,10 +2800,8 @@ int KYTY_SYSV_ABI KernelDirectMemoryQuery(int64_t offset, int flags, void* info,
return OK;
}
int KYTY_SYSV_ABI KernelAllocateDirectMemory(int64_t search_start, int64_t search_end, size_t len,
size_t alignment, int memory_type,
int64_t* phys_addr_out) {
PRINT_NAME();
int AllocateDirectMemory(int64_t search_start, int64_t search_end, size_t len, size_t alignment,
int memory_type, int64_t* phys_addr_out, bool automatic) {
std::lock_guard<std::recursive_mutex> memory_operation_lock(g_memory_operation_mutex);
@@ -2748,7 +2820,10 @@ int KYTY_SYSV_ABI KernelAllocateDirectMemory(int64_t search_start, int64_t searc
}
uint64_t addr = 0;
if (!g_physical_memory->Alloc(search_start, search_end, len, alignment, &addr, memory_type)) {
const auto kind = automatic ? PhysicalMemory::AllocationKind::Automatic
: PhysicalMemory::AllocationKind::Direct;
if (!g_physical_memory->Alloc(search_start, search_end, len, alignment, &addr, memory_type,
kind)) {
LOGF_COLOR(Log::Color::Red, "\t[Fail]\n");
return KERNEL_ERROR_EAGAIN;
}
@@ -2760,6 +2835,14 @@ int KYTY_SYSV_ABI KernelAllocateDirectMemory(int64_t search_start, int64_t searc
return OK;
}
int KYTY_SYSV_ABI KernelAllocateDirectMemory(int64_t search_start, int64_t search_end, size_t len,
size_t alignment, int memory_type,
int64_t* phys_addr_out) {
PRINT_NAME();
return AllocateDirectMemory(search_start, search_end, len, alignment, memory_type,
phys_addr_out);
}
int KYTY_SYSV_ABI KernelAllocateMainDirectMemory(size_t len, size_t alignment, int memory_type,
int64_t* phys_addr_out) {
PRINT_NAME();
@@ -3082,6 +3165,36 @@ int KYTY_SYSV_ABI KernelMapDirectMemory2(void** addr, size_t len, int type, int
return ret;
}
int MapAutomaticMemory(uint64_t vaddr, size_t size, int type, int prot) {
std::lock_guard<std::recursive_mutex> memory_operation_lock(g_memory_operation_mutex);
constexpr uint64_t page_size = 0x4000;
if (size == 0 || (vaddr & (page_size - 1)) != 0 || (size & (page_size - 1)) != 0 ||
vaddr < kExtendedMemoryBase || vaddr >= kExtendedMemoryBase + kExtendedMemorySize ||
size > kExtendedMemoryBase + kExtendedMemorySize - vaddr) {
return KERNEL_ERROR_EINVAL;
}
PhysicalMemory::PhysicalRanges ranges;
if (!g_physical_memory->ReserveAutomatic(size, &ranges)) {
return KERNEL_ERROR_EAGAIN;
}
uint64_t mapped = 0;
for (const auto& [offset, length]: ranges) {
void* address = reinterpret_cast<void*>(vaddr + mapped);
const int result = KernelMapDirectMemory2(&address, length, type, prot, 0x10,
static_cast<int64_t>(offset), page_size);
if (result != OK) {
if (mapped != 0) {
EXIT_IF(KernelMunmap(vaddr, mapped) != OK);
}
g_physical_memory->RestoreAutomatic(ranges);
return result;
}
mapped += length;
}
return OK;
}
int KYTY_SYSV_ABI KernelMapNamedDirectMemory(void** addr, size_t len, int prot, int flags,
int64_t direct_memory_start, size_t alignment,
const char* name) {
@@ -3892,7 +4005,7 @@ int KYTY_SYSV_ABI KernelMemoryPoolExpand(int64_t search_start, int64_t search_en
uint64_t phys_addr = 0;
if (!g_physical_memory->Alloc(static_cast<uint64_t>(search_start),
static_cast<uint64_t>(search_end), len, effective_alignment,
&phys_addr, 0, true)) {
&phys_addr, 0, PhysicalMemory::AllocationKind::Pooled)) {
return KERNEL_ERROR_ENOMEM;
}
+5
View File
@@ -24,6 +24,8 @@ struct Lifecycle {
using callback_func_t = void (*)(uintptr_t addr, size_t size);
constexpr uint32_t KERNEL_MAXIMUM_NAME_LENGTH = 32;
constexpr uint64_t kExtendedMemoryBase = 0x080000000000ull;
constexpr uint64_t kExtendedMemorySize = 512ull * 1024 * 1024 * 1024;
struct VirtualQueryInfo {
uintptr_t start;
@@ -107,6 +109,9 @@ static_assert(sizeof(KernelMemoryPoolBlockStats) == 16,
void RegisterCallbacks(callback_func_t alloc_func, callback_func_t free_func);
void SetFlexibleMemorySize(uint64_t size);
int AllocateDirectMemory(int64_t search_start, int64_t search_end, size_t size, size_t alignment,
int memory_type, int64_t* phys_addr_out, bool automatic = false);
int MapAutomaticMemory(uint64_t vaddr, size_t size, int type, int prot);
bool TryWriteBacking(uint64_t vaddr, const void* data, uint64_t size);
bool TryReadBacking(uint64_t vaddr, void* data, uint64_t size);
bool TryReadGpuCleanBacking(uint64_t vaddr, void* data, uint64_t size);
+37 -38
View File
@@ -3,7 +3,6 @@
static constexpr uint64_t HOST_SYSTEM_MANAGED_MIN = 0x0000040000ull;
static constexpr uint64_t HOST_SYSTEM_MANAGED_MAX = 0x07ffffbfffull;
static constexpr uint64_t HOST_SYSTEM_RESERVED_MIN = HOST_SYSTEM_MANAGED_MAX + 1u;
#if defined(__APPLE__) && (defined(__x86_64__) || defined(_M_X64))
static constexpr uint64_t HOST_SYSTEM_RESERVED_MAX = 0x0fbfffffffull;
static constexpr uint64_t HOST_USER_MIN = 0x7000000000ull;
@@ -1077,51 +1076,51 @@ private:
#endif
void ReserveGuestRegions() {
const std::array<std::pair<uint64_t, uint64_t>, 3> regions {{
{HOST_SYSTEM_MANAGED_MIN, HOST_SYSTEM_RESERVED_MAX + 1u},
{HOST_USER_MIN, HOST_USER_MAX + 1u},
{kExtendedMemoryBase, kExtendedMemoryBase + kExtendedMemorySize},
}};
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
constexpr uint64_t MinimumRegionSize = 0x1000000;
const auto process = GetCurrentProcess();
const auto guest_end = HOST_USER_MAX + 1u;
auto next_addr = HOST_SYSTEM_MANAGED_MIN;
while (next_addr < guest_end) {
MEMORY_BASIC_INFORMATION info {};
if (VirtualQuery(reinterpret_cast<const void*>(next_addr), &info, sizeof(info)) == 0) {
EXIT("failed to inspect guest address space at 0x%016" PRIx64 "\n", next_addr);
}
const auto region_base = reinterpret_cast<uint64_t>(info.BaseAddress);
const auto region_end =
std::min<uint64_t>(guest_end, region_base + static_cast<uint64_t>(info.RegionSize));
const auto reserve_start =
AlignUp(std::max<uint64_t>(region_base, HOST_SYSTEM_MANAGED_MIN), m_granularity);
const auto reserve_end = AlignDown(region_end, m_granularity);
if (info.State == MEM_FREE && reserve_end > reserve_start &&
reserve_end - reserve_start > MinimumRegionSize) {
auto* ptr = VirtualAlloc2(
process, reinterpret_cast<void*>(reserve_start), reserve_end - reserve_start,
MEM_RESERVE | MEM_RESERVE_PLACEHOLDER, PAGE_NOACCESS, nullptr, 0);
if (reinterpret_cast<uint64_t>(ptr) != reserve_start) {
EXIT("failed to reserve guest address space at 0x%016" PRIx64
", size 0x%016" PRIx64 "\n",
reserve_start, reserve_end - reserve_start);
for (const auto& [guest_start, guest_end]: regions) {
auto next_addr = guest_start;
while (next_addr < guest_end) {
MEMORY_BASIC_INFORMATION info {};
if (VirtualQuery(reinterpret_cast<const void*>(next_addr), &info, sizeof(info)) == 0) {
EXIT("failed to inspect guest address space at 0x%016" PRIx64 "\n", next_addr);
}
AddFreeUnlocked(reserve_start, reserve_end - reserve_start);
m_owned.emplace_back(reserve_start, reserve_end - reserve_start);
}
if (region_end <= next_addr) {
EXIT("guest address-space scan did not advance at 0x%016" PRIx64 "\n", next_addr);
const auto region_base = reinterpret_cast<uint64_t>(info.BaseAddress);
const auto region_end = std::min<uint64_t>(
guest_end, region_base + static_cast<uint64_t>(info.RegionSize));
const auto reserve_start =
AlignUp(std::max(region_base, guest_start), m_granularity);
const auto reserve_end = AlignDown(region_end, m_granularity);
if (info.State == MEM_FREE && reserve_end > reserve_start &&
reserve_end - reserve_start > MinimumRegionSize) {
auto* ptr = VirtualAlloc2(
process, reinterpret_cast<void*>(reserve_start), reserve_end - reserve_start,
MEM_RESERVE | MEM_RESERVE_PLACEHOLDER, PAGE_NOACCESS, nullptr, 0);
if (reinterpret_cast<uint64_t>(ptr) != reserve_start) {
EXIT("failed to reserve guest address space at 0x%016" PRIx64
", size 0x%016" PRIx64 "\n",
reserve_start, reserve_end - reserve_start);
}
AddFreeUnlocked(reserve_start, reserve_end - reserve_start);
m_owned.emplace_back(reserve_start, reserve_end - reserve_start);
}
if (region_end <= next_addr) {
EXIT("guest address-space scan did not advance at 0x%016" PRIx64 "\n", next_addr);
}
next_addr = region_end;
}
next_addr = region_end;
}
#else
const std::array<std::pair<uint64_t, uint64_t>, 3> regions {{
{HOST_SYSTEM_MANAGED_MIN, HOST_SYSTEM_MANAGED_MAX + 1u},
{HOST_SYSTEM_RESERVED_MIN, HOST_SYSTEM_RESERVED_MAX + 1u},
{HOST_USER_MIN, HOST_USER_MAX + 1u},
}};
#if KYTY_PLATFORM == KYTY_PLATFORM_LINUX && !defined(__APPLE__)
std::vector<std::pair<uint64_t, uint64_t>> occupied;
FILE* maps = fopen("/proc/self/maps", "r");
+1 -30
View File
@@ -512,15 +512,6 @@ static void SchedulerBackoffOnce() {
#endif
}
static bool SleepMicroSchedulerBackoff(uint64_t microseconds) {
if (microseconds > 1) {
return false;
}
SchedulerBackoffOnce();
return true;
}
static void SleepMicroWithSignalPoll(uint64_t microseconds) {
if (microseconds == 0) {
KernelDispatchPendingSignalForCurrentThread();
@@ -529,9 +520,7 @@ static void SleepMicroWithSignalPoll(uint64_t microseconds) {
while (microseconds > 0) {
const auto step = std::min<uint64_t>(microseconds, SIGNAL_APC_POLL_MICROS);
if (!SleepMicroSchedulerBackoff(step)) {
Common::Thread::SleepMicro(step);
}
Common::Thread::SleepMicro(step);
microseconds -= step;
KernelDispatchPendingSignalForCurrentThread();
}
@@ -3842,28 +3831,16 @@ void KYTY_SYSV_ABI KernelSetThreadDtors(thread_dtors_func_t dtors) {
}
int KYTY_SYSV_ABI KernelUsleep(KernelUseconds microseconds) {
Common::Timer t;
t.Start();
SleepMicroWithSignalPoll(microseconds);
// double ts = t.GetTimeS();
// LOGF("\tactual: %g microseconds\n", ts * 1000000.0);
return OK;
}
unsigned int KYTY_SYSV_ABI KernelSleep(unsigned int seconds) {
PRINT_NAME();
LOGF("\tsleep: %u\n", seconds);
Common::Timer t;
t.Start();
SleepMicroWithSignalPoll(static_cast<uint64_t>(seconds) * 1000000ull);
double ts = t.GetTimeS();
LOGF("\tactual: %g seconds\n", ts);
return OK;
}
int KYTY_SYSV_ABI KernelNanosleep(const KernelTimespec* rqtp, KernelTimespec* rmtp) {
PRINT_NAME();
if (rqtp == nullptr) {
return KERNEL_ERROR_EFAULT;
}
@@ -3880,13 +3857,7 @@ int KYTY_SYSV_ABI KernelNanosleep(const KernelTimespec* rqtp, KernelTimespec* rm
uint64_t nanos =
static_cast<uint64_t>(rqtp->tv_sec) * 1000000000ull + static_cast<uint64_t>(rqtp->tv_nsec);
LOGF("\tnanosleep: %" PRIu64 "\n", nanos);
Common::Timer t;
t.Start();
SleepNanoWithSignalPoll(nanos);
double ts = t.GetTimeS();
LOGF("\tactual: %g nanoseconds\n", ts * 1000000000.0);
if (rmtp != nullptr) {
rmtp->tv_sec = 0;
+1 -1
View File
@@ -20,7 +20,7 @@ add_executable(launcher WIN32
)
target_link_libraries(launcher Qt6::Concurrent Qt6::Network Qt6::Widgets Vulkan::Headers SDL3::SDL3-static kyty_git_version)
target_link_libraries(launcher Qt6::Concurrent Qt6::Network Qt6::Widgets Vulkan::Headers SDL3::SDL3-static kyty_git_version common)
if(WIN32)
target_link_libraries(launcher setupapi)
endif()
+3 -1
View File
@@ -1,6 +1,7 @@
#ifndef CONFIGURATION_ITEM_H
#define CONFIGURATION_ITEM_H
#include <QIcon>
#include <QObject>
#include <QTreeWidgetItem>
@@ -18,7 +19,7 @@ public:
explicit ConfigurationItem(std::unique_ptr<Configuration> info, QTreeWidget* parent);
~ConfigurationItem() override;
void Update();
void Update(bool reload_icon = false);
bool operator<(const QTreeWidgetItem& other) const override;
Configuration& GetInfo() { return *m_info; }
@@ -35,6 +36,7 @@ private:
void UpdateStatusIndicator();
std::unique_ptr<Configuration> m_info;
QIcon m_icon;
bool m_running = false;
QComboBox* m_status_combo = nullptr;
QLabel* m_status_indicator = nullptr;
+28
View File
@@ -0,0 +1,28 @@
#ifndef GAME_CONTENT_H
#define GAME_CONTENT_H
#include <QByteArray>
#include <QString>
#include <QStringList>
#include <cstdint>
#include <filesystem>
namespace GameContent {
inline constexpr uint64_t MaxMetadataSize = uint64_t {1} << 20u;
inline constexpr uint64_t MaxImageSize = uint64_t {32} << 20u;
inline constexpr uint64_t MaxTrophyPackageSize = uint64_t {128} << 20u;
[[nodiscard]] std::filesystem::path ToPath(const QString& path);
[[nodiscard]] QString FromPath(const std::filesystem::path& path);
[[nodiscard]] bool IsArchive(const QString& base);
[[nodiscard]] std::filesystem::path Resolve(const QString& base, const QString& relative);
[[nodiscard]] bool FileExists(const QString& base, const QString& relative);
[[nodiscard]] QByteArray ReadPath(const std::filesystem::path& path, uint64_t max_size);
[[nodiscard]] QByteArray ReadFile(const QString& base, const QString& relative, uint64_t max_size);
[[nodiscard]] QStringList ListFiles(const QString& base, const QString& relative);
} // namespace GameContent
#endif // GAME_CONTENT_H
+8 -5
View File
@@ -1,6 +1,8 @@
#ifndef GAME_LIST_TREE_WIDGET_H
#define GAME_LIST_TREE_WIDGET_H
#include "gameContent.h"
#include <QPaintEvent>
#include <QPainter>
#include <QPalette>
@@ -17,14 +19,15 @@ public:
viewport()->setAutoFillBackground(false);
}
void SetBackgroundImage(const QString& path) {
if (m_path == path) {
void SetBackgroundImage(const QString& key) {
if (m_path == key) {
return;
}
m_path = path;
m_path = key;
m_source = QPixmap();
if (!path.isEmpty()) {
m_source.load(path);
if (!key.isEmpty()) {
m_source.loadFromData(GameContent::ReadFile(key, QStringLiteral("sce_sys/pic0.png"),
GameContent::MaxImageSize));
}
UpdateScaledBackground();
viewport()->update();
+20 -6
View File
@@ -1,6 +1,7 @@
#include "configurationItem.h"
#include "configuration.h"
#include "gameContent.h"
#include <QApplication>
#include <QComboBox>
@@ -14,6 +15,7 @@
#include <QLabel>
#include <QLineEdit>
#include <QLocale>
#include <QPixmap>
#include <QSize>
#include <QStringList>
#include <QStyle>
@@ -154,7 +156,7 @@ ConfigurationItem::ConfigurationItem(std::unique_ptr<Configuration> info, QTreeW
m_comment_edit->setFrame(false);
parent->setItemWidget(this, CommentsColumn, m_comment_edit);
Update();
Update(true);
SetRunning(false);
setData(SizeColumn, Qt::UserRole, qint64(-1));
@@ -170,7 +172,14 @@ ConfigurationItem::ConfigurationItem(std::unique_ptr<Configuration> info, QTreeW
watcher->deleteLater();
});
watcher->setFuture(QtConcurrent::run([path = m_info->basedir]() -> qint64 {
if (path.isEmpty() || !QDir(path).exists()) {
if (path.isEmpty()) {
return -1;
}
const QFileInfo info(path);
if (GameContent::IsArchive(path)) {
return info.size();
}
if (!QDir(path).exists()) {
return -1;
}
qint64 bytes = 0;
@@ -188,7 +197,7 @@ ConfigurationItem::ConfigurationItem(std::unique_ptr<Configuration> info, QTreeW
ConfigurationItem::~ConfigurationItem() = default;
void ConfigurationItem::Update() {
void ConfigurationItem::Update(bool reload_icon) {
const auto display_text = GetDisplayText(*m_info);
const auto path = GetPathText(*m_info);
@@ -209,6 +218,12 @@ void ConfigurationItem::Update() {
m_comment_edit->setText(m_info->game_comment);
}
if (reload_icon) {
const auto icon_data = GameContent::ReadFile(
m_info->basedir, QStringLiteral("sce_sys/icon0.png"), GameContent::MaxImageSize);
QPixmap icon;
m_icon = !icon_data.isEmpty() && icon.loadFromData(icon_data) ? QIcon(icon) : QIcon {};
}
UpdateIcon();
UpdateStatusIndicator();
}
@@ -267,9 +282,8 @@ void ConfigurationItem::SetCompatibilityEditable(bool editable) {
}
void ConfigurationItem::UpdateIcon() {
const QString icon_file = QDir(m_info->basedir).filePath(QStringLiteral("sce_sys/icon0.png"));
if (QFileInfo::exists(icon_file)) {
setIcon(NameColumn, QIcon(icon_file));
if (!m_icon.isNull()) {
setIcon(NameColumn, m_icon);
return;
}
+106 -84
View File
@@ -1,9 +1,11 @@
#include "configurationListWidget.h"
#include "common/archive.h"
#include "compatibilityDatabase.h"
#include "configuration.h"
#include "configurationEditDialog.h"
#include "configurationItem.h"
#include "gameContent.h"
#include "gameListTreeWidget.h"
#include "inputMappingDialog.h"
#include "mainDialog.h"
@@ -127,15 +129,6 @@ static QStringList SettingsStringList(const QVariant& value) {
return text.isEmpty() ? QStringList() : QStringList({text});
}
static QString GetPic0Path(const Configuration& info) {
if (info.basedir.isEmpty()) {
return {};
}
const auto path = QDir(info.basedir).filePath(QStringLiteral("sce_sys/pic0.png"));
return QFileInfo::exists(path) ? QFileInfo(path).absoluteFilePath() : QString();
}
static void ConfigureGameList(Ui::ConfigurationListWidget* ui) {
ui->cfgs_list->setAlternatingRowColors(false);
ui->cfgs_list->setAllColumnsShowFocus(true);
@@ -490,17 +483,16 @@ static QString GetFirmwareVersion(const QJsonObject& root) {
return version;
}
static GameMetadata GetGameMetadata(const QString& param_file, const QString& fallback) {
static GameMetadata GetGameMetadata(const QByteArray& param_data, const QString& fallback) {
GameMetadata ret;
ret.title_name = fallback;
QFile param(param_file);
if (!param.open(QIODevice::ReadOnly)) {
if (param_data.isEmpty()) {
return ret;
}
QJsonParseError parse_error;
const auto doc = QJsonDocument::fromJson(param.readAll(), &parse_error);
const auto doc = QJsonDocument::fromJson(param_data, &parse_error);
if (parse_error.error != QJsonParseError::NoError || !doc.isObject()) {
return ret;
}
@@ -522,18 +514,18 @@ static GameMetadata GetGameMetadata(const QString& param_file, const QString& fa
}
static void SetGameFiles(Configuration& info, const QString& game_dir, const QString& game_path,
const GameMetadata& metadata) {
QDir game(game_dir);
const GameMetadata& metadata, bool archive) {
const QFileInfo game(game_dir);
info.game_path = game_path;
info.basedir = game.absolutePath();
info.basedir = archive ? game.absoluteFilePath() : QDir(game_dir).absolutePath();
info.name = metadata.title_name;
info.title_id = metadata.title_id;
info.gameVersion = metadata.gameVersion;
info.firmwareVer = metadata.firmwareVer;
if (info.name.isEmpty()) {
info.name = game.dirName();
info.name = archive ? game.completeBaseName() : QDir(game_dir).dirName();
}
}
@@ -609,6 +601,94 @@ void ConfigurationListWidget::ScanGameDirectory() {
const QString eboot_name = QStringLiteral("eboot.bin");
QSet<QString> found_games;
const auto add_game = [this, &found_games, &eboot_name,
&running_items](const QString& base, const QString& game_path,
const QString& legacy_game_path, const QString& fallback,
bool archive) {
const QString game_key = PathKey(game_path);
if (game_key.isEmpty() || found_games.contains(game_key)) {
return;
}
// Keep the index and decompression cache alive through validation, metadata and icon
// reads, after rejecting duplicate candidates from overlapping game folders.
const auto reader = archive ? Common::OpenArchive(GameContent::ToPath(base)) : nullptr;
if (archive && (reader == nullptr || !GameContent::FileExists(base, eboot_name))) {
return;
}
found_games.insert(game_key);
const auto metadata =
GetGameMetadata(GameContent::ReadFile(base, QStringLiteral("sce_sys/param.json"),
GameContent::MaxMetadataSize),
fallback);
auto info = std::make_unique<Configuration>();
info->custom_settings =
FindCustomInfo(&m_custom_infos, game_path, legacy_game_path) != nullptr;
SetGameFiles(*info, base, game_path, metadata, archive);
const auto* compatibility = m_compatibility->Find(info->title_id);
if (compatibility != nullptr) {
info->game_status = compatibility->status;
info->game_comment = compatibility->comment;
}
if (auto* item = running_items.value(game_key); item != nullptr) {
// Refresh metadata without losing the running row's identity.
const QSignalBlocker status_blocker(item->GetStatusCombo());
item->GetInfo().CopyGameInfoFrom(*info);
item->Update(true);
item->SetCompatibilityEditable(m_compatibility->IsLocal() &&
!item->GetInfo().title_id.trimmed().isEmpty());
return;
}
auto* item = new ConfigurationItem(std::move(info), m_ui->cfgs_list);
item->SetCompatibilityEditable(m_compatibility->IsLocal() &&
!item->GetInfo().title_id.trimmed().isEmpty());
connect(item->GetStatusCombo(), &QComboBox::currentIndexChanged, item,
[this, item](int /*index*/) {
if (!m_compatibility->IsLocal()) {
return;
}
const auto& title_id = item->GetInfo().title_id;
if (title_id.trimmed().isEmpty()) {
return;
}
item->GetInfo().game_status = static_cast<Configuration::GameStatus>(
item->GetStatusCombo()->currentData().toInt());
item->Update();
m_compatibility->SetStatus(title_id, item->GetInfo().game_status);
m_ui->cfgs_list->setCurrentItem(item);
SelectItem(item);
});
connect(item->GetCommentEdit(), &QLineEdit::editingFinished, item, [this, item]() {
if (!m_compatibility->IsLocal()) {
return;
}
const auto& title_id = item->GetInfo().title_id;
if (title_id.trimmed().isEmpty()) {
return;
}
item->GetInfo().game_comment = item->GetCommentEdit()->text();
item->Update();
m_compatibility->SetComment(title_id, item->GetInfo().game_comment);
m_ui->cfgs_list->setCurrentItem(item);
SelectItem(item);
});
};
const auto scan_archives = [&add_game](const QDir& directory, const QDir& root) {
const auto entries = directory.entryInfoList(QDir::Files | QDir::NoSymLinks, QDir::Name);
for (const auto& archive: entries) {
const auto base = archive.absoluteFilePath();
if (!Common::IsSupportedArchive(GameContent::ToPath(base))) {
continue;
}
add_game(base, QDir::cleanPath(base), root.relativeFilePath(base),
archive.completeBaseName(), true);
}
};
for (const auto& root_path: m_game_dirs) {
QDir root(root_path);
@@ -616,6 +696,7 @@ void ConfigurationListWidget::ScanGameDirectory() {
continue;
}
scan_archives(root, root);
QList<QDir> pending_dirs;
const auto root_subdirs =
root.entryInfoList(QDir::Dirs | QDir::NoDotAndDotDot | QDir::NoSymLinks);
@@ -625,74 +706,13 @@ void ConfigurationListWidget::ScanGameDirectory() {
while (!pending_dirs.isEmpty()) {
QDir game_dir = pending_dirs.takeFirst();
scan_archives(game_dir, root);
if (game_dir.exists(eboot_name)) {
const QString game_path = NormalizeGameDirectory(game_dir.absolutePath());
const QString legacy_game_path = root.relativeFilePath(game_dir.absolutePath());
const QString game_key = PathKey(game_path);
if (game_key.isEmpty() || found_games.contains(game_key)) {
continue;
}
found_games.insert(game_key);
auto metadata = GetGameMetadata(
game_dir.filePath(QStringLiteral("sce_sys/param.json")), game_dir.dirName());
auto info = std::make_unique<Configuration>();
info->custom_settings =
FindCustomInfo(&m_custom_infos, game_path, legacy_game_path) != nullptr;
SetGameFiles(*info, game_dir.absolutePath(), game_path, metadata);
const auto* compatibility = m_compatibility->Find(info->title_id);
if (compatibility != nullptr) {
info->game_status = compatibility->status;
info->game_comment = compatibility->comment;
}
if (auto* item = running_items.value(game_key); item != nullptr) {
// Refresh metadata without losing the running row's identity.
const QSignalBlocker status_blocker(item->GetStatusCombo());
item->GetInfo().CopyGameInfoFrom(*info);
item->Update();
item->SetCompatibilityEditable(m_compatibility->IsLocal() &&
!item->GetInfo().title_id.trimmed().isEmpty());
continue;
}
auto* item = new ConfigurationItem(std::move(info), m_ui->cfgs_list);
item->SetCompatibilityEditable(m_compatibility->IsLocal() &&
!item->GetInfo().title_id.trimmed().isEmpty());
connect(item->GetStatusCombo(), &QComboBox::currentIndexChanged, item,
[this, item](int /*index*/) {
if (!m_compatibility->IsLocal()) {
return;
}
const auto& title_id = item->GetInfo().title_id;
if (title_id.trimmed().isEmpty()) {
return;
}
item->GetInfo().game_status = static_cast<Configuration::GameStatus>(
item->GetStatusCombo()->currentData().toInt());
item->Update();
m_compatibility->SetStatus(title_id, item->GetInfo().game_status);
m_ui->cfgs_list->setCurrentItem(item);
SelectItem(item);
});
connect(item->GetCommentEdit(), &QLineEdit::editingFinished, item, [this, item]() {
if (!m_compatibility->IsLocal()) {
return;
}
const auto& title_id = item->GetInfo().title_id;
if (title_id.trimmed().isEmpty()) {
return;
}
item->GetInfo().game_comment = item->GetCommentEdit()->text();
item->Update();
m_compatibility->SetComment(title_id, item->GetInfo().game_comment);
m_ui->cfgs_list->setCurrentItem(item);
SelectItem(item);
});
add_game(game_dir.absolutePath(), game_path, legacy_game_path, game_dir.dirName(),
false);
continue;
}
@@ -805,7 +825,8 @@ void ConfigurationListWidget::open_game_folder() {
return;
}
const QDir game_dir(item->GetInfo().basedir);
const auto base = item->GetInfo().basedir;
const QDir game_dir(GameContent::IsArchive(base) ? QFileInfo(base).absolutePath() : base);
if (!game_dir.exists()) {
QMessageBox::warning(this, tr("Open game folder"), tr("Game folder does not exist."));
return;
@@ -896,7 +917,7 @@ void ConfigurationListWidget::SelectItem(QTreeWidgetItem* witem) {
m_ui->edit_button->setEnabled(!item->IsRunning());
m_selected_item = item;
m_ui->cfgs_list->SetBackgroundImage(GetPic0Path(item->GetInfo()));
m_ui->cfgs_list->SetBackgroundImage(item->GetInfo().basedir);
emit Select();
}
@@ -959,7 +980,8 @@ void ConfigurationListWidget::show_context_menu(const QPoint& pos) {
if (item != nullptr) {
action_run->setDisabled(item->IsRunning());
action_open_folder->setDisabled(!QDir(item->GetInfo().basedir).exists());
const auto& base = item->GetInfo().basedir;
action_open_folder->setDisabled(!QDir(base).exists() && !GameContent::IsArchive(base));
action_view_trophies->setDisabled(!has_trophy_data);
action_remove_save_data->setDisabled(item->IsRunning() || save_data_dirs.isEmpty());
action_edit->setDisabled(item->IsRunning());
+80
View File
@@ -0,0 +1,80 @@
#include "gameContent.h"
#include "common/archive.h"
#include "common/file.h"
#include "common/stringUtils.h"
#include <QFileInfo>
#include <limits>
namespace GameContent {
std::filesystem::path ToPath(const QString& path) {
#if defined(_WIN32)
return std::filesystem::path(path.toStdWString());
#else
return std::filesystem::path(path.toStdString());
#endif
}
QString FromPath(const std::filesystem::path& path) {
#if defined(_WIN32)
return QString::fromStdWString(path.wstring());
#else
return QString::fromStdString(Common::PathToString(path));
#endif
}
bool IsArchive(const QString& base) {
const QFileInfo info(base);
return info.isFile() && Common::IsSupportedArchive(ToPath(base));
}
std::filesystem::path Resolve(const QString& base, const QString& relative) {
const auto root = ToPath(base);
return IsArchive(base) ? Common::MakeArchivePath(root, ToPath(relative))
: root / ToPath(relative);
}
bool FileExists(const QString& base, const QString& relative) {
return Common::File::IsFileExisting(Resolve(base, relative));
}
QByteArray ReadFile(const QString& base, const QString& relative, uint64_t max_size) {
return ReadPath(Resolve(base, relative), max_size);
}
QByteArray ReadPath(const std::filesystem::path& path, uint64_t max_size) {
Common::File file(path, Common::File::Mode::Read);
if (file.IsInvalid()) {
return {};
}
const auto size = file.Size();
if (size > max_size || size > static_cast<uint64_t>(std::numeric_limits<int>::max())) {
return {};
}
QByteArray data(static_cast<int>(size), Qt::Uninitialized);
uint32_t bytes_read = 0;
file.Read(data.data(), static_cast<uint32_t>(size), &bytes_read);
return bytes_read == size ? data : QByteArray {};
}
QStringList ListFiles(const QString& base, const QString& relative) {
const auto directory = Resolve(base, relative);
const bool archive = Common::IsArchivePath(directory);
QStringList result;
for (const auto& entry: Common::File::GetDirEntries(directory)) {
if (entry.is_file) {
const auto path = FromPath(directory / Common::PathFromUtf8(entry.name));
if (archive || !QFileInfo(path).isSymLink()) {
result.append(path);
}
}
}
return result;
}
} // namespace GameContent
+3 -2
View File
@@ -3,6 +3,7 @@
#include "configuration.h"
#include "configurationItem.h"
#include "configurationListWidget.h"
#include "gameContent.h"
#include "patchesDialog.h"
#include "updateChecker.h"
@@ -260,7 +261,7 @@ static QStringList CreateEmulatorArgs(const Configuration& info) {
}
QString game = info.basedir;
if (!info.elf.isEmpty()) {
if (!info.elf.isEmpty() && !GameContent::IsArchive(info.basedir)) {
game = QDir(info.basedir).filePath(info.elf);
}
args << "--game" << game;
@@ -507,7 +508,7 @@ void MainDialogPrivate::Update() {
if (run_enabled) {
const auto& info = item->GetInfo();
auto dir = info.basedir;
run_enabled = !dir.isEmpty() && QDir(dir).exists();
run_enabled = !dir.isEmpty() && (QDir(dir).exists() || GameContent::IsArchive(dir));
}
m_ui->widget->SetRunEnabled(run_enabled);
+54 -23
View File
@@ -1,13 +1,14 @@
#include "trophyViewerDialog.h"
#include "common/archive.h"
#include "configuration.h"
#include "gameContent.h"
#include <QAbstractItemView>
#include <QBrush>
#include <QByteArray>
#include <QDialogButtonBox>
#include <QDir>
#include <QFile>
#include <QFileInfo>
#include <QFont>
#include <QHeaderView>
@@ -39,12 +40,14 @@
namespace {
constexpr quint32 UCP_MAGIC = 0xb228c60a;
constexpr quint32 UCP_VERSION = 1;
constexpr int UCP_HEADER_LEN = 0x40;
constexpr int UCP_TOC_SKIP = 0x20;
constexpr int UCP_ENTRY_LEN = 0x40;
constexpr int UCP_NAME_LEN = 0x20;
constexpr quint32 UCP_MAGIC = 0xb228c60a;
constexpr quint32 UCP_VERSION = 1;
constexpr int UCP_HEADER_LEN = 0x40;
constexpr int UCP_TOC_SKIP = 0x20;
constexpr int UCP_ENTRY_LEN = 0x40;
constexpr int UCP_NAME_LEN = 0x20;
constexpr quint32 UCP_MAX_FILES = 4096;
constexpr quint64 UCP_MAX_EXTRACTED_SIZE = quint64 {64} << 20u;
struct UcpEntry {
QString name;
@@ -98,14 +101,22 @@ static QString ReadFixedString(const QByteArray& data, qsizetype offset, qsizety
return QString::fromLatin1(data.constData() + offset, len);
}
static bool IsUsedUcpEntry(const QString& name) {
const auto lower = name.toCaseFolded();
return lower == QStringLiteral("tropconf.json") || lower == QStringLiteral("tropmeta.json") ||
(lower.startsWith(QStringLiteral("tropmeta_")) &&
lower.endsWith(QStringLiteral(".json"))) ||
(lower.startsWith(QStringLiteral("trop")) && lower.endsWith(QStringLiteral(".png")));
}
static bool ReadUcp(const QString& file_name, QMap<QString, QByteArray>& files, QString& error) {
QFile file(file_name);
if (!file.open(QIODevice::ReadOnly)) {
const QByteArray data =
GameContent::ReadPath(GameContent::ToPath(file_name), GameContent::MaxTrophyPackageSize);
if (data.isEmpty()) {
error = QObject::tr("Could not open %1").arg(QDir::toNativeSeparators(file_name));
return false;
}
const QByteArray data = file.readAll();
if (data.size() < UCP_HEADER_LEN) {
error = QObject::tr("%1 is too small to be a trophy package.")
.arg(QFileInfo(file_name).fileName());
@@ -127,14 +138,18 @@ static bool ReadUcp(const QString& file_name, QMap<QString, QByteArray>& files,
}
const auto declared_size = qFromBigEndian<quint64>(data.constData() + 0x08);
if (declared_size > static_cast<quint64>(data.size())) {
if (declared_size < UCP_HEADER_LEN || declared_size > static_cast<quint64>(data.size())) {
error = QObject::tr("%1 is truncated.").arg(QFileInfo(file_name).fileName());
return false;
}
const auto file_count = qFromBigEndian<quint32>(data.constData() + 0x10);
if (file_count > UCP_MAX_FILES) {
error = QObject::tr("%1 contains too many files.").arg(QFileInfo(file_name).fileName());
return false;
}
const auto toc_offset = static_cast<quint64>(qFromBigEndian<quint32>(data.constData() + 0x14));
const auto data_size = static_cast<quint64>(data.size());
const auto data_size = declared_size;
const quint64 table_size = UCP_TOC_SKIP + static_cast<quint64>(file_count) * UCP_ENTRY_LEN;
if (toc_offset > data_size || table_size > data_size - toc_offset ||
@@ -144,6 +159,7 @@ static bool ReadUcp(const QString& file_name, QMap<QString, QByteArray>& files,
return false;
}
quint64 extracted_size = 0;
for (quint32 i = 0; i < file_count; i++) {
const auto entry_offset = static_cast<qsizetype>(toc_offset + UCP_TOC_SKIP +
static_cast<quint64>(i) * UCP_ENTRY_LEN);
@@ -161,9 +177,22 @@ static bool ReadUcp(const QString& file_name, QMap<QString, QByteArray>& files,
.arg(QFileInfo(file_name).fileName(), entry.name);
return false;
}
if (!IsUsedUcpEntry(entry.name)) {
continue;
}
const auto entry_limit = entry.name.endsWith(QStringLiteral(".png"), Qt::CaseInsensitive)
? GameContent::MaxImageSize
: GameContent::MaxMetadataSize;
if (entry.size > entry_limit || entry.size > UCP_MAX_EXTRACTED_SIZE - extracted_size) {
error = QObject::tr("%1 contains an oversized entry for %2.")
.arg(QFileInfo(file_name).fileName(), entry.name);
return false;
}
files.insert(entry.name.toCaseFolded(), data.mid(static_cast<qsizetype>(entry.offset),
static_cast<qsizetype>(entry.size)));
extracted_size += entry.size;
}
return true;
@@ -392,30 +421,31 @@ static QStringList FindTrophyFiles(const Configuration* info) {
return {};
}
const QDir trophy_dir(QDir(info->basedir).filePath(QStringLiteral("sce_sys/trophy2")));
if (!trophy_dir.exists()) {
return {};
QStringList files;
for (const auto& file:
GameContent::ListFiles(info->basedir, QStringLiteral("sce_sys/trophy2"))) {
const auto name = QFileInfo(file).fileName();
if (name.startsWith(QStringLiteral("trophy"), Qt::CaseInsensitive) &&
name.endsWith(QStringLiteral(".ucp"), Qt::CaseInsensitive)) {
files.append(file);
}
}
const auto files =
trophy_dir.entryInfoList({QStringLiteral("Trophy*.ucp"), QStringLiteral("trophy*.ucp")},
QDir::Files | QDir::NoSymLinks, QDir::Name | QDir::IgnoreCase);
files.sort(Qt::CaseInsensitive);
QStringList trophy_files;
QSet<QString> seen;
for (const auto& file: files) {
auto key = file.canonicalFilePath();
auto key = QFileInfo(file).canonicalFilePath();
if (key.isEmpty()) {
key = file.absoluteFilePath();
key = file;
}
key = QDir::cleanPath(key).toCaseFolded();
if (!seen.contains(key)) {
seen.insert(key);
trophy_files.append(file.absoluteFilePath());
trophy_files.append(file);
}
}
return trophy_files;
}
@@ -506,6 +536,7 @@ void TrophyViewerDialog::ShowForGame(const Configuration* info, QWidget* parent)
}
bool TrophyViewerDialog::LoadGame(const Configuration& info, QString& error) {
const auto reader = Common::OpenArchive(GameContent::ToPath(info.basedir));
const auto trophy_files = FindTrophyFiles(&info);
if (trophy_files.isEmpty()) {
error = tr("No trophy package found in sce_sys/trophy2.");
+1 -3
View File
@@ -161,10 +161,8 @@ int KYTY_SYSV_ABI AjmDecAt9ParseConfigData(const void* config_data,
return AJM_ERROR_INVALID_PARAMETER;
}
uint8_t config[ATRAC9_CONFIG_DATA_SIZE] {};
std::memcpy(config, config_data, sizeof(config));
Atrac9CodecInfo codec_info {};
const int init_result = Atrac9InitDecoder(handle, config);
const int init_result = AjmAt9InitDecoder(handle, static_cast<const uint8_t*>(config_data));
const int info_result =
init_result == 0 ? Atrac9GetCodecInfo(handle, &codec_info) : init_result;
Atrac9ReleaseHandle(handle);
+14 -1
View File
@@ -18,6 +18,19 @@ extern "C" {
namespace Libs::Audio::Ajm {
inline int AjmAt9InitDecoder(void* handle, const uint8_t* config_data) {
// Keep the guest configuration unchanged for metadata and decoder resets.
uint8_t config[ATRAC9_CONFIG_DATA_SIZE];
std::memcpy(config, config_data, sizeof(config));
const auto channel_config = (config[1] >> 1u) & 7u;
if (channel_config >= 6u) {
// LibAtrac9 lacks vibration layouts; their blocks match ordinary mono/dual-mono.
// Translate only the channel field, preserving the other configuration bits.
config[1] -= 6u << 1u;
}
return Atrac9InitDecoder(handle, config);
}
struct AjmDecAt9InitializeParameters {
uint8_t config_data[ATRAC9_CONFIG_DATA_SIZE];
uint32_t reserved;
@@ -278,7 +291,7 @@ private:
std::memcpy(m_config_data, config_data, ATRAC9_CONFIG_DATA_SIZE);
m_has_config = true;
const int init_ret = Atrac9InitDecoder(m_handle, m_config_data);
const int init_ret = AjmAt9InitDecoder(m_handle, m_config_data);
if (init_ret != 0) {
m_is_initialized = false;
result->result = AJM_RESULT_CODEC_ERROR | AJM_RESULT_INVALID_DATA;
+22 -10
View File
@@ -32,6 +32,7 @@ constexpr int AUDIO_OUT_PORT_TYPE_AUDIO3D = 126;
constexpr int AUDIO_OUT_PORT_TYPE_AUX = 127;
constexpr uint32_t AUDIO_OUT_PARAM_FORMAT_MASK = 0x000000ffu;
constexpr uint64_t AUDIO_OUT_TARGET_LATENCY_US = 40000;
constexpr int AUDIO_IN_SILENT_STATE_DEVICE_NONE = 0x1;
constexpr uint32_t AUDIO_IN_GRAIN_MAX_ASYNC = 384;
@@ -368,11 +369,11 @@ bool Audio::QueueSdlAudio(PortOut* port, const void* data, bool blocking) {
uint32_t min_queued_size = 0;
if (blocking) {
constexpr uint64_t target_latency_us = 40000;
const auto buffer_us = port->freq != 0 ? (1000000ULL * port->samples_num) / port->freq : 0;
const auto buffers =
buffer_us != 0 ? static_cast<uint32_t>((target_latency_us + buffer_us - 1) / buffer_us)
: 2u;
buffer_us != 0
? static_cast<uint32_t>((AUDIO_OUT_TARGET_LATENCY_US + buffer_us - 1) / buffer_us)
: 2u;
min_queued_size = prepared_size * std::clamp(buffers, 2u, 16u);
const auto wait_start = LibKernel::KernelGetProcessTime();
auto queued = SDL_GetAudioStreamQueued(port->stream);
@@ -440,11 +441,14 @@ Audio::Id Audio::AudioOutOpen(int type, uint32_t samples_num, uint32_t freq, For
port.volume[i] = 32768;
}
if (type == AUDIO_OUT_PORT_TYPE_VIBRATION) {
port.haptics = Controller::DualSenseHaptics::Open(freq);
} else {
// Pad speaker ports keep the main output too; it plays them whenever no DualSense does.
if (type != AUDIO_OUT_PORT_TYPE_VIBRATION) {
OpenSdlDevice(&port);
}
if (type == AUDIO_OUT_PORT_TYPE_VIBRATION || type == AUDIO_OUT_PORT_TYPE_PADSPK) {
port.haptics =
Controller::DualSenseHaptics::Open(freq, type == AUDIO_OUT_PORT_TYPE_PADSPK);
}
return Id::Create(id);
}
@@ -554,14 +558,22 @@ uint32_t Audio::AudioOutOutputs(OutputParam* params, uint32_t num, bool blocking
for (uint32_t i = 0; i < num; i++) {
auto& port = m_out_ports[params[i].handle.GetId()];
if (port.type == AUDIO_OUT_PORT_TYPE_VIBRATION) {
// Haptics never pace output; keep the stream alive against close and volume changes.
uint64_t controller_queued_us = 0;
if (port.haptics != nullptr) {
// Keep the stream alive against close and volume changes.
Common::LockGuard lock(m_mutex);
Controller::DualSenseHaptics::Queue(
controller_queued_us = Controller::DualSenseHaptics::Queue(
port.haptics, Controller::GetActiveControllerId(), params[i].data, port.samples_num,
static_cast<uint32_t>(port.channels_num), FormatIsFloat(port.format), port.volume);
} else {
}
if (controller_queued_us == 0) {
// No DualSense took it (e.g. it was unplugged); a pad speaker port plays on the main
// output instead, and a vibration port has none.
QueueSdlAudio(&port, params[i].data, blocking);
} else if (blocking && port.type == AUDIO_OUT_PORT_TYPE_PADSPK &&
controller_queued_us > AUDIO_OUT_TARGET_LATENCY_US) {
// Vibration never paces output; the speaker is audible, so pace it like other ports.
Common::Thread::SleepMicro(controller_queued_us - AUDIO_OUT_TARGET_LATENCY_US);
}
}
+3
View File
@@ -70,6 +70,9 @@ int KYTY_SYSV_ABI AudioOut2PortGetState(AudioOut2PortHandle port, AudioOut2PortS
int KYTY_SYSV_ABI AudioOut2GetSystemState(AudioOut2SystemState* state);
int KYTY_SYSV_ABI AudioOut2UserCreate(uint32_t user_id, AudioOut2UserHandle* handle);
int KYTY_SYSV_ABI AudioOut2UserDestroy(AudioOut2UserHandle handle);
int KYTY_SYSV_ABI AudioOut2UserGetSupportedAttributes(AudioOut2UserHandle handle,
uint32_t* context_attributes,
uint32_t* port_attributes);
size_t KYTY_SYSV_ABI AudioOut2GetSpeakerArrayMemorySize(uint32_t num_speakers, uint8_t is_3d,
uint8_t is_ambisonics);
int KYTY_SYSV_ABI AudioOut2SpeakerArrayCreate(AudioOut2SpeakerArrayHandle* handle,
+72 -51
View File
@@ -1,6 +1,6 @@
#include "common/common.h"
#include "common/file.h"
#include "common/logging/log.h"
#include "common/stringUtils.h"
#include "kernel/fileSystem.h"
#include "kernel/pthread.h"
#include "libs/audio.h"
@@ -9,12 +9,14 @@
#include <algorithm>
#include <atomic>
#include <cctype>
#include <cerrno>
#include <condition_variable>
#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <cstring>
#include <deque>
#include <limits>
#include <magic_enum.hpp>
#include <memory>
#include <mutex>
@@ -577,30 +579,40 @@ public:
explicit FileStreamer(AvPlayerFileReplacement f): file(f) {}
~FileStreamer() {
if (ctx != nullptr) {
av_freep(&ctx->buffer);
avio_context_free(&ctx);
}
if (opened && file.close != nullptr) {
if (opened) {
file.close(file.object_pointer);
}
}
bool Init(const std::string& path) {
if (file.open == nullptr || file.close == nullptr || file.read_offset == nullptr ||
file.size == nullptr) {
if (file.open != nullptr) {
if (file.close == nullptr || file.read_offset == nullptr || file.size == nullptr ||
file.open(file.object_pointer, path.c_str()) < 0) {
return false;
}
opened = true;
size = file.size(file.object_pointer);
} else {
if (!local_file.Open(LibKernel::FileSystem::GetRealFilename(path),
Common::File::Mode::Read)) {
return false;
}
size = local_file.Size();
}
if (size == 0 || size > static_cast<uint64_t>(std::numeric_limits<int64_t>::max())) {
return false;
}
if (file.open(file.object_pointer, path.c_str()) < 0) {
return false;
}
opened = true;
size = file.size(file.object_pointer);
if (size == 0) {
return false;
}
auto* buf = static_cast<uint8_t*>(av_malloc(4096));
constexpr int buffer_size = 64 * 1024;
auto* buf = static_cast<uint8_t*>(av_malloc(buffer_size));
if (buf == nullptr) {
return false;
}
ctx = avio_alloc_context(buf, 4096, 0, this, Read, nullptr, Seek);
ctx = avio_alloc_context(buf, buffer_size, 0, this, Read, nullptr, Seek);
if (ctx == nullptr) {
av_free(buf);
}
return ctx != nullptr;
}
AVIOContext* Context() const { return ctx; }
@@ -611,33 +623,50 @@ private:
if (s->pos >= s->size) {
return AVERROR_EOF;
}
len = static_cast<int>(std::min<uint64_t>(len, s->size - s->pos));
auto r =
s->file.read_offset(s->file.object_pointer, buf, s->pos, static_cast<uint32_t>(len));
if (r <= 0) {
return r == 0 ? AVERROR_EOF : r;
len = static_cast<int>(std::min<uint64_t>(len, s->size - s->pos));
int read = 0;
if (s->opened) {
read = s->file.read_offset(s->file.object_pointer, buf, s->pos,
static_cast<uint32_t>(len));
} else {
uint32_t bytes = 0;
s->local_file.Read(buf, static_cast<uint32_t>(len), &bytes);
read = static_cast<int>(bytes);
}
s->pos += static_cast<uint64_t>(r);
return r;
if (read <= 0 || read > len) {
return read < 0 ? read : AVERROR(EIO);
}
s->pos += static_cast<uint64_t>(read);
return read;
}
static int64_t Seek(void* opaque, int64_t off, int whence) {
static int64_t Seek(void* opaque, int64_t offset, int whence) {
auto* s = static_cast<FileStreamer*>(opaque);
if ((whence & AVSEEK_SIZE) != 0) {
return static_cast<int64_t>(s->size);
}
int64_t p = whence == SEEK_SET
? off
: (whence == SEEK_CUR
? static_cast<int64_t>(s->pos) + off
: (whence == SEEK_END ? static_cast<int64_t>(s->size) + off : -1));
if (p < 0) {
return -1;
whence &= ~AVSEEK_FORCE;
uint64_t base = 0;
switch (whence) {
case SEEK_SET: break;
case SEEK_CUR: base = s->pos; break;
case SEEK_END: base = s->size; break;
default: return AVERROR(EINVAL);
}
p = std::min<int64_t>(p, static_cast<int64_t>(s->size));
s->pos = static_cast<uint64_t>(p);
return p;
const auto distance = offset < 0 ? uint64_t {0} - static_cast<uint64_t>(offset)
: static_cast<uint64_t>(offset);
if ((offset < 0 && distance > base) ||
(offset >= 0 && distance > std::numeric_limits<int64_t>::max() - base)) {
return AVERROR(EINVAL);
}
const auto position = offset < 0 ? base - distance : base + distance;
if (!s->opened && !s->local_file.Seek(position)) {
return AVERROR(EIO);
}
s->pos = position;
return static_cast<int64_t>(position);
}
AvPlayerFileReplacement file;
Common::File local_file;
bool opened = false;
uint64_t pos = 0;
uint64_t size = 0;
@@ -675,25 +704,17 @@ public:
return static_cast<Source*>(opaque)->interrupt_io.load() ? 1 : 0;
};
raw->interrupt_callback.opaque = this;
if (file.open != nullptr) {
streamer = std::make_unique<FileStreamer>(file);
if (!streamer->Init(path)) {
avformat_free_context(raw);
return AVPLAYER_ERROR_OPERATION_FAILED;
}
raw->pb = streamer->Context();
if (auto rc = avformat_open_input(&raw, nullptr, nullptr, nullptr); rc < 0) {
LOGF("\t avformat_open_input callback failed: %s\n", fferr(rc).c_str());
return AVPLAYER_ERROR_OPERATION_FAILED;
}
} else {
auto real = LibKernel::FileSystem::GetRealFilename(std::string(path.c_str()));
auto real_str = Common::PathToString(real);
if (auto rc = avformat_open_input(&raw, real_str.c_str(), nullptr, nullptr); rc < 0) {
LOGF("\t avformat_open_input failed: %s path=%s\n", fferr(rc).c_str(),
real_str.c_str());
return AVPLAYER_ERROR_OPERATION_FAILED;
}
streamer = std::make_unique<FileStreamer>(file);
if (!streamer->Init(path)) {
avformat_free_context(raw);
return AVPLAYER_ERROR_OPERATION_FAILED;
}
raw->pb = streamer->Context();
if (const auto rc = avformat_open_input(&raw, file.open == nullptr ? path.c_str() : nullptr,
nullptr, nullptr);
rc < 0) {
LOGF("\t avformat_open_input failed: %s path=%s\n", fferr(rc).c_str(), path.c_str());
return AVPLAYER_ERROR_OPERATION_FAILED;
}
fmt = raw;
if (auto rc = avformat_find_stream_info(fmt, nullptr); rc < 0) {
+29 -14
View File
@@ -6,6 +6,7 @@
#include "libs/errno.h"
#include "libs/libs.h"
#include <atomic>
#include <cstdio>
#include <cstring>
#include <mutex>
@@ -306,6 +307,7 @@ LIB_NAME("SaveDataDialog", "SaveDataDialog");
constexpr int SAVE_STATUS_NONE = 0;
constexpr int SAVE_STATUS_INITIALIZED = 1;
constexpr int SAVE_STATUS_RUNNING = 2;
constexpr int SAVE_STATUS_FINISHED = 3;
constexpr int SAVE_RESULT_OK = 0;
constexpr int SAVE_BUTTON_ID_OK = 1;
@@ -351,18 +353,20 @@ struct SaveDataDialogResult {
uint8_t reserved[32];
};
static int g_save_status = SAVE_STATUS_NONE;
static int g_save_mode = 0;
static void* g_save_user_data = nullptr;
static char g_save_dir_name[sizeof(SaveDataDirName::data)] {};
static std::atomic<int> g_save_status = SAVE_STATUS_NONE;
static bool g_save_running_polled = false;
static int g_save_mode = 0;
static void* g_save_user_data = nullptr;
static char g_save_dir_name[sizeof(SaveDataDirName::data)] {};
int KYTY_SYSV_ABI SaveDataDialogInitialize() {
PRINT_NAME();
g_save_status = SAVE_STATUS_INITIALIZED;
g_save_mode = 0;
g_save_user_data = nullptr;
g_save_dir_name[0] = '\0';
g_save_status = SAVE_STATUS_INITIALIZED;
g_save_running_polled = false;
g_save_mode = 0;
g_save_user_data = nullptr;
g_save_dir_name[0] = '\0';
return OK;
}
@@ -376,6 +380,14 @@ int KYTY_SYSV_ABI SaveDataDialogGetStatus() {
int KYTY_SYSV_ABI SaveDataDialogUpdateStatus() {
PRINT_NAME();
// Some games require a RUNNING update before FINISHED.
if (g_save_status == SAVE_STATUS_RUNNING) {
if (g_save_running_polled) {
g_save_status = SAVE_STATUS_FINISHED;
}
g_save_running_polled = true;
}
return g_save_status;
}
@@ -431,7 +443,8 @@ int KYTY_SYSV_ABI SaveDataDialogOpen(const void* param) {
}
}
g_save_status = SAVE_STATUS_FINISHED;
g_save_status = SAVE_STATUS_RUNNING;
g_save_running_polled = false;
return OK;
}
@@ -441,7 +454,8 @@ int KYTY_SYSV_ABI SaveDataDialogClose(const void* close_param) {
LOGF("\t close_param = 0x%016" PRIx64 "\n", reinterpret_cast<uint64_t>(close_param));
g_save_status = SAVE_STATUS_FINISHED;
g_save_status = SAVE_STATUS_FINISHED;
g_save_running_polled = false;
return OK;
}
@@ -455,10 +469,11 @@ int KYTY_SYSV_ABI SaveDataDialogIsReadyToDisplay() {
int KYTY_SYSV_ABI SaveDataDialogTerminate() {
PRINT_NAME();
g_save_status = SAVE_STATUS_NONE;
g_save_mode = 0;
g_save_user_data = nullptr;
g_save_dir_name[0] = '\0';
g_save_status = SAVE_STATUS_NONE;
g_save_running_polled = false;
g_save_mode = 0;
g_save_user_data = nullptr;
g_save_dir_name[0] = '\0';
return 0;
}
+171 -53
View File
@@ -5,13 +5,17 @@
#include <SDL3/SDL.h>
#include <algorithm>
#include <array>
#include <cmath>
#include <map>
#include <vector>
namespace Libs::Controller::DualSenseHaptics {
struct Stream {
uint32_t freq = 0;
bool speaker = false;
int controller = -1;
SDL_AudioStream* sdl = nullptr;
SDL_AudioDeviceID device = 0;
uint64_t next_check = 0;
@@ -20,19 +24,26 @@ struct Stream {
namespace {
constexpr int FRAME_BYTES = 4 * sizeof(float);
Common::Mutex g_mutex;
int g_controller = -1;
int g_playback_streams = 0;
uint8_t g_large_motor = 0, g_small_motor = 0;
uint64_t g_rumble_until = 0, g_haptics_until = 0;
constexpr int FRAME_BYTES = 4 * sizeof(float);
Common::Mutex g_mutex;
int g_controller = -1;
int g_haptics_streams = 0;
uint8_t g_large_motor = 0, g_small_motor = 0;
uint64_t g_rumble_until = 0, g_haptics_until = 0;
int g_speaker_controller = -1;
std::map<int, int> g_speaker_streams;
void ApplyRumble() { // Caller holds g_mutex; no calls into Audio or Controller.
void ApplyRumble(bool force = false) { // Caller holds g_mutex; no calls into Audio or Controller.
SDL_LockJoysticks();
if (auto* pad = SDL_GetGamepadFromID(static_cast<SDL_JoystickID>(g_controller));
pad != nullptr) {
const auto now = SDL_GetTicks();
const bool muted = g_haptics_until > now || g_rumble_until <= now;
if (force && !muted) {
// SDL caches motor strengths. Force a report after raw speaker routing changed the
// controller's motor mode without changing that cache.
(void)SDL_RumbleGamepad(pad, 0, 0, 0);
}
(void)SDL_RumbleGamepad(
pad, muted ? 0 : g_large_motor * 0x101U, muted ? 0 : g_small_motor * 0x101U,
g_rumble_until > now ? static_cast<uint32_t>(g_rumble_until - now) : 0);
@@ -42,14 +53,52 @@ void ApplyRumble() { // Caller holds g_mutex; no calls into Audio or Controller.
void SelectController(int controller) {
if (g_controller != controller) {
g_haptics_until = 0;
g_haptics_until = g_rumble_until = 0;
ApplyRumble();
g_controller = controller;
g_large_motor = g_small_motor = 0;
g_rumble_until = 0;
}
}
bool RouteSpeaker(int controller, bool speaker) { // Caller holds g_mutex.
// Route the right channel to the speaker and mute the headphones with the values Linux uses
// without headphones (output report 0x02). Plugged-in headphones are not checked.
std::array<uint8_t, 38> report {};
report[0] = 0x80; // Audio control valid
if (speaker) {
report[0] |= 0x20; // Speaker volume valid
report[1] = 0x80; // Audio control 2 valid
report[5] = 0x64; // Speaker volume
report[7] = 0x30; // Output path: right channel to the speaker, headphones muted
report[37] = 0x02; // Speaker preamp +6 dB
}
bool routed = false;
SDL_LockJoysticks();
if (auto* pad = SDL_GetGamepadFromID(static_cast<SDL_JoystickID>(controller)); pad != nullptr) {
routed = SDL_SendGamepadEffect(pad, report.data(), static_cast<int>(report.size()));
}
SDL_UnlockJoysticks();
if (routed && controller == g_controller) {
ApplyRumble(true);
}
return routed;
}
bool SelectSpeaker(int controller) { // Caller holds g_mutex; -1 restores headphone routing.
if (g_speaker_controller == controller) {
return true;
}
if (g_speaker_controller != -1) {
RouteSpeaker(g_speaker_controller, false);
}
g_speaker_controller = -1;
if (controller != -1 && !RouteSpeaker(controller, true)) {
return false;
}
g_speaker_controller = controller;
return true;
}
void SDLCALL UpdateRumble(void*, SDL_AudioStream*, int, int) {
Common::LockGuard lock(g_mutex);
if (g_haptics_until != 0 && SDL_GetTicks() >= g_haptics_until) {
@@ -64,22 +113,55 @@ void CloseDevice(Stream* stream) {
SDL_DestroyAudioStream(stream->sdl);
stream->sdl = nullptr;
Common::LockGuard lock(g_mutex);
if (--g_playback_streams == 0 && g_haptics_until != 0) {
if (!stream->speaker && --g_haptics_streams == 0 && g_haptics_until != 0) {
g_haptics_until = 0;
ApplyRumble();
}
// Route back to the headphones once no speaker port plays on the controller.
if (stream->speaker && --g_speaker_streams.at(stream->controller) == 0) {
g_speaker_streams.erase(stream->controller);
if (g_speaker_controller == stream->controller) {
SelectSpeaker(-1);
}
}
}
}
bool CanUseDevice(int controller) {
// SDL exposes no portable association between a gamepad and its USB audio endpoint.
// Require one wired DualSense; guessing with multiple pads could play on the wrong one.
SDL_LockJoysticks();
auto* pad = SDL_GetGamepadFromID(static_cast<SDL_JoystickID>(controller));
bool usable =
pad != nullptr && SDL_GetGamepadConnectionState(pad) == SDL_JOYSTICK_CONNECTION_WIRED;
int count = 0;
auto* pads = SDL_GetGamepads(&count);
int dualsense_count = 0;
bool found = false;
for (int i = 0; i < count; i++) {
if (SDL_GetGamepadTypeForID(pads[i]) == SDL_GAMEPAD_TYPE_PS5) {
dualsense_count++;
found |= pads[i] == static_cast<SDL_JoystickID>(controller);
}
}
SDL_free(pads);
SDL_UnlockJoysticks();
return usable && found && dualsense_count == 1;
}
SDL_AudioDeviceID FindDevice() {
int count = 0;
auto* devices = SDL_GetAudioPlaybackDevices(&count);
SDL_AudioDeviceID device = 0;
for (int i = 0; i < count && device == 0; i++) {
for (int i = 0; i < count; i++) {
const char* name = SDL_GetAudioDeviceName(devices[i]);
SDL_AudioSpec spec {};
if (name != nullptr && SDL_strcasestr(name, "DualSense") != nullptr &&
SDL_GetAudioDeviceFormat(devices[i], &spec, nullptr) && spec.channels == 4) {
if (device != 0) {
device = 0; // Multiple matching endpoints cannot be associated reliably either.
break;
}
device = devices[i];
}
}
@@ -87,13 +169,59 @@ SDL_AudioDeviceID FindDevice() {
return device;
}
void RefreshDevice(Stream* stream, int controller) {
if (stream->controller != controller) {
CloseDevice(stream);
stream->controller = controller;
stream->next_check = 0;
}
const auto now = SDL_GetTicks();
if (now < stream->next_check) {
return;
}
stream->next_check = now + 2000;
const auto device = FindDevice();
if (stream->device == device && stream->sdl != nullptr) {
return;
}
CloseDevice(stream);
stream->device = device;
if (device == 0) {
return;
}
const SDL_AudioSpec desired {SDL_AUDIO_F32, 4, static_cast<int>(stream->freq)};
auto* sdl = SDL_OpenAudioDeviceStream(device, &desired, UpdateRumble, nullptr);
SDL_AudioSpec actual {};
bool ready = sdl != nullptr &&
SDL_GetAudioDeviceFormat(SDL_GetAudioStreamDevice(sdl), &actual, nullptr) &&
actual.channels == 4 && SDL_ResumeAudioStreamDevice(sdl);
if (ready) {
Common::LockGuard lock(g_mutex);
if (stream->speaker) {
ready = SelectSpeaker(controller);
if (ready) {
g_speaker_streams[controller]++;
}
} else {
g_haptics_streams++;
}
}
if (ready) {
stream->sdl = sdl;
LOGF("DualSenseHaptics: playing on '%s'\n", SDL_GetAudioDeviceName(device));
} else {
SDL_DestroyAudioStream(sdl);
LOGF("DualSenseHaptics: cannot open quad output: %s\n", SDL_GetError());
}
}
} // namespace
Stream* Open(uint32_t freq) {
Stream* Open(uint32_t freq, bool speaker) {
if (freq == 0 || !SDL_InitSubSystem(SDL_INIT_AUDIO)) {
return nullptr;
}
return new Stream {.freq = freq};
return new Stream {.freq = freq, .speaker = speaker};
}
void Close(Stream* stream) {
@@ -104,44 +232,30 @@ void Close(Stream* stream) {
}
}
void Queue(Stream* stream, int controller, const void* data, uint32_t frames, uint32_t channels,
bool is_float, const int* volume) {
uint64_t Queue(Stream* stream, int controller, const void* data, uint32_t frames, uint32_t channels,
bool is_float, const int* volume) {
if (stream == nullptr || data == nullptr || frames == 0 || channels == 0 || volume == nullptr) {
return;
return 0;
}
if (SDL_GetGamepadTypeForID(static_cast<SDL_JoystickID>(controller)) != SDL_GAMEPAD_TYPE_PS5) {
if (!CanUseDevice(controller)) {
CloseDevice(stream);
return;
}
const auto now = SDL_GetTicks();
if (now >= stream->next_check) {
stream->next_check = now + 2000;
const auto device = FindDevice();
if (stream->device != device || stream->sdl == nullptr) {
CloseDevice(stream);
stream->device = device;
if (device != 0) {
const SDL_AudioSpec desired {SDL_AUDIO_F32, 4, static_cast<int>(stream->freq)};
auto* sdl = SDL_OpenAudioDeviceStream(device, &desired, UpdateRumble, nullptr);
SDL_AudioSpec actual {};
if (sdl != nullptr &&
SDL_GetAudioDeviceFormat(SDL_GetAudioStreamDevice(sdl), &actual, nullptr) &&
actual.channels == 4 && SDL_ResumeAudioStreamDevice(sdl)) {
stream->sdl = sdl;
{
Common::LockGuard lock(g_mutex);
g_playback_streams++;
}
LOGF("DualSenseHaptics: playing on '%s'\n", SDL_GetAudioDeviceName(device));
} else {
SDL_DestroyAudioStream(sdl);
LOGF("DualSenseHaptics: cannot open quad output: %s\n", SDL_GetError());
}
}
}
stream->next_check = 0; // Look for the device as soon as a DualSense is active again.
return 0;
}
RefreshDevice(stream, controller);
if (stream->sdl == nullptr) {
return;
return 0;
}
if (stream->speaker) {
bool routed;
{
Common::LockGuard lock(g_mutex);
routed = SelectSpeaker(controller);
}
if (!routed) {
CloseDevice(stream);
return 0;
}
}
stream->buffer.assign(static_cast<size_t>(frames) * 4, 0.0f);
bool audible = false;
@@ -152,7 +266,8 @@ void Queue(Stream* stream, int controller, const void* data, uint32_t frames, ui
float value = is_float ? static_cast<const float*>(data)[index]
: static_cast<const int16_t*>(data)[index] / 32768.0f;
value *= volume[src_ch] / 32768.0f;
stream->buffer[static_cast<size_t>(frame) * 4 + ch + 2] = value;
// The speaker plays the right front channel; the actuators play the back ones.
stream->buffer[static_cast<size_t>(frame) * 4 + ch + (stream->speaker ? 0 : 2)] = value;
audible |= std::fabs(value) > 1.0f / 1024;
}
}
@@ -161,26 +276,29 @@ void Queue(Stream* stream, int controller, const void* data, uint32_t frames, ui
SDL_ClearAudioStream(stream->sdl);
queued = 0;
}
const int bytes = static_cast<int>(frames * FRAME_BYTES);
if (audible) {
const int bytes = static_cast<int>(frames * FRAME_BYTES);
const auto queued_us =
static_cast<uint64_t>(queued + bytes) * 1000000 / (stream->freq * FRAME_BYTES);
if (audible && !stream->speaker) {
Common::LockGuard lock(g_mutex);
if (SDL_GetGamepadTypeForID(static_cast<SDL_JoystickID>(controller)) !=
SDL_GAMEPAD_TYPE_PS5) {
return;
return 0;
}
SelectController(controller);
const auto playback_now = SDL_GetTicks();
const bool starting = g_haptics_until <= playback_now;
const auto duration = (queued + bytes) * 1000ULL / (stream->freq * FRAME_BYTES);
g_haptics_until =
std::max(g_haptics_until, playback_now + std::max<uint64_t>(duration, 250));
std::max(g_haptics_until, playback_now + std::max<uint64_t>(queued_us / 1000, 250));
if (starting) {
ApplyRumble();
}
}
if (!SDL_PutAudioStreamData(stream->sdl, stream->buffer.data(), bytes)) {
CloseDevice(stream);
return 0;
}
return queued_us;
}
bool SetVibration(int controller, uint8_t large_motor, uint8_t small_motor) {
@@ -198,8 +316,8 @@ bool SetVibration(int controller, uint8_t large_motor, uint8_t small_motor) {
void Shutdown() {
Common::LockGuard lock(g_mutex);
g_rumble_until = 0;
SelectController(-1);
SelectSpeaker(-1);
}
} // namespace Libs::Controller::DualSenseHaptics

Some files were not shown because too many files have changed in this diff Show More