Spdlog migration

[Niram7777]

TODO:

• filter
• duplicate
• update doc
• group log config
• refactor to avoid get -> TODO separate PR, no perf diff seen
• someone should test Windows
• fix truncated log on SDL exit button (already was there before)
• async

Various info:

• in async, the log thread is the 3rd one but it lost his name (thread 1 Main, thread 2 Tracy)
• bitfields have to be promoted to underlying type with + (I havent found variadic template with bitfield overloading possible)
• control logging:

shadps4 "SPDLOG_LEVEL=off,Render.Vulkan=debug,Lib.AudioOut=info"
or
SPDLOG_LEVEL="warning,logger1=debug,logger2=info" shadps4
or config file
filter = "trace"

• default:

[Log]
append = false
enable = true
filter = ""
maxSkipDuration = 5000
separate = false
sizeLimit = 104857600
skipDuplicate = true
sync = true
type = "wincolor" # Windows only

• log level first letter is lowercase
• maybe we should use utf-8 on Windows? https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page

[Niram7777]

@OFFTKP @viniciuslrangel you remember exactly why native string was not enough ?

6295d6c
I dont see what is the issue you had (but Windows is encoding UTF strangely to me)
https://godbolt.org/z/6Td1EooeT

[OFFTKP]

@Niram7777
IIRC, the commit you linked removes occurences of my_path.string() because on Windows if my_path contains Greek or Cyrillic or other weird characters converting to regular string would error, and wstring is used instead.

[OFFTKP]

But looking at the godbolt example it doesn't seem to have issues, so idk. This was a long time ago.

[Niram7777]

But looking at the godbolt example it doesn't seem to have issues, so idk. This was a long time ago.

yeah I cant reproduce the issue, once the PR is ready (hopefully next weekend) if you could test it, it would be great

[OFFTKP]

@Niram7777 The issue might've been related with path->qstring and qstring->path conversions. Anyway, there was some bug where opening a game that is inside a path that contained non-ascii characters would cause issues. It could've been only during path to qstring and backwards conversion that has an issue but we ended up modifying all places that convert path to string due to misunderstanding.

[Niram7777]

I would need people to test on Windows with different combination of

[Log]
filter = ""
skip = true
type = "sync"

both with crash and closing window

[Niram7777]

this might fix #3585 (to be tested)

[StevenMiller123]

Seems to work fine on Windows, here are several logs using different log settings to demonstrate.
Only real nitpick I have from testing is that there's no limit on log size, some games can get spammy enough to hit extremely large logs without a proper log size limit.

shad_log.zip
shad_log.txt
shad_log.txt
shad_log.txt
shad_log.txt

CUSA15081.log
CUSA15081.log
CUSA15436.zip
CUSA02012.log

[tochnonechelovek]

Great PR, now CUSA05485 doesn't lag the Qt launcher itself as it did with previous logs. Also, the logs just feel more performant. Not sure if it makes much sense saying something about Qt launcher or if it even was considered, but it is affected by this PR.

1. Logs are not colored in Qt launcher on Windows (at the bottom part of the window).
2. "Skip" setting name is probably confusing, but maybe just for me. As far as I understand, it means "grouping" for logs.
3. CUSA01483 is supposed to restart the game after an error and it doesn't on this PR. Also, no log like this is written to log file, but is written in the console/Qt launcher:
   [Common] <Info> (Game:Main) emulator.cpp:493 Restart: Restarting the emulator with args: --log-append --game [path] --override-root [path] --wait-for-pid [pid]

[Niram7777]

Seems to work fine on Windows, here are several logs using different log settings to demonstrate. Only real nitpick I have from testing is that there's no limit on log size, some games can get spammy enough to hit extremely large logs without a proper log size limit.

shad_log.zip shad_log.txt shad_log.txt shad_log.txt shad_log.txt

CUSA15081.log CUSA15081.log CUSA15436.zip CUSA02012.log

done, configurable with sizeLimit = 104857600 (bytes)

[Niram7777]

Great PR, now CUSA05485 doesn't lag the Qt launcher itself as it did with previous logs. Also, the logs just feel more performant. Not sure if it makes much sense saying something about Qt launcher or if it even was considered, but it is affected by this PR.

1. Logs are not colored in Qt launcher on Windows (at the bottom part of the window).
2. "Skip" setting name is probably confusing, but maybe just for me. As far as I understand, it means "grouping" for logs.
3. CUSA01483 is supposed to restart the game after an error and it doesn't on this PR. Also, no log like this is written to log file, but is written in the console/Qt launcher:
   [Common] <Info> (Game:Main) emulator.cpp:493 Restart: Restarting the emulator with args: --log-append --game [path] --override-root [path] --wait-for-pid [pid]

1. it should be fixed now
2. I have renamed it skipDuplicate because grouping was the name I used before but now altough the difference is subtle, the logs are not grouped, they are skiped - meaning logs are written immediately and after we check do we skip
3. did you added --log-append ? because I have moved it from the cli to config file. Could you retry latest version and give more logs ?
   -> I think I found out the issue

[tochnonechelovek]

1. it should be fixed now
2. I have renamed it skipDuplicate because grouping was the name I used before but now altough the difference is subtle, the logs are not grouped, they are skiped - meaning logs are written immediately and after we check do we skip
3. did you added --log-append ? because I have moved it from the cli to config file. Could you retry latest version and give more logs ?
   -> I think I found out the issue

1. I tried the latest commit of this PR in Qt launcher (main), but now it doesn't show any logs in it at all. I tried using Qt launcher from PR 262 by downloading artifacts and using the latest commit of this PR, but no luck getting logs shown in the launcher again. The logs still are saved as a file, though. Maybe I'm doing something wrong, but I would like to know what to do differently if so.

2. This is much better, cool 👍🏻

3. I haven't used the --log-append before; the emulator just restarted with these settings by itself. Right now, for some reason, when using this PR, CUSA01483 restarts the game only after using the "Separate Log Files" setting, no matter if it's main launcher or PR 262 launcher. On main commit of shadPS4, the game restarts no matter what launcher.

[Niram7777]

1. it should be fixed now
2. I have renamed it skipDuplicate because grouping was the name I used before but now altough the difference is subtle, the logs are not grouped, they are skiped - meaning logs are written immediately and after we check do we skip
3. did you added --log-append ? because I have moved it from the cli to config file. Could you retry latest version and give more logs ?
   -> I think I found out the issue

1. I tried the latest commit of this PR in Qt launcher (main), but now it doesn't show any logs in it at all. I tried using Qt launcher from PR 262 by downloading artifacts and using the latest commit of this PR, but no luck getting logs shown in the launcher again. The logs still are saved as a file, though. Maybe I'm doing something wrong, but I would like to know what to do differently if so.
2. This is much better, cool 👍🏻
3. I haven't used the --log-append before; the emulator just restarted with these settings by itself. Right now, for some reason, when using this PR, CUSA01483 restarts the game only after using the "Separate Log Files" setting, no matter if it's main launcher or PR 262 launcher. On main commit of shadPS4, the game restarts no matter what launcher.

thanks for the tests

1. indeed, on Linux there is no color either with Qt launcher, I am investigating
2. it looks like an issue with the shadPs4.txt log file between opened twice on Windows, I have an idea I will check

[Niram7777]

PR ready, regression should be fixed, I would like to avoid keeping those PRs for too long
@tochnonechelovek could you test CUSA01483 ?

[tochnonechelovek]

Sorry, the logs are still not appearing in the Qt launcher, no matter if it's 262 or main launcher. Settings don't matter in that case. Not sure if it's an issue for this PR or the Qt launcher one.

CUSA01483 on 262 still doesn't restart the game without "Separate Log Files". Tried all other log settings and log filters: the results are the same.

[tochnonechelovek]

Now everything works fine. No regressions in my titles, logs now show in Qt launcher colored and CUSA01483 restarts when emulator tells it to. Great work! 🎉

[georgemoralis]

ok now if you can rebase to latest i think we are ready for merge :D

[Niram7777]

ok now if you can rebase to latest i think we are ready for merge :D

I need to split out the non-log stuff in another PR