-
Notifications
You must be signed in to change notification settings - Fork 106
Resource Download table #1165
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jw098
wants to merge
85
commits into
PokemonAutomation:main
Choose a base branch
from
jw098:download-table
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Resource Download table #1165
Changes from 72 commits
Commits
Show all changes
85 commits
Select commit
Hold shift + click to select a range
af8be50
add resource downloading table - minimal implementation
jw098 34fd060
move ResourceDownloadTable.h to ResourceDownload folder
jw098 14bb494
add DownloadButton and DownloadButtonWidget
jw098 9c8044d
add Delete Button
jw098 9bb0892
add_rows_from_resource_list_json
jw098 98aee6c
refactor so that we keep a handle on m_resources and m_resource_rows.
jw098 4c1d407
check if resource is downloaded
jw098 6554806
check if downloaded. add "Version" column
jw098 60c4024
add DOWNLOADED_RESOURCE_PATH()
jw098 f7a6652
initial implementation of updating "Version" column on a separate thr…
jw098 e582f0f
initialize worker thread outside of constructor
jw098 64ae1e6
convert member fields to PIMPL
jw098 be129ed
add version number to the JSON
jw098 058e8c1
update version_num, version_status for each row
jw098 b136c2f
refactor Row class into its own file
jw098 767d148
clicking Download button disables it until action is done
jw098 008317c
add download pop-up
jw098 881349b
move functions to Helper class to avoid circular dependency. lazy ini…
jw098 d30759c
cleanup headers
jw098 18362a0
print to console when throwing ParseException
jw098 1736cb8
update exception handling within the thread. add predownload_warning_…
jw098 8b9e41e
FileDownloader::download_file_to_disk
jw098 fafd581
initial progress bar implementation
jw098 6d6d9ad
fix Download button
jw098 2ad2b73
added static progress bar to UI
jw098 d63ed48
refactor. move functions out of DownloadButton into Row. Move buttons…
jw098 07450a8
more refactor of ResourceDownloadRow class. pass local_metadata to it…
jw098 c91ece7
working implementation of Progress bar. fix inheritance for DownloadW…
jw098 5e9d39d
download failed pop-up if internet disconnected.
jw098 ee4fd7e
delete folder prior to downloading. create parent directory during th…
jw098 a154fbe
catch errors with get_resource_version_num()
jw098 0b4a5f9
add unzip progress bar
jw098 17a280d
minor UI changes with the progress bar
jw098 eaa37db
minor fix
jw098 d9012ca
update the Version and Downloaded column after downloading
jw098 918f18f
add cancel button
jw098 7889422
add cancel callback to download routine
jw098 8913a4b
add cancel callback to unzip function
jw098 4196b88
refactor: move logic from DownloadButton class to DownloadRow class.
jw098 a61d90b
implement delete action for Delete button
jw098 861031d
unzip routine will throw OperationCancelledException if cancelled. an…
jw098 8620003
adjust logic that updates button state
jw098 30adc57
update UI state for progress bar
jw098 3821cc0
adjust text and width of buttons
jw098 bd42eca
get SHA 256 after each download
jw098 9ab8984
compare with expected hash
jw098 ab0a65e
minior UI changes. update file size in table to KiB/MiB/GiB.
jw098 fccc36a
minor changes
jw098 be7f2ab
remove tabs
jw098 e29ba6c
fix build
jw098 66dca0d
fix build
jw098 a3d376f
use Filesystem::Path instead of implicitly using std::filesystem
jw098 0932d46
fix build
jw098 4b82781
draft: use CancellableScope
jw098 80e2d68
move thread into DownloadThread class
jw098 bdcf4bc
refactor DownloadThread and its lifetime
jw098 0f3578c
update initialization of DownloadThread to use factory method to ensu…
jw098 df47629
fix ResourceDownload buttons, which fixes the issue with blank box in…
jw098 741d1c6
add missing semicolon
jw098 92621b6
add DownloadListener. first step of migration away from Qt signals
jw098 772573e
remove Qt signals from DownloadRow. remove slots from DownloadWidget
jw098 e9d4712
fix build
jw098 d2f708a
minor refactor
jw098 0e07929
update table labels
jw098 8e09153
minor
jw098 d2071fc
change progress updates from percentage to bytes_done, total_bytes
jw098 b69fcbb
remove extra semi colons
jw098 5255230
remove qint64 from header
jw098 061e8de
LabelCellWidget: remove listener
jw098 41cd51b
remove spaces
jw098 a102cfc
Merge branch 'main' into download-table
jw098 1132e99
remove spaces
jw098 17c1d20
remove ResourceType::ZIP_FILE
jw098 2bd1f08
Merge branch 'main' into download-table
jw098 992bc2c
minor fixes
jw098 175fe11
add MAX_CONCURRENT_DOWNLOADS. DownloadThreads check their position in…
jw098 5f55e7a
Merge branch 'main' into download-table
jw098 6f7a4e9
check ActionState before proceeding with the action
jw098 1b06111
add PRE_DELETE, PRE_DOWNLOAD ActionState
jw098 89bd2ca
Block button actions if not in the correct ActionState
jw098 d4a128d
minor update
jw098 1bec6ce
ensure correct ActionState at start of ensure_remote_metadata_loaded
jw098 8786b50
change unique_lock to lock_guard
jw098 4d182b2
add pop-up for cancelling download
jw098 0c083b8
remove old comments
jw098 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
141 changes: 141 additions & 0 deletions
141
SerialPrograms/Source/CommonFramework/ResourceDownload/ResourceDownloadHelpers.cpp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| /* Resource Download Helpers | ||
| * | ||
| * From: https://github.com/PokemonAutomation/ | ||
| * | ||
| */ | ||
|
|
||
| #include "CommonFramework/Globals.h" | ||
| #include "CommonFramework/Logging/Logger.h" | ||
| // #include "CommonFramework/Tools/GlobalThreadPools.h" | ||
| #include "CommonFramework/Tools/FileDownloader.h" | ||
| #include "CommonFramework/Exceptions/OperationFailedException.h" | ||
| #include "Common/Cpp/Json/JsonArray.h" | ||
| #include "Common/Cpp/Json/JsonObject.h" | ||
| #include "Common/Cpp/Filesystem.h" | ||
| #include "ResourceDownloadHelpers.h" | ||
|
|
||
| // #include <filesystem> | ||
| // #include <thread> | ||
| // #include <unordered_set> | ||
|
|
||
| #include <iostream> | ||
| using std::cout; | ||
| using std::endl; | ||
|
|
||
| namespace PokemonAutomation{ | ||
|
|
||
|
|
||
| ResourceType get_resource_type_from_string(std::string type){ | ||
| if (type == "ZippedFolder"){ | ||
| return ResourceType::ZIP_FILE; | ||
| }else{ | ||
| throw InternalProgramError(nullptr, PA_CURRENT_FUNCTION, "get_resource_type_from_string: Unknown string."); | ||
| } | ||
|
|
||
| } | ||
|
|
||
| std::vector<DownloadedResourceMetadata> deserialize_resource_list_json(const JsonValue& json){ | ||
| std::vector<DownloadedResourceMetadata> resources; | ||
|
|
||
| try{ | ||
| const JsonObject& obj = json.to_object_throw(); | ||
| const JsonArray& resource_list = obj.get_array_throw("resourceList"); | ||
| for (const JsonValue& resource_val : resource_list){ | ||
| const JsonObject& resource_obj = resource_val.to_object_throw(); | ||
|
|
||
| std::string resource_name = resource_obj.get_string_throw("resourceName"); | ||
| std::optional<uint16_t> version_num = (uint16_t)resource_obj.get_integer_throw("version"); | ||
| ResourceType resource_type = get_resource_type_from_string(resource_obj.get_string_throw("Type")); | ||
| size_t compressed_bytes = (size_t)resource_obj.get_integer_throw("CompressedBytes"); | ||
| size_t decompressed_bytes = (size_t)resource_obj.get_integer_throw("DecompressedBytes"); | ||
| std::string url = resource_obj.get_string_throw("URL"); | ||
| std::string sha_256 = resource_obj.get_string_throw("SHA_256"); | ||
|
|
||
| DownloadedResourceMetadata resource = { | ||
| resource_name, | ||
| version_num, | ||
| resource_type, | ||
| compressed_bytes, | ||
| decompressed_bytes, | ||
| url, | ||
| sha_256 | ||
| }; | ||
|
|
||
| resources.emplace_back(std::move(resource)); | ||
|
|
||
| } | ||
|
|
||
| }catch (ParseException&){ | ||
| std::cerr << "JSON parsing error. Given JSON file doesn't match the expected format." << endl; | ||
| // throw ParseException("JSON parsing error. Given JSON file doesn't match the expected format."); | ||
| return std::vector<DownloadedResourceMetadata>(); | ||
| } | ||
|
|
||
| return resources; | ||
| } | ||
|
|
||
|
|
||
| const std::vector<DownloadedResourceMetadata>& local_resource_download_list(){ | ||
| // cout << "local_resource_download_list" << endl; | ||
| static std::vector<DownloadedResourceMetadata> local_resources = deserialize_resource_list_json(load_json_file(RESOURCE_PATH() + "ResourceDownloadList.json")); | ||
|
|
||
| return local_resources; | ||
| } | ||
|
|
||
|
|
||
| JsonValue fetch_resource_download_list_json_from_remote(){ | ||
| Logger& logger = global_logger_tagged(); | ||
| JsonValue json = | ||
| FileDownloader::download_json_file( | ||
| logger, | ||
| "https://raw.githubusercontent.com/jw098/Packages/refs/heads/download/Resources/ResourceDownloadList.json" | ||
| ); | ||
|
|
||
| return json; | ||
| } | ||
|
|
||
| const JsonValue& remote_resource_download_list_json(){ | ||
| static const JsonValue json = fetch_resource_download_list_json_from_remote(); | ||
|
|
||
| return json; | ||
| } | ||
|
|
||
| const std::vector<DownloadedResourceMetadata>& remote_resource_download_list(){ | ||
| // cout << "remote_resource_download_list" << endl; | ||
| static std::vector<DownloadedResourceMetadata> remote_resources = deserialize_resource_list_json(remote_resource_download_list_json()); | ||
|
|
||
| return remote_resources; | ||
| } | ||
|
|
||
| std::optional<uint16_t> get_resource_version_num(Filesystem::Path folder_path){ | ||
| try{ | ||
| std::string file_name = folder_path.string() + "/version.json"; | ||
| const JsonValue& json = load_json_file(file_name); | ||
|
|
||
| const JsonObject& obj = json.to_object_throw(); | ||
| uint16_t version_num = (uint16_t)obj.get_integer_throw("version"); | ||
| return version_num; | ||
| }catch(...){ | ||
| std::cerr << "Unable to determine the version number from version.json." << endl; | ||
| return std::nullopt; | ||
| } | ||
|
|
||
| } | ||
|
|
||
| ResourceVersionStatus get_version_status(uint16_t expected_version_num, std::optional<uint16_t> current_version_num){ | ||
| if (!current_version_num.has_value()){ | ||
| return ResourceVersionStatus::NOT_APPLICABLE; | ||
| } | ||
|
|
||
| if (current_version_num < expected_version_num){ | ||
| return ResourceVersionStatus::OUTDATED; | ||
| }else if (current_version_num == expected_version_num){ | ||
| return ResourceVersionStatus::CURRENT; | ||
| }else{ // current > expected | ||
| return ResourceVersionStatus::FUTURE_VERSION; | ||
| } | ||
| } | ||
|
|
||
|
|
||
|
|
||
| } | ||
61 changes: 61 additions & 0 deletions
61
SerialPrograms/Source/CommonFramework/ResourceDownload/ResourceDownloadHelpers.h
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| /* Resource Download Helpers | ||
|
jw098 marked this conversation as resolved.
|
||
| * | ||
| * From: https://github.com/PokemonAutomation/ | ||
| * | ||
| */ | ||
|
|
||
| #ifndef PokemonAutomation_ResourceDownloadHelpers_H | ||
| #define PokemonAutomation_ResourceDownloadHelpers_H | ||
|
|
||
| #include <string> | ||
| #include <optional> | ||
|
|
||
|
|
||
| namespace PokemonAutomation{ | ||
|
|
||
| namespace Filesystem{ | ||
| class Path; | ||
| } | ||
|
|
||
|
|
||
| enum class ResourceType{ | ||
| ZIP_FILE, | ||
| }; | ||
|
|
||
| struct DownloadedResourceMetadata{ | ||
| std::string resource_name; | ||
| std::optional<uint16_t> version_num; | ||
| ResourceType resource_type; | ||
| size_t size_compressed_bytes; | ||
| size_t size_decompressed_bytes; | ||
| std::string url; | ||
| std::string sha_256; | ||
| }; | ||
|
|
||
| enum class ResourceVersionStatus{ | ||
| CURRENT, | ||
| OUTDATED, // still used, but newer version available | ||
| FUTURE_VERSION, // current version number is greater than the expected version number | ||
| NOT_APPLICABLE, // resource not downloaded locally, so can't get its version | ||
| // RETIRED, // no longer used | ||
| // BLANK, // not yet fetched version info from remote | ||
| }; | ||
|
|
||
| enum class RemoteMetadataStatus{ | ||
| UNINITIALIZED, | ||
| NOT_AVAILABLE, | ||
| AVAILABLE, | ||
| }; | ||
| struct RemoteMetadata { | ||
| RemoteMetadataStatus status = RemoteMetadataStatus::UNINITIALIZED; | ||
| DownloadedResourceMetadata metadata; | ||
| }; | ||
|
|
||
|
|
||
| const std::vector<DownloadedResourceMetadata>& local_resource_download_list(); | ||
| const std::vector<DownloadedResourceMetadata>& remote_resource_download_list(); | ||
| std::optional<uint16_t> get_resource_version_num(Filesystem::Path folder_path); | ||
| ResourceVersionStatus get_version_status(uint16_t expected_version_num, std::optional<uint16_t> current_version_num); | ||
|
|
||
| } | ||
| #endif | ||
46 changes: 46 additions & 0 deletions
46
SerialPrograms/Source/CommonFramework/ResourceDownload/ResourceDownloadOptions.cpp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| /* Resource Download Row | ||
| * | ||
| * From: https://github.com/PokemonAutomation/ | ||
| * | ||
| */ | ||
|
|
||
| #include "ResourceDownloadRow.h" | ||
| #include "ResourceDownloadOptions.h" | ||
|
|
||
| // #include <thread> | ||
|
|
||
| #include <iostream> | ||
| using std::cout; | ||
| using std::endl; | ||
|
|
||
| namespace PokemonAutomation{ | ||
|
|
||
| // ResourceDownloadButton::~ResourceDownloadButton(){} | ||
| ResourceDownloadButton::ResourceDownloadButton(ResourceDownloadRow& p_row) | ||
| : ConfigOptionImpl<ResourceDownloadButton>(LockMode::UNLOCK_WHILE_RUNNING) | ||
| , row(p_row) | ||
| , m_enabled(true) | ||
| {} | ||
|
|
||
|
|
||
| ResourceDeleteButton::ResourceDeleteButton(ResourceDownloadRow& p_row) | ||
| : ConfigOptionImpl<ResourceDeleteButton>(LockMode::UNLOCK_WHILE_RUNNING) | ||
| , row(p_row) | ||
| , m_enabled(true) | ||
| {} | ||
|
|
||
|
|
||
| ResourceCancelButton::ResourceCancelButton(ResourceDownloadRow& p_row) | ||
| : ConfigOptionImpl<ResourceCancelButton>(LockMode::UNLOCK_WHILE_RUNNING) | ||
| , row(p_row) | ||
| , m_enabled(true) | ||
| {} | ||
|
|
||
|
|
||
| ResourceProgressBar::ResourceProgressBar(ResourceDownloadRow& p_row) | ||
| : ConfigOptionImpl<ResourceProgressBar>(LockMode::UNLOCK_WHILE_RUNNING) | ||
| , row(p_row) | ||
| {} | ||
|
|
||
|
|
||
| } |
89 changes: 89 additions & 0 deletions
89
SerialPrograms/Source/CommonFramework/ResourceDownload/ResourceDownloadOptions.h
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| /* Resource Download Row | ||
| * | ||
| * From: https://github.com/PokemonAutomation/ | ||
| * | ||
| */ | ||
|
|
||
| #ifndef PokemonAutomation_ResourceDownloadOptions_H | ||
| #define PokemonAutomation_ResourceDownloadOptions_H | ||
|
|
||
| #include <QObject> | ||
| // #include "Common/Cpp/Containers/Pimpl.h" | ||
| // #include "Common/Cpp/Concurrency/AsyncTask.h" | ||
| // #include "Common/Cpp/Options/StaticTableOption.h" | ||
| // #include "ResourceDownloadHelpers.h" | ||
|
|
||
|
|
||
| namespace PokemonAutomation{ | ||
|
|
||
| class ResourceDownloadRow; | ||
|
|
||
|
|
||
| class ResourceDownloadButton : public QObject, public ConfigOptionImpl<ResourceDownloadButton>{ | ||
| Q_OBJECT | ||
|
jw098 marked this conversation as resolved.
Outdated
|
||
| public: | ||
| // ~ResourceDownloadButton(); | ||
| ResourceDownloadButton(ResourceDownloadRow& p_row); | ||
|
|
||
| public: | ||
| inline bool get_enabled(){ return m_enabled; } | ||
| inline void set_enabled(bool enabled){ | ||
| m_enabled = enabled; | ||
| } | ||
|
|
||
| public: | ||
| ResourceDownloadRow& row; | ||
|
|
||
| private: | ||
| bool m_enabled; // button should be blocked during an active task. m_enabled is false when blocked | ||
|
|
||
|
|
||
|
|
||
|
|
||
| }; | ||
|
|
||
| class ResourceDeleteButton : public ConfigOptionImpl<ResourceDeleteButton>{ | ||
| public: | ||
| ResourceDeleteButton(ResourceDownloadRow& p_row); | ||
|
|
||
| public: | ||
| inline bool get_enabled(){ return m_enabled; } | ||
| inline void set_enabled(bool enabled){ | ||
| m_enabled = enabled; | ||
| } | ||
|
|
||
| public: | ||
| ResourceDownloadRow& row; | ||
|
|
||
| private: | ||
| bool m_enabled; | ||
| }; | ||
|
|
||
| class ResourceCancelButton : public ConfigOptionImpl<ResourceCancelButton>{ | ||
| public: | ||
| ResourceCancelButton(ResourceDownloadRow& p_row); | ||
|
|
||
| public: | ||
| inline bool get_enabled(){ return m_enabled; } | ||
| inline void set_enabled(bool enabled){ | ||
| m_enabled = enabled; | ||
| } | ||
|
|
||
| public: | ||
| ResourceDownloadRow& row; | ||
|
|
||
| private: | ||
| bool m_enabled; | ||
| }; | ||
|
|
||
| class ResourceProgressBar : public ConfigOptionImpl<ResourceProgressBar>{ | ||
| public: | ||
| ResourceProgressBar(ResourceDownloadRow& p_row); | ||
|
|
||
| ResourceDownloadRow& row; | ||
| }; | ||
|
|
||
|
|
||
|
|
||
| } | ||
| #endif | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.