diff --git a/README.md b/README.md index 0ff1bf3..3e5e495 100644 --- a/README.md +++ b/README.md @@ -47,7 +47,7 @@ You can use any of the following methods to build. - 5. If the building process is successful, you’ll get your APKs in the
+ 5. If the building process is successful, you'll get your APKs in the
6. Make sure to do below steps once in a while(daily or weekly) to keep the builder bug free.
@@ -134,20 +134,20 @@ You can use any of the following methods to build. `*` - Can be overridden for individual app. ### App Level Config -| Env Name | Description | Default | -|:--------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------:|:-------------------------------| -| [~~**APP_NAME**_CLI_DL~~](#global-resources) | DL for CLI to be used for patching **APP_NAME**.(Disabled Temp) | GLOBAL_CLI_DL | -| [**APP_NAME**_PATCHES_DL](#global-resources) | DL for Patches to be used for patching **APP_NAME**. | GLOBAL_PATCHES_DL | -| [**APP_NAME**_SPACE_FORMATTED_PATCHES](#global-resources) | Whether patches are space formatted. **APP_NAME**. | GLOBAL_SPACE_FORMATTED_PATCHES | -| [**APP_NAME**_KEYSTORE_FILE_NAME](#global-keystore-file-name) | Key file to be used for signing **APP_NAME**. | GLOBAL_KEYSTORE_FILE_NAME | -| [**APP_NAME**_OLD_KEY](#global-keystore-file-name) | Whether key used was generated with cli > v4(new)

**APP_NAME**.

| GLOBAL_OLK_KEY | -| [**APP_NAME**_ARCHS_TO_BUILD](#global-archs-to-build) | Arch to keep in the patched **APP_NAME**. | GLOBAL_ARCHS_TO_BUILD | -| [**APP_NAME**_EXCLUDE_PATCH**](#custom-exclude-patching) | Patches to exclude while patching **APP_NAME**. | [] | -| [**APP_NAME**_INCLUDE_PATCH**](#custom-include-patching) | Patches to include while patching **APP_NAME**. | [] | -| [**APP_NAME**_VERSION](#app-version) | Version to use for download for patching. | Recommended by patch resources | -| [**APP_NAME**_PACKAGE_NAME***](#any-patch-apps) | Package name of the app to be patched | None | -| [**APP_NAME**_DL_SOURCE***](#any-patch-apps) | Download source of any of the supported scrapper | None | -| [**APP_NAME**_DL***](#app-dl) | Direct download Link for clean apk | None | +| Env Name | Description | Default | +|:--------------------------------------------------------------|:----------------------------------------------------------------------------------------------------:|:-------------------------------| +| [~~**APP_NAME**_CLI_DL~~](#global-resources) | DL for CLI to be used for patching **APP_NAME**.(Disabled Temp) | GLOBAL_CLI_DL | +| [**APP_NAME**_PATCHES_DL](#global-resources) | DL for Patches to be used for patching **APP_NAME**. Supports multiple bundles via comma separation. | GLOBAL_PATCHES_DL | +| [**APP_NAME**_SPACE_FORMATTED_PATCHES](#global-resources) | Whether patches are space formatted. **APP_NAME**. | GLOBAL_SPACE_FORMATTED_PATCHES | +| [**APP_NAME**_KEYSTORE_FILE_NAME](#global-keystore-file-name) | Key file to be used for signing **APP_NAME**. | GLOBAL_KEYSTORE_FILE_NAME | +| [**APP_NAME**_OLD_KEY](#global-keystore-file-name) | Whether key used was generated with cli > v4(new)

**APP_NAME**.

| GLOBAL_OLK_KEY | +| [**APP_NAME**_ARCHS_TO_BUILD](#global-archs-to-build) | Arch to keep in the patched **APP_NAME**. | GLOBAL_ARCHS_TO_BUILD | +| [**APP_NAME**_EXCLUDE_PATCH**](#custom-exclude-patching) | Patches to exclude while patching **APP_NAME**. | [] | +| [**APP_NAME**_INCLUDE_PATCH**](#custom-include-patching) | Patches to include while patching **APP_NAME**. | [] | +| [**APP_NAME**_VERSION](#app-version) | Version to use for download for patching. | Recommended by patch resources | +| [**APP_NAME**_PACKAGE_NAME***](#any-patch-apps) | Package name of the app to be patched | None | +| [**APP_NAME**_DL_SOURCE***](#any-patch-apps) | Download source of any of the supported scrapper | None | +| [**APP_NAME**_DL***](#app-dl) | Direct download Link for clean apk | None | `**` - By default all patches for a given app are included.
`**` - Can be used to included universal patch.
@@ -308,10 +308,17 @@ You can use any of the following methods to build. ``` With the config tool will try to patch YouTube with resources from inotia00 while other global resource will used for patching other apps.
+ + **Multi-Patching Support**: You can now use multiple patch bundles from different creators for the same app: + ```dotenv + # Comma-separated URLs + YOUTUBE_PATCHES_DL=https://github.com/ReVanced/revanced-patches,https://github.com/indrastorm/Dropped-patches + ``` + The tool will download all specified patch bundles and apply them together using the ReVanced CLI's multiple `-p` argument support.
If you have want to provide resource locally in the apks folder. You can specify that by mentioning filename prefixed with `local://`.
- *Note* - The link provided must be DLs. Unless they are from GitHub.
- *Note* - If your patches resource are available on GitHub and you want to select latest resource without excluding + _Note_ - The link provided must be DLs. Unless they are from GitHub.
+ _Note_ - If your patches resource are available on GitHub and you want to select latest resource without excluding pre-release you can add `latest-prerelease` to the URL. Example: ```dotenv @@ -323,7 +330,7 @@ You can use any of the following methods to build. ``` For above example tool while selecting latest patches will exclude any pre-release/beta ie. will consider only stable releases..
- *Note* - Some of the patch source like inotia00 still provides **-** separated patches while revanced shifted to + _Note_ - Some of the patch source like inotia00 still provides **-** separated patches while revanced shifted to Space formatted patches. Use `SPACE_FORMATTED_PATCHES` to define the type of patches. 8. If you don't want to use default keystore. You can provide your own by @@ -371,7 +378,7 @@ You can use any of the following methods to build. ```dotenv YOUTUBE_ARCHS_TO_BUILD=arm64-v8a,armeabi-v7a ``` - *Note* - + _Note_ - 1. Possible values are: `armeabi-v7a`,`x86`,`x86_64`,`arm64-v8a` 2. Make sure the patching resource(CLI) support this feature. 11. If you want to include any extra file to the Github upload. Set comma arguments diff --git a/src/app.py b/src/app.py index 2cff5a3..51ebca7 100644 --- a/src/app.py +++ b/src/app.py @@ -31,10 +31,17 @@ class APP(object): self.app_version = config.env.str(f"{app_name}_VERSION".upper(), None) self.experiment = False self.cli_dl = config.env.str(f"{app_name}_CLI_DL".upper(), config.global_cli_dl) - self.patches_dl = config.env.str(f"{app_name}_PATCHES_DL".upper(), config.global_patches_dl) + + # Support multiple patch bundles via comma-separated URLs + patches_dl_raw = config.env.str(f"{app_name}_PATCHES_DL".upper(), config.global_patches_dl) + self.patches_dl_list = [url.strip() for url in patches_dl_raw.split(",") if url.strip()] + # Keep backward compatibility + self.patches_dl = patches_dl_raw + self.exclude_request: list[str] = config.env.list(f"{app_name}_EXCLUDE_PATCH".upper(), []) self.include_request: list[str] = config.env.list(f"{app_name}_INCLUDE_PATCH".upper(), []) self.resource: dict[str, dict[str, str]] = {} + self.patch_bundles: list[dict[str, str]] = [] # Store multiple patch bundles self.no_of_patches: int = 0 self.keystore_name = config.env.str(f"{app_name}_KEYSTORE_FILE_NAME".upper(), config.global_keystore_name) self.archs_to_build = config.env.list(f"{app_name}_ARCHS_TO_BUILD".upper(), config.global_archs_to_build) @@ -113,18 +120,18 @@ class APP(object): ---------- url : str The `url` parameter is a string that represents the URL of the resource you want to download. - It can be a URL from GitHub or a local file URL. + It can be a URL from GitHub or a local file URL. config : RevancedConfig The `config` parameter is an instance of the `RevancedConfig` class. It is used to provide - configuration settings for the download process. + configuration settings for the download process. assets_filter : str The `assets_filter` parameter is a string that is used to filter the assets to be downloaded - from a GitHub repository. It is used when the `url` parameter starts with "https://github". The - `assets_filter` string is matched against the names of the assets in the repository, and only - file_name : str + from a GitHub repository. It is used when the `url` parameter starts with "https://github". The + `assets_filter` string is matched against the names of the assets in the repository, and only + file_name : str The `file_name` parameter is a string that represents the name of the file that will be - downloaded. If no value is provided for `file_name`, the function will generate a filename based - on the URL of the file being downloaded. + downloaded. If no value is provided for `file_name`, the function will generate a filename based + on the URL of the file being downloaded. Returns ------- @@ -148,6 +155,58 @@ class APP(object): Downloader(config).direct_download(url, file_name) return tag, file_name + def _setup_download_tasks(self: Self) -> list[tuple[str, str, None, str]]: + """Setup download tasks for CLI and patch bundles.""" + download_tasks = [ + ("cli", self.cli_dl, None, ".*jar"), + ] + + # Download multiple patch bundles + for i, patches_url in enumerate(self.patches_dl_list): + bundle_name = f"patches_{i}" if len(self.patches_dl_list) > 1 else "patches" + download_tasks.append((bundle_name, patches_url, None, ".*rvp")) + + return download_tasks + + def _handle_cached_resource(self: Self, resource_name: str, tag: str, file_name: str) -> None: + """Handle cached resource and update appropriate data structures.""" + if resource_name.startswith("patches"): + self.patch_bundles.append( + { + "name": resource_name, + "file_name": file_name, + "version": tag, + }, + ) + # Keep backward compatibility for single bundle + if resource_name == "patches" or len(self.patches_dl_list) == 1: + self.resource["patches"] = { + "file_name": file_name, + "version": tag, + } + else: + self.resource[resource_name] = { + "file_name": file_name, + "version": tag, + } + + def _handle_downloaded_resource( + self: Self, + resource_name: str, + tag: str, + file_name: str, + download_tasks: list[tuple[str, str, RevancedConfig, str]], + resource_cache: dict[str, tuple[str, str]], + ) -> None: + """Handle newly downloaded resource and update cache.""" + self._handle_cached_resource(resource_name, tag, file_name) + + # Update cache for the corresponding URL + for task_name, task_url, _, _ in download_tasks: + if task_name == resource_name: + resource_cache[task_url.strip()] = (tag, file_name) + break + def download_patch_resources( self: Self, config: RevancedConfig, @@ -159,14 +218,15 @@ class APP(object): ---------- config : RevancedConfig The `config` parameter is an instance of the `RevancedConfig` class. It is used to provide - configuration settings for the resource download tasks. + configuration settings for the resource download tasks. resource_cache: dict[str, tuple[str, str]] """ logger.info("Downloading resources for patching.") - download_tasks = [ - ("cli", self.cli_dl, config, ".*jar"), - ("patches", self.patches_dl, config, ".*rvp"), + base_tasks = self._setup_download_tasks() + # Update download tasks with config + download_tasks: list[tuple[str, str, RevancedConfig, str]] = [ + (name, url, config, filter_pattern) for name, url, _, filter_pattern in base_tasks ] with ThreadPoolExecutor(1) as executor: @@ -177,10 +237,7 @@ class APP(object): if url in resource_cache: logger.info(f"Skipping {resource_name} download, using cached resource: {url}") tag, file_name = resource_cache[url] - self.resource[resource_name] = { - "file_name": file_name, - "version": tag, - } + self._handle_cached_resource(resource_name, tag, file_name) continue futures[resource_name] = executor.submit(self.download, url, cfg, assets_filter) @@ -190,14 +247,7 @@ class APP(object): for resource_name, future in futures.items(): try: tag, file_name = future.result() - self.resource[resource_name] = { - "file_name": file_name, - "version": tag, - } - resource_cache[download_tasks[["cli", "patches"].index(resource_name)][1].strip()] = ( - tag, - file_name, - ) + self._handle_downloaded_resource(resource_name, tag, file_name, download_tasks, resource_cache) except BuilderError as e: msg = f"Failed to download {resource_name} resource." raise PatchingFailedError(msg) from e diff --git a/src/downloader/github.py b/src/downloader/github.py index 214825c..e0cdf76 100644 --- a/src/downloader/github.py +++ b/src/downloader/github.py @@ -51,7 +51,9 @@ class Github(Downloader): """Extract repo owner and url from github url.""" parsed_url = urlparse(url) path_segments = parsed_url.path.strip("/").split("/") - + if len(path_segments) < 2: + msg = f"Invalid GitHub URL format: {url}" + raise DownloadError(msg) github_repo_owner = path_segments[0] github_repo_name = path_segments[1] tag_position = 3 diff --git a/src/exceptions.py b/src/exceptions.py index 8e0fc31..f3b95e8 100644 --- a/src/exceptions.py +++ b/src/exceptions.py @@ -130,4 +130,4 @@ class PatchesJsonLoadError(BuilderError): def __str__(self: Self) -> str: """Exception message.""" base_message = super().__str__() - return f"Message - {base_message} Url - {self.file_name}" + return f"Message - {base_message} File - {self.file_name}" diff --git a/src/parser.py b/src/parser.py index 16f2ccb..cc265a8 100644 --- a/src/parser.py +++ b/src/parser.py @@ -24,6 +24,9 @@ class Parser(object): OUTPUT_ARG = "-o" KEYSTORE_ARG = "--keystore" OPTIONS_ARG = "-O" + ENABLE_ARG = "-e" + DISABLE_ARG = "-d" + EXCLUSIVE_ARG = "--exclusive" def __init__(self: Self, patcher: Patches, config: RevancedConfig) -> None: self._PATCHES: list[str] = [] @@ -71,7 +74,7 @@ class Parser(object): for opt in options: pair = self.format_option(opt) self._PATCHES[:0] = [self.OPTIONS_ARG, pair] - self._PATCHES[:0] = ["-e", name] + self._PATCHES[:0] = [self.ENABLE_ARG, name] def exclude(self: Self, name: str) -> None: """The `exclude` function adds a given patch to the list of excluded patches. @@ -81,7 +84,7 @@ class Parser(object): name : str The `name` parameter is a string that represents the name of the patch to be excluded. """ - self._PATCHES.extend(["-d", name]) + self._PATCHES.extend([self.DISABLE_ARG, name]) self._EXCLUDED.append(name) def get_excluded_patches(self: Self) -> list[str]: @@ -119,22 +122,27 @@ class Parser(object): name = name.lower().replace(" ", "-") indices = [i for i in range(len(self._PATCHES)) if self._PATCHES[i] == name] for patch_index in indices: - if self._PATCHES[patch_index - 1] == "-e": - self._PATCHES[patch_index - 1] = "-d" + if self._PATCHES[patch_index - 1] == self.ENABLE_ARG: + self._PATCHES[patch_index - 1] = self.DISABLE_ARG else: - self._PATCHES[patch_index - 1] = "-e" + self._PATCHES[patch_index - 1] = self.ENABLE_ARG except ValueError: return False else: return True - def exclude_all_patches(self: Self) -> None: - """The function `exclude_all_patches` exclude all the patches.""" - for idx, item in enumerate(self._PATCHES): - if idx == 0: - continue - if item == "-e": - self._PATCHES[idx] = "-d" + def enable_exclusive_mode(self: Self) -> None: + """Enable exclusive mode - only explicitly enabled patches will run, all others disabled by default.""" + logger.info("Enabling exclusive mode for fast testing - only keeping one patch enabled.") + # Clear all patches and keep only the first one enabled + if self._PATCHES: + # Find the first enable argument and its patch name + for idx in range(0, len(self._PATCHES), 2): + if idx < len(self._PATCHES) and self._PATCHES[idx] == self.ENABLE_ARG and idx + 1 < len(self._PATCHES): + first_patch = self._PATCHES[idx + 1] + # Clear all patches and set only the first one + self._PATCHES = [self.ENABLE_ARG, first_patch] + break def fetch_patch_options(self: Self, name: str, options_list: list[dict[str, Any]]) -> dict[str, Any]: """The function `fetch_patch_options` finds patch options for the patch. @@ -219,15 +227,27 @@ class Parser(object): app.resource["cli"]["file_name"], apk_arg, app.download_file_name, - self.PATCHES_ARG, - app.resource["patches"]["file_name"], - self.OUTPUT_ARG, - app.get_output_file_name(), - self.KEYSTORE_ARG, - app.keystore_name, - exp, ] - args[1::2] = map(self.config.temp_folder.joinpath, args[1::2]) + + # Add multiple patch bundles using -p argument + if hasattr(app, "patch_bundles") and app.patch_bundles: + # Use multiple -p arguments for multiple bundles + for bundle in app.patch_bundles: + args.extend([self.PATCHES_ARG, bundle["file_name"]]) + else: + # Fallback to single bundle for backward compatibility + args.extend([self.PATCHES_ARG, app.resource["patches"]["file_name"]]) + + args.extend( + [ + self.OUTPUT_ARG, + app.get_output_file_name(), + self.KEYSTORE_ARG, + app.keystore_name, + exp, + ], + ) + args[1::2] = [str(self.config.temp_folder.joinpath(arg)) for arg in args[1::2]] if app.old_key: # https://github.com/ReVanced/revanced-cli/issues/272#issuecomment-1740587534 old_key_flags = [ @@ -237,7 +257,7 @@ class Parser(object): ] args.extend(old_key_flags) if self.config.ci_test: - self.exclude_all_patches() + self.enable_exclusive_mode() if self._PATCHES: args.extend(self._PATCHES) if app.app_name in self.config.rip_libs_apps: diff --git a/src/patches.py b/src/patches.py index 16eb9b8..214af47 100644 --- a/src/patches.py +++ b/src/patches.py @@ -1,7 +1,7 @@ """Revanced Patches.""" import contextlib -from typing import ClassVar, Self +from typing import Any, ClassVar, Self from loguru import logger @@ -125,11 +125,35 @@ class Patches(object): The `app` parameter is of type `APP`. It represents an instance of the `APP` class. """ self.patches_dict[app.app_name] = [] - patches = convert_command_output_to_json( - f"{config.temp_folder}/{app.resource["cli"]["file_name"]}", - f"{config.temp_folder}/{app.resource["patches"]["file_name"]}", - ) + # Handle multiple patch bundles + if hasattr(app, "patch_bundles") and app.patch_bundles: + for bundle in app.patch_bundles: + patches = convert_command_output_to_json( + f"{config.temp_folder}/{app.resource["cli"]["file_name"]}", + f"{config.temp_folder}/{bundle["file_name"]}", + ) + self._process_patches(patches, app) + elif "patches" in app.resource: + # Fallback to single bundle for backward compatibility + patches = convert_command_output_to_json( + f"{config.temp_folder}/{app.resource["cli"]["file_name"]}", + f"{config.temp_folder}/{app.resource["patches"]["file_name"]}", + ) + self._process_patches(patches, app) + + app.no_of_patches = len(self.patches_dict[app.app_name]) + + def _process_patches(self: Self, patches: list[dict[Any, Any]], app: APP) -> None: + """Process patches from a single bundle and add them to the patches dict. + + Parameters + ---------- + patches : list[dict[Any, Any]] + List of patches from a bundle + app : APP + The app instance + """ for patch in patches: if not patch["compatiblePackages"]: p = {x: patch[x] for x in ["name", "description"]} @@ -142,9 +166,9 @@ class Patches(object): p = {x: patch[x] for x in ["name", "description"]} p["app"] = compatible_package p["version"] = version[-1] if version else "all" - self.patches_dict[app.app_name].append(p) - - app.no_of_patches = len(self.patches_dict[app.app_name]) + # Avoid duplicate patches from multiple bundles + if not any(existing["name"] == p["name"] for existing in self.patches_dict[app.app_name]): + self.patches_dict[app.app_name].append(p) def __init__(self: Self, config: RevancedConfig, app: APP) -> None: self.patches_dict: dict[str, list[dict[str, str]]] = {"universal_patch": []}