Support TLS socket, enhance utilities#4
Open
mikecovlee wants to merge 29 commits into
Open
Conversation
…tionalities - Added `http_client` class for handling HTTP requests, including methods for parsing URLs, reading responses, and managing connections. - Introduced `openai_client` class extending `http_client` for interacting with OpenAI's API, including methods for sending messages and handling responses. - Enhanced `network.cpp` with SSL options parsing and improved error handling for network operations. - Added tests for HTTP client header parsing, URL parsing, OpenAI client functionality, and TLS trust self-check. - Created a sample DeepSeek chat client for testing API interactions. - Improved error reporting and handling in network operations.
- Introduced `run_tests.sh` as a test runner script for executing unit and integration tests. - Added unit tests for URL parsing, header parsing, OpenAI client functionality, and asynchronous TCP operations. - Implemented integration tests for TLS trust verification. - Removed outdated tests related to DeepSeek API and simplified client-server communication tests. - Enhanced error handling and reporting in tests to improve feedback on failures.
There was a problem hiding this comment.
Pull request overview
This pull request introduces a structured test suite and CI for the CovScript Network extension, while expanding the networking stack with TLS trust reporting, SSL option parsing/exposure, and an OpenAI-API-compatible HTTP client in netutils.
Changes:
- Added cross-platform test runners plus new unit/integration tests (URL parsing, header parsing, OpenAI client, async TCP, TLS trust).
- Extended the C++ network extension with TLS trust-mode options, trust reports, SSL-aware async read/write, and
connect_ssl. - Updated build + packaging metadata (OpenSSL linkage, version bumps) and added a GitHub Actions CI matrix.
Reviewed changes
Copilot reviewed 23 out of 26 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_url_parse.csc | New unit test coverage for http_client.parse_url. |
| tests/test_tls_trust.csc | New integration test to validate TLS trust configuration + reporting. |
| tests/test_simple_server.ecs | Removed older simple HTTP server test harness. |
| tests/test_simple_client.ecs | Removed older simple HTTP client concurrency test harness. |
| tests/test_server.csc | Removed older async TCP server test. |
| tests/test_openai_client.csc | New unit test coverage for openai_client behavior and defaults. |
| tests/test_header_parser.csc | New unit tests for HTTP response header parsing. |
| tests/test_deepseek.csc | New integration test exercising OpenAI-compatible client against DeepSeek (key-gated). |
| tests/test_client.csc | Removed older async TCP client test. |
| tests/test_async_tcp.csc | New async TCP unit tests (accept/connect/read/write round-trips). |
| run_tests.sh | Bash test runner to execute unit + integration tests locally. |
| run_tests.bat | Windows test runner to execute unit + integration tests locally (key-gated DeepSeek). |
| network.cpp | Exposes TLS option parsing + trust reporting + SSL-aware async operations through CNI. |
| netutils.ecs | Adds http_client and openai_client implementations used by new tests/integration flows. |
| netutils.csym | Updates generated symbol/module metadata to include new client APIs. |
| netutils.csp | Updates generated compiled output to include new client APIs. |
| include/network/network.hpp | Removed old header path (replaced by include/network.hpp). |
| include/network.hpp | New consolidated networking header with TLS trust modes and reporting plumbing. |
| include/asio.hpp | Adds top-level Asio umbrella header for local Asio tree usage. |
| csbuild/network.json | Bumps network package version. |
| csbuild/netutils.json | Bumps netutils package version. |
| csbuild/netutils_csym.json | Bumps netutils.csym package version. |
| CMakeLists.txt | Requires + links OpenSSL and adjusts Windows link libs for TLS/trust features. |
| bench_concurrent.sh | Adds a wrk-based concurrency benchmarking helper script. |
| .gitignore | Ignores compiled CovScript output artifact(s). |
| .github/workflows/ci.yml | Adds CI matrix (3 OSes × release/nightly) building CovScript + extension and running tests. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
mikecovlee
changed the title
run_tests.sh as a test runner script for executing unit and integration tests.
- Added unit tests for URL parsing, header parsing, OpenAI client functionality, and asynchronous TCP operations.
- Implemented integration tests for TLS trust verification.
- Removed outdated tests related to DeepSeek API and simplified client-server communication tests.
- Enhanced error handling and reporting in tests to improve feedback on failures.- Implemented `KeyStore` class to manage argument options and values. - Developed `ArgumentParser` class with methods to add arguments, options, set defaults, and print help messages. - Introduced functions for filtering strings, removing prefixes, and padding strings for help output formatting. - Added JSON configuration files for `argparse` and `argparse.csym` packages. - Created examples for a simple TLS implementation using the new argparse functionality. - Enhanced error handling and validation for command-line arguments and options. - Updated network error messages for clarity and consistency.
- Introduced new test files for fiber socket, HTTP round-trip, TCP sync, UDP, and TLS error handling. - Enhanced `run_tests.sh` to include new test cases and conditionally skip deepseek tests if the API key is not set. - Implemented various test scenarios including async operations, error handling, and socket options for both TCP and UDP protocols. - Added utility tests for hex conversion and hostname resolution. - Ensured all tests report pass/fail status and handle exceptions gracefully.
- Updated CNI_API.md to reflect changes in netutils HTTPS behavior (v1.2.1) regarding SSL certificate verification. - Added FreeBSD support in network.hpp for SSL certificate verification paths. - Improved error handling and thread safety in network.cpp, including adjustments to async operations and shutdown procedures. - Refactored test cases in test_fiber_socket.csc for better clarity and structure, including new functions for checking true/false conditions. - Enhanced test_http_roundtrip.csc with additional checks and improved error handling. - Introduced a new test_http_server.csc to validate HTTP server functionality and response handling. - Cleaned up test_tcp_sync.csc by removing redundant comments and improving test structure. - Updated test_tls_errors.csc to streamline error handling and improve clarity in TLS-related tests. - Refined test_udp.csc by removing unnecessary sections and enhancing the focus on UDP functionality. - Improved test_utils.csc by consolidating utility function tests and ensuring consistent usage of network module functions.
…nsion - Introduced comprehensive documentation outlining the asynchronous I/O operations, lifecycle, and state management in the CovScript Network Extension. - Included detailed sequence diagrams for asynchronous TCP client and server processes, as well as UDP operations. - Documented the state object lifecycle and event loop management, enhancing understanding of async operations. fix: Improve Windows certificate loading in network.hpp - Modified the `load_windows_root_certs` function to load certificates from multiple Windows system certificate stores (ROOT, CA, AuthRoot) for better coverage. - Enhanced error reporting for certificate loading failures. fix: Ensure safe shutdown and close operations in network.cpp - Updated `close()` and `shutdown()` methods to prevent execution when async operations are still in flight, throwing runtime errors if called prematurely. - Implemented time-bounded checks in `safe_shutdown()` to avoid indefinite blocking under sustained async load. refactor: Enhance async job management in network.cpp - Improved the handling of async job counters and introduced atomic operations for thread safety. - Added checks to prevent use-after-free errors when closing sockets with pending async operations. test: Update tests for fiber and async operations - Refined tests for fiber-based socket operations to ensure proper completion and error handling. - Added checks for server response validation and improved error reporting in TLS handshake tests.
…ror handling - Bump server version to 1.4 and introduce new configuration constants for HTTP client. - Modify content reception functions to use a configurable framing size. - Enhance safe shutdown procedures in various worker functions to log potential issues. - Refactor HTTP client to utilize default ports and configurable read chunk sizes. - Update network.cpp to improve error handling and ensure consistent port validation. - Introduce optional timeout handling in wait functions for better control over asynchronous operations. - Adjust test_fiber_socket.csc to streamline error handling and improve readability.
Comment on lines
+1070
to
+1074
| while need > 0 | ||
| var step = need | ||
| if step > 8192 | ||
| step = http_client_read_chunk | ||
| end |
Comment on lines
+107
to
+109
| # Read response | ||
| var response = client.receive(1024) | ||
| check_not_null("S02-02: received response", response) |
Comment on lines
+128
to
+129
| var response2 = client2.receive(1024) | ||
| check_not_null("S03-02: received 404 response", response2) |
Comment on lines
+68
to
+73
| if !connected | ||
| error_msg = "TCP connect failed" | ||
| else | ||
| sock.connect_ssl(host, options) | ||
| handshake_ok = true | ||
| end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.