refactor: DRY Page class, fix bugs, enforce autoload convention - #16
Merged
Conversation
Parse Nokogiri document once and reuse across all extraction methods (title, links, meta_tags, headings, images, forms, scripts, microposts). This eliminates 9 redundant HTML parses per page. Also: - Deduplicate to_h/as_json via transform_values - Fix resolve_page_url to match http:/https: explicitly instead of bare 'http' prefix (could match 'httpserver.com' incorrectly)
BulkDownloader: Move tmp_path cleanup from fetch_and_save (where it was out of scope and never executed) into write_page_file where the variable is actually defined. DownloadState: DRY cleanup_stale to reuse stale_entries instead of duplicating the file-existence logic.
Extract CoverageAnalyzer from coverage_report.rb into its own file. Extract HealthReport and HealthDetail Structs from archive_health_check.rb into health_report.rb. Remove redundant inline 'require nokogiri' from SnapshotDiff. Ensures each autoloaded constant maps to its own file, following the Archaeo convention established in lib/archaeo.rb.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Architecture and code quality improvements identified during comprehensive TODO parity audit.
1. Page class DRY refactor
to_h/as_jsonviatransform_valuesresolve_page_urlprefix matching:"http"matchedhttpserver.comincorrectly. Now explicitly checks"http:"and"https:".2. Bug fixes
tmp_pathcleanup was infetch_and_saverescue buttmp_pathis a local variable inwrite_page_file—defined?(tmp_path)was always false. Moved cleanup into the correct scope.cleanup_staleduplicated file-existence logic fromstale_entries. Now reuses it (DRY).3. Class-per-file autoload convention
CoverageAnalyzerfromcoverage_report.rbintocoverage_analyzer.rbHealthReport/HealthDetailStructs fromarchive_health_check.rbintohealth_report.rbrequire "nokogiri"fromSnapshotDiffTest Results
561 examples, 0 failures