Skip to content

PoC: Test migration to subdirectory - #1485

Closed
jscotka wants to merge 1 commit into
rhinstaller:mainfrom
jscotka:test_migration_from_main
Closed

PoC: Test migration to subdirectory#1485
jscotka wants to merge 1 commit into
rhinstaller:mainfrom
jscotka:test_migration_from_main

Conversation

@jscotka

@jscotka jscotka commented Aug 11, 2025

Copy link
Copy Markdown

Used AI to reorganize structure of project

AI summary:

  • Move all test shell scripts (*.sh) from root to tests/ directory
  • Move all kickstart templates (*.ks.in) from root to tests/ directory
  • Move library files from lib/ to tests/libs/ directory
  • Update test runner scripts for new directory structure:
    • scripts/run_kickstart_tests.sh: Updated test discovery and path resolution
    • scripts/test_manager/collector.py: Updated kickstart template discovery
  • Update path references in test files:
    • Updated library paths in proxy-auth.sh and driverdisk tests
    • Updated path references in kickstart templates
  • Add comprehensive documentation:
    • REORGANIZATION_SUMMARY.md: Complete overview of changes
  • Preserve all functionality:
    • 295 tests discovered correctly for RHEL10
    • 27 tests properly skipped (RHEL/manual exclusions)
    • All test execution patterns maintained
    • Enhanced command-line test path resolution

This reorganization improves project structure while maintaining full backward compatibility and test functionality.

- Move all test shell scripts (*.sh) from root to tests/ directory
- Move all kickstart templates (*.ks.in) from root to tests/ directory
- Move library files from lib/ to tests/libs/ directory
- Update test runner scripts for new directory structure:
  - scripts/run_kickstart_tests.sh: Updated test discovery and path resolution
  - scripts/test_manager/collector.py: Updated kickstart template discovery
- Update path references in test files:
  - Updated library paths in proxy-auth.sh and driverdisk tests
  - Updated path references in kickstart templates
- Add comprehensive documentation:
  - REORGANIZATION_SUMMARY.md: Complete overview of changes
  - TEST_RUNNER_FIX_SUMMARY.md: Technical details of runner fixes
- Preserve all functionality:
  - 295 tests discovered correctly for RHEL10
  - 27 tests properly skipped (RHEL/manual exclusions)
  - All test execution patterns maintained
  - Enhanced command-line test path resolution

This reorganization improves project structure while maintaining
full backward compatibility and test functionality.
@abadger

abadger commented Aug 11, 2025

Copy link
Copy Markdown
Member

This is going to be hard to review. Some ideas:

  • Instead of letting AI make the changes directly, Have it write scripts with mv + wildcards to move files and sed commands to update script logic with the new locations. Then run those to make the changes. If we do this, we can check the logic for errors and feel better about doing a less thorough job of checking the output.
  • If there's any way to break this up into separate PRs, that would help for review purposes. I realize that it would be less efficient in some ways (having to touch the same file several times, PRs that have to be reviewed in a certain order, etc) but one can only hold so many ideas in their head at one time. If we have only one or two rules we have to remember to check for each file being touched (example: Is this a library? Then it is being moved to test/libs with no changes? Otherwise, is this a file referencing the libraries? So the only change is updating the path to test/libs?) then the task becomes more managable.

@jscotka

jscotka commented Aug 12, 2025

Copy link
Copy Markdown
Author

This is going to be hard to review. Some ideas:

* Instead of letting AI make the changes directly,  Have it write scripts with `mv` + wildcards to move files and `sed` commands to update script logic with the new locations.  Then run those to make the changes.  If we do this, we can check the logic for errors and feel better about doing a less thorough job of checking the output.

* If there's any way to break this up into separate PRs, that would help for review purposes.  I realize that it would be less efficient in some ways (having to touch the same file several times, PRs that have to be reviewed in a certain order, etc) but one can only hold so many ideas in their head at one time. If we have only one or two rules we have to remember to check for each file being touched (example: Is this a library?  Then it is being moved to `test/libs` with no changes?  Otherwise, is this a file referencing the libraries?  So the only change is updating the path to `test/libs`?) then the task becomes more managable.

Yes I can try to do it in this way and help to improve this part of project. and make it more manageable for changes.
I took it as part of my day of learning and try to focus more on AI, try to improve my experience and learn how to use it.

I understand that change is so massive to do review, but when I've looked into touched files and changes, everything saw more less simple changes, what just improved paths.

@abadger

abadger commented Aug 12, 2025

Copy link
Copy Markdown
Member

Yeah. I think with a script or trusted contributor I would be okay with some spot checking and assuming the rest of the changes were the same and wouldn't cause problems. But since AI is unpredictable, I would feel like I have to check that every change it made is the same.

@jscotka

jscotka commented Aug 12, 2025

Copy link
Copy Markdown
Author

Hi @abadger ,
I've created next PR, #1487
as your suggested changes. If you think we can close this PR, and let just second one. I think it is better to have it separated, as this is completely new approach. so that does not sense to mix these changes together in one PR.

Crucial part is now that I've let AI to create script what can migrate tests on behalf and tooling is backward compatible, so that this is important to review (as this is relatively small and easy chnage), but migration script is just something what does not affect workflow anyhow, it is big as we need to check crossdeps when migration. and I've removed the part with library location changes as this caused more troubles when migrated.

@KKoukiou KKoukiou closed this Aug 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants