Skip to content

review: test: Fixed nondeterministic failures in ContractOnSettersParametrizedTest.data() #6499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yonghanlin wants to merge 2 commits into
base: master
Choose a base branch
from
Open

review: test: Fixed nondeterministic failures in ContractOnSettersParametrizedTest.data() #6499

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
31e3b1e
fix: ensure ContractOnSettersParametrizedTest only invokes valid IMPO…
yonghanlin Oct 27, 2025
358493e
Merge branch 'master' into fix/ContractOnSettersParametrizedTest-nond…
monperrus Nov 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion src/test/java/spoon/test/parent/ContractOnSettersParametrizedTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,9 @@ public static Object createCompatibleObject(CtTypeReference<?> parameterType) {
fooBar.delete(); // removing from default package
return fooBar; // createNewClass implictly needs a CtClass
}
if (spoon.reflect.reference.CtReference.class.equals(c)) {
return f.Type().createReference("java.lang.String");
}
for (CtType<?> t : allInstantiableMetamodelInterfaces) {
if (c.isAssignableFrom(t.getActualClass())) {
CtElement argument = factory.Core().create((Class<? extends CtElement>) t.getActualClass());
Expand Down Expand Up @@ -157,7 +160,7 @@ public static Object createCompatibleObject(CtTypeReference<?> parameterType) {

if (Set.class.isAssignableFrom(c)) {
// we create one set with one element
HashSet<Object> objects = new HashSet<>();
java.util.LinkedHashSet<Object> objects = new java.util.LinkedHashSet<>();
objects.add(createCompatibleObject(parameterType.getActualTypeArguments().get(0)));
return objects;
}
Expand Down Expand Up @@ -220,6 +223,35 @@ private void testContract(CtType<?> toTest) throws Throwable {
int nBefore = changeListener.nbCallsToOnAction;
changeListener.changedElements = new ArrayList<>();

Copy link
Copy Markdown
Collaborator

@monperrus monperrus Nov 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a fat comment of what you do here and why we need it?

It's unclear how this relates to the problem described in the PR description.

Thanks!

Copy link
Copy Markdown
Contributor Author

@yonghanlin yonghanlin Nov 19, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your review! The code below filters out invalid (receiver, method, argument) combinations before invoking setters via reflection in ContractOnSettersParametrizedTest. This test generates candidate combinations, and the order of these candidates depends on iteration over unordered collections. When NonDex randomizes iteration order, some seeds produce combinations that violate Spoon's API contracts, which leads to nondeterministic test failures.

For IMPORT_REFERENCE, the parameter type is a CtReference, but Spoon only accepts four specific subtypes: CtTypeReference, CtExecutableReference, CtFieldReference, and CtPackageReference. When createCompatibleObject() produced other CtReference subtypes, the reflective call triggered contract violations. The code below explicitly checks for these four allowed types and skip all other arguments.

Class<?> ctImport = Class.forName("spoon.reflect.declaration.CtImport");
if (ctImport.isInstance(receiver) && "setReference".equals(actualMethod.getName())) {
    Class<?> typeRef = Class.forName("spoon.reflect.reference.CtTypeReference");
    Class<?> execRef = Class.forName("spoon.reflect.reference.CtExecutableReference");
    Class<?> fieldRef = Class.forName("spoon.reflect.reference.CtFieldReference");
    Class<?> pkgRef = Class.forName("spoon.reflect.reference.CtPackageReference");
    boolean ok = typeRef.isInstance(argument)
            || execRef.isInstance(argument)
            || fieldRef.isInstance(argument)
            || pkgRef.isInstance(argument);
    if (!ok) continue;
}

For pattern arguments, the generator sometimes produces a CtUnnamedPattern as the value for this parameter. However, CtCasePattern cannot directly take a CtUnnamedPattern as its pattern. It is only valid when nested inside a record pattern. When this invalid combination appears under certain NonDex seeds, the reflective call fails. The code below therefore skips this specific combination.

Class<?> casePattern = Class.forName("spoon.reflect.code.CtCasePattern");
Class<?> unnamedPattern = Class.forName("spoon.reflect.code.CtUnnamedPattern");
if (casePattern.isInstance(receiver)
        && "setPattern".equals(actualMethod.getName())
        && unnamedPattern.isInstance(argument)) {
    continue;
}

Please let me know if you need any more details. Thank you again for pointing out the unclear part.

Class<?> paramClass = actualMethod.getParameterTypes()[0];
if (!paramClass.isPrimitive() && !paramClass.isInstance(argument)) {
continue;
}

try {
Class<?> ctImport = Class.forName("spoon.reflect.declaration.CtImport");
if (ctImport.isInstance(receiver) && "setReference".equals(actualMethod.getName())) {
Class<?> typeRef = Class.forName("spoon.reflect.reference.CtTypeReference");
Class<?> execRef = Class.forName("spoon.reflect.reference.CtExecutableReference");
Class<?> fieldRef = Class.forName("spoon.reflect.reference.CtFieldReference");
Class<?> pkgRef = Class.forName("spoon.reflect.reference.CtPackageReference");
boolean ok = typeRef.isInstance(argument)
|| execRef.isInstance(argument)
|| fieldRef.isInstance(argument)
|| pkgRef.isInstance(argument);
if (!ok) continue;
}
} catch (ClassNotFoundException ignore) {}
try {
Class<?> casePattern = Class.forName("spoon.reflect.code.CtCasePattern");
Class<?> unnamedPattern = Class.forName("spoon.reflect.code.CtUnnamedPattern");
if (casePattern.isInstance(receiver)
&& "setPattern".equals(actualMethod.getName())
&& unnamedPattern.isInstance(argument)) {
continue;
}
} catch (ClassNotFoundException ignore) {}

// here we actually call the setter
actualMethod.invoke(receiver, argument);

Expand Down
Loading