Add --config_source pretrained to capture full model config via model_id - #181
Draft
kiszk wants to merge 3 commits into
Draft
Add --config_source pretrained to capture full model config via model_id#181kiszk wants to merge 3 commits into
kiszk wants to merge 3 commits into
Conversation
Signed-off-by: Kazuaki Ishizaki <ishizaki@jp.ibm.com>
Signed-off-by: Kazuaki Ishizaki <ishizaki@jp.ibm.com>
kiszk
force-pushed
the
feat/config-source-from-pretrained
branch
from
July 15, 2026 07:46
f99ca9a to
b43aa98
Compare
Signed-off-by: Kazuaki Ishizaki <ishizaki@jp.ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
--config_sourceoption toauto_generate_module_config.pythatcontrols how a module's
PretrainedConfigconstructor argument is recorded inthe generated YAML.
kwargs(default, unchanged): rebuild the config from a handful of extracteddimensions (
config_path+config_kwargs). Library defaults fill ineverything else — the historical behaviour.
pretrained(new): record only themodel_idso the OOT test framework canload the full, faithful config with
AutoConfig.from_pretrained(model_id).Motivation
The
kwargspath extracts only ~6 fields (hidden_size,num_attention_heads,num_key_value_heads,intermediate_size,max_position_embeddings,_attn_implementation). Every other field(
rope_theta,rope_scaling,attention_bias,head_dim,logits_scaling,...) is dropped to library defaults, so a module rebuilt from the YAML can
diverge from the real model. Recording the
model_idand reloading the configat test time preserves every field the model actually had.
There are two other motivations for this PR (out of scope of this PR)
LLM()for instead of instantiatingconfig_clsto support this architectureChanges (
utils/module_discovery/auto_generate_module_config.py)CONFIG_SOURCE_KWARGS/CONFIG_SOURCE_PRETRAINEDconstants and a shared_build_config_arg_spec(config, config_source, model_id)helper that producesthe config-arg spec, replacing the two inlined spec-building blocks in
capture_constructor_info.pretrainedmode the spec emitsmodel_idplus a smallconfig_overridescarrying the resolved
_attn_implementation(whichfrom_pretrainedmayleave as
None), keeping the standalone module on the same attention pathused at capture time.
config_pathis still recorded as a human-readablehint.
ModuleInfoCapture.__init__gainsconfig_source/model_id(validated),so switching works both from the CLI and from the driver API used by
generate_*_config.pyscripts._convert_constructor_arg_to_sample_inputemits themodel_id/config_overridesform when present, else the existingconfig_kwargsform.--config_source {kwargs,pretrained}CLI flag (defaultkwargs);mainpasses it plus
model_id=args.model_pathintoModuleInfoCapture.Fully backward compatible: the default (
kwargs) reproduces the previous outputbyte-for-byte, and existing generated YAMLs are unaffected.
Related
This is the generator half of the change. The paired consumer change (the OOT
framework resolving
model_idviaAutoConfig.from_pretrained) is submittedseparately against the torch-spyre repo.
#173
Testing
kwargsandpretrainedproduce theexpected shapes,
pretrainedwithout amodel_idraises, andModuleInfoCapturerejects an invalidconfig_source.python utils/module_discovery/auto_generate_module_config.py --model_path ibm-granite/granite-3.3-8b-instruct --config_source pretrained -o /tmp/granite_pretrained.yaml