Maximum Recursion Depth Exceeded Issue

[jainronit]

Hi,

I am running ShapeMapper2 on two samples (treated vs. untreated) but am running into a problem where the maximum recursion depth for a Python object is exceeded (see below for full error output). Would you be able to advise on what the issue might be/any potential workarounds? Thank you, I appreciate it!

args: --target ./data/pool1_finalized.fasta --amplicon --modified --U ./data/Pool1_NoPUS_NAI.fq --untreated --U ./data/Pool1_NoPUS_DMSO.fq --output-counted-mutations --per-read-histograms --min-mapq 0 --min-qual-to-count 0 --min-qual-to-trim 0
Traceback (most recent call last):
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/cli.py", line 278, in
run(sys.argv)
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/cli.py", line 75, in run
pipeline, arg_dict = ap.construct(rest_args)
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline_arg_parser.py", line 567, in construct
return (build_pipeline(**kw),
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline_builder.py", line 509, in build_pipeline
pipeline.setup()
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline.py", line 503, in setup
self.calc_run_order(serial_mode=serial_mode)
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline.py", line 237, in calc_run_order
self.calc_run_order(run_order=updated_run_order,
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline.py", line 237, in calc_run_order
self.calc_run_order(run_order=updated_run_order,
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline.py", line 237, in calc_run_order
self.calc_run_order(run_order=updated_run_order,
[Previous line repeated 985 more times]
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline.py", line 225, in calc_run_order
comps = self.sort_components(comps)
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline.py", line 95, in sort_components
recurse(self)
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline.py", line 94, in recurse
recurse(m, tree_path=updated_tree_path)
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline.py", line 94, in recurse
recurse(m, tree_path=updated_tree_path)
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline.py", line 94, in recurse
recurse(m, tree_path=updated_tree_path)
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/pipeline.py", line 88, in recurse
o.pipeline_location = tree_path
File "/oak/stanford/groups/nicolemm/ronitj/shapemapper2-2.3/internals/python/pyshapemap/component.py", line 103, in setattr
for i in range(len(self.internal_components)):
RecursionError: maximum recursion depth exceeded while calling a Python object

[Psirving]

We've seen this before in issue #48. That resolution was not documented because they emailed me directly.

What happened in that case is that there were too many sequences in the fasta file given to --targets. I have no idea what the upper limit is. If your fasta --target ./data/pool1_finalized.fasta has too many entries (based on the name "pool1", seems like this might be the case), try splitting the file into smaller pools and run each as a seperate Shapemapper run.

If you find an approximate limit to the number of entries here, let us know.

[jainronit]

Hi @Psirving, thanks so much for your help - I split the FASTA file into smaller chunks and that enabled ShapeMapper to run successfully. FYI, it seems that the limit to the number of entries is between 200 and 300. I'm wondering, however, if there is a way to map to the entire pool of sequences (perhaps run bowtie independently of Shapemapper and then feed in the resultant BAM file)? I'm concerned that the mapping might be impacted by the fact that I'm no longer mapping to the whole pool (i.e. a read could map to a sequence in the first set of 200 sequences, but then map again to a different sequence in the second set of 200 sequences whereas it would've uniquely mapped if all the pool sequences were present in the FASTA). Could you advise on how best to address this?

[Psirving]

If you're mapping quality threshold is set to default, this should only happen if the read is really multimapping. I think Shapemapper2 tosses multi-mapped reads, or maybe its that they take a penalty to the mapping quality score. Either way, what you're trying to do sounds similar to a transcriptome-wide study.

In our lab, we've been using shapemapper-txome. It does a pseudoalignment on the entire set, bins reads by transcript, then performs ShapeMapper on each individually.

There's also a published method that is more manual work and less computationally efficient, but basically does what you're suggesting: https://doi.org/10.1038/s41586-023-06500-y

[jainronit]

Great, thanks for sharing @Psirving - it looks like shapemapper-txome provides exactly the functionality I'm looking for, so I'd like to give that a try. However, I wasn't able to find any information on the Github page on how to install shapemapper-txome - would you be able to point me to where I might be able to find that info? I understand it's a wrapper for shapemapper, but not sure if it requires a separate installation.

[Psirving]

Unfortunately, shapemapper-txome isn't nicely packaged like ShapeMapper2. You'll need to install all of the dependencies manually (there are links in the README) and download the git repository (green "<> code" link at the top). Let me know if you have any specific troubles and I can try to help.