Refactor barcodeBERT tokenizer: move to dataset and fix performance issue - #34

Open

charlie1587 wants to merge 49 commits into

mainfrom

before_merge_with_BarcodeBERT_5M

Commits on Apr 4, 2025

Add new branch.
zmgong
committed
4d925c6

Commits on Apr 15, 2025

Merge commit '7fb8e020f139aceee889f206a42e006b618a38bc' into before_merge_with_BarcodeBERT_5M
Chuanqi
committed
47577ad
fix(config): Add missing config
Chuanqi
committed
4e967fb
fix(training): Fix new barcodeBERT tokenizer
Chuanqi
committed
51b4e4e

Commits on Apr 16, 2025

Commits on Apr 17, 2025

Merge commit '51b4e4eb62bffaf5c76fb524664b7b5e878ebc71' into before_merge_with_BarcodeBERT_5M
Chuanqi
committed
00a492e
refactor(dataset): Move tokenizer to dataset.py
Chuanqi
committed
3916b2b
feat(dataset): Support batch for new barcodebert tokenizer
Chuanqi
committed
6974b88
Merge commit '6fd7556946d5e07c7bb7801aab459c2e37d8a891' into before_merge_with_BarcodeBERT_5M
Chuanqi
committed
6d64867
style(dataset): Remove comment
Chuanqi
committed
710f57f
fix(dataset): Fixed the problem of tokenizer loading data too slowly
Chuanqi
committed
950a4a1

Commits on Apr 18, 2025

style(dataset): Delete unused tokenizers and format the code
Chuanqi
committed
7c0d919
style(training): Format the code
Chuanqi
committed
736e499
fix(inference): Add missing library
Chuanqi
committed
fac118b

Commits on Apr 21, 2025

Update the font size of some figure plotting scripts.
zmgong
committed
462a599
update the model names to more proper names in multiple config files.
zmgong
committed
779da4f
Remove the finished tod comment.
zmgong
committed
5802ac8
fix(dataset): Correct typo
Chuanqi
committed
d831530

Commits on Apr 23, 2025

Commits on Apr 27, 2025

Commits on May 4, 2025

Commits on May 5, 2025

Merge branch '33-update-the-softmax-issue-with-dna-encoder' into before_merge_with_BarcodeBERT_5M
Chuanqi
committed
080a775
fix: Resolve tensor resizing error in data loading
Chuanqi
committed
e2acf1c