Cryptobib is comprehensive and fairly popular. In this issue, I want to discuss some of the potential improvements that we could make. I want to positioning this as a starting point for a discussion, and I also don't think all of these issues need to be tackled at once. In my mind, we will create separate issues for work items from this discussion.
I can think of a few problems/points of friction:
- Adding a 40MB file to your bibtex processing takes a fair bit of compute, taking upwards of a minute with
biber
- Editing a 30MB
crypto_db.bib file is fairly horrible with many editors
- Because we're directly storing the database in
bibtex we have to do weird things to preserve compatibility with older versions of bibtex while not being able to support fancier features supported by biblatex (e.g. custom date formatting with may # "~27--28," shenanigans instead of date fields supporting date ranges)
- Due to features added at different times it's hard to e.g. go back and add
doi to entries that are currently lacking them or to otherwise try to update metadata. Once it's in, it's in.
- IACR ePrint updates to papers are not tracked
- IACR ePrint versions of papers are not tracked
On the maintenance side, I can think of the following issues:
- The current scripts work fine but are a bit annoying at times because we need to parse bibtex.
- We could use a bit of a cleanup in terms of virtual environments and stuff (mostly more modern tools like
uv, ruff)
- We should use the ePrint API (but that needs API secrets)
- The
export repositories are manually generated and should probably be done using CI
I think the following would be nice-to-have:
export-compat and export-biblatex variants of cryptobib that you can get the modern biblatex features (I have hacked this together already in this fork: https://github.com/thomwiggers/cryptobib-export/). Maybe also bibxml for biber, which should be faster to parse.- A variant that separates all conferences and years into separate
.bib files so that you don't have to parse all conferences.
- This could be combined with a LaTeX package that automatically imports the right bibliography files.
On the strategy for getting this done, I would like to lean into the use of vibecoding. For example, Claude is pretty good at writing scripts for scraping, parsing and processing data. While I of course don't want to merge PRs without review, I actually don't think that I care too much about the code quality here — we don't run production systems on these scripts, and I also think that if the vibed thing turns into a mess, throwing things away and starting over is something that has gotten super cheap with AI.
Cryptobib is comprehensive and fairly popular. In this issue, I want to discuss some of the potential improvements that we could make. I want to positioning this as a starting point for a discussion, and I also don't think all of these issues need to be tackled at once. In my mind, we will create separate issues for work items from this discussion.
I can think of a few problems/points of friction:
bibercrypto_db.bibfile is fairly horrible with many editorsbibtexwe have to do weird things to preserve compatibility with older versions ofbibtexwhile not being able to support fancier features supported bybiblatex(e.g. custom date formatting withmay # "~27--28,"shenanigans instead ofdatefields supporting date ranges)eventdatefieldsdoito entries that are currently lacking them or to otherwise try to update metadata. Once it's in, it's in.On the maintenance side, I can think of the following issues:
uv,ruff)exportrepositories are manually generated and should probably be done using CII think the following would be nice-to-have:
export-compatandexport-biblatexvariants ofcryptobibthat you can get the modern biblatex features (I have hacked this together already in this fork: https://github.com/thomwiggers/cryptobib-export/). Maybe alsobibxmlfor biber, which should be faster to parse..bibfiles so that you don't have to parse all conferences.On the strategy for getting this done, I would like to lean into the use of vibecoding. For example, Claude is pretty good at writing scripts for scraping, parsing and processing data. While I of course don't want to merge PRs without review, I actually don't think that I care too much about the code quality here — we don't run production systems on these scripts, and I also think that if the vibed thing turns into a mess, throwing things away and starting over is something that has gotten super cheap with AI.