Skip to content

[micromega] Switch from Big_int to ZArith.#8743

Merged
fajb merged 6 commits intorocq-prover:masterfrom
ejgallego:zarith
Sep 16, 2020
Merged

[micromega] Switch from Big_int to ZArith.#8743
fajb merged 6 commits intorocq-prover:masterfrom
ejgallego:zarith

Conversation

@ejgallego
Copy link
Copy Markdown
Contributor

@ejgallego ejgallego commented Oct 16, 2018

We replace micromega use of Num by Zarith.

@ejgallego ejgallego added kind: cleanup Code removal, deprecation, refactorings, etc. kind: infrastructure CI, build tools, development tools. needs: documentation Documentation was not added or updated. labels Oct 16, 2018
@ejgallego ejgallego added this to the 8.10+beta1 milestone Oct 16, 2018
@ejgallego ejgallego force-pushed the zarith branch 2 times, most recently from 2a2f144 to 1294956 Compare October 16, 2018 13:08
@ejgallego ejgallego requested review from a team and thery as code owners October 16, 2018 13:08
@ejgallego

This comment has been minimized.

@vbgl

This comment has been minimized.

@ejgallego

This comment has been minimized.

@ejgallego
Copy link
Copy Markdown
Contributor Author

Take care of Nix if you want on the way.

Comment thread plugins/extraction/big.ml Outdated
@thery
Copy link
Copy Markdown
Contributor

thery commented Oct 16, 2018

is it ok with everybody if we add a new dependency (gmp)?

@maximedenes
Copy link
Copy Markdown
Member

is it ok with everybody if we add a new dependency (gmp)?

Note that we are just replacing dependencies by others. I would much prefer depending on ZArith + GMP than on Num.

Copy link
Copy Markdown
Member

@ppedrot ppedrot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No opinion.

@thery
Copy link
Copy Markdown
Contributor

thery commented Oct 16, 2018

Num was not the home cooked stuff by Valerie Menissier? gmp seems a bigger chunk

@maximedenes
Copy link
Copy Markdown
Member

Here's what the Num README says:

This is a legacy library. It used to be part of the core OCaml distribution (in otherlibs/num) but is now distributed separately. New applications that need arbitrary-precision arithmetic should use the Zarith library (https://github.com/ocaml/Zarith) instead of the Num library, and older applications that already use Num are encouraged to switch to Zarith. Zarith delivers much better performance than Num and has a nicer API.

@ejgallego
Copy link
Copy Markdown
Contributor Author

@thery has a point in that the transition won't be fully pain-free.

Also, I am not sure about the dependency of the frontend on zarith, but I guess we could factor that out. That being said, I do think the change is worth it.

@ejgallego ejgallego added the needs: benchmarking Performance testing is required. label Oct 16, 2018
@ejgallego
Copy link
Copy Markdown
Contributor Author

Performance will be interesting, please add to the CI / Bench cool devs in that regard.

@thery
Copy link
Copy Markdown
Contributor

thery commented Oct 16, 2018

If the motivation is micromega, I am not sure it will make much difference but I can be wrong.

@ejgallego

This comment has been minimized.

@fajb
Copy link
Copy Markdown
Contributor

fajb commented Sep 9, 2020

I am looking into it.

It could well be some serialization problem?

It does not seem so. Disabling caches does not solve the problem...

Comment thread plugins/micromega/numCompat.ml Outdated
@ejgallego
Copy link
Copy Markdown
Contributor Author

Ok one worrying failure:

assertion failure at gcd
File "./src/Helpers/ModArith.v", line 57, characters 2-6:

@ejgallego
Copy link
Copy Markdown
Contributor Author

Gonna run a CI with #13002 added and without the debug code so we can bench and also get a complete CI.

@ejgallego
Copy link
Copy Markdown
Contributor Author

For perennial, the problem is:

assertion failure at gcd, [18446744073709551616] vs [-18446744073709551616]
File "./src/Helpers/ModArith.v", line 57, characters 2-6:

so likely some silly typo again.

@ejgallego
Copy link
Copy Markdown
Contributor Author

Ok, this seems finally ready for review and bench. Note that the issue in ocaml/Zarith#58 created some problems in testing, but none in practice; anyways we've added a workaround.

Debug code lives in https://github.com/ejgallego/coq/tree/zarith%2Bdebug

@fajb
Copy link
Copy Markdown
Contributor

fajb commented Sep 11, 2020

There is still a problem with the .csdp.cache of the test-suite?
I don't remember how this is handled.
Is it supposed to be already there or regenerated?

@vbgl
Copy link
Copy Markdown
Contributor

vbgl commented Sep 11, 2020 via email

@ejgallego
Copy link
Copy Markdown
Contributor Author

The test suite is actually failing due to out of memory! It worked in my machine.

Maybe some zarith bug ? Seems pretty weird indeed.

@ejgallego
Copy link
Copy Markdown
Contributor Author

It should be already there, named: test-suite/.csdp.cache.test-suite

@fajb I updated that file, things were working fine in my machine, but indeed the memory consumption is strange as our test suite has noticed.

Bench is over by the way:

┌─────────────────────────────┬─────────────────────────┬─────────────────────────────────────────────┬─────────────────────────────────────────────┬───────────────────────────────┬───────────────────┐
│                             │      user time [s]      │                 CPU cycles                  │              CPU instructions               │     max resident mem [KB]     │    mem faults     │
│                             │                         │                                             │                                             │                               │                   │
│                package_name │     NEW     OLD PDIFF   │               NEW               OLD PDIFF   │               NEW               OLD PDIFF   │        NEW        OLD PDIFF   │  NEW  OLD PDIFF   │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                 coq-bignums │   57.52   58.14 -1.07 % │      159473439997      160743140667 -0.79 % │      215256412407      215405481574 -0.07 % │     472000     471360 +0.14 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                    coq-hott │  313.44  315.87 -0.77 % │      852125813994      854152466163 -0.24 % │     1372337168651     1372139577735 +0.01 % │     635796     676888 -6.07 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                 coq-coqutil │   59.07   59.49 -0.71 % │      164066630624      165249278506 -0.72 % │      204867373818      207850670552 -1.44 % │     528876     533292 -0.83 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│             coq-fiat-crypto │ 4933.56 4964.10 -0.62 % │    13703805770489    13780276215807 -0.55 % │    23632356922890    23808769949152 -0.74 % │    2326364    2326560 -0.01 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                coq-coqprime │   80.62   81.02 -0.49 % │      221772219356      222290123321 -0.23 % │      310222771856      310606456790 -0.12 % │     768808     768732 +0.01 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│              coq-coquelicot │   69.59   69.91 -0.46 % │      191199255689      190908716758 +0.15 % │      245176134888      245136030621 +0.02 % │     585232     587252 -0.34 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│       coq-mathcomp-fingroup │   47.92   48.00 -0.17 % │      133034489372      132729052337 +0.23 % │      183737227625      183653953473 +0.05 % │     486296     486292 +0.00 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                coq-bedrock2 │  220.50  220.74 -0.11 % │      609995514725      610764130742 -0.13 % │     1009815212129     1010897419381 -0.11 % │    1087348    1087252 +0.01 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                         coq │  542.33  542.88 -0.10 % │     1515870429882     1517190268687 -0.09 % │     2073129885186     2075059428096 -0.09 % │     592384     592152 +0.04 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│              coq-verdi-raft │ 1253.31 1254.45 -0.09 % │     3491485199898     3492305794495 -0.02 % │     4896883656609     4898588982213 -0.03 % │     934516     934728 -0.02 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│          coq-mathcomp-field │  290.97  291.20 -0.08 % │      810371423923      809839193393 +0.07 % │     1249424876124     1249297149216 +0.01 % │     825928     825676 +0.03 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│  coq-performance-tests-lite │  543.97  544.36 -0.07 % │     1520691914914     1520116420525 +0.04 % │     2795268552108     2796442877783 -0.04 % │     747332     747384 -0.01 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│      coq-mathcomp-odd-order │ 1070.16 1070.87 -0.07 % │     2980695269124     2981616599146 -0.03 % │     4750209131583     4750233816415 -0.00 % │     992036     992080 -0.00 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│            coq-fiat-parsers │  643.44  643.84 -0.06 % │     1802603676169     1802453523131 +0.01 % │     2717331471990     2717370622014 -0.00 % │    2998104    2997984 +0.00 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│               coq-fiat-core │  105.45  105.51 -0.06 % │      301576060554      300758643181 +0.27 % │      401371168791      401333791593 +0.01 % │     487060     487068 -0.00 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│       coq-mathcomp-solvable │  172.38  172.44 -0.03 % │      479703885177      479233959314 +0.10 % │      677556062495      677468542362 +0.01 % │     691832     691720 +0.02 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│       coq-engine-bench-lite │  380.02  380.05 -0.01 % │     1069023511982     1068285842923 +0.07 % │     1838925945920     1838247888520 +0.04 % │    3508668    3508732 -0.00 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│               coq-fourcolor │ 2518.64 2518.32 +0.01 % │     7022901241797     7019509600213 +0.05 % │    11447737865163    11447223653710 +0.00 % │     774876     775024 -0.02 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│             coq-lambda-rust │ 1262.98 1262.76 +0.02 % │     3515011128111     3513461858344 +0.04 % │     4949338217145     4946675717171 +0.05 % │    1159380    1159368 +0.00 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│ coq-rewriter-perf-SuperFast │ 1057.38 1056.95 +0.04 % │     2940976697968     2940895197848 +0.00 % │     4629379715349     4629740954555 -0.01 % │    1070804    1070672 +0.01 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                  coq-geocoq │ 1469.75 1469.07 +0.05 % │     4102093064533     4097309838052 +0.12 % │     6222263366825     6222091958462 +0.00 % │    1190120    1146084 +3.84 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                coq-rewriter │  610.51  610.22 +0.05 % │     1694597080132     1692249153052 +0.14 % │     2583731738860     2583918291568 -0.01 % │    1036784     987712 +4.97 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                    coq-corn │ 1707.32 1705.26 +0.12 % │     4754184603733     4750701788800 +0.07 % │     7278243980988     7278474379110 -0.00 % │     855684     855604 +0.01 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                   coq-flocq │  159.48  159.28 +0.13 % │      443219035633      442805735154 +0.09 % │      552990623634      553358794765 -0.07 % │     822204     822128 +0.01 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                 coq-unimath │ 3993.34 3987.00 +0.16 % │    11111629365061    11095643671726 +0.14 % │    21160840068546    21166523184354 -0.03 % │    1113380    1116692 -0.30 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                coq-compcert │  663.84  662.62 +0.18 % │     1848990291336     1848235883695 +0.04 % │     2555820339465     2555802379899 +0.00 % │    1131576    1131244 +0.03 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│        coq-mathcomp-algebra │  140.40  140.14 +0.19 % │      389907236902      389371135024 +0.14 % │      503642194141      503577259557 +0.01 % │     562908     564560 -0.29 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                   coq-verdi │  120.83  120.51 +0.27 % │      332438231484      332729584726 -0.09 % │      413031222258      413028766973 +0.00 % │     584464     585620 -0.20 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│      coq-mathcomp-character │  160.69  160.25 +0.27 % │      446290493749      445446063798 +0.19 % │      618941348991      618925419293 +0.00 % │     734624     735040 -0.06 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│                   coq-color │  618.37  616.30 +0.34 % │     1723501371646     1718757700920 +0.28 % │     2190345082800     2190190676093 +0.01 % │    1455192    1455288 -0.01 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│      coq-mathcomp-ssreflect │   50.39   50.14 +0.50 % │      138115038496      137633047582 +0.35 % │      176210590083      176150819239 +0.03 % │     521688     521560 +0.02 % │    0    0  +nan % │
├─────────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼───────────────────┤
│            coq-math-classes │  190.48  189.52 +0.51 % │      530678395166      529859918698 +0.15 % │      700214855833      700186292794 +0.00 % │     504956     504808 +0.03 % │    0    0  +nan % │
└─────────────────────────────┴─────────────────────────┴─────────────────────────────────────────────┴─────────────────────────────────────────────┴───────────────────────────────┴───────────────────┘

No significative change IMO.

@fajb
Copy link
Copy Markdown
Contributor

fajb commented Sep 11, 2020

@ejgallego the bench looks good.
commit 229350b has a fresh .csdp.cache. It works even when csdp is uninstalled.
I have also added zarith to merlin.in and make clean removes .csdp.cache from the test-suite.

@silene
Copy link
Copy Markdown
Contributor

silene commented Sep 11, 2020

the bench looks good

Should we worry that the memory footprint of coq-geocoq and coq-rewriter has increased by 5%?

@ejgallego
Copy link
Copy Markdown
Contributor Author

Should we worry that the memory footprint of coq-geocoq and coq-rewriter has increased by 5%?

If not reproducible likely not; I don't know how to trigger a bench for 2 development only now tho.

@fajb
Copy link
Copy Markdown
Contributor

fajb commented Sep 11, 2020

@ejgallego No idea. I have never looked at memory consumption.
My feeling was that zarith was designed to be GC friendly.

@ejgallego
Copy link
Copy Markdown
Contributor Author

@ejgallego the bench looks good.
commit 229350b has a fresh .csdp.cache. It works even when csdp is uninstalled.
I have also added zarith to merlin.in and make clean removes .csdp.cache from the test-suite.

Thanks, I've launched the CI.

@ejgallego No idea. I have never looked at memory consumption.
My feeling was that zarith was designed to be GC friendly.

Indeed it seems the OOM could maybe a problem with the csdp cache? Let see what the re-run says.

Comment thread plugins/micromega/numCompat.ml Outdated
ejgallego and others added 6 commits September 15, 2020 15:56
We still link num in `coqc` , that will be removed in a separate step.

Co-authored-by: Vincent Laporte <Vincent.Laporte@fondation-inria.fr>
-  merlin.in : added zarith
- test-suite/Makefile remove .csdp.cache on make clean

updated .csdp.cache.test-suite
In particular, behavior of `Z.gcd` and `Z.lcm` has been fixed in
1.10, see

ocaml/Zarith#58
@ejgallego
Copy link
Copy Markdown
Contributor Author

@fajb I think we are close to a fixpoint here, would you like to assign?

@fajb
Copy link
Copy Markdown
Contributor

fajb commented Sep 15, 2020

@ejgallego fixpoint, eventually! I'll merge as soon as the CI passes.

@ejgallego
Copy link
Copy Markdown
Contributor Author

@ejgallego fixpoint, eventually! I'll merge as soon as the CI passes.

Failure on coq_tools is spurious [failing everywhere] and not related to this PR.

@fajb
Copy link
Copy Markdown
Contributor

fajb commented Sep 16, 2020

@ejgallego well done.

@ejgallego
Copy link
Copy Markdown
Contributor Author

Thanks @fajb !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind: cleanup Code removal, deprecation, refactorings, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.