Skip to content
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,11 @@
4. `dcast()` and `melt()` "just work" when passed a data.frame, not just data.tables, with no need for coercion, [#7614](https://github.com/Rdatatable/data.table/issues/7614). Thanks @MichaelChirico for the suggestion and @manmita for the PR. Note that to avoid potential conflicts with {reshape2}'s data.frame methods, we do the dispatch to the data.table method manually.

5. `tables()` can now optionally report `data.table` objects stored one level deep inside list objects when `depth=1L`, [#2606](https://github.com/Rdatatable/data.table/issues/2606). Thanks @MichaelChirico for the report and @manmita for the PR

Comment thread
jangorecki marked this conversation as resolved.
Outdated
6. `yearqtr()` and `yearmon()` now gain an optional format specifier [#7694](https://github.com/Rdatatable/data.table/issues/7694). 'numeric' is the default, which preserves the original behavior, but 'character' formats `yearqtr()` as YYYYQ# (e.g. 2025Q2) and `yearmon()` as YYYYM## (e.g. 2025M02, 2025M10). Thanks to @jan-swissre for the report and @LunaticSage218 for the implementation.

7. `frolladaptivefun()` and `frollfun()` adjusted print statements for verbose messages to be more user friendly, [#7021](https://github.com/Rdatatable/data.table/issues/7021) Thanks @jangorecki for initiating the issue.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also needs your name or github tag for thanking yourself.

For the news phrasing itself, maybe something like?

Verbose outputs from `frolladaptivefun()` and `frollfun()` are now clearer and more user friendly


### BUG FIXES

1. `fread()` with `skip=0` and `(header=TRUE|FALSE)` no longer skips the first row when it has fewer fields than subsequent rows, [#7463](https://github.com/Rdatatable/data.table/issues/7463). Thanks @emayerhofer for the report and @ben-schwen for the fix.
Expand Down
116 changes: 58 additions & 58 deletions inst/tests/froll.Rraw
Original file line number Diff line number Diff line change
Expand Up @@ -308,36 +308,36 @@ test(6000.0671, frollmean(c(1:2,NA,4:10), 4), c(rep(NA_real_, 6), 5.5, 6.5, 7.5,
"frollfunR: 1:",
"frollmeanFast: running for input length 10, window 4, hasnf 0, narm 0",
"frollmeanFast: non-finite values are present in input, skip non-finite unaware attempt and run with extra care for NFs straighaway",
"frollfun: processing fun 0 algo 0 took.*",
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"
))
))
Comment thread
jangorecki marked this conversation as resolved.
Outdated
test(6000.0672, frollmean(c(1:2,NA,4:10), 4, has.nf=FALSE), c(rep(NA_real_, 6), 5.5, 6.5, 7.5, 8.5), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*sequentially.*single rolling computation.*",
"frollfunR: 1:",
"frollmeanFast: running for input length 10, window 4, hasnf -1, narm 0",
"frollmeanFast: non-finite values are present in input, skip non-finite unaware attempt and run with extra care for NFs straighaway",
"frollfun: processing fun 0 algo 0 took.*",
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"
), warning="has.nf=FALSE used but non-finite values are present in input, use default has.nf=NA to avoid this warning")
), warning="has.nf=FALSE used but non-finite values are present in input, use default has.nf=NA to avoid this warning")
test(6000.0673, frollmean(c(1:2,NA,4:10), 2, has.nf=FALSE), c(NA, 1.5, NA, NA, 4.5, 5.5, 6.5, 7.5, 8.5, 9.5), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*sequentially.*single rolling computation.*",
"frollfunR: 1:",
"frollmeanFast: running for input length 10, window 2, hasnf -1, narm 0",
"frollmeanFast: non-finite values are present in input, re-running with extra care for NFs",
"frollfun: processing fun 0 algo 0 took.*",
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"
), warning="has.nf=FALSE used but non-finite values are present in input, use default has.nf=NA to avoid this warning")
), warning="has.nf=FALSE used but non-finite values are present in input, use default has.nf=NA to avoid this warning")
test(6000.0674, frollmean(c(1:2,NA,4:10), 4, align="center"), c(rep(NA_real_, 4), 5.5, 6.5, 7.5, 8.5, NA, NA), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*sequentially.*single rolling computation.*",
"frollmeanFast: running for input length 10, window 4, hasnf 0, narm 0",
"frollmeanFast: non-finite values are present in input, skip non-finite unaware attempt and run with extra care for NFs straighaway",
"frollfun: align 0, shift answer by -2",
"frollfun: processing fun 0 algo 0 took.*",
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"
))
))
options(datatable.verbose=FALSE)

#### fill constant
Expand Down Expand Up @@ -436,17 +436,17 @@ test(6000.1196, frollmean(c(1:5,NA), 1:6, algo="exact", na.rm=TRUE, adaptive=TRU
"frollfunR: 1:",
"frolladaptivemeanExact: running in parallel for input length 6, hasnf 0, narm 1",
"frolladaptivemeanExact: non-finite values are present in input, re-running with extra care for NFs",
"frolladaptivefun: processing fun 0 algo 1 took.*",
"frolladaptivefun: processing fun MEAN algo exact took.*",
"frollfunR: processing.*took.*"
))
))
#### exact na.rm=TRUE verbose=TRUE
test(6000.1197, frollmean(c(1:5,NA), 2, algo="exact", na.rm=TRUE), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*algo='exact' is already parallelised.*",
"frollfunR: 1:",
"frollmeanExact: running in parallel for input length 6, window 2, hasnf 0, narm 1",
"frollmeanExact: non-finite values are present in input, re-running with extra care for NFs",
"frollfun: processing fun 0 algo 1 took.*",
"frollfun: processing fun MEAN algo exact took.*",
"frollfunR: processing.*took.*"
))
options(datatable.verbose=FALSE)
Expand Down Expand Up @@ -675,82 +675,82 @@ test(6000.171, frollmean(x, n), output=c(
"frollfunR: .*sequentially.*single rolling computation.*",
"frollfunR: 1:",
"frollmeanFast: running for input length 10, window 3, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 0 took.*",
"frollfunR: processing.*took.*"))
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"))
test(6000.172, frollmean(list(x, x+1), n), output=c(
"frollfunR: allocating memory for results 2x1",
"frollfunR: .*in parallel.*",
"frollfunR: 1:",
"frollmeanFast: running for input length 10, window 3, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 0 took.*",
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: 2:",
"frollmeanFast: running for input length 10, window 3, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 0 took.*",
"frollfunR: processing.*took.*"))
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"))
test(6000.173, frollmean(x, c(n, n+1)), output=c(
"frollfunR: allocating memory for results 1x2",
"frollfunR: .*in parallel.*",
"frollfunR: 1:",
"frollmeanFast: running for input length 10, window 3, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 0 took.*",
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: 2:",
"frollmeanFast: running for input length 10, window 4, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 0 took.*",
"frollfunR: processing.*took.*"))
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"))
test(6000.174, frollmean(list(x, x+1), c(n, n+1)), output=c(
"frollfunR: allocating memory for results 2x2",
"frollfunR: .*in parallel.*",
"frollfunR: 1:",
"frollmeanFast: running for input length 10, window 3, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 0 took.*",
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: 2:",
"frollmeanFast: running for input length 10, window 4, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 0 took.*",
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: 3:",
"frollmeanFast: running for input length 10, window 3, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 0 took.*",
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: 4:",
"frollmeanFast: running for input length 10, window 4, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 0 took.*",
"frollfunR: processing.*took.*"))
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"))
test(6000.175, frollmean(x, n, algo="exact"), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*algo='exact' is already parallelised.*",
"frollfunR: 1:",
"frollmeanExact: running in parallel for input length 10, window 3, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 1 took.*",
"frollfunR: processing.*took.*"))
"frollfun: processing fun MEAN algo exact took.*",
"frollfunR: processing.*took.*"))
test(6000.176, frollmean(x, n, align="center"), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*sequentially.*single rolling computation.*",
"frollfunR: 1:",
"frollmeanFast: running for input length 10, window 3, hasnf 0, narm 0",
"frollfun: align 0, shift answer by -1",
"frollfun: processing fun 0 algo 0 took.*",
"frollfunR: processing.*took.*"))
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"))
test(6000.177, frollmean(x, n, align="left"), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*sequentially.*single rolling computation.*",
"frollfunR: 1:",
"frollmeanFast: running for input length 10, window 3, hasnf 0, narm 0",
"frollfun: align -1, shift answer by -2",
"frollfun: processing fun 0 algo 0 took.*",
"frollfunR: processing.*took.*"))
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"))
nn = c(1:4,2:3,1:4)
test(6000.178, frollmean(x, nn, adaptive=TRUE), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*sequentially because adaptive.*",
"frollfunR: 1:",
"frolladaptivemeanFast: running for input length 10, hasnf 0, narm 0",
"frolladaptivefun: processing fun 0 algo 0 took.*",
"frollfunR: processing.*took.*"))
"frolladaptivefun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"))
test(6000.179, frollmean(x, nn, algo="exact", adaptive=TRUE), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*algo='exact' is already parallelised.*",
"frollfunR: 1:",
"frolladaptivemeanExact: running in parallel for input length 10, hasnf 0, narm 0",
"frolladaptivefun: processing fun 0 algo 1 took.*",
"frollfunR: processing.*took.*"))
"frolladaptivefun: processing fun MEAN algo exact took.*",
"frollfunR: processing.*took.*"))

x[8] = NA
test(6000.180, frollmean(x, n), output=c(
Expand All @@ -759,59 +759,59 @@ test(6000.180, frollmean(x, n), output=c(
"frollfunR: 1:",
"frollmeanFast: running for input length 10, window 3, hasnf 0, narm 0",
"frollmeanFast: non-finite values are present in input, re-running with extra care for NFs",
"frollfun: processing fun 0 algo 0 took.*",
"frollfunR: processing.*took.*"))
"frollfun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"))
test(6000.181, frollmean(x, n, algo="exact"), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*algo='exact' is already parallelised.*",
"frollfunR: 1:",
"frollmeanExact: running in parallel for input length 10, window 3, hasnf 0, narm 0",
"frollmeanExact: non-finite values are present in input, na.rm=FALSE and algo='exact' propagates NFs properply, no need to re-run",
"frollfun: processing fun 0 algo 1 took.*",
"frollfunR: processing.*took.*"))
"frollfun: processing fun MEAN algo exact took.*",
"frollfunR: processing.*took.*"))
test(6000.182, frollmean(x, nn, adaptive=TRUE), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*sequentially because adaptive.*",
"frollfunR: 1:",
"frolladaptivemeanFast: running for input length 10, hasnf 0, narm 0",
"frolladaptivemeanFast: non-finite values are present in input, re-running with extra care for NFs",
"frolladaptivefun: processing fun 0 algo 0 took.*",
"frollfunR: processing.*took.*"))
"frolladaptivefun: processing fun MEAN algo fast took.*",
"frollfunR: processing.*took.*"))
test(6000.183, frollmean(x, nn, algo="exact", adaptive=TRUE), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*algo='exact' is already parallelised.*",
"frollfunR: 1:",
"frolladaptivemeanExact: running in parallel for input length 10, hasnf 0, narm 0",
"frolladaptivemeanExact: non-finite values are present in input, na.rm=FALSE and algo='exact' propagates NFs properply, no need to re-run",
"frolladaptivefun: processing fun 0 algo 1 took.*",
"frollfunR: processing.*took.*"))
"frolladaptivefun: processing fun MEAN algo exact took.*",
"frollfunR: processing.*took.*"))

d = as.data.table(list(1:10/2, 10:1/4))
test(6000.184, frollmean(d[,1], 3, algo="exact"), output=c(
"frollfunR: allocating memory for results 1x1",
"frollfunR: .*algo='exact' is already parallelised.*",
"frollfunR: 1:",
"frollmeanExact: running in parallel for input length 10, window 3, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 1 took.*",
"frollfun: processing fun MEAN algo exact took.*",
"frollfunR: processing.*took.*"
))
))
test(6000.185, frollmean(d, 3:4, algo="exact"), output=c(
"frollfunR: allocating memory for results 2x2",
"frollfunR: .*sequentially.*algo='exact'.*already parallelised.*",
"frollfunR: 1:",
"frollmeanExact: running in parallel for input length 10, window 3, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 1 took.*",
"frollfun: processing fun MEAN algo exact took.*",
"frollfunR: 2:",
"frollmeanExact: running in parallel for input length 10, window 4, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 1 took.*",
"frollfun: processing fun MEAN algo exact took.*",
"frollfunR: 3:",
"frollmeanExact: running in parallel for input length 10, window 3, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 1 took.*",
"frollfun: processing fun MEAN algo exact took.*",
"frollfunR: 4:",
"frollmeanExact: running in parallel for input length 10, window 4, hasnf 0, narm 0",
"frollfun: processing fun 0 algo 1 took.*",
"frollfun: processing fun MEAN algo exact took.*",
"frollfunR: processing.*took.*"
))
))
options(datatable.verbose=FALSE)

## test warnings
Expand Down Expand Up @@ -861,9 +861,9 @@ options(datatable.verbose=FALSE)

## frollmax adaptive
options(datatable.verbose=TRUE) ## adaptive frollmax no fast algo
test(6000.3, frollmax(1:4, c(2,2,2,2), adaptive=TRUE), output="frolladaptivefun: algo 0 not implemented, fall back to 1")
test(6000.3001, frollmax(1:4, c(2,2,2,2), algo="fast", adaptive=TRUE), output="frolladaptivefun: algo 0 not implemented, fall back to 1")
test(6000.3002, frollmax(1:4, c(2,2,2,2), algo="exact", adaptive=TRUE), notOutput="frolladaptivefun: algo 0 not implemented, fall back to 1")
test(6000.3, frollmax(1:4, c(2,2,2,2), adaptive=TRUE), output="frolladaptivefun: algo fast not implemented, fall back to exact")
test(6000.3001, frollmax(1:4, c(2,2,2,2), algo="fast", adaptive=TRUE), output="frolladaptivefun: algo fast not implemented, fall back to exact")
test(6000.3002, frollmax(1:4, c(2,2,2,2), algo="exact", adaptive=TRUE), notOutput="frolladaptivefun: algo fast not implemented, fall back to exact")
options(datatable.verbose=FALSE)
n = c(3,2,2,4,2,1,4,8)
x = c(7,2,3,6,3,2,6,6) # no NA
Expand Down Expand Up @@ -1022,9 +1022,9 @@ test(6000.564, frollapply(FUN=mean, 1:3, list(c(0,-1,1)), adaptive=TRUE), error=

## frollmin adaptive
options(datatable.verbose=TRUE) ## adaptive frollmin no fast algo
test(6000.6, frollmin(1:4, c(2,2,2,2), adaptive=TRUE), output="frolladaptivefun: algo 0 not implemented, fall back to 1")
test(6000.6001, frollmin(1:4, c(2,2,2,2), algo="fast", adaptive=TRUE), output="frolladaptivefun: algo 0 not implemented, fall back to 1")
test(6000.6002, frollmin(1:4, c(2,2,2,2), algo="exact", adaptive=TRUE), notOutput="frolladaptivefun: algo 0 not implemented, fall back to 1")
test(6000.6, frollmin(1:4, c(2,2,2,2), adaptive=TRUE), output="frolladaptivefun: algo fast not implemented, fall back to exact")
test(6000.6001, frollmin(1:4, c(2,2,2,2), algo="fast", adaptive=TRUE), output="frolladaptivefun: algo fast not implemented, fall back to exact")
test(6000.6002, frollmin(1:4, c(2,2,2,2), algo="exact", adaptive=TRUE), notOutput="frolladaptivefun: algo fast not implemented, fall back to exact")
options(datatable.verbose=FALSE)
n = c(3,2,2,4,2,1,4,8)
x = c(7,2,3,6,3,2,6,6) # no NA
Expand Down Expand Up @@ -1200,7 +1200,7 @@ test(6000.931, frollprod(1:3, 2), c(NA, 2, 6), output="frollprodFast: running fo
test(6000.932, frollprod(1:3, 2, align="left"), c(2, 6, NA), output="frollfun: align")
test(6000.933, frollprod(c(1,2,NA), 2), c(NA, 2, NA), output="non-finite values are present in input, re-running with extra care for NFs")
test(6000.934, frollprod(c(NA,2,3), 2), c(NA, NA, 6), output="non-finite values are present in input, skip non-finite inaware attempt and run with extra care for NFs straighaway")
test(6000.935, frollprod(1:3, c(2,2,2), adaptive=TRUE), c(NA, 2, 6), output="algo 0 not implemented, fall back to 1")
test(6000.935, frollprod(1:3, c(2,2,2), adaptive=TRUE), c(NA, 2, 6), output="algo fast not implemented, fall back to exact")
test(6000.936, frollprod(c(NA,2,3), c(2,2,2), adaptive=TRUE), c(NA, NA, 6), output="non-finite values are present in input, na.rm=FALSE and algo='exact' propagates NFs properply, no need to re-run")
options(datatable.verbose=FALSE)
# floating point overflow
Expand Down Expand Up @@ -1433,7 +1433,7 @@ test(6001.715, frollvar(1:3, 0, algo="exact"), c(NA_real_,NA_real_,NA_real_), op
test(6001.716, frollvar(c(1:2,NA), 0, algo="exact"), c(NA_real_,NA_real_,NA_real_))
test(6001.717, frollvar(c(1:2,NA), 0, algo="exact", na.rm=TRUE), c(NA_real_,NA_real_,NA_real_))
test(6001.718, frollvar(c(1:2,NA), 2), c(NA,0.5,NA), options=c("datatable.verbose"=TRUE), output="redirecting to frollvarExact")
test(6001.721, frollvar(adaptive=TRUE, 1:3, c(2,0,2)), c(NA,NA,0.5), options=c("datatable.verbose"=TRUE), output="not implemented, fall back to")
test(6001.721, frollvar(adaptive=TRUE, 1:3, c(2,0,2)), c(NA,NA,0.5), options=c("datatable.verbose"=TRUE), output="algo fast not implemented, fall back to exact")
test(6001.722, frollvar(adaptive=TRUE, 1:3, c(2,0,2), fill=99), c(99,NA,0.5))
test(6001.723, frollvar(adaptive=TRUE, c(1:2,NA), c(2,0,2)), c(NA_real_,NA_real_,NA_real_))
test(6001.724, frollvar(adaptive=TRUE, c(1:2,NA), c(2,0,2), na.rm=TRUE), c(NA_real_,NA_real_,NA_real_))
Expand Down Expand Up @@ -1492,7 +1492,7 @@ test(6001.8194, frollsd(c(NA,2:3), 2, has.nf=FALSE), c(NA,NA,sqrt(0.5)), warning
test(6001.8195, frollsd(c(NA,2:3), 2), c(NA,NA,sqrt(0.5)), options=c("datatable.verbose"=TRUE), output="skip non-finite inaware attempt and run with extra care")
test(6001.8196, frollsd(c(NA,2:3), 2, has.nf=FALSE, algo="exact"), c(NA,NA,sqrt(0.5)), warning="used but non-finite values are present in input")
test(6001.8197, frollsd(c(NA,2:3), 2, algo="exact", na.rm=TRUE), c(NA,NA,sqrt(0.5)), options=c("datatable.verbose"=TRUE), output="re-running with extra care for NF")
test(6001.8201, frollsd(adaptive=TRUE, 1:3, c(2,2,2)), c(NA,sqrt(0.5),sqrt(0.5)), options=c("datatable.verbose"=TRUE), output="frolladaptivefun: algo 0 not implemented, fall back to 1")
test(6001.8201, frollsd(adaptive=TRUE, 1:3, c(2,2,2)), c(NA,sqrt(0.5),sqrt(0.5)), options=c("datatable.verbose"=TRUE), output="frolladaptivefun: algo fast not implemented, fall back to exact")
test(6001.8202, frollsd(adaptive=TRUE, 1:3, c(2,2,2)), c(NA,sqrt(0.5),sqrt(0.5)), options=c("datatable.verbose"=TRUE), output="frolladaptivesdExact: calling sqrt(frolladaptivevarExact(...))")
test(6001.821, frollsd(adaptive=TRUE, 1:3, c(2,0,2)), c(NA,NA,sqrt(0.5)))
test(6001.822, frollsd(adaptive=TRUE, 1:3, c(2,0,2), fill=99), c(99,NA,sqrt(0.5)))
Expand Down
Loading
Loading