Junk String from Native::resolve(argc, argv, self)?

[stellarpower]

Hi,

Pretty new to Rice, so far I've been using it with ruby-bindgen on this project and it's worked well. I build my application inside a container, and it's run fine on the host even with different interpreter versions.

However, I have just started testing on my laptop, and I am getting a string containing junk passed through to my C++ function. At first I figured this might be a version mismatch, as when I looked, the laptop has been upgraded to ruby 4.0 (and might add, nice work between both ruby and rice that it seems to get this far, when e.g. Python is such a mess between API versions!!), but I've used RBEnv to build and install the same 3.4.8 that is on my desktop, where it was built, and that's not made a difference.

I know little about the internals of the interpreter, but, from the backtrace it seems to be happening very early in the process, when we receive the array of VALUEs from Ruby as argc and argv

Stopped on this call,

parray 6 argv
(VALUE *) $0 = 0x00007f20620a5048 {
  [0] = 139776605055480
  [1] = 88065
  [2] = 129
  [3] = 3
  [4] = 1
  [5] = 33
}

p rb_type(argv[0])
(ruby_value_type) RUBY_T_STRING
p RSTRING_PTR(argv[0])
(char *) 0x00007f2046e46a08 "\U00000016"
p RSTRING_LEN(argv[0])
(long) 1

p (char*)rb_class2name(klass)
(char *) 0x00007f206209d230 "Kernel"
p (char*)rb_class2name(self)
(char *) 0x00007........... "Object"

(The function in question is a free function, so that is as expected)

Values 1 to 5 are as expected (low-valued integers), but 0 should be a string literal of a filesystem path, and instead I get just two characters of apparent rubbish. By the looks of it, it seems the VALUE should (almost/partly/sometimes?) match Object#object_id, so, that first value seems suspiciously high. Playing with a few string literals in the interpreter, I got relatively low object_ids by comparison.

Is this an issue between where I am building and where I am deploying on my laptop, should Rice-built modules be stable across different Ruby API versions? Or is this a bug within Rice? What interests me is that it seems to be coming straight out of the C API untouched

Rice verison: 4.12.0
Ruby version: 3.1.2 (build), 3.4.8, 4.0.2
Arch: linux AMD64

Thanks!

[cfis]

Ruby extensions are not stable across different Ruby API versions, so Rice extensions would not be either. You'll want to recompile the extension against the version of Ruby you are using (easiest is to recompile it on the machine you are testing with).

Not sure if that is the problem in this case, but start there.

Glad to hear ruby-bindgen is working, I am about to make an official 1.0 release (finally!).

[stellarpower]

Thanks. Well, unless I just messed something up, I thought it was working on 3.4 on the host but not on the laptop. But downgrading to the exact same as in the build container seems to have made a difference, so, will keep an eye on it. Out of interest, do you know if there's any specific areas within the API that mean we have to rebuild for different versions? I was under the impression it was a lot more stable than many scripting languages, certainly the language itself has changed little by comparison and miuch of it seems to have been got right from early on. Glossing over what Rice is doing and having had a play before with casting VALUEs it seems remarkably straightforward.

Also as this failed silently, might be worth adding a warning or similar if the runtime major + minor version does not match what was built against on import.

I was also having a play at integrating bindgen with xmake, if I end up polishing that over time can let you know. I prefer and now always use xmake as a build system on new projects, but also found the cmake file bindgen produced seemed to have a good few holes that stopped it from building. That said, with C++26 coming soon, it might soon be a much easier option to do all this within the compiler itself.

[cfis]

Ruby will not load an extension built for a different major version (say 3.x versus 4.x), you should get an error message saying the extension did not load (it is not always obvious why though).

You can use whatever build system you would like of course, but the problem is you need RubyGems to actually run it when your extension gem is installed. You can hack extconf.rb or you can actually hack rubygems, like Rice does (well the hack is only needed for older RubyGem versions).

C++ adoption is slow, I would be happy to bump Rice to C++20 but it still feels premature. C++26 will be more like 2036 before its the baseline :(

[stellarpower]

you should get an error message saying the extension did not load (it is not always obvious why though).

Afraid not, it required it just fine, hence why I figured maybe it Just Works ™️. If I didn't have something in the chain that throws if the first argument exists on the filesystem then I would've been able to call all the way through. I'll check if anything else is missing in the build configuration.

You can hack extconf.rb

Maybe my current one is too simplistic, I basically decoupled it as much as possible to start with. Some point with time I'd like to make a custom shell, levering a parser into IRB to make it easier to run system commands day-to-day (`ls` with the extra backticks isn't gonna work and I want normal bareword parsing, tab completion of files, etc. to work, and a few extra ideas pulled in from some of the other new shells that have proved not so great once you start using them) but giving me immediate access to a proper language, i.e., Ruby. Bit like xonsh is for Python. So I was just toying with what if I bind to Ruby rather than messing around with CLI arguments within C++ as it wastes time. I inevitably get lazy and hardcode some assumptions that end up changing, so would be nicer if my program is an object and I interact with it in the shell to change the setup.

Anyway, it basically seemed dependency inversion, I have all the complications of what my build needs within the current setup, so I just regenerate the source files, tell the compiler where to find Rice's headers, dig out the Ruby headers from the currently active interpreter, and build it all into a shared library. Again maybe it's too trivial but so far it seems to work, I'll expand it out if I am missing further steps.

C++26 will be more like 2036 before its the baseline

Yeah agreed. Meanwhile I already have Bloomberg's modified clang and am just waiting for the upstream STL fixes (mainly constexpr dynamic memory) before I'll be using that as my daily driver, cause, why not automate things with the nice new features? 😅 Contracts could also be a useful thing for bindings, we might need to sanity-check a few things or concepts that don't map so neatly across two given languages at the border and ensure we fail early.