Limited C API is not as abstract as HPy. Most notably Limited C API still exposes reference counting as memory management mechanism, HPy abstracts that. However, ecosystem wide adoption of limited C API and stable ABI would already improve things significantly.
There is basic GraalPy support in Maturin[0] and PyO3[1], the problem is often that packages require older Maturin/PyO3 versions and/or they use CPython-isms, semi-public APIs, etc., but it is getting better, for example [2].
It is fair to say that large projects with a huge set of dependencies will likely face some compatibility issues, but we're working on ironing this out. There is GraalPy support in setup-python GitHub action. GraalPy is supported in the manylinux image [3]. Hopefully soon also in cibuildwheel [4].
RDS files are a common way of sharing serialized R objects. Promises are valid R objects and supported by this serialization format. They always have been and I believe it is an intentional feature. The problem is that some people may think of RDS files as more convenient CSV files, but they are not.
No, you'd have to take all the profit of all the companies in the country and divide that by the number of citizens. That would be true socialism. Like a kibbutz in Israel. IMO it is as much utopia as 100% free market capitalism. Almost everywhere you have something in between, including in the US.
FastR doesn't alter the semantics of R, so when dplyr copies a vector in GNU-R then FastR has to copy it too. However, FastR does use reference counting (not sure if that's turned on in GNU-R 3.5.1 now) so it may avoid some unnecessary copies.
There is a plot of warm-up curves for this specific example. Search for "To make the analysis of that benchmark complete, here is a plot with warm-up curves".
However, it is true that the warm-up and memory usage are something we need to improve. We're working on providing native image [1] of FastR. With that, both the warm-up and memory usage shold get close to GNU-R.
FastR can actually run all tests of the development version of dplyr with a simple patch. We're working on removing the need for that patch altogether.
data.table is a different beast and we will probably provide and maintain patched version for FastR. They do things like casting data of internal R structure to byte array and then memcopy it to another R structure. This is very tricky to emulate if your data structures actually live on Java side and you're handing out only some handles to the native code.
FastR also uses native BLAS and LAPACK libraries. It should be possible to link it with Intel MKL as well.
We didn't want to include comparison to R-3.5.X, because FastR itself is based on the base library of 3.4.0, but the results for GNU-R 3.5.1 almost the same as for R-3.4.0.
AFAIK ALTREP is not used that much yet inside GNU-R itself. They can now do efficient integer sequences (i.e. 1:1000 does not allocate 1000 integers unless necessary), which would save a little bit of memory in this example, but that's about it. FastR also plans to implement the ALTREP interface for packages. Internally, we've been already using things like compact sequences.
