Adding Dockerfile and bash scripts to build and run Docker containers #93
Conversation
…demos with the new one, using findepi's provided docker images Tested against one example, continue to test against the remaining non-GUI based examples.
|
Testing build docker container against the below examples:
Below demos run partially but fails to complete other checks, need further investigation:
Below demos are GUI based and might be non-trivial to run inside a docker container atm:
In case, the test fails or cannot be tested, could be due to one or all of these reasons:
|
|
@neomatrix369 , thanks! I'll review asap. |
…the docker image name (under version), makes it easier to identify graalvm-demo images
Thanks, try to run this on your own machine and see how it works, I will continue to test other examples using the docker container, please don't merge it yet (even if you are happy with the current PR). |
|
@olyagpl we can now build and run any version of GraalVM (community version, provided they are available on |
…nodejs component using gu, also listen on port 8088 as per requirements of one of the examples.
- testing simple example hello_graal - install python using gu - install maven as part of the graalvm-demos docker image - set the appropriate paths and add them to the PATH env variable
…VM_HOME - fixes maven build issues. Add a couple of files to the ignore list. Add 'wrk' installation step(s) to the Dockerfile.
…e-images, and built the main docker image based on these two dependencies.
…ckerImage.sh shell-script
…t folder 'shared'
… added folders to ignore list
…g the scala cache folders to the shared folder
neomatrix369
force-pushed
the
provide-Docker-scripts
branch
from
a0cad5c
to
2025341
Compare
…CRONAUT_HOME value via the ENV directive
…so the more recent versions
… installed. Also changed the base image, to fix debian source.list related issues. Replaced old/broken links with new ones.
neomatrix369
force-pushed
the
provide-Docker-scripts
branch
from
b0b320c
to
56c2e7b
Compare
…s:stretch-scm Minor updates to other install commands, wget needed the flag to disable certificate checks
…ecessary locale related info for installing gems
…images in both local and remote repositories when attempting to run a container
…e installation and rebuilding of Ruby component and ssl library
…the installation of python packages using graalpython
…tyasai/git-repos/OpenJDK/graal-stuff/graalvm-demos/docker-images for the venv path
…t complain about missing/invalid certificates, also making download more secure. graalpython: temporarily removing the step to pre-install/pre-bake python packages into the image so container runtime gestation is reduced, atm the graalpython -m command starts the reinstall step again
…o install dependencies before running the demo. This step also helps understand if there is a failure what or why things failed.
…ing pulled, also fix the logic to check for absence of image in the local and remote repo
neomatrix369
force-pushed
the
provide-Docker-scripts
branch
from
e7e11cc
to
3e36c7c
Compare
…mo, dependency required for the R aspect of the build process
| FROM ${DOCKER_USER_NAME}/micronaut-starter:${FULL_GRAALVM_VERSION} as micronaut-starter-image | ||
| FROM ${DOCKER_USER_NAME}/workload-generator as workload-generator | ||
|
|
||
| FROM debian:buster |
There was a problem hiding this comment.
Is there a reason to use debian here instead of e.g. the official GraalVM Docker image from https://github.com/graalvm/container/pkgs/container/graalvm-ce ?
That might make things easier, because we test on OracleLinux in CI.
There was a problem hiding this comment.
Good point, no we can use GraalVM as the base image, thanks for confirming this saves building time.
The original code evolved such that I swayed away from this approach due to some issues but I can revert to your suggestion.
There was a problem hiding this comment.
How do you install packages in the Graalvm docker image, apt or aptitude is not available?
There was a problem hiding this comment.
There was a problem hiding this comment.
microdfn isn't able to find g++, libc++-dev, libssl-dev, the latter fixed the TruffleRuby and FastR issues we had with galaaz demos
There was a problem hiding this comment.
The image I'm building is using an official GraalVM images if you see the buildDockerImage.sh and Dockerfile and Dockerfile-mn in the docker-images/ sub-folder.
In this Dockerfile, it's reusing some files from the offical Docker images, but transplanting on debian:buster.
That's absolutely not guaranteed to work, it feels pretty hacky honestly. For instance the files in the official images might already be tweaked to run on OracleLinux, for instance https://github.com/graalvm/container/blob/326e236ee937de37769e0709505dafdb7a6a9f78/community/Dockerfile.java11#L44-L53.
I'm checking with @ezzarghili, I think there is a way to install rpm's for each component/language, and that should automatically install all required system dependencies. They are currently working on having 21.3 Docker images with those rpm packages.
Maybe we can even have a Docker image with all components pre-installed, that sounds ideal for this use-case, isn't it?
IMHO trying to minimize dependencies below what is marked as requirement for each component is not a good thing and will just into problems, e.g., I think nobody is happy to help you debug if you on purpose skipped some documented dependency.
I don't understand the concern about image size here. It's a repo of demos with all languages, of course the image will be big, and I don't think it's much of an issue.
I understand it's easier for you for now to use debian-based images and you don't want to change everything right now.
But if we get something like Docker images with rpms or already all components we should use that, instead of hardcoding and manually finding system dependencies in yet another place.
There was a problem hiding this comment.
But if we get something like Docker images with rpms or already all components we should use that, instead of hardcoding and manually finding system dependencies in yet another place.
I agree with using them - as this would be ideal and makes it consistent. If a standard image/config for Docker or anything of that sort is available, I would just like to use it as it would work out-of-the-box as you mentioned earlier.
I don't fully agree about your comment about "hacky" in this context, I will agree it's less optimum or less efficient, but it's done in the absence of the standard image or the know-how about it.
I will say the same about the docs, they are not necessary if you have config files and single-line commands to create images using those config files (I see you have these in different places in some form or shape). Incorporating existing config or images and building on top of them is more appropriate. That idea is still in the files/scripts in the script, maybe it's not visible, reusing components is definitely the way to go.
What has happened here is actually a bit of reinvention of the wheel, which is debatable if it could have been avoided but now we learning from what's available, happy to adapt them. This will also reduce the calls to various teams for help in case something does not work - I do not wish to do this either, it can hold up the process on this end as well.
After all of this, there is still chances of something not working but then those are much less. Also, peer review helps iron out these glitches, and it's a one-off because when this works it will always work unless the upstream regresses or something specific has been changed downstream.
There was a problem hiding this comment.
Maybe we can even have a Docker image with all components pre-installed, that sounds ideal for this use-case, isn't it?
This is something I would have been doing in my second refactor round for these scripts, but if this is already in the planning then I can wait and adapt those images and build on top of them - thanks for letting me know I will keep an eye for it
There was a problem hiding this comment.
I understand it's easier for you for now to use debian-based images and you don't want to change everything right now.
Only for now, but it's on my mental list to change it. But I'm always happy to change the docker and shell scripts -- if we can reduce all the lines written to a few lines then that's a better solution and reusing existing images could be a way towards that.
There was a problem hiding this comment.
It's a repo of demos with all languages, of course the image will be big, and I don't think it's much of an issue.
Thinking of this again, you are right, I'll take that back -- image size should not be an issue, especially to avoid inconsistency and be able to reproduce issues on the same image type. A whole chain of things can happen if images differ in signature, and be hard to trackback or investigate, as the number of moving parts are increasing.
…ge, and as a result removing the -no-check-certificate flag from wget and curl commands, as this is not necessary and also good to not use them anymore
…ying the layer from a ready-to-use graalvm docker image
|
Thanks for your feedback @eregon I really appreciate it Especially you are nitpicking and finding cruft, which I would also like to remove from the scripts and config files -- a second pair of eyes surely helps 🙏 🎉 |
|
We will gradually improve and make things less and less manual as we go along. I'm merging this PR. |
|
Thanks @olpaw for the https://github.com/graalvm/graalvm-demos/tree/master/native-hello-module demo example, I have tested it in our Docker container for GraalVM 21.3.0, and it works fine. You can also try it on this demo or other ones by doing this: $ git clone https://github.com/graalvm/graalvm-demos/
$ ./runDockerImage.sh "java11-21.3.0"
### At the prompt inside the container do:
root@.....:/graalvm-demos# cd native-hello-module
### perform all the steps for the demo
root@.....:/graalvm-demos# |
@eregon As I'm getting ready to continue from this point on again, I wanted to check if the above thing you mentioned sometime back is available with the latest GraalVM 21.3 docker images? I have just built a 21.3 docker image for GraalVM demos. If not, it's fine, I can build on top of the GraalVM docker image (as you were suggesting earlier) and maybe it needs fewer dependencies - which could be to our advantage. |
|
Hey @ezzarghili would you be able to help me out with this query #93 (comment) as you manage all things docker for the GraalVM team |
|
Yes, @ezzarghili should be the best contact about this and in general if you have any issue related to these Docker images, https://github.com/graalvm/container is the place to report them. |
|
Public yum rpms already exist on https://yum.oracle.com/repo/OracleLinux/OL7/graalvm/community/x86_64/index.html and https://yum.oracle.com/repo/OracleLinux/OL8/graalvm/community/x86_64/index.html https://github.com/graalvm/container/pkgs/container/jdk is the minimal image to build on, but there is also the native-image and nodejs ones. Please keep in mind that for experimental languages (graalpython, truffleruby...) the rpms are only available for Year.N.0 releases (i.e 21.3.0 but not 21.3.1+). |
|
I've tried the above and to give an idea how it works: You need to be careful to match the graalvm and java versions, otherwise you'll end up installing another graalvm in that image, i.e., installing another |
|
Currently it's necessary to manually run the post-install hook for ruby, but it shouldn't (filed internally as GR-35132). after that And currently you also need to set the locale manually: |
@olyagpl continuing from PR #31, as per our last conversation: #31 (comment)
Handy Docker scripts to facilitate running the
graalvmdemos in a clean and isolated environment. The below defaults to GraalVm 21.2.0 Java 11Build a docker image with GraalVM runtime in it:
Run a docker container with GraalVM runtime in it (from the root directory of the project):
Run a docker container with GraalVM runtime in it (from the root directory of the project) for GUI-based apps:
To build and run examples using another version of GraalVM runtime (i.e. GraalVM 21.2.0 for Java 11), the usage is as follows:
Build a docker image with GraalVM runtime in it:
Run a docker container with GraalVM runtime in it (from the root directory of the project):
Run a docker container with GraalVM runtime in it (from the root directory of the project) for GUI-based apps:
Note: See https://github.com/graalvm/container/pkgs/container/graalvm-ce, for a list of complete tag names to be used at CLI parameters to the above shell-scripts. A number of free or commercial VNC Viewers can be found online and are fairly easy to use.
Thanks to @findepi for providing the GraalVM community with the complete set of lightweight GraalVM docker images. We have now migrated the scripts to use GraalVM's own home built docker images, your images have been helpful so far to test out many of the demos, thank you for that, please take this migration as a positive gesture.
In addition to the version of GraalVM binaries, the
graalvm-democontainer contains the following at runtime: