Is it possible to make karaf run a set of command line commands?

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Is it possible to make karaf run a set of command line commands?

Steinar Bang
Is it possible to make a karaf installation run a set of console
commands?

What I would like to do, is to try to create a dockerfile setting up a
docker image for my webapp:
       https://github.com/steinarb/ukelonn

And the steps listed in 6. and 8. below are run from the karaf console.
Is it possible to run them from an outside script somehow?

What I would need to do to create a docker image, is to:
 1. Start with a minimal debian installation
 2. Add the default location of postgres databases as a volume
    (ie. mounted from the host's file system)
 3. Add /etc/karaf as a volume
 4. Add the repository for the unofficial karaf package
     echo "deb http://apt.bang.priv.no/public stable main" >>/etc/apt/sources.list
     wget -O - https://apt.bang.priv.no/apt_pub.gpg | apt-key add -
 5. Install karaf and postgresql and all of their dependencies
     apt-get update
     apt-get install -yq --no-install-recommends postgresql karaf
 6. Configure up an extra maven repository from the karaf console:
     config:edit org.ops4j.pax.url.mvn
     config:property-append org.ops4j.pax.url.mvn.repositories ", https://maven.bang.priv.no/repository/@id=ukelonn@snapshots, http://maven.vaadin.com/vaadin-addons@id=vaadin"
     config:property-set org.ops4j.pax.url.mvn.globalUpdatePolicy always
     config:update
 7. Create an empty database karaf owned by user karaf
     /usr/bin/sudo -u postgres createuser karaf
     /usr/bin/sudo -u postgres createdb -O karaf ukelonn
 8. Add the feature repository and install the features
     feature:repo-add mvn:no.priv.bang.ukelonn/ukelonn.karaf/LATEST/xml/features
     feature:install ukelonn-db-postgresql
     feature:install ukelonn



Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to make karaf run a set of command line commands?

fpapon
Hi Steinar,

May be you can run Karaf console commands using the ssh client in the
docker compose ? (I did not try this before and I'm interesting to know
if it work's ;)

François


Le 02/03/2018 à 01:10, Steinar Bang a écrit :

> Is it possible to make a karaf installation run a set of console
> commands?
>
> What I would like to do, is to try to create a dockerfile setting up a
> docker image for my webapp:
>        https://github.com/steinarb/ukelonn
>
> And the steps listed in 6. and 8. below are run from the karaf console.
> Is it possible to run them from an outside script somehow?
>
> What I would need to do to create a docker image, is to:
>  1. Start with a minimal debian installation
>  2. Add the default location of postgres databases as a volume
>     (ie. mounted from the host's file system)
>  3. Add /etc/karaf as a volume
>  4. Add the repository for the unofficial karaf package
>      echo "deb http://apt.bang.priv.no/public stable main" >>/etc/apt/sources.list
>      wget -O - https://apt.bang.priv.no/apt_pub.gpg | apt-key add -
>  5. Install karaf and postgresql and all of their dependencies
>      apt-get update
>      apt-get install -yq --no-install-recommends postgresql karaf
>  6. Configure up an extra maven repository from the karaf console:
>      config:edit org.ops4j.pax.url.mvn
>      config:property-append org.ops4j.pax.url.mvn.repositories ", https://maven.bang.priv.no/repository/@id=ukelonn@snapshots, http://maven.vaadin.com/vaadin-addons@id=vaadin"
>      config:property-set org.ops4j.pax.url.mvn.globalUpdatePolicy always
>      config:update
>  7. Create an empty database karaf owned by user karaf
>      /usr/bin/sudo -u postgres createuser karaf
>      /usr/bin/sudo -u postgres createdb -O karaf ukelonn
>  8. Add the feature repository and install the features
>      feature:repo-add mvn:no.priv.bang.ukelonn/ukelonn.karaf/LATEST/xml/features
>      feature:install ukelonn-db-postgresql
>      feature:install ukelonn
>
>
>

François Papon
fpapon@apache.org
Yupiik - https://www.yupiik.com
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to make karaf run a set of command line commands?

jbonofre
In reply to this post by Steinar Bang
Hi,

you can change in etc/shell.init.script.

Here you can define a set of commands executed at startup. You can also create a
script wrapping all your commands.

Regards
JB

On 03/01/2018 10:10 PM, Steinar Bang wrote:

> Is it possible to make a karaf installation run a set of console
> commands?
>
> What I would like to do, is to try to create a dockerfile setting up a
> docker image for my webapp:
>        https://github.com/steinarb/ukelonn
>
> And the steps listed in 6. and 8. below are run from the karaf console.
> Is it possible to run them from an outside script somehow?
>
> What I would need to do to create a docker image, is to:
>  1. Start with a minimal debian installation
>  2. Add the default location of postgres databases as a volume
>     (ie. mounted from the host's file system)
>  3. Add /etc/karaf as a volume
>  4. Add the repository for the unofficial karaf package
>      echo "deb http://apt.bang.priv.no/public stable main" >>/etc/apt/sources.list
>      wget -O - https://apt.bang.priv.no/apt_pub.gpg | apt-key add -
>  5. Install karaf and postgresql and all of their dependencies
>      apt-get update
>      apt-get install -yq --no-install-recommends postgresql karaf
>  6. Configure up an extra maven repository from the karaf console:
>      config:edit org.ops4j.pax.url.mvn
>      config:property-append org.ops4j.pax.url.mvn.repositories ", https://maven.bang.priv.no/repository/@id=ukelonn@snapshots, http://maven.vaadin.com/vaadin-addons@id=vaadin"
>      config:property-set org.ops4j.pax.url.mvn.globalUpdatePolicy always
>      config:update
>  7. Create an empty database karaf owned by user karaf
>      /usr/bin/sudo -u postgres createuser karaf
>      /usr/bin/sudo -u postgres createdb -O karaf ukelonn
>  8. Add the feature repository and install the features
>      feature:repo-add mvn:no.priv.bang.ukelonn/ukelonn.karaf/LATEST/xml/features
>      feature:install ukelonn-db-postgresql
>      feature:install ukelonn
>
>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to make karaf run a set of command line commands?

Steinar Bang
In reply to this post by fpapon
>>>>> Francois Papon <[hidden email]>:

> May be you can run Karaf console commands using the ssh client in the
> docker compose ? (I did not try this before and I'm interesting to know
> if it work's ;)

With the help off sshpass, this works for simple one-liners, like
"feature:install ukelonn".

First install sshpass (and, of course ssh)
 apt-get install openssh-client sshpass

One line commands like this works:
 sb@lorenzo:~$ sshpass -p karaf ssh -p 8101 karaf@localhost 'feature:repo-add mvn:no.priv.bang.sonar.sonar-collector/sonar-collector-webhook/LATEST/xml/features'
 Password authentication
 Adding feature url mvn:no.priv.bang.sonar.sonar-collector/sonar-collector-webhook/LATEST/xml/features
 sb@lorenzo:~$ sshpass -p karaf ssh -p 8101 karaf@localhost 'feature:install sonar-collector-webhook'
 Password authentication
 sb@lorenzo:~$

However, this approach fails on the multi line commands like the config edit:
 sb@lorenzo:~$ sshpass -p karaf ssh -p 8101 karaf@localhost 'config:edit org.ops4j.pax.url.mvn'
 Password authentication
 sb@lorenzo:~$ sshpass -p karaf ssh -p 8101 karaf@localhost 'config:property-append org.ops4j.pax.url.mvn.repositories ", https://maven.bang.priv.no/repository/@id=ukelonn@snapshots, http://maven.vaadin.com/vaadin-addons@id=vaadin"'
 Password authentication
 No configuration is being edited--run the edit command first
 sb@lorenzo:~$

Of course, the /etc/karaf/org.ops4j.pax.url.mvn.cfg file could be edited
directly with a sed script...

I will try both this approach and JB's approach and see what works best.

Thanks!

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to make karaf run a set of command line commands?

fpapon
Ok, thanks for your feeback ;)

François


Le 02/03/2018 à 20:57, Steinar Bang a écrit :

>>>>>> Francois Papon <[hidden email]>:
>> May be you can run Karaf console commands using the ssh client in the
>> docker compose ? (I did not try this before and I'm interesting to know
>> if it work's ;)
> With the help off sshpass, this works for simple one-liners, like
> "feature:install ukelonn".
>
> First install sshpass (and, of course ssh)
>  apt-get install openssh-client sshpass
>
> One line commands like this works:
>  sb@lorenzo:~$ sshpass -p karaf ssh -p 8101 karaf@localhost 'feature:repo-add mvn:no.priv.bang.sonar.sonar-collector/sonar-collector-webhook/LATEST/xml/features'
>  Password authentication
>  Adding feature url mvn:no.priv.bang.sonar.sonar-collector/sonar-collector-webhook/LATEST/xml/features
>  sb@lorenzo:~$ sshpass -p karaf ssh -p 8101 karaf@localhost 'feature:install sonar-collector-webhook'
>  Password authentication
>  sb@lorenzo:~$
>
> However, this approach fails on the multi line commands like the config edit:
>  sb@lorenzo:~$ sshpass -p karaf ssh -p 8101 karaf@localhost 'config:edit org.ops4j.pax.url.mvn'
>  Password authentication
>  sb@lorenzo:~$ sshpass -p karaf ssh -p 8101 karaf@localhost 'config:property-append org.ops4j.pax.url.mvn.repositories ", https://maven.bang.priv.no/repository/@id=ukelonn@snapshots, http://maven.vaadin.com/vaadin-addons@id=vaadin"'
>  Password authentication
>  No configuration is being edited--run the edit command first
>  sb@lorenzo:~$
>
> Of course, the /etc/karaf/org.ops4j.pax.url.mvn.cfg file could be edited
> directly with a sed script...
>
> I will try both this approach and JB's approach and see what works best.
>
> Thanks!
>

François Papon
fpapon@apache.org
Yupiik - https://www.yupiik.com
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to make karaf run a set of command line commands?

Michal Hlaváč
In reply to this post by Steinar Bang

you can use docker exec or compose exec to run multiple commands from file.

create configure.cmd file with commands e.g.

 

config:edit com.example.config

config:property-set example.property value
config:update

then run:

 

docker-compose exec -T container /opt/karaf/bin/client -b < configure.cmd

 

m.

 

On štvrtok, 1. marca 2018 22:10:18 CET Steinar Bang wrote:

> Is it possible to make a karaf installation run a set of console

> commands?

>

> What I would like to do, is to try to create a dockerfile setting up a

> docker image for my webapp:

> https://github.com/steinarb/ukelonn

>

> And the steps listed in 6. and 8. below are run from the karaf console.

> Is it possible to run them from an outside script somehow?

>

> What I would need to do to create a docker image, is to:

> 1. Start with a minimal debian installation

> 2. Add the default location of postgres databases as a volume

> (ie. mounted from the host's file system)

> 3. Add /etc/karaf as a volume

> 4. Add the repository for the unofficial karaf package

> echo "deb http://apt.bang.priv.no/public stable main" >>/etc/apt/sources.list

> wget -O - https://apt.bang.priv.no/apt_pub.gpg | apt-key add -

> 5. Install karaf and postgresql and all of their dependencies

> apt-get update

> apt-get install -yq --no-install-recommends postgresql karaf

> 6. Configure up an extra maven repository from the karaf console:

> config:edit org.ops4j.pax.url.mvn

> config:property-append org.ops4j.pax.url.mvn.repositories ", https://maven.bang.priv.no/repository/@id=ukelonn@snapshots, http://maven.vaadin.com/vaadin-addons@id=vaadin"

> config:property-set org.ops4j.pax.url.mvn.globalUpdatePolicy always

> config:update

> 7. Create an empty database karaf owned by user karaf

> /usr/bin/sudo -u postgres createuser karaf

> /usr/bin/sudo -u postgres createdb -O karaf ukelonn

> 8. Add the feature repository and install the features

> feature:repo-add mvn:no.priv.bang.ukelonn/ukelonn.karaf/LATEST/xml/features

> feature:install ukelonn-db-postgresql

> feature:install ukelonn

>

>

>

>

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to make karaf run a set of command line commands?

mhus_
I managed the problem starting karaf in the background and installing features via client after startup. Therefore you need to wait until karaf starts and installed default packages.


Important: if you plan to use local only bundles, you need to copy it into the project before you start docker build. You need a create.sh script to do the full image creation.

The script is not finished at all but it’s working.
 
Mike.

On 17. Mar 2018, at 23:18, Michal Hlavac <[hidden email]> wrote:

you can use docker exec or compose exec to run multiple commands from file.
create configure.cmd file with commands e.g.

 

config:edit com.example.config
config:property-set example.property value
config:update
then run:

 

docker-compose exec -T container /opt/karaf/bin/client -b < configure.cmd

 

m.

 

On štvrtok, 1. marca 2018 22:10:18 CET Steinar Bang wrote:
> Is it possible to make a karaf installation run a set of console
> commands?
>
> What I would like to do, is to try to create a dockerfile setting up a
> docker image for my webapp:
>
> And the steps listed in 6. and 8. below are run from the karaf console.
> Is it possible to run them from an outside script somehow?
>
> What I would need to do to create a docker image, is to:
> 1. Start with a minimal debian installation
> 2. Add the default location of postgres databases as a volume
> (ie. mounted from the host's file system)
> 3. Add /etc/karaf as a volume
> 4. Add the repository for the unofficial karaf package
> echo "deb http://apt.bang.priv.no/public stable main" >>/etc/apt/sources.list
> wget -O - https://apt.bang.priv.no/apt_pub.gpg | apt-key add -
> 5. Install karaf and postgresql and all of their dependencies
> apt-get update
> apt-get install -yq --no-install-recommends postgresql karaf
> 6. Configure up an extra maven repository from the karaf console:
> config:edit org.ops4j.pax.url.mvn
> config:property-set org.ops4j.pax.url.mvn.globalUpdatePolicy always
> config:update
> 7. Create an empty database karaf owned by user karaf
> /usr/bin/sudo -u postgres createuser karaf
> /usr/bin/sudo -u postgres createdb -O karaf ukelonn
> 8. Add the feature repository and install the features
> feature:repo-add mvn:no.priv.bang.ukelonn/ukelonn.karaf/LATEST/xml/features
> feature:install ukelonn-db-postgresql
> feature:install ukelonn
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to make karaf run a set of command line commands?

jbonofre
By the way to know when Karaf is fully started you have the Bootstrap service. It's what we use for the startup progress bar (like in ServiceMix).

Regards
JB
Le 23 avr. 2018, à 11:17, Mike Hummel <[hidden email]> a écrit:
I managed the problem starting karaf in the background and installing features via client after startup. Therefore you need to wait until karaf starts and installed default packages.


Important: if you plan to use local only bundles, you need to copy it into the project before you start docker build. You need a create.sh script to do the full image creation.

The script is not finished at all but it’s working.
 
Mike.

On 17. Mar 2018, at 23:18, Michal Hlavac <[hidden email]> wrote:

you can use docker exec or compose exec to run multiple commands from file.
create configure.cmd file with commands e.g.

 

config:edit com.example.config
config:property-set example.property value
config:update
then run:

 

docker-compose exec -T container /opt/karaf/bin/client -b < configure.cmd

 

m.

 

On štvrtok, 1. marca 2018 22:10:18 CET Steinar Bang wrote:
> Is it possible to make a karaf installation run a set of console
> commands?
>
> What I would like to do, is to try to create a dockerfile setting up a
> docker image for my webapp:
>
> And the steps listed in 6. and 8. below are run from the karaf console.
> Is it possible to run them from an outside script somehow?
>
> What I would need to do to create a docker image, is to:
> 1. Start with a minimal debian installation
> 2. Add the default location of postgres databases as a volume
> (ie. mounted from the host's file system)
> 3. Add /etc/karaf as a volume
> 4. Add the repository for the unofficial karaf package
> echo "deb http://apt.bang.priv.no/public stable main" >>/etc/apt/sources.list
> wget -O - https://apt.bang.priv.no/apt_pub.gpg | apt-key add -
> 5. Install karaf and postgresql and all of their dependencies
> apt-get update
> apt-get install -yq --no-install-recommends postgresql karaf
> 6. Configure up an extra maven repository from the karaf console:
> config:edit org.ops4j.pax.url.mvn
> config:property-set org.ops4j.pax.url.mvn.globalUpdatePolicy always
> config:update
> 7. Create an empty database karaf owned by user karaf
> /usr/bin/sudo -u postgres createuser karaf
> /usr/bin/sudo -u postgres createdb -O karaf ukelonn
> 8. Add the feature repository and install the features
> feature:repo-add mvn:no.priv.bang.ukelonn/ukelonn.karaf/LATEST/xml/features
> feature:install ukelonn-db-postgresql
> feature:install ukelonn
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Creating docker image from docker hub karaf image (Was: Is it possible to make karaf run a set of command line commands?)

Steinar Bang
In reply to this post by jbonofre
>>>>> Jean-Baptiste Onofré <[hidden email]>:

> Hi,
> you can change in etc/shell.init.script.

> Here you can define a set of commands executed at startup. You can also create a
> script wrapping all your commands.

Tried this approach and I had some success.  The "success" is in that I
managed to end up with a working application-in-karaf in a docker
container.

The "some" is because I had to ssh in to the karaf-in-docker instance to
complete the application install.


 1. Copied etc/org.ops4j.pax.url.mvn.cfg from a karaf 4.2.8 instance,
    and added my own maven repo
     https://gist.github.com/steinarb/62a070f6481d3e84c334a78f1c771e80#file-org-ops4j-pax-url-mvn-cfg-L113

 2. Copied shell.init.script and added the feature:repository-add and
    feature:install commands at the end
     https://gist.github.com/steinarb/f6b5b1b76a4943fe43f1ff387f38332e#file-shell-init-script-L67

 3. Created a Dockerfile adding to the official docker hub karaf 4.2.8
    image, copying in the two modified config files
     https://gist.github.com/steinarb/3e0197f2805439861d6df0b05f1419d7

 4. Built the image from the Dockerfile
     docker build -t steinarb/ukelonn-demo:v1 .

 5. Started the image
     docker run -p 8101:8101 -p 8181:8181 -d steinarb/ukelonn-demo:v1

 6. Nothing was listening on port 8181

 7. Started logging in follow mode (used "docker ps" to find the
    container id)
     docker logs -f ffc851f87207

 8. SSH'd in to karaf running inside the docker container
     ssh -p 8101 karaf@localhost

 9. The log in follow mode showed the feature being installed

10. Now I could connect to http://localhost:8181/ukelonn where I found
    the demo version of my application running

Reply | Threaded
Open this post in threaded view
|

Re: Creating docker image from docker hub karaf image

Steinar Bang
The karaf-based docker examples I have seen build a custom karaf
distribution and then creates a Dockerfile that copies the custom karaf
distribution into /opt/apache-karaf in an image based one openjdk:8-jre.

What I would like to do, is to start with the official karaf image on
docker hub https://hub.docker.com/r/apache/karaf and then add a feature
as a part of the build process.

Unless I've misunderstood, the images will be a lot smaller, because
they will only hold the difference between the base image and the
current state of the image file system at the end of the build.

Here's what I have tried:
 1. Add the feature to shell.init.script (partly works, see previous
    email. But loads the feature at startup time so the behaviour may be
    different to what was expected)

 2. Fire up karaf and do commands to set the feature repository and add
    the feature (this failed miserably (as I expected, but I had to try))
     FROM apache/karaf:4.2.8
     COPY org.ops4j.pax.url.mvn.cfg /opt/apache-karaf/etc
     RUN karaf daemon
     RUN feature:repo-add mvn:no.priv.bang.ukelonn/karaf/LATEST/xml/features
     RUN feature:install ukelonn-with-derby
     RUN exit

 3. Fire up karaf as a deamon and then do ssh in to add the feature
    repository and install the feature (this started up the daemon and
    then got stuck)
     FROM apache/karaf:4.2.8
     COPY org.ops4j.pax.url.mvn.cfg /opt/apache-karaf/etc
     RUN karaf daemon
     RUN sshpass -p karaf ssh -p 8101 karaf@localhost 'feature:repo-add mvn:no.priv.bang.ukelonn/karaf/LATEST/xml/features'
     RUN sshpass -p karaf ssh -p 8101 karaf@localhost 'feature:install ukelonn-with-derby'
     RUN karaf stop

Are there any other ways this could be done?
Is there a way to copy in the bundles as loaded by a feature?

Reply | Threaded
Open this post in threaded view
|

Re: Creating docker image from docker hub karaf image

Steinar Bang
>>>>> Steinar Bang <[hidden email]>:

> Are there any other ways this could be done?

One possible way would be if the karaf-maven-plugin could build an
assembly that is identical to the assembly used to make the docker hub
image, and then add the new feature(s).

Then the diff to the docker hub image should only be the new feature(s).

Reply | Threaded
Open this post in threaded view
|

Re: Creating docker image from docker hub karaf image

Steinar Bang
In reply to this post by Steinar Bang
>>>>> Steinar Bang <[hidden email]>:

> What I would like to do, is to start with the official karaf image on
> docker hub https://hub.docker.com/r/apache/karaf and then add a feature
> as a part of the build process.

> Unless I've misunderstood, the images will be a lot smaller, because
> they will only hold the difference between the base image and the
> current state of the image file system at the end of the build.

Hm... I've looked at the file system state of a priestine karaf 4.2.8
docker image, and maybe this wasn't such a good idea after all?
 https://hub.docker.com/layers/apache/karaf/4.2.8/images/sha256-90ab8758ddddf4f0b28afe7ca24b67456a9a8619be4cf5fc2fa90e3b26e64adf?context=explore

A priestine karaf 4.2.8 image as an almost empty data directory, and the
first thing that happens when karaf is started, is that e.g. data/cache/
is filled up with the running bundles, so the file system delta will be
pretty big anyway.

I.e. there will not be much space saved by this approach.

Reply | Threaded
Open this post in threaded view
|

Re: Creating docker image from docker hub karaf image

Steinar Bang
I have successfully created a docker image for my demo application based
on the official karaf 4.2.8 docker hub image.

Here's what I did:

 1. Copied etc/org.ops4j.pax.url.mvn.cfg from a karaf 4.2.8 instance,
    and added my own maven repo
     https://gist.github.com/steinarb/62a070f6481d3e84c334a78f1c771e80#file-org-ops4j-pax-url-mvn-cfg-L113

 2. Copied org.apache.karaf.features.cfg and added the feature repo of
    the sample application
     https://gist.github.com/steinarb/fec331bcd37a0a7bdba850d33fcd0555#file-org-apache-karaf-features-cfg-L28
    and added the sample application to the list of boot features
     https://gist.github.com/steinarb/fec331bcd37a0a7bdba850d33fcd0555#file-org-apache-karaf-features-cfg-L52

 3. Created a Dockerfile adding to the official docker hub karaf 4.2.8
    image, copying in the two modified config files
     https://gist.github.com/steinarb/444ad82a63dcf156101a064666ed590c#file-dockerfile-L1

 4. Built the image from the Dockerfile
     docker build -t steinarb/ukelonn-demo:v2 .

 5. Started the image
     docker run -p 8101:8101 -p 8181:8181 -d steinarb/ukelonn-demo:v2

 6. The container took a while to start but eventually I could connect
    to http://localhost:8181/ukelonn where I found the demo version of
    my application running

The downsides of this approach, are:
 1. Container startup is slow since features and their dependencies must
    be downloaded

 2. The maven repons may not be reachable at startup time and container
    startup will fail

 3. What the maven repos serve may potensially not be the same on each
    startup

All of the above is, I guess, sort of contrary to docker philosophy,
which is to pack up everything the application needs to start, so that
the application can be started in a predictable and reproducable manner.

However, for this particular application, this is actually what I want:
I can create a docker image and deploy to docker hub, and on any docker
installation that can pull from dockerhub and can reach internet, I can
spin up the latest snapshot with the two commands "docker pull" and
"docker run".

Reply | Threaded
Open this post in threaded view
|

Re: Creating docker image from docker hub karaf image

jbonofre
Hi,

For the startup, you can already populate the system folder (by hand or using mvn deploy:deploy-file or mvn install:install-file).

That’s what I’m doing in docker image preparation.

Regards
JB

> Le 13 avr. 2020 à 21:09, Steinar Bang <[hidden email]> a écrit :
>
> I have successfully created a docker image for my demo application based
> on the official karaf 4.2.8 docker hub image.
>
> Here's what I did:
>
> 1. Copied etc/org.ops4j.pax.url.mvn.cfg from a karaf 4.2.8 instance,
>    and added my own maven repo
>     https://gist.github.com/steinarb/62a070f6481d3e84c334a78f1c771e80#file-org-ops4j-pax-url-mvn-cfg-L113
>
> 2. Copied org.apache.karaf.features.cfg and added the feature repo of
>    the sample application
>     https://gist.github.com/steinarb/fec331bcd37a0a7bdba850d33fcd0555#file-org-apache-karaf-features-cfg-L28
>    and added the sample application to the list of boot features
>     https://gist.github.com/steinarb/fec331bcd37a0a7bdba850d33fcd0555#file-org-apache-karaf-features-cfg-L52
>
> 3. Created a Dockerfile adding to the official docker hub karaf 4.2.8
>    image, copying in the two modified config files
>     https://gist.github.com/steinarb/444ad82a63dcf156101a064666ed590c#file-dockerfile-L1
>
> 4. Built the image from the Dockerfile
>     docker build -t steinarb/ukelonn-demo:v2 .
>
> 5. Started the image
>     docker run -p 8101:8101 -p 8181:8181 -d steinarb/ukelonn-demo:v2
>
> 6. The container took a while to start but eventually I could connect
>    to http://localhost:8181/ukelonn where I found the demo version of
>    my application running
>
> The downsides of this approach, are:
> 1. Container startup is slow since features and their dependencies must
>    be downloaded
>
> 2. The maven repons may not be reachable at startup time and container
>    startup will fail
>
> 3. What the maven repos serve may potensially not be the same on each
>    startup
>
> All of the above is, I guess, sort of contrary to docker philosophy,
> which is to pack up everything the application needs to start, so that
> the application can be started in a predictable and reproducable manner.
>
> However, for this particular application, this is actually what I want:
> I can create a docker image and deploy to docker hub, and on any docker
> installation that can pull from dockerhub and can reach internet, I can
> spin up the latest snapshot with the two commands "docker pull" and
> "docker run".
>

Reply | Threaded
Open this post in threaded view
|

Re: Creating docker image from docker hub karaf image

Alex Soto
I am working now on a similar task. I have pre-populated the maven repository directory (~/.m2/repository) from the container that made the build (I am using multi-stage images),  and set

org.ops4j.pax.url.mvn.localRepository=/root/.m2/repository

In `etc/org.ops4j.pax.url.mvn.cfg`.   


Is there anything else I need to make the image self contained?  JB, can you elaborate about your reference to the system folder?


Best regards,
Alex soto




On Apr 14, 2020, at 2:13 AM, Jean-Baptiste Onofre <[hidden email]> wrote:

Hi,

For the startup, you can already populate the system folder (by hand or using mvn deploy:deploy-file or mvn install:install-file).

That’s what I’m doing in docker image preparation.

Regards
JB

Le 13 avr. 2020 à 21:09, Steinar Bang <[hidden email]> a écrit :

I have successfully created a docker image for my demo application based
on the official karaf 4.2.8 docker hub image.

Here's what I did:

1. Copied etc/org.ops4j.pax.url.mvn.cfg from a karaf 4.2.8 instance,
  and added my own maven repo
   https://gist.github.com/steinarb/62a070f6481d3e84c334a78f1c771e80#file-org-ops4j-pax-url-mvn-cfg-L113

2. Copied org.apache.karaf.features.cfg and added the feature repo of
  the sample application
   https://gist.github.com/steinarb/fec331bcd37a0a7bdba850d33fcd0555#file-org-apache-karaf-features-cfg-L28
  and added the sample application to the list of boot features
   https://gist.github.com/steinarb/fec331bcd37a0a7bdba850d33fcd0555#file-org-apache-karaf-features-cfg-L52

3. Created a Dockerfile adding to the official docker hub karaf 4.2.8
  image, copying in the two modified config files
   https://gist.github.com/steinarb/444ad82a63dcf156101a064666ed590c#file-dockerfile-L1

4. Built the image from the Dockerfile
   docker build -t steinarb/ukelonn-demo:v2 .

5. Started the image
   docker run -p 8101:8101 -p 8181:8181 -d steinarb/ukelonn-demo:v2

6. The container took a while to start but eventually I could connect
  to http://localhost:8181/ukelonn where I found the demo version of
  my application running

The downsides of this approach, are:
1. Container startup is slow since features and their dependencies must
  be downloaded

2. The maven repons may not be reachable at startup time and container
  startup will fail

3. What the maven repos serve may potensially not be the same on each
  startup

All of the above is, I guess, sort of contrary to docker philosophy,
which is to pack up everything the application needs to start, so that
the application can be started in a predictable and reproducable manner.

However, for this particular application, this is actually what I want:
I can create a docker image and deploy to docker hub, and on any docker
installation that can pull from dockerhub and can reach internet, I can
spin up the latest snapshot with the two commands "docker pull" and
"docker run".



Reply | Threaded
Open this post in threaded view
|

Re: Creating docker image from docker hub karaf image

jbonofre
What do you think about a small example/tooling addition to simplify this ?

By the way, a bit related, I will send updates and details about Karaf DevX by the end of this week. Stay tuned ;) 

Regards
JB

Le 14 avr. 2020 à 14:58, Alex Soto <[hidden email]> a écrit :

I am working now on a similar task. I have pre-populated the maven repository directory (~/.m2/repository) from the container that made the build (I am using multi-stage images),  and set

org.ops4j.pax.url.mvn.localRepository=/root/.m2/repository

In `etc/org.ops4j.pax.url.mvn.cfg`.   


Is there anything else I need to make the image self contained?  JB, can you elaborate about your reference to the system folder?


Best regards,
Alex soto




On Apr 14, 2020, at 2:13 AM, Jean-Baptiste Onofre <[hidden email]> wrote:

Hi,

For the startup, you can already populate the system folder (by hand or using mvn deploy:deploy-file or mvn install:install-file).

That’s what I’m doing in docker image preparation.

Regards
JB

Le 13 avr. 2020 à 21:09, Steinar Bang <[hidden email]> a écrit :

I have successfully created a docker image for my demo application based
on the official karaf 4.2.8 docker hub image.

Here's what I did:

1. Copied etc/org.ops4j.pax.url.mvn.cfg from a karaf 4.2.8 instance,
  and added my own maven repo
   https://gist.github.com/steinarb/62a070f6481d3e84c334a78f1c771e80#file-org-ops4j-pax-url-mvn-cfg-L113

2. Copied org.apache.karaf.features.cfg and added the feature repo of
  the sample application
   https://gist.github.com/steinarb/fec331bcd37a0a7bdba850d33fcd0555#file-org-apache-karaf-features-cfg-L28
  and added the sample application to the list of boot features
   https://gist.github.com/steinarb/fec331bcd37a0a7bdba850d33fcd0555#file-org-apache-karaf-features-cfg-L52

3. Created a Dockerfile adding to the official docker hub karaf 4.2.8
  image, copying in the two modified config files
   https://gist.github.com/steinarb/444ad82a63dcf156101a064666ed590c#file-dockerfile-L1

4. Built the image from the Dockerfile
   docker build -t steinarb/ukelonn-demo:v2 .

5. Started the image
   docker run -p 8101:8101 -p 8181:8181 -d steinarb/ukelonn-demo:v2

6. The container took a while to start but eventually I could connect
  to http://localhost:8181/ukelonn where I found the demo version of
  my application running

The downsides of this approach, are:
1. Container startup is slow since features and their dependencies must
  be downloaded

2. The maven repons may not be reachable at startup time and container
  startup will fail

3. What the maven repos serve may potensially not be the same on each
  startup

All of the above is, I guess, sort of contrary to docker philosophy,
which is to pack up everything the application needs to start, so that
the application can be started in a predictable and reproducable manner.

However, for this particular application, this is actually what I want:
I can create a docker image and deploy to docker hub, and on any docker
installation that can pull from dockerhub and can reach internet, I can
spin up the latest snapshot with the two commands "docker pull" and
"docker run".




Reply | Threaded
Open this post in threaded view
|

Re: Creating docker image from docker hub karaf image

Steinar Bang
>>>>> Jean-Baptiste Onofre <[hidden email]>:

> What do you think about a small example/tooling addition to simplify this ?

I think that if you can use karaf's feature resolver to populate the
system directory, that would be awsome! :-)


Reply | Threaded
Open this post in threaded view
|

Re: Creating docker image from docker hub karaf image

Alex Soto
Agree, and perhaps the "data/cache”, and/or whatever else is needed to run without connecting the internet



On Apr 14, 2020, at 11:22 AM, Steinar Bang <[hidden email]> wrote:

Jean-Baptiste Onofre <[hidden email]>:

What do you think about a small example/tooling addition to simplify this ?

I think that if you can use karaf's feature resolver to populate the
system directory, that would be awsome! :-)



Reply | Threaded
Open this post in threaded view
|

Re: Creating docker image from docker hub karaf image

Mike Hummel
Hi,

I did not read the hole tread but I solved a lot of problems with karaf in docker containers already.

There is also a small readme and also examples...

Container Sources:

Hub:

Example:

The idea is to provide a karaf docker base container. You can add the assembly and configurations (and profiles). And start it while build one time to collect all the resources you need. Background is a fast startup in kubernetes.

You can also mount a local repository to use bundle:watch for development. - That was the point - I think so. As described in the readme.

Regards,
Mike



On 14. Apr 2020, at 18:53, Alex Soto <[hidden email]> wrote:

Agree, and perhaps the "data/cache”, and/or whatever else is needed to run without connecting the internet



On Apr 14, 2020, at 11:22 AM, Steinar Bang <[hidden email]> wrote:

Jean-Baptiste Onofre <[hidden email]>:

What do you think about a small example/tooling addition to simplify this ?

I think that if you can use karaf's feature resolver to populate the
system directory, that would be awsome! :-)





signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

My first real karaf-based docker image: sonar-collector

Steinar Bang
In reply to this post by Steinar Bang
I have created my first real karaf-based docker image:
 https://github.com/steinarb/sonar-collector#run-the-sonar-metrics-collector-in-docker

The sonar-collector is intended as a webhook called by sonar on
completing a build, it will store key sonar measurements as a row in a
database table.  

Sonar has these numbers internally but doesn't have a way to get at
them. At first we typed these numbers manually into a spreadsheet to get
statistics on a set of analyzed programs.  But that was tedious, so we
created sonar-collector to capture the numbers instead.

The docker image is built with maven against a local docker and then
after testing, the image was pushed to docker hub:
 https://github.com/steinarb/sonar-collector#building-the-docker-image

The docker image is a very thin layer on top of the official karaf 4.2.8
docker image
 https://github.com/steinarb/sonar-collector/blob/master/docker/docker/Dockerfile#L1

It copies in two files into the image's karaf etc directory:
 1. org.apache.karaf.features.cfg
    a. Adds a feature repository for sonar-collector
        https://github.com/steinarb/sonar-collector/blob/master/docker/docker/org.apache.karaf.features.cfg#L28
    b. Adds two features to the boot features
       i. An enviroment variable overloadable DataSourceFactory feature
          that defaults to postgresql
            https://github.com/steinarb/sonar-collector/blob/master/docker/docker/org.apache.karaf.features.cfg#L52
       ii. The sonar-collector-webhook-with-postgresql feature
            https://github.com/steinarb/sonar-collector/blob/master/docker/docker/org.apache.karaf.features.cfg#L53

 2. org.ops4j.datasource-sonar-collector.cfg which is a modified version
    of the file generated by pax-jdbc on first start, and the JDBC
    driver name and JDBC connection info overridable by system
    environment variables
     https://github.com/steinarb/sonar-collector/blob/master/docker/docker/org.ops4j.datasource-sonar-collector.cfg#L2

This image will provision itself from maven central (sonar-collector is
released to maven-central) and will always use the latest released
version, so there is no need to build a new docker image if a new
version of sonar-collector is released to maven central.

Since the database schema is set up by liquibase and the SQL used is
trivial (inserts rows into a single table), it should in theory be
possible to use a different database than PostgreSQL, and I have
provided the possibility to override the database used when doing
"docker run" of the image.

But using other databases than PostgreSQL is something I haven't
extensively tested.

12