Install feature as boot features with command line

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Install feature as boot features with command line

nicolas.brasey
Hi all,

I'm trying to find out if there is way to install a feature and make it as a boot feature without manually altering the feature cfg file (org.apache.karaf.features.cfg). Checking in karaf's code seems to indicate there is no way to do this programmatically.

Ideally, it would be a flag in the feature:install command to indicate to add this feature as a boot feature.

The reason we need this is that our solution is an integrated solution which is delivered by different departments:

1) Product 1 (kar 1) => dev team A
2) Product 2 (kar 2) => dev team B
3) Integration layer (camel routes essentially) (kar 3) => integration team

All these different teams delivering a self contained kar file with a feature which should be installed and started when karaf starts in order to have the global solution running.

We are using karaf v.4.1.2 which does not seems to persist which features have been installed (only the boot features). I'm not sure about the v.4.2.x...

I know Karaf since not so long, but I believe Karaf has been designed so that the delivery team is supposed to create a Karaf distribution and assembling the required boot features at build time. If this is true, then it is not ideal according to how our internal process is made.

Any thoughts?
Thanks!

Best regards,
Nicolas 




 
Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

jbonofre
Hi Nicolas

The purpose of boot features is to start in early stage, so it uses the
cfg file as definition.

In your case, you should use inner and prerequisite feature.

Do you have some dependencies between kar ?

Why don't you use directly feature instead of packaging as a kar ?

Can you explain exactly what you are trying to do ?

Karaf always store the feature state (in the resolver), so, if you first
install kar1, then kar2, when you restart, the resolver will
reinstall/start those features.

So, you don't have to use a boot feature: if you don't remove the data
folder, the installed features and repositories are stored.

Regards
JB

On 05/07/2018 13:52, Nicolas Brasey wrote:

> Hi all,
>
> I'm trying to find out if there is way to install a feature and make it
> as a boot feature without manually altering the feature cfg file
> (org.apache.karaf.features.cfg). Checking in karaf's code seems to
> indicate there is no way to do this programmatically.
>
> Ideally, it would be a flag in the feature:install command to indicate
> to add this feature as a boot feature.
>
> The reason we need this is that our solution is an integrated solution
> which is delivered by different departments:
>
> 1) Product 1 (kar 1) => dev team A
> 2) Product 2 (kar 2) => dev team B
> 3) Integration layer (camel routes essentially) (kar 3) => integration team
>
> All these different teams delivering a self contained kar file with a
> feature which should be installed and started when karaf starts in order
> to have the global solution running.
>
> We are using karaf v.4.1.2 which does not seems to persist which
> features have been installed (only the boot features). I'm not sure
> about the v.4.2.x...
>
> I know Karaf since not so long, but I believe Karaf has been designed so
> that the delivery team is supposed to create a Karaf distribution and
> assembling the required boot features at build time. If this is true,
> then it is not ideal according to how our internal process is made.
>
> Any thoughts?
> Thanks!
>
> Best regards,
> Nicolas 
>
>
>
>
>  

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

nicolas.brasey
Hi Jean-Baptiste, 

Yes, our different kars have dependencies, so they must installed in a certain order.

The features.xml only is not possible in our case because our target machines are running on private networks without internet access, so the kars must contain all the runtime transitive dependencies.

But to explain exactly what I'm trying to achieve today, I need to explain the history of our solution. It started 3 years ago as a single application, at that time, the dev team A was delivering the app as a turnkey solution with a custom karaf distribution. The installation was very easy for the delivery team => Move the distribution to the server, unzip and start, the application jars were located in the system repository and the app features marked as boot features.

Over the last months, we extended the core application by providing plugin mechanisms by using SPI's, implementing the white-board pattern (we like this a lot by the way :)). So now dev team B is implementing different plugins for specific needs. Those plugins have a different life cycle that the core app, and they are released as kars.

Lately we found a bug in the main app which needed to be fixed and patched in a prod environment. We provided the patch with a new custom karaf distribution which the delivery installed. Of course, this new karaf distribution used another data directory and did not reinstall the features of the kars of the different plugins. So manually reinstalling the plugin features is absolutely needed in this scenario or the behavior of the application can differ from before the patching, which is obviously dangerous. 

Like I said, our main application is still released as part of our custom karaf distribution, this should be changed and we will probably release it a kar as well.

So I try to achieve 3 things:

1) Find the optimal packaging for our app and plugins so that it can be install / upgraded easily
2) Hide Karaf (OSGi) complexity from the delivery team, and they don't want to know about it :-)
3) Find the optimal way to set the bundle versions in our feature files using ranges

What would be a good practice in a scenario like ours?

Thanks again.
Regards
Nicolas



On Thu, Jul 5, 2018 at 2:31 PM Jean-Baptiste Onofré <[hidden email]> wrote:
Hi Nicolas

The purpose of boot features is to start in early stage, so it uses the
cfg file as definition.

In your case, you should use inner and prerequisite feature.

Do you have some dependencies between kar ?

Why don't you use directly feature instead of packaging as a kar ?

Can you explain exactly what you are trying to do ?

Karaf always store the feature state (in the resolver), so, if you first
install kar1, then kar2, when you restart, the resolver will
reinstall/start those features.

So, you don't have to use a boot feature: if you don't remove the data
folder, the installed features and repositories are stored.

Regards
JB

On 05/07/2018 13:52, Nicolas Brasey wrote:
> Hi all,
>
> I'm trying to find out if there is way to install a feature and make it
> as a boot feature without manually altering the feature cfg file
> (org.apache.karaf.features.cfg). Checking in karaf's code seems to
> indicate there is no way to do this programmatically.
>
> Ideally, it would be a flag in the feature:install command to indicate
> to add this feature as a boot feature.
>
> The reason we need this is that our solution is an integrated solution
> which is delivered by different departments:
>
> 1) Product 1 (kar 1) => dev team A
> 2) Product 2 (kar 2) => dev team B
> 3) Integration layer (camel routes essentially) (kar 3) => integration team
>
> All these different teams delivering a self contained kar file with a
> feature which should be installed and started when karaf starts in order
> to have the global solution running.
>
> We are using karaf v.4.1.2 which does not seems to persist which
> features have been installed (only the boot features). I'm not sure
> about the v.4.2.x...
>
> I know Karaf since not so long, but I believe Karaf has been designed so
> that the delivery team is supposed to create a Karaf distribution and
> assembling the required boot features at build time. If this is true,
> then it is not ideal according to how our internal process is made.
>
> Any thoughts?
> Thanks!
>
> Best regards,
> Nicolas 
>
>
>
>
>  

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

Guillaume Nodet-2
In reply to this post by nicolas.brasey
Have you tried simply dropping the kars in the deploy folder ?
This should install / start them automatically without the need to create a custom distribution.

Guillaume

Le jeu. 5 juil. 2018 à 13:53, Nicolas Brasey <[hidden email]> a écrit :
Hi all,

I'm trying to find out if there is way to install a feature and make it as a boot feature without manually altering the feature cfg file (org.apache.karaf.features.cfg). Checking in karaf's code seems to indicate there is no way to do this programmatically.

Ideally, it would be a flag in the feature:install command to indicate to add this feature as a boot feature.

The reason we need this is that our solution is an integrated solution which is delivered by different departments:

1) Product 1 (kar 1) => dev team A
2) Product 2 (kar 2) => dev team B
3) Integration layer (camel routes essentially) (kar 3) => integration team

All these different teams delivering a self contained kar file with a feature which should be installed and started when karaf starts in order to have the global solution running.

We are using karaf v.4.1.2 which does not seems to persist which features have been installed (only the boot features). I'm not sure about the v.4.2.x...

I know Karaf since not so long, but I believe Karaf has been designed so that the delivery team is supposed to create a Karaf distribution and assembling the required boot features at build time. If this is true, then it is not ideal according to how our internal process is made.

Any thoughts?
Thanks!

Best regards,
Nicolas 




 


--
------------------------
Guillaume Nodet

Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

nicolas.brasey
Yes we tried but had problems with the KarService implementation of karaf v.4.1.2 which had some issues with starting our features, there was some kind of loop which ended-up installing/uninstalling many time the same features, it was not working for us, so we have now our own implementation of a KarService which only unpacks the kar into a repository directory outside of the karaf distribution. The installation of the feature is made manually in a second stage. So at the moment we use the Kar as only a zip container as maven repository.

I saw the implementation of the KarService changed in the latest version of Karaf, so I've not tried again since.

Is it possible to tell the karaf feature resolver to persist the state of the features outside of the karaf distribution ? This would be helpful for us.

Thanks,
Nicolas




 

On Fri, Jul 6, 2018 at 2:34 PM Guillaume Nodet <[hidden email]> wrote:
Have you tried simply dropping the kars in the deploy folder ?
This should install / start them automatically without the need to create a custom distribution.

Guillaume

Le jeu. 5 juil. 2018 à 13:53, Nicolas Brasey <[hidden email]> a écrit :
Hi all,

I'm trying to find out if there is way to install a feature and make it as a boot feature without manually altering the feature cfg file (org.apache.karaf.features.cfg). Checking in karaf's code seems to indicate there is no way to do this programmatically.

Ideally, it would be a flag in the feature:install command to indicate to add this feature as a boot feature.

The reason we need this is that our solution is an integrated solution which is delivered by different departments:

1) Product 1 (kar 1) => dev team A
2) Product 2 (kar 2) => dev team B
3) Integration layer (camel routes essentially) (kar 3) => integration team

All these different teams delivering a self contained kar file with a feature which should be installed and started when karaf starts in order to have the global solution running.

We are using karaf v.4.1.2 which does not seems to persist which features have been installed (only the boot features). I'm not sure about the v.4.2.x...

I know Karaf since not so long, but I believe Karaf has been designed so that the delivery team is supposed to create a Karaf distribution and assembling the required boot features at build time. If this is true, then it is not ideal according to how our internal process is made.

Any thoughts?
Thanks!

Best regards,
Nicolas 




 


--
------------------------
Guillaume Nodet

Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

fpapon

Hi Nicolas,

When target machines are on a private network, it's usefull to have an instance of Nexus used like a proxy for the externals dependencies and the dev team can also publish their bundles on this Nexus.

regards,

François Papon
[hidden email]
Yupiik - https://www.yupiik.com
Le 06/07/2018 à 16:55, Nicolas Brasey a écrit :
Yes we tried but had problems with the KarService implementation of karaf v.4.1.2 which had some issues with starting our features, there was some kind of loop which ended-up installing/uninstalling many time the same features, it was not working for us, so we have now our own implementation of a KarService which only unpacks the kar into a repository directory outside of the karaf distribution. The installation of the feature is made manually in a second stage. So at the moment we use the Kar as only a zip container as maven repository.

I saw the implementation of the KarService changed in the latest version of Karaf, so I've not tried again since.

Is it possible to tell the karaf feature resolver to persist the state of the features outside of the karaf distribution ? This would be helpful for us.

Thanks,
Nicolas




 

On Fri, Jul 6, 2018 at 2:34 PM Guillaume Nodet <[hidden email]> wrote:
Have you tried simply dropping the kars in the deploy folder ?
This should install / start them automatically without the need to create a custom distribution.

Guillaume

Le jeu. 5 juil. 2018 à 13:53, Nicolas Brasey <[hidden email]> a écrit :
Hi all,

I'm trying to find out if there is way to install a feature and make it as a boot feature without manually altering the feature cfg file (org.apache.karaf.features.cfg). Checking in karaf's code seems to indicate there is no way to do this programmatically.

Ideally, it would be a flag in the feature:install command to indicate to add this feature as a boot feature.

The reason we need this is that our solution is an integrated solution which is delivered by different departments:

1) Product 1 (kar 1) => dev team A
2) Product 2 (kar 2) => dev team B
3) Integration layer (camel routes essentially) (kar 3) => integration team

All these different teams delivering a self contained kar file with a feature which should be installed and started when karaf starts in order to have the global solution running.

We are using karaf v.4.1.2 which does not seems to persist which features have been installed (only the boot features). I'm not sure about the v.4.2.x...

I know Karaf since not so long, but I believe Karaf has been designed so that the delivery team is supposed to create a Karaf distribution and assembling the required boot features at build time. If this is true, then it is not ideal according to how our internal process is made.

Any thoughts?
Thanks!

Best regards,
Nicolas 




 


--
------------------------
Guillaume Nodet


François Papon
fpapon@apache.org
Yupiik - https://www.yupiik.com
Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

nicolas.brasey
Hi Francois,

This is not possible, the servers are running in security zones without internet access, no proxying or tunneling is possible. But this is not really the problem as we use the kars which contain all we need.

Nicolas

On Fri, Jul 6, 2018 at 3:26 PM Francois Papon <[hidden email]> wrote:

Hi Nicolas,

When target machines are on a private network, it's usefull to have an instance of Nexus used like a proxy for the externals dependencies and the dev team can also publish their bundles on this Nexus.

regards,

François Papon
[hidden email]
Yupiik - https://www.yupiik.com
Le 06/07/2018 à 16:55, Nicolas Brasey a écrit :
Yes we tried but had problems with the KarService implementation of karaf v.4.1.2 which had some issues with starting our features, there was some kind of loop which ended-up installing/uninstalling many time the same features, it was not working for us, so we have now our own implementation of a KarService which only unpacks the kar into a repository directory outside of the karaf distribution. The installation of the feature is made manually in a second stage. So at the moment we use the Kar as only a zip container as maven repository.

I saw the implementation of the KarService changed in the latest version of Karaf, so I've not tried again since.

Is it possible to tell the karaf feature resolver to persist the state of the features outside of the karaf distribution ? This would be helpful for us.

Thanks,
Nicolas




 

On Fri, Jul 6, 2018 at 2:34 PM Guillaume Nodet <[hidden email]> wrote:
Have you tried simply dropping the kars in the deploy folder ?
This should install / start them automatically without the need to create a custom distribution.

Guillaume

Le jeu. 5 juil. 2018 à 13:53, Nicolas Brasey <[hidden email]> a écrit :
Hi all,

I'm trying to find out if there is way to install a feature and make it as a boot feature without manually altering the feature cfg file (org.apache.karaf.features.cfg). Checking in karaf's code seems to indicate there is no way to do this programmatically.

Ideally, it would be a flag in the feature:install command to indicate to add this feature as a boot feature.

The reason we need this is that our solution is an integrated solution which is delivered by different departments:

1) Product 1 (kar 1) => dev team A
2) Product 2 (kar 2) => dev team B
3) Integration layer (camel routes essentially) (kar 3) => integration team

All these different teams delivering a self contained kar file with a feature which should be installed and started when karaf starts in order to have the global solution running.

We are using karaf v.4.1.2 which does not seems to persist which features have been installed (only the boot features). I'm not sure about the v.4.2.x...

I know Karaf since not so long, but I believe Karaf has been designed so that the delivery team is supposed to create a Karaf distribution and assembling the required boot features at build time. If this is true, then it is not ideal according to how our internal process is made.

Any thoughts?
Thanks!

Best regards,
Nicolas 




 


--
------------------------
Guillaume Nodet


Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

fpapon

Ok, sorry for the noise ;)

May be you can write your own Deployer as a workaround, there is an example here : https://github.com/jbonofre/karaf/tree/DEV_GUIDE/examples/karaf-deployer-example

François


    
Le 06/07/2018 à 17:33, Nicolas Brasey a écrit :
Hi Francois,

This is not possible, the servers are running in security zones without internet access, no proxying or tunneling is possible. But this is not really the problem as we use the kars which contain all we need.

Nicolas

On Fri, Jul 6, 2018 at 3:26 PM Francois Papon <[hidden email]> wrote:

Hi Nicolas,

When target machines are on a private network, it's usefull to have an instance of Nexus used like a proxy for the externals dependencies and the dev team can also publish their bundles on this Nexus.

regards,

François Papon
[hidden email]
Yupiik - https://www.yupiik.com
Le 06/07/2018 à 16:55, Nicolas Brasey a écrit :
Yes we tried but had problems with the KarService implementation of karaf v.4.1.2 which had some issues with starting our features, there was some kind of loop which ended-up installing/uninstalling many time the same features, it was not working for us, so we have now our own implementation of a KarService which only unpacks the kar into a repository directory outside of the karaf distribution. The installation of the feature is made manually in a second stage. So at the moment we use the Kar as only a zip container as maven repository.

I saw the implementation of the KarService changed in the latest version of Karaf, so I've not tried again since.

Is it possible to tell the karaf feature resolver to persist the state of the features outside of the karaf distribution ? This would be helpful for us.

Thanks,
Nicolas




 

On Fri, Jul 6, 2018 at 2:34 PM Guillaume Nodet <[hidden email]> wrote:
Have you tried simply dropping the kars in the deploy folder ?
This should install / start them automatically without the need to create a custom distribution.

Guillaume

Le jeu. 5 juil. 2018 à 13:53, Nicolas Brasey <[hidden email]> a écrit :
Hi all,

I'm trying to find out if there is way to install a feature and make it as a boot feature without manually altering the feature cfg file (org.apache.karaf.features.cfg). Checking in karaf's code seems to indicate there is no way to do this programmatically.

Ideally, it would be a flag in the feature:install command to indicate to add this feature as a boot feature.

The reason we need this is that our solution is an integrated solution which is delivered by different departments:

1) Product 1 (kar 1) => dev team A
2) Product 2 (kar 2) => dev team B
3) Integration layer (camel routes essentially) (kar 3) => integration team

All these different teams delivering a self contained kar file with a feature which should be installed and started when karaf starts in order to have the global solution running.

We are using karaf v.4.1.2 which does not seems to persist which features have been installed (only the boot features). I'm not sure about the v.4.2.x...

I know Karaf since not so long, but I believe Karaf has been designed so that the delivery team is supposed to create a Karaf distribution and assembling the required boot features at build time. If this is true, then it is not ideal according to how our internal process is made.

Any thoughts?
Thanks!

Best regards,
Nicolas 




 


--
------------------------
Guillaume Nodet



François Papon
fpapon@apache.org
Yupiik - https://www.yupiik.com
Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

Guillaume Nodet-2
In reply to this post by nicolas.brasey


Le ven. 6 juil. 2018 à 14:56, Nicolas Brasey <[hidden email]> a écrit :
Yes we tried but had problems with the KarService implementation of karaf v.4.1.2 which had some issues with starting our features, there was some kind of loop which ended-up installing/uninstalling many time the same features, it was not working for us, so we have now our own implementation of a KarService which only unpacks the kar into a repository directory outside of the karaf distribution. The installation of the feature is made manually in a second stage. So at the moment we use the Kar as only a zip container as maven repository.

So what are you distributing exactly ? Because if you use the kar deployer, it means you *have* to use a manual step after that because the features won't be available at boot stage.  If you repackage the application after having installed the kars manually, you're basically creating a custom distro, else, persisting something which has not taken place yet won't help...
 

I saw the implementation of the KarService changed in the latest version of Karaf, so I've not tried again since.

Is it possible to tell the karaf feature resolver to persist the state of the features outside of the karaf distribution ? This would be helpful for us.

Thanks,
Nicolas




 

On Fri, Jul 6, 2018 at 2:34 PM Guillaume Nodet <[hidden email]> wrote:
Have you tried simply dropping the kars in the deploy folder ?
This should install / start them automatically without the need to create a custom distribution.

Guillaume

Le jeu. 5 juil. 2018 à 13:53, Nicolas Brasey <[hidden email]> a écrit :
Hi all,

I'm trying to find out if there is way to install a feature and make it as a boot feature without manually altering the feature cfg file (org.apache.karaf.features.cfg). Checking in karaf's code seems to indicate there is no way to do this programmatically.

Ideally, it would be a flag in the feature:install command to indicate to add this feature as a boot feature.

The reason we need this is that our solution is an integrated solution which is delivered by different departments:

1) Product 1 (kar 1) => dev team A
2) Product 2 (kar 2) => dev team B
3) Integration layer (camel routes essentially) (kar 3) => integration team

All these different teams delivering a self contained kar file with a feature which should be installed and started when karaf starts in order to have the global solution running.

We are using karaf v.4.1.2 which does not seems to persist which features have been installed (only the boot features). I'm not sure about the v.4.2.x...

I know Karaf since not so long, but I believe Karaf has been designed so that the delivery team is supposed to create a Karaf distribution and assembling the required boot features at build time. If this is true, then it is not ideal according to how our internal process is made.

Any thoughts?
Thanks!

Best regards,
Nicolas 




 


--
------------------------
Guillaume Nodet



--
------------------------
Guillaume Nodet

Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

jbonofre
In reply to this post by nicolas.brasey
Hi Nicolas,

I would recommend:

1. To take a look if you can't populate the system folder or even use
Cave on Karaf instance, acting as Maven repository that you can populated

Anyway, Kar is also an option but it needs a clean packaging and
dependency on kar file:

1. Your can package all as a kar but you are loosing "flexibility"
2. In your case, I would recommend to use kar (with deploy
folder/deployer or kar:* commands for deployment). A first kar could be
a common kar providing packages/services that other kar will use. Other
kar are atomic and isolated from the others.
So you first deploy the first common kar and then the other kars (the
order should not matter).

Kar works fine but a kar is supposed to be atomic: that's why I asked if
you have dependencies between kars.

Regards
JB

On 06/07/2018 13:55, Nicolas Brasey wrote:

> Hi Jean-Baptiste, 
>
> Yes, our different kars have dependencies, so they must installed in a
> certain order.
>
> The features.xml only is not possible in our case because our target
> machines are running on private networks without internet access, so the
> kars must contain all the runtime transitive dependencies.
>
> But to explain exactly what I'm trying to achieve today, I need to
> explain the history of our solution. It started 3 years ago as a single
> application, at that time, the dev team A was delivering the app as a
> turnkey solution with a custom karaf distribution. The installation was
> very easy for the delivery team => Move the distribution to the server,
> unzip and start, the application jars were located in the system
> repository and the app features marked as boot features.
>
> Over the last months, we extended the core application by providing
> plugin mechanisms by using SPI's, implementing the white-board pattern
> (we like this a lot by the way :)). So now dev team B is implementing
> different plugins for specific needs. Those plugins have a different
> life cycle that the core app, and they are released as kars.
>
> Lately we found a bug in the main app which needed to be fixed and
> patched in a prod environment. We provided the patch with a new custom
> karaf distribution which the delivery installed. Of course, this new
> karaf distribution used another data directory and did not reinstall the
> features of the kars of the different plugins. So manually reinstalling
> the plugin features is absolutely needed in this scenario or the
> behavior of the application can differ from before the patching, which
> is obviously dangerous. 
>
> Like I said, our main application is still released as part of our
> custom karaf distribution, this should be changed and we will probably
> release it a kar as well.
>
> So I try to achieve 3 things:
>
> 1) Find the optimal packaging for our app and plugins so that it can be
> install / upgraded easily
> 2) Hide Karaf (OSGi) complexity from the delivery team, and they don't
> want to know about it :-)
> 3) Find the optimal way to set the bundle versions in our feature files
> using ranges
>
> What would be a good practice in a scenario like ours?
>
> Thanks again.
> Regards
> Nicolas
>
>
>
> On Thu, Jul 5, 2018 at 2:31 PM Jean-Baptiste Onofré <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Nicolas
>
>     The purpose of boot features is to start in early stage, so it uses the
>     cfg file as definition.
>
>     In your case, you should use inner and prerequisite feature.
>
>     Do you have some dependencies between kar ?
>
>     Why don't you use directly feature instead of packaging as a kar ?
>
>     Can you explain exactly what you are trying to do ?
>
>     Karaf always store the feature state (in the resolver), so, if you first
>     install kar1, then kar2, when you restart, the resolver will
>     reinstall/start those features.
>
>     So, you don't have to use a boot feature: if you don't remove the data
>     folder, the installed features and repositories are stored.
>
>     Regards
>     JB
>
>     On 05/07/2018 13:52, Nicolas Brasey wrote:
>     > Hi all,
>     >
>     > I'm trying to find out if there is way to install a feature and
>     make it
>     > as a boot feature without manually altering the feature cfg file
>     > (org.apache.karaf.features.cfg). Checking in karaf's code seems to
>     > indicate there is no way to do this programmatically.
>     >
>     > Ideally, it would be a flag in the feature:install command to indicate
>     > to add this feature as a boot feature.
>     >
>     > The reason we need this is that our solution is an integrated solution
>     > which is delivered by different departments:
>     >
>     > 1) Product 1 (kar 1) => dev team A
>     > 2) Product 2 (kar 2) => dev team B
>     > 3) Integration layer (camel routes essentially) (kar 3) =>
>     integration team
>     >
>     > All these different teams delivering a self contained kar file with a
>     > feature which should be installed and started when karaf starts in
>     order
>     > to have the global solution running.
>     >
>     > We are using karaf v.4.1.2 which does not seems to persist which
>     > features have been installed (only the boot features). I'm not sure
>     > about the v.4.2.x...
>     >
>     > I know Karaf since not so long, but I believe Karaf has been
>     designed so
>     > that the delivery team is supposed to create a Karaf distribution and
>     > assembling the required boot features at build time. If this is true,
>     > then it is not ideal according to how our internal process is made.
>     >
>     > Any thoughts?
>     > Thanks!
>     >
>     > Best regards,
>     > Nicolas 
>     >
>     >
>     >
>     >
>     >  
>
>     --
>     Jean-Baptiste Onofré
>     [hidden email] <mailto:[hidden email]>
>     http://blog.nanthrax.net
>     Talend - http://www.talend.com
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

nicolas.brasey
In reply to this post by fpapon
Hi Francois,

No worries, thanks for the link, I'll have a look!

Nicolas

On Fri, Jul 6, 2018 at 3:43 PM Francois Papon <[hidden email]> wrote:

Ok, sorry for the noise ;)

May be you can write your own Deployer as a workaround, there is an example here : https://github.com/jbonofre/karaf/tree/DEV_GUIDE/examples/karaf-deployer-example

François


    
Le 06/07/2018 à 17:33, Nicolas Brasey a écrit :
Hi Francois,

This is not possible, the servers are running in security zones without internet access, no proxying or tunneling is possible. But this is not really the problem as we use the kars which contain all we need.

Nicolas

On Fri, Jul 6, 2018 at 3:26 PM Francois Papon <[hidden email]> wrote:

Hi Nicolas,

When target machines are on a private network, it's usefull to have an instance of Nexus used like a proxy for the externals dependencies and the dev team can also publish their bundles on this Nexus.

regards,

François Papon
[hidden email]
Yupiik - https://www.yupiik.com
Le 06/07/2018 à 16:55, Nicolas Brasey a écrit :
Yes we tried but had problems with the KarService implementation of karaf v.4.1.2 which had some issues with starting our features, there was some kind of loop which ended-up installing/uninstalling many time the same features, it was not working for us, so we have now our own implementation of a KarService which only unpacks the kar into a repository directory outside of the karaf distribution. The installation of the feature is made manually in a second stage. So at the moment we use the Kar as only a zip container as maven repository.

I saw the implementation of the KarService changed in the latest version of Karaf, so I've not tried again since.

Is it possible to tell the karaf feature resolver to persist the state of the features outside of the karaf distribution ? This would be helpful for us.

Thanks,
Nicolas




 

On Fri, Jul 6, 2018 at 2:34 PM Guillaume Nodet <[hidden email]> wrote:
Have you tried simply dropping the kars in the deploy folder ?
This should install / start them automatically without the need to create a custom distribution.

Guillaume

Le jeu. 5 juil. 2018 à 13:53, Nicolas Brasey <[hidden email]> a écrit :
Hi all,

I'm trying to find out if there is way to install a feature and make it as a boot feature without manually altering the feature cfg file (org.apache.karaf.features.cfg). Checking in karaf's code seems to indicate there is no way to do this programmatically.

Ideally, it would be a flag in the feature:install command to indicate to add this feature as a boot feature.

The reason we need this is that our solution is an integrated solution which is delivered by different departments:

1) Product 1 (kar 1) => dev team A
2) Product 2 (kar 2) => dev team B
3) Integration layer (camel routes essentially) (kar 3) => integration team

All these different teams delivering a self contained kar file with a feature which should be installed and started when karaf starts in order to have the global solution running.

We are using karaf v.4.1.2 which does not seems to persist which features have been installed (only the boot features). I'm not sure about the v.4.2.x...

I know Karaf since not so long, but I believe Karaf has been designed so that the delivery team is supposed to create a Karaf distribution and assembling the required boot features at build time. If this is true, then it is not ideal according to how our internal process is made.

Any thoughts?
Thanks!

Best regards,
Nicolas 




 


--
------------------------
Guillaume Nodet



Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

nicolas.brasey
In reply to this post by Guillaume Nodet-2
Our ideas is to provide maven repositories for each "application". These repos should contains features.xml which refers to jars which are also in the same repos or jars that are already available. I wanted to leverage the fact that each kar repo is automatically available as remote repo by the resolver. 

I think Cave is definitely an option but I wanted to avoid the need to maintain yet another server in prod and potential point of failures. I wanted to keep it simple for now...





On Fri, Jul 6, 2018 at 3:48 PM Guillaume Nodet <[hidden email]> wrote:


Le ven. 6 juil. 2018 à 14:56, Nicolas Brasey <[hidden email]> a écrit :
Yes we tried but had problems with the KarService implementation of karaf v.4.1.2 which had some issues with starting our features, there was some kind of loop which ended-up installing/uninstalling many time the same features, it was not working for us, so we have now our own implementation of a KarService which only unpacks the kar into a repository directory outside of the karaf distribution. The installation of the feature is made manually in a second stage. So at the moment we use the Kar as only a zip container as maven repository.

So what are you distributing exactly ? Because if you use the kar deployer, it means you *have* to use a manual step after that because the features won't be available at boot stage.  If you repackage the application after having installed the kars manually, you're basically creating a custom distro, else, persisting something which has not taken place yet won't help...
 

I saw the implementation of the KarService changed in the latest version of Karaf, so I've not tried again since.

Is it possible to tell the karaf feature resolver to persist the state of the features outside of the karaf distribution ? This would be helpful for us.

Thanks,
Nicolas




 

On Fri, Jul 6, 2018 at 2:34 PM Guillaume Nodet <[hidden email]> wrote:
Have you tried simply dropping the kars in the deploy folder ?
This should install / start them automatically without the need to create a custom distribution.

Guillaume

Le jeu. 5 juil. 2018 à 13:53, Nicolas Brasey <[hidden email]> a écrit :
Hi all,

I'm trying to find out if there is way to install a feature and make it as a boot feature without manually altering the feature cfg file (org.apache.karaf.features.cfg). Checking in karaf's code seems to indicate there is no way to do this programmatically.

Ideally, it would be a flag in the feature:install command to indicate to add this feature as a boot feature.

The reason we need this is that our solution is an integrated solution which is delivered by different departments:

1) Product 1 (kar 1) => dev team A
2) Product 2 (kar 2) => dev team B
3) Integration layer (camel routes essentially) (kar 3) => integration team

All these different teams delivering a self contained kar file with a feature which should be installed and started when karaf starts in order to have the global solution running.

We are using karaf v.4.1.2 which does not seems to persist which features have been installed (only the boot features). I'm not sure about the v.4.2.x...

I know Karaf since not so long, but I believe Karaf has been designed so that the delivery team is supposed to create a Karaf distribution and assembling the required boot features at build time. If this is true, then it is not ideal according to how our internal process is made.

Any thoughts?
Thanks!

Best regards,
Nicolas 




 


--
------------------------
Guillaume Nodet



--
------------------------
Guillaume Nodet

Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

nicolas.brasey
In reply to this post by jbonofre
Thanks for that.

I'll experiment a little bit this week and come back to you in case I need more guidance.

Thanks again.

Nicolas

On Fri, Jul 6, 2018 at 3:56 PM Jean-Baptiste Onofré <[hidden email]> wrote:
Hi Nicolas,

I would recommend:

1. To take a look if you can't populate the system folder or even use
Cave on Karaf instance, acting as Maven repository that you can populated

Anyway, Kar is also an option but it needs a clean packaging and
dependency on kar file:

1. Your can package all as a kar but you are loosing "flexibility"
2. In your case, I would recommend to use kar (with deploy
folder/deployer or kar:* commands for deployment). A first kar could be
a common kar providing packages/services that other kar will use. Other
kar are atomic and isolated from the others.
So you first deploy the first common kar and then the other kars (the
order should not matter).

Kar works fine but a kar is supposed to be atomic: that's why I asked if
you have dependencies between kars.

Regards
JB

On 06/07/2018 13:55, Nicolas Brasey wrote:
> Hi Jean-Baptiste, 
>
> Yes, our different kars have dependencies, so they must installed in a
> certain order.
>
> The features.xml only is not possible in our case because our target
> machines are running on private networks without internet access, so the
> kars must contain all the runtime transitive dependencies.
>
> But to explain exactly what I'm trying to achieve today, I need to
> explain the history of our solution. It started 3 years ago as a single
> application, at that time, the dev team A was delivering the app as a
> turnkey solution with a custom karaf distribution. The installation was
> very easy for the delivery team => Move the distribution to the server,
> unzip and start, the application jars were located in the system
> repository and the app features marked as boot features.
>
> Over the last months, we extended the core application by providing
> plugin mechanisms by using SPI's, implementing the white-board pattern
> (we like this a lot by the way :)). So now dev team B is implementing
> different plugins for specific needs. Those plugins have a different
> life cycle that the core app, and they are released as kars.
>
> Lately we found a bug in the main app which needed to be fixed and
> patched in a prod environment. We provided the patch with a new custom
> karaf distribution which the delivery installed. Of course, this new
> karaf distribution used another data directory and did not reinstall the
> features of the kars of the different plugins. So manually reinstalling
> the plugin features is absolutely needed in this scenario or the
> behavior of the application can differ from before the patching, which
> is obviously dangerous. 
>
> Like I said, our main application is still released as part of our
> custom karaf distribution, this should be changed and we will probably
> release it a kar as well.
>
> So I try to achieve 3 things:
>
> 1) Find the optimal packaging for our app and plugins so that it can be
> install / upgraded easily
> 2) Hide Karaf (OSGi) complexity from the delivery team, and they don't
> want to know about it :-)
> 3) Find the optimal way to set the bundle versions in our feature files
> using ranges
>
> What would be a good practice in a scenario like ours?
>
> Thanks again.
> Regards
> Nicolas
>
>
>
> On Thu, Jul 5, 2018 at 2:31 PM Jean-Baptiste Onofré <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Nicolas
>
>     The purpose of boot features is to start in early stage, so it uses the
>     cfg file as definition.
>
>     In your case, you should use inner and prerequisite feature.
>
>     Do you have some dependencies between kar ?
>
>     Why don't you use directly feature instead of packaging as a kar ?
>
>     Can you explain exactly what you are trying to do ?
>
>     Karaf always store the feature state (in the resolver), so, if you first
>     install kar1, then kar2, when you restart, the resolver will
>     reinstall/start those features.
>
>     So, you don't have to use a boot feature: if you don't remove the data
>     folder, the installed features and repositories are stored.
>
>     Regards
>     JB
>
>     On 05/07/2018 13:52, Nicolas Brasey wrote:
>     > Hi all,
>     >
>     > I'm trying to find out if there is way to install a feature and
>     make it
>     > as a boot feature without manually altering the feature cfg file
>     > (org.apache.karaf.features.cfg). Checking in karaf's code seems to
>     > indicate there is no way to do this programmatically.
>     >
>     > Ideally, it would be a flag in the feature:install command to indicate
>     > to add this feature as a boot feature.
>     >
>     > The reason we need this is that our solution is an integrated solution
>     > which is delivered by different departments:
>     >
>     > 1) Product 1 (kar 1) => dev team A
>     > 2) Product 2 (kar 2) => dev team B
>     > 3) Integration layer (camel routes essentially) (kar 3) =>
>     integration team
>     >
>     > All these different teams delivering a self contained kar file with a
>     > feature which should be installed and started when karaf starts in
>     order
>     > to have the global solution running.
>     >
>     > We are using karaf v.4.1.2 which does not seems to persist which
>     > features have been installed (only the boot features). I'm not sure
>     > about the v.4.2.x...
>     >
>     > I know Karaf since not so long, but I believe Karaf has been
>     designed so
>     > that the delivery team is supposed to create a Karaf distribution and
>     > assembling the required boot features at build time. If this is true,
>     > then it is not ideal according to how our internal process is made.
>     >
>     > Any thoughts?
>     > Thanks!
>     >
>     > Best regards,
>     > Nicolas 
>     >
>     >
>     >
>     >
>     >  
>
>     --
>     Jean-Baptiste Onofré
>     [hidden email] <mailto:[hidden email]>
>     http://blog.nanthrax.net
>     Talend - http://www.talend.com
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

jbonofre
Cool !

Don't hesitate to ping me (even on Hangout or whatever).

I'm always happy to help !

Regards
JB

On 09/07/2018 19:30, Nicolas Brasey wrote:

> Thanks for that.
>
> I'll experiment a little bit this week and come back to you in case I
> need more guidance.
>
> Thanks again.
>
> Nicolas
>
> On Fri, Jul 6, 2018 at 3:56 PM Jean-Baptiste Onofré <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Nicolas,
>
>     I would recommend:
>
>     1. To take a look if you can't populate the system folder or even use
>     Cave on Karaf instance, acting as Maven repository that you can
>     populated
>
>     Anyway, Kar is also an option but it needs a clean packaging and
>     dependency on kar file:
>
>     1. Your can package all as a kar but you are loosing "flexibility"
>     2. In your case, I would recommend to use kar (with deploy
>     folder/deployer or kar:* commands for deployment). A first kar could be
>     a common kar providing packages/services that other kar will use. Other
>     kar are atomic and isolated from the others.
>     So you first deploy the first common kar and then the other kars (the
>     order should not matter).
>
>     Kar works fine but a kar is supposed to be atomic: that's why I asked if
>     you have dependencies between kars.
>
>     Regards
>     JB
>
>     On 06/07/2018 13:55, Nicolas Brasey wrote:
>     > Hi Jean-Baptiste, 
>     >
>     > Yes, our different kars have dependencies, so they must installed in a
>     > certain order.
>     >
>     > The features.xml only is not possible in our case because our target
>     > machines are running on private networks without internet access,
>     so the
>     > kars must contain all the runtime transitive dependencies.
>     >
>     > But to explain exactly what I'm trying to achieve today, I need to
>     > explain the history of our solution. It started 3 years ago as a
>     single
>     > application, at that time, the dev team A was delivering the app as a
>     > turnkey solution with a custom karaf distribution. The
>     installation was
>     > very easy for the delivery team => Move the distribution to the
>     server,
>     > unzip and start, the application jars were located in the system
>     > repository and the app features marked as boot features.
>     >
>     > Over the last months, we extended the core application by providing
>     > plugin mechanisms by using SPI's, implementing the white-board pattern
>     > (we like this a lot by the way :)). So now dev team B is implementing
>     > different plugins for specific needs. Those plugins have a different
>     > life cycle that the core app, and they are released as kars.
>     >
>     > Lately we found a bug in the main app which needed to be fixed and
>     > patched in a prod environment. We provided the patch with a new custom
>     > karaf distribution which the delivery installed. Of course, this new
>     > karaf distribution used another data directory and did not
>     reinstall the
>     > features of the kars of the different plugins. So manually
>     reinstalling
>     > the plugin features is absolutely needed in this scenario or the
>     > behavior of the application can differ from before the patching, which
>     > is obviously dangerous. 
>     >
>     > Like I said, our main application is still released as part of our
>     > custom karaf distribution, this should be changed and we will probably
>     > release it a kar as well.
>     >
>     > So I try to achieve 3 things:
>     >
>     > 1) Find the optimal packaging for our app and plugins so that it
>     can be
>     > install / upgraded easily
>     > 2) Hide Karaf (OSGi) complexity from the delivery team, and they don't
>     > want to know about it :-)
>     > 3) Find the optimal way to set the bundle versions in our feature
>     files
>     > using ranges
>     >
>     > What would be a good practice in a scenario like ours?
>     >
>     > Thanks again.
>     > Regards
>     > Nicolas
>     >
>     >
>     >
>     > On Thu, Jul 5, 2018 at 2:31 PM Jean-Baptiste Onofré
>     <[hidden email] <mailto:[hidden email]>
>     > <mailto:[hidden email] <mailto:[hidden email]>>> wrote:
>     >
>     >     Hi Nicolas
>     >
>     >     The purpose of boot features is to start in early stage, so it
>     uses the
>     >     cfg file as definition.
>     >
>     >     In your case, you should use inner and prerequisite feature.
>     >
>     >     Do you have some dependencies between kar ?
>     >
>     >     Why don't you use directly feature instead of packaging as a kar ?
>     >
>     >     Can you explain exactly what you are trying to do ?
>     >
>     >     Karaf always store the feature state (in the resolver), so, if
>     you first
>     >     install kar1, then kar2, when you restart, the resolver will
>     >     reinstall/start those features.
>     >
>     >     So, you don't have to use a boot feature: if you don't remove
>     the data
>     >     folder, the installed features and repositories are stored.
>     >
>     >     Regards
>     >     JB
>     >
>     >     On 05/07/2018 13:52, Nicolas Brasey wrote:
>     >     > Hi all,
>     >     >
>     >     > I'm trying to find out if there is way to install a feature and
>     >     make it
>     >     > as a boot feature without manually altering the feature cfg file
>     >     > (org.apache.karaf.features.cfg). Checking in karaf's code
>     seems to
>     >     > indicate there is no way to do this programmatically.
>     >     >
>     >     > Ideally, it would be a flag in the feature:install command
>     to indicate
>     >     > to add this feature as a boot feature.
>     >     >
>     >     > The reason we need this is that our solution is an
>     integrated solution
>     >     > which is delivered by different departments:
>     >     >
>     >     > 1) Product 1 (kar 1) => dev team A
>     >     > 2) Product 2 (kar 2) => dev team B
>     >     > 3) Integration layer (camel routes essentially) (kar 3) =>
>     >     integration team
>     >     >
>     >     > All these different teams delivering a self
>     contained kar file with a
>     >     > feature which should be installed and started when
>     karaf starts in
>     >     order
>     >     > to have the global solution running.
>     >     >
>     >     > We are using karaf v.4.1.2 which does not seems to persist which
>     >     > features have been installed (only the boot features). I'm
>     not sure
>     >     > about the v.4.2.x...
>     >     >
>     >     > I know Karaf since not so long, but I believe Karaf has been
>     >     designed so
>     >     > that the delivery team is supposed to create a Karaf
>     distribution and
>     >     > assembling the required boot features at build time. If this
>     is true,
>     >     > then it is not ideal according to how our internal process
>     is made.
>     >     >
>     >     > Any thoughts?
>     >     > Thanks!
>     >     >
>     >     > Best regards,
>     >     > Nicolas 
>     >     >
>     >     >
>     >     >
>     >     >
>     >     >  
>     >
>     >     --
>     >     Jean-Baptiste Onofré
>     >     [hidden email] <mailto:[hidden email]>
>     <mailto:[hidden email] <mailto:[hidden email]>>
>     >     http://blog.nanthrax.net
>     >     Talend - http://www.talend.com
>     >
>
>     --
>     Jean-Baptiste Onofré
>     [hidden email] <mailto:[hidden email]>
>     http://blog.nanthrax.net
>     Talend - http://www.talend.com
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Install feature as boot features with command line

fpapon
In reply to this post by nicolas.brasey

Yes, Cave is a good option, especially with the last Cave Deployer feature included by JB that is really interesting :)


Le 09/07/2018 à 21:29, Nicolas Brasey a écrit :
Our ideas is to provide maven repositories for each "application". These repos should contains features.xml which refers to jars which are also in the same repos or jars that are already available. I wanted to leverage the fact that each kar repo is automatically available as remote repo by the resolver. 

I think Cave is definitely an option but I wanted to avoid the need to maintain yet another server in prod and potential point of failures. I wanted to keep it simple for now...





On Fri, Jul 6, 2018 at 3:48 PM Guillaume Nodet <[hidden email]> wrote:


Le ven. 6 juil. 2018 à 14:56, Nicolas Brasey <[hidden email]> a écrit :
Yes we tried but had problems with the KarService implementation of karaf v.4.1.2 which had some issues with starting our features, there was some kind of loop which ended-up installing/uninstalling many time the same features, it was not working for us, so we have now our own implementation of a KarService which only unpacks the kar into a repository directory outside of the karaf distribution. The installation of the feature is made manually in a second stage. So at the moment we use the Kar as only a zip container as maven repository.

So what are you distributing exactly ? Because if you use the kar deployer, it means you *have* to use a manual step after that because the features won't be available at boot stage.  If you repackage the application after having installed the kars manually, you're basically creating a custom distro, else, persisting something which has not taken place yet won't help...
 

I saw the implementation of the KarService changed in the latest version of Karaf, so I've not tried again since.

Is it possible to tell the karaf feature resolver to persist the state of the features outside of the karaf distribution ? This would be helpful for us.

Thanks,
Nicolas




 

On Fri, Jul 6, 2018 at 2:34 PM Guillaume Nodet <[hidden email]> wrote:
Have you tried simply dropping the kars in the deploy folder ?
This should install / start them automatically without the need to create a custom distribution.

Guillaume

Le jeu. 5 juil. 2018 à 13:53, Nicolas Brasey <[hidden email]> a écrit :
Hi all,

I'm trying to find out if there is way to install a feature and make it as a boot feature without manually altering the feature cfg file (org.apache.karaf.features.cfg). Checking in karaf's code seems to indicate there is no way to do this programmatically.

Ideally, it would be a flag in the feature:install command to indicate to add this feature as a boot feature.

The reason we need this is that our solution is an integrated solution which is delivered by different departments:

1) Product 1 (kar 1) => dev team A
2) Product 2 (kar 2) => dev team B
3) Integration layer (camel routes essentially) (kar 3) => integration team

All these different teams delivering a self contained kar file with a feature which should be installed and started when karaf starts in order to have the global solution running.

We are using karaf v.4.1.2 which does not seems to persist which features have been installed (only the boot features). I'm not sure about the v.4.2.x...

I know Karaf since not so long, but I believe Karaf has been designed so that the delivery team is supposed to create a Karaf distribution and assembling the required boot features at build time. If this is true, then it is not ideal according to how our internal process is made.

Any thoughts?
Thanks!

Best regards,
Nicolas 




 


--
------------------------
Guillaume Nodet



--
------------------------
Guillaume Nodet


François Papon
fpapon@apache.org
Yupiik - https://www.yupiik.com