[PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

[PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

jbonofre
Hi guys,

Recently, we received a lot of questions around how to create Karaf
custom distribution based on karaf-maven-plugin, and how to use the
static profile to create "standalone/static" distribution.

If the plugin works fine, it's not easy to understand some "details",
like the dependency scope impact, or providing the set of default
features repos and features. I already helped users (in private
communication) to fix their custom distributions.

Obviously, we should simplify the way of creating custom distribution,
especially with the new tooling & feature we now provide around Docker.

I would like to propose the following:

1. Set the default behavior of the assembly goal to create a custom
distribution based on standard. For the user, instead of providing
(again) all framework, standard, enterprise features repos and all
standard boot features (shell, ...), it will just specify the tar.gz/zip
base and his own features repo/repos (or the goal will use the same
version of the goal plugin itself). All the rest will be done by the
plugin for him. Use the karaf packaging as default to define this. At
the end of the day, the user pom.xml will look like:

<project>
        <groupId>foo</groupId>
        <artifactId>bar</artifactId>
        <version>1.0-SNAPSHOT</version>
        <packaging>karaf</packaging>

        <dependencies>
                <dependency>
                        <groupId>org.apache.karaf</groupId>
                        <artifactId>apache-karaf</artifactId>
                        <version>4.2.1</version>
                        <type>tar.gz</type>
                </dependency>
                <dependency>
                        <groupId>foo</groupId>
                        <artifactId>my</artifactId>
                        <version>1.0-SNAPSHOT</version>
                        <classifier>features</classifier>
                        <type>xml</type>
                </dependency>
        </dependencies>

        <build>
                <plugins>
                        <plugin>
                                <groupId>org.apache.karaf.tooling</groupId>
<artifactId>karaf-maven-plugin</artifactId>
<extensions>true</extensions>
<inherited>true</inherited>
<configuration>
<bootFeatures>
        <feature>my</feature>
</bootFeatures>
<installedFeatures>
        <feature>my-other</feature>
</installedFeatures>
</configuration>
                        </plugin>
                </plugins>
        </build>

</project>

The idea is to automatically execute install-kar + assembly for the
karaf packaging and let the user focus on its own resources (features,
config, ...) just providing the base Karaf archive.
The user will be able to use src/main/resources to provide any files in
etc, bin, or whatever in the resulting custom distribution.

2. Improve a bit the features XML generation
If the custom distribution is the highest priority, just after the
improvements on this area, I would like to improve the way of creating
features XML.
Now, to be honest, almost all of us write features repos XML by hand. It
gives us the maximum of flexibility. However, on the other hand, the
features XML and code contain should be sync.
I would like to improve the generate features MOJO, however leveraging
most of all functionalities around features (prerequisites, dependency
flag, inner features, ...).

I have to dig a little bit around that, but if you want some ideas
already, please let me know.

Thoughts ?

Regards
JB
--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

fpapon
Hi JB,

I agree about this, we have to focus on the user friendly stuff and I
have an additional point :

  * Add a simple way to update/add properties in the default
    configuration files of the standard distribution in the assembly
    (like system.properties....)

I'm not sure this is the correct discussion but I think we also have to
work about helping user to upgrade the Karaf version of their custom
distributions.

regards,

François Papon
[hidden email]

Le 13/09/2018 à 15:51, Jean-Baptiste Onofré a écrit :

> Hi guys,
>
> Recently, we received a lot of questions around how to create Karaf
> custom distribution based on karaf-maven-plugin, and how to use the
> static profile to create "standalone/static" distribution.
>
> If the plugin works fine, it's not easy to understand some "details",
> like the dependency scope impact, or providing the set of default
> features repos and features. I already helped users (in private
> communication) to fix their custom distributions.
>
> Obviously, we should simplify the way of creating custom distribution,
> especially with the new tooling & feature we now provide around Docker.
>
> I would like to propose the following:
>
> 1. Set the default behavior of the assembly goal to create a custom
> distribution based on standard. For the user, instead of providing
> (again) all framework, standard, enterprise features repos and all
> standard boot features (shell, ...), it will just specify the tar.gz/zip
> base and his own features repo/repos (or the goal will use the same
> version of the goal plugin itself). All the rest will be done by the
> plugin for him. Use the karaf packaging as default to define this. At
> the end of the day, the user pom.xml will look like:
>
> <project>
> <groupId>foo</groupId>
> <artifactId>bar</artifactId>
> <version>1.0-SNAPSHOT</version>
> <packaging>karaf</packaging>
>
> <dependencies>
> <dependency>
> <groupId>org.apache.karaf</groupId>
> <artifactId>apache-karaf</artifactId>
> <version>4.2.1</version>
> <type>tar.gz</type>
> </dependency>
> <dependency>
> <groupId>foo</groupId>
> <artifactId>my</artifactId>
> <version>1.0-SNAPSHOT</version>
> <classifier>features</classifier>
> <type>xml</type>
> </dependency>
> </dependencies>
>
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.karaf.tooling</groupId>
> <artifactId>karaf-maven-plugin</artifactId>
> <extensions>true</extensions>
> <inherited>true</inherited>
> <configuration>
> <bootFeatures>
> <feature>my</feature>
> </bootFeatures>
> <installedFeatures>
> <feature>my-other</feature>
> </installedFeatures>
> </configuration>
> </plugin>
> </plugins>
> </build>
>
> </project>
>
> The idea is to automatically execute install-kar + assembly for the
> karaf packaging and let the user focus on its own resources (features,
> config, ...) just providing the base Karaf archive.
> The user will be able to use src/main/resources to provide any files in
> etc, bin, or whatever in the resulting custom distribution.
>
> 2. Improve a bit the features XML generation
> If the custom distribution is the highest priority, just after the
> improvements on this area, I would like to improve the way of creating
> features XML.
> Now, to be honest, almost all of us write features repos XML by hand. It
> gives us the maximum of flexibility. However, on the other hand, the
> features XML and code contain should be sync.
> I would like to improve the generate features MOJO, however leveraging
> most of all functionalities around features (prerequisites, dependency
> flag, inner features, ...).
>
> I have to dig a little bit around that, but if you want some ideas
> already, please let me know.
>
> Thoughts ?
>
> Regards
> JB

François Papon
fpapon@apache.org
Yupiik - https://www.yupiik.com
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

Grzegorz Grzybek
In reply to this post by jbonofre
Hello

Thanks JBO for the idea. True - custom distro generation isn't that
intuitive as it could be. Personally I missed the flexibility, not
simplicity, that's why I added <framework>custom</framework> in
"[KARAF-5468] Cleaning up AssemblyMojo, Profiles and profile Builder".

Without it, the implied value for "framework" property was "framework" or
"static-framework" depending on whether you had dependency on
mvn:org.apache.karaf.features/framework/VERSION/kar or
mvn:org.apache.karaf.features/static/VERSION/kar in your POM. The
discovered "framework" property was added as "startup feature" which was
very confusing (mixing "framework" and "feature" concepts).

I can't tell much about improvements for
karaf-maven-plugin:features-generate-descriptor, because I always preferred
manual creation of feature XMLs.

However having dependency on existing karaf distro ("standard" apache-karaf
or apache-karaf-minimal) is good idea! karaf-maven-plugin:assembly could
search zip or tar.gz or tgz kind of dependencies (with "provided" or any
other scope) and:
 - unpack it
 - check etc/profile.cfg

etc/profile.cfg is (since 4.2.0) nicely written as (official
apache-karaf-minimal distro):

#
# Profile generated by Karaf Assembly Builder
#

# Parent profiles
attribute.parents = generated-startup generated-boot generated-installed

# Attributes
attribute.overlay = true

# Feature XML repositories
repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
mvn:org.apache.karaf.features/framework/4.2.0/xml/features
repository.mvn\:org.apache.karaf.features/standard/4.2.0/xml/features =
mvn:org.apache.karaf.features/standard/4.2.0/xml/features
repository.mvn\:org.apache.karaf.features/spring/4.2.0/xml/features =
mvn:org.apache.karaf.features/spring/4.2.0/xml/features

# Features
feature.framework = framework
feature.jaas = jaas
feature.shell = shell
feature.feature = feature
feature.ssh = ssh
feature.bundle = bundle
feature.config = config
feature.log = log

However, with complex distros it can look like this (my distro) - see all
the blacklisted items and even special configuration options - these are
all generated from pom.xml:

#
# Profile generated by Karaf Assembly Builder
#

# Parent profiles
attribute.parents = generated-startup generated-boot generated-installed

# Attributes
attribute.overlay = true

# Feature XML repositories
repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
mvn:org.apache.karaf.features/framework/4.2.0/xml/features
...
# Features
feature.framework = framework
feature.patch-management = patch-management
...
feature.pax-jms-config = pax-jms-config
feature.pax-jms-pool-narayana = pax-jms-pool-narayana
feature.pax-jms-pool-transx = pax-jms-pool-transx

# Bundles
...
bundle.mvn\:org.bouncycastle/bcprov-jdk15on/1.60 =
mvn:org.bouncycastle/bcprov-jdk15on/1.60
bundle.mvn\:org.bouncycastle/bcpkix-jdk15on/1.60 =
mvn:org.bouncycastle/bcpkix-jdk15on/1.60

# Configuration properties for etc/config.properties
config.karaf.delay.console = true

# Blacklisted repositories
blacklisted.repository.mvn\:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
= mvn:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
...

# Blacklisted features
blacklisted.feature.httplite = httplite
blacklisted.feature.jetty/[8,9) = jetty/[8,9)
blacklisted.feature.pax-*jetty* = pax-*jetty*
blacklisted.feature.cxf-*-jetty = cxf-*-jetty
...

# Blacklisted bundles
blacklisted.bundle.mvn\:org.ops4j.pax.cdi/pax-cdi-jetty-weld =
mvn:org.ops4j.pax.cdi/pax-cdi-jetty-weld

This etc/profile.cfg can be treated simply as it was added in
karaf-maven-plugin:assembly configuration itself!

<configuration>
    <profilesUris>
        <uri>path/to/profile.cfg</uri>
    </profilesUris>
</configuration>

best regards
Grzegorz Grzybek

czw., 13 wrz 2018 o 13:51 Jean-Baptiste Onofré <[hidden email]> napisał(a):

> Hi guys,
>
> Recently, we received a lot of questions around how to create Karaf
> custom distribution based on karaf-maven-plugin, and how to use the
> static profile to create "standalone/static" distribution.
>
> If the plugin works fine, it's not easy to understand some "details",
> like the dependency scope impact, or providing the set of default
> features repos and features. I already helped users (in private
> communication) to fix their custom distributions.
>
> Obviously, we should simplify the way of creating custom distribution,
> especially with the new tooling & feature we now provide around Docker.
>
> I would like to propose the following:
>
> 1. Set the default behavior of the assembly goal to create a custom
> distribution based on standard. For the user, instead of providing
> (again) all framework, standard, enterprise features repos and all
> standard boot features (shell, ...), it will just specify the tar.gz/zip
> base and his own features repo/repos (or the goal will use the same
> version of the goal plugin itself). All the rest will be done by the
> plugin for him. Use the karaf packaging as default to define this. At
> the end of the day, the user pom.xml will look like:
>
> <project>
>         <groupId>foo</groupId>
>         <artifactId>bar</artifactId>
>         <version>1.0-SNAPSHOT</version>
>         <packaging>karaf</packaging>
>
>         <dependencies>
>                 <dependency>
>                         <groupId>org.apache.karaf</groupId>
>                         <artifactId>apache-karaf</artifactId>
>                         <version>4.2.1</version>
>                         <type>tar.gz</type>
>                 </dependency>
>                 <dependency>
>                         <groupId>foo</groupId>
>                         <artifactId>my</artifactId>
>                         <version>1.0-SNAPSHOT</version>
>                         <classifier>features</classifier>
>                         <type>xml</type>
>                 </dependency>
>         </dependencies>
>
>         <build>
>                 <plugins>
>                         <plugin>
>                                 <groupId>org.apache.karaf.tooling</groupId>
> <artifactId>karaf-maven-plugin</artifactId>
> <extensions>true</extensions>
> <inherited>true</inherited>
> <configuration>
> <bootFeatures>
>         <feature>my</feature>
> </bootFeatures>
> <installedFeatures>
>         <feature>my-other</feature>
> </installedFeatures>
> </configuration>
>                         </plugin>
>                 </plugins>
>         </build>
>
> </project>
>
> The idea is to automatically execute install-kar + assembly for the
> karaf packaging and let the user focus on its own resources (features,
> config, ...) just providing the base Karaf archive.
> The user will be able to use src/main/resources to provide any files in
> etc, bin, or whatever in the resulting custom distribution.
>
> 2. Improve a bit the features XML generation
> If the custom distribution is the highest priority, just after the
> improvements on this area, I would like to improve the way of creating
> features XML.
> Now, to be honest, almost all of us write features repos XML by hand. It
> gives us the maximum of flexibility. However, on the other hand, the
> features XML and code contain should be sync.
> I would like to improve the generate features MOJO, however leveraging
> most of all functionalities around features (prerequisites, dependency
> flag, inner features, ...).
>
> I have to dig a little bit around that, but if you want some ideas
> already, please let me know.
>
> Thoughts ?
>
> Regards
> JB
> --
> Jean-Baptiste Onofré
> [hidden email]
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

jbonofre
Thanks for your feedback Greg.

I started a new local branch with "new" karaf-maven-plugin custom
distribution approach. This branch also contains an example.

I will create a corresponding PR and an update on the mailing list to be
able to discuss all together.

Regards
JB

On 13/09/2018 14:18, Grzegorz Grzybek wrote:

> Hello
>
> Thanks JBO for the idea. True - custom distro generation isn't that
> intuitive as it could be. Personally I missed the flexibility, not
> simplicity, that's why I added <framework>custom</framework> in
> "[KARAF-5468] Cleaning up AssemblyMojo, Profiles and profile Builder".
>
> Without it, the implied value for "framework" property was "framework" or
> "static-framework" depending on whether you had dependency on
> mvn:org.apache.karaf.features/framework/VERSION/kar or
> mvn:org.apache.karaf.features/static/VERSION/kar in your POM. The
> discovered "framework" property was added as "startup feature" which was
> very confusing (mixing "framework" and "feature" concepts).
>
> I can't tell much about improvements for
> karaf-maven-plugin:features-generate-descriptor, because I always preferred
> manual creation of feature XMLs.
>
> However having dependency on existing karaf distro ("standard" apache-karaf
> or apache-karaf-minimal) is good idea! karaf-maven-plugin:assembly could
> search zip or tar.gz or tgz kind of dependencies (with "provided" or any
> other scope) and:
>  - unpack it
>  - check etc/profile.cfg
>
> etc/profile.cfg is (since 4.2.0) nicely written as (official
> apache-karaf-minimal distro):
>
> #
> # Profile generated by Karaf Assembly Builder
> #
>
> # Parent profiles
> attribute.parents = generated-startup generated-boot generated-installed
>
> # Attributes
> attribute.overlay = true
>
> # Feature XML repositories
> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
> repository.mvn\:org.apache.karaf.features/standard/4.2.0/xml/features =
> mvn:org.apache.karaf.features/standard/4.2.0/xml/features
> repository.mvn\:org.apache.karaf.features/spring/4.2.0/xml/features =
> mvn:org.apache.karaf.features/spring/4.2.0/xml/features
>
> # Features
> feature.framework = framework
> feature.jaas = jaas
> feature.shell = shell
> feature.feature = feature
> feature.ssh = ssh
> feature.bundle = bundle
> feature.config = config
> feature.log = log
>
> However, with complex distros it can look like this (my distro) - see all
> the blacklisted items and even special configuration options - these are
> all generated from pom.xml:
>
> #
> # Profile generated by Karaf Assembly Builder
> #
>
> # Parent profiles
> attribute.parents = generated-startup generated-boot generated-installed
>
> # Attributes
> attribute.overlay = true
>
> # Feature XML repositories
> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
> ...
> # Features
> feature.framework = framework
> feature.patch-management = patch-management
> ...
> feature.pax-jms-config = pax-jms-config
> feature.pax-jms-pool-narayana = pax-jms-pool-narayana
> feature.pax-jms-pool-transx = pax-jms-pool-transx
>
> # Bundles
> ...
> bundle.mvn\:org.bouncycastle/bcprov-jdk15on/1.60 =
> mvn:org.bouncycastle/bcprov-jdk15on/1.60
> bundle.mvn\:org.bouncycastle/bcpkix-jdk15on/1.60 =
> mvn:org.bouncycastle/bcpkix-jdk15on/1.60
>
> # Configuration properties for etc/config.properties
> config.karaf.delay.console = true
>
> # Blacklisted repositories
> blacklisted.repository.mvn\:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
> = mvn:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
> ...
>
> # Blacklisted features
> blacklisted.feature.httplite = httplite
> blacklisted.feature.jetty/[8,9) = jetty/[8,9)
> blacklisted.feature.pax-*jetty* = pax-*jetty*
> blacklisted.feature.cxf-*-jetty = cxf-*-jetty
> ...
>
> # Blacklisted bundles
> blacklisted.bundle.mvn\:org.ops4j.pax.cdi/pax-cdi-jetty-weld =
> mvn:org.ops4j.pax.cdi/pax-cdi-jetty-weld
>
> This etc/profile.cfg can be treated simply as it was added in
> karaf-maven-plugin:assembly configuration itself!
>
> <configuration>
>     <profilesUris>
>         <uri>path/to/profile.cfg</uri>
>     </profilesUris>
> </configuration>
>
> best regards
> Grzegorz Grzybek
>
> czw., 13 wrz 2018 o 13:51 Jean-Baptiste Onofré <[hidden email]> napisał(a):
>
>> Hi guys,
>>
>> Recently, we received a lot of questions around how to create Karaf
>> custom distribution based on karaf-maven-plugin, and how to use the
>> static profile to create "standalone/static" distribution.
>>
>> If the plugin works fine, it's not easy to understand some "details",
>> like the dependency scope impact, or providing the set of default
>> features repos and features. I already helped users (in private
>> communication) to fix their custom distributions.
>>
>> Obviously, we should simplify the way of creating custom distribution,
>> especially with the new tooling & feature we now provide around Docker.
>>
>> I would like to propose the following:
>>
>> 1. Set the default behavior of the assembly goal to create a custom
>> distribution based on standard. For the user, instead of providing
>> (again) all framework, standard, enterprise features repos and all
>> standard boot features (shell, ...), it will just specify the tar.gz/zip
>> base and his own features repo/repos (or the goal will use the same
>> version of the goal plugin itself). All the rest will be done by the
>> plugin for him. Use the karaf packaging as default to define this. At
>> the end of the day, the user pom.xml will look like:
>>
>> <project>
>>         <groupId>foo</groupId>
>>         <artifactId>bar</artifactId>
>>         <version>1.0-SNAPSHOT</version>
>>         <packaging>karaf</packaging>
>>
>>         <dependencies>
>>                 <dependency>
>>                         <groupId>org.apache.karaf</groupId>
>>                         <artifactId>apache-karaf</artifactId>
>>                         <version>4.2.1</version>
>>                         <type>tar.gz</type>
>>                 </dependency>
>>                 <dependency>
>>                         <groupId>foo</groupId>
>>                         <artifactId>my</artifactId>
>>                         <version>1.0-SNAPSHOT</version>
>>                         <classifier>features</classifier>
>>                         <type>xml</type>
>>                 </dependency>
>>         </dependencies>
>>
>>         <build>
>>                 <plugins>
>>                         <plugin>
>>                                 <groupId>org.apache.karaf.tooling</groupId>
>> <artifactId>karaf-maven-plugin</artifactId>
>> <extensions>true</extensions>
>> <inherited>true</inherited>
>> <configuration>
>> <bootFeatures>
>>         <feature>my</feature>
>> </bootFeatures>
>> <installedFeatures>
>>         <feature>my-other</feature>
>> </installedFeatures>
>> </configuration>
>>                         </plugin>
>>                 </plugins>
>>         </build>
>>
>> </project>
>>
>> The idea is to automatically execute install-kar + assembly for the
>> karaf packaging and let the user focus on its own resources (features,
>> config, ...) just providing the base Karaf archive.
>> The user will be able to use src/main/resources to provide any files in
>> etc, bin, or whatever in the resulting custom distribution.
>>
>> 2. Improve a bit the features XML generation
>> If the custom distribution is the highest priority, just after the
>> improvements on this area, I would like to improve the way of creating
>> features XML.
>> Now, to be honest, almost all of us write features repos XML by hand. It
>> gives us the maximum of flexibility. However, on the other hand, the
>> features XML and code contain should be sync.
>> I would like to improve the generate features MOJO, however leveraging
>> most of all functionalities around features (prerequisites, dependency
>> flag, inner features, ...).
>>
>> I have to dig a little bit around that, but if you want some ideas
>> already, please let me know.
>>
>> Thoughts ?
>>
>> Regards
>> JB
>> --
>> Jean-Baptiste Onofré
>> [hidden email]
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

jbonofre
In reply to this post by fpapon
Hi,

that's a valid point. What do you think if the karaf-maven-plugin tries
to do a merge ?

I'm also thinking about supporting git repository for resources storage.

Regards
JB

On 13/09/2018 14:15, Francois Papon wrote:

> Hi JB,
>
> I agree about this, we have to focus on the user friendly stuff and I
> have an additional point :
>
>   * Add a simple way to update/add properties in the default
>     configuration files of the standard distribution in the assembly
>     (like system.properties....)
>
> I'm not sure this is the correct discussion but I think we also have to
> work about helping user to upgrade the Karaf version of their custom
> distributions.
>
> regards,
>
> François Papon
> [hidden email]
>
> Le 13/09/2018 à 15:51, Jean-Baptiste Onofré a écrit :
>> Hi guys,
>>
>> Recently, we received a lot of questions around how to create Karaf
>> custom distribution based on karaf-maven-plugin, and how to use the
>> static profile to create "standalone/static" distribution.
>>
>> If the plugin works fine, it's not easy to understand some "details",
>> like the dependency scope impact, or providing the set of default
>> features repos and features. I already helped users (in private
>> communication) to fix their custom distributions.
>>
>> Obviously, we should simplify the way of creating custom distribution,
>> especially with the new tooling & feature we now provide around Docker.
>>
>> I would like to propose the following:
>>
>> 1. Set the default behavior of the assembly goal to create a custom
>> distribution based on standard. For the user, instead of providing
>> (again) all framework, standard, enterprise features repos and all
>> standard boot features (shell, ...), it will just specify the tar.gz/zip
>> base and his own features repo/repos (or the goal will use the same
>> version of the goal plugin itself). All the rest will be done by the
>> plugin for him. Use the karaf packaging as default to define this. At
>> the end of the day, the user pom.xml will look like:
>>
>> <project>
>> <groupId>foo</groupId>
>> <artifactId>bar</artifactId>
>> <version>1.0-SNAPSHOT</version>
>> <packaging>karaf</packaging>
>>
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.karaf</groupId>
>> <artifactId>apache-karaf</artifactId>
>> <version>4.2.1</version>
>> <type>tar.gz</type>
>> </dependency>
>> <dependency>
>> <groupId>foo</groupId>
>> <artifactId>my</artifactId>
>> <version>1.0-SNAPSHOT</version>
>> <classifier>features</classifier>
>> <type>xml</type>
>> </dependency>
>> </dependencies>
>>
>> <build>
>> <plugins>
>> <plugin>
>> <groupId>org.apache.karaf.tooling</groupId>
>> <artifactId>karaf-maven-plugin</artifactId>
>> <extensions>true</extensions>
>> <inherited>true</inherited>
>> <configuration>
>> <bootFeatures>
>> <feature>my</feature>
>> </bootFeatures>
>> <installedFeatures>
>> <feature>my-other</feature>
>> </installedFeatures>
>> </configuration>
>> </plugin>
>> </plugins>
>> </build>
>>
>> </project>
>>
>> The idea is to automatically execute install-kar + assembly for the
>> karaf packaging and let the user focus on its own resources (features,
>> config, ...) just providing the base Karaf archive.
>> The user will be able to use src/main/resources to provide any files in
>> etc, bin, or whatever in the resulting custom distribution.
>>
>> 2. Improve a bit the features XML generation
>> If the custom distribution is the highest priority, just after the
>> improvements on this area, I would like to improve the way of creating
>> features XML.
>> Now, to be honest, almost all of us write features repos XML by hand. It
>> gives us the maximum of flexibility. However, on the other hand, the
>> features XML and code contain should be sync.
>> I would like to improve the generate features MOJO, however leveraging
>> most of all functionalities around features (prerequisites, dependency
>> flag, inner features, ...).
>>
>> I have to dig a little bit around that, but if you want some ideas
>> already, please let me know.
>>
>> Thoughts ?
>>
>> Regards
>> JB
>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

fpapon
Nice idea for using git :)
It would be usefull for the user because he doesn't have to manualy merge when upgrading the main Karaf version.
regards,
Francois 
Envoyé depuis mon smartphone Samsung Galaxy.
-------- Message d'origine --------De : Jean-Baptiste Onofré <[hidden email]> Date : 13/09/2018  18:53  (GMT+04:00) À : [hidden email] Objet : Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create
  custom distribution
Hi,

that's a valid point. What do you think if the karaf-maven-plugin tries
to do a merge ?

I'm also thinking about supporting git repository for resources storage.

Regards
JB

On 13/09/2018 14:15, Francois Papon wrote:

> Hi JB,
>
> I agree about this, we have to focus on the user friendly stuff and I
> have an additional point :
>
>   * Add a simple way to update/add properties in the default
>     configuration files of the standard distribution in the assembly
>     (like system.properties....)
>
> I'm not sure this is the correct discussion but I think we also have to
> work about helping user to upgrade the Karaf version of their custom
> distributions.
>
> regards,
>
> François Papon
> [hidden email]
>
> Le 13/09/2018 à 15:51, Jean-Baptiste Onofré a écrit :
>> Hi guys,
>>
>> Recently, we received a lot of questions around how to create Karaf
>> custom distribution based on karaf-maven-plugin, and how to use the
>> static profile to create "standalone/static" distribution.
>>
>> If the plugin works fine, it's not easy to understand some "details",
>> like the dependency scope impact, or providing the set of default
>> features repos and features. I already helped users (in private
>> communication) to fix their custom distributions.
>>
>> Obviously, we should simplify the way of creating custom distribution,
>> especially with the new tooling & feature we now provide around Docker.
>>
>> I would like to propose the following:
>>
>> 1. Set the default behavior of the assembly goal to create a custom
>> distribution based on standard. For the user, instead of providing
>> (again) all framework, standard, enterprise features repos and all
>> standard boot features (shell, ...), it will just specify the tar.gz/zip
>> base and his own features repo/repos (or the goal will use the same
>> version of the goal plugin itself). All the rest will be done by the
>> plugin for him. Use the karaf packaging as default to define this. At
>> the end of the day, the user pom.xml will look like:
>>
>> <project>
>> <groupId>foo</groupId>
>> <artifactId>bar</artifactId>
>> <version>1.0-SNAPSHOT</version>
>> <packaging>karaf</packaging>
>>
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.karaf</groupId>
>> <artifactId>apache-karaf</artifactId>
>> <version>4.2.1</version>
>> <type>tar.gz</type>
>> </dependency>
>> <dependency>
>> <groupId>foo</groupId>
>> <artifactId>my</artifactId>
>> <version>1.0-SNAPSHOT</version>
>> <classifier>features</classifier>
>> <type>xml</type>
>> </dependency>
>> </dependencies>
>>
>> <build>
>> <plugins>
>> <plugin>
>> <groupId>org.apache.karaf.tooling</groupId>
>> <artifactId>karaf-maven-plugin</artifactId>
>> <extensions>true</extensions>
>> <inherited>true</inherited>
>> <configuration>
>> <bootFeatures>
>> <feature>my</feature>
>> </bootFeatures>
>> <installedFeatures>
>> <feature>my-other</feature>
>> </installedFeatures>
>> </configuration>
>> </plugin>
>> </plugins>
>> </build>
>>
>> </project>
>>
>> The idea is to automatically execute install-kar + assembly for the
>> karaf packaging and let the user focus on its own resources (features,
>> config, ...) just providing the base Karaf archive.
>> The user will be able to use src/main/resources to provide any files in
>> etc, bin, or whatever in the resulting custom distribution.
>>
>> 2. Improve a bit the features XML generation
>> If the custom distribution is the highest priority, just after the
>> improvements on this area, I would like to improve the way of creating
>> features XML.
>> Now, to be honest, almost all of us write features repos XML by hand. It
>> gives us the maximum of flexibility. However, on the other hand, the
>> features XML and code contain should be sync.
>> I would like to improve the generate features MOJO, however leveraging
>> most of all functionalities around features (prerequisites, dependency
>> flag, inner features, ...).
>>
>> I have to dig a little bit around that, but if you want some ideas
>> already, please let me know.
>>
>> Thoughts ?
>>
>> Regards
>> JB
>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
François Papon
fpapon@apache.org
Yupiik - https://www.yupiik.com
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

jbonofre
In reply to this post by Grzegorz Grzybek
For the feature generation, I have two ideas that I would like to PoC:

- use a CLI/interactive mode. For instance, if the user runs mvn
karaf:feature-generate, the MOJO will ask some questions. The purpose is
to "abstract" some concepts to simplify for the users.
- additionally, we can imagine to have template (feature-template.xml or
a yaml). The user can provide a template instead of answering questions
(a kind of response file).
- this could be used for bridge the gap with bndtools as well

I would like to PoC this.

Regards
JB

On 13/09/2018 14:18, Grzegorz Grzybek wrote:

> Hello
>
> Thanks JBO for the idea. True - custom distro generation isn't that
> intuitive as it could be. Personally I missed the flexibility, not
> simplicity, that's why I added <framework>custom</framework> in
> "[KARAF-5468] Cleaning up AssemblyMojo, Profiles and profile Builder".
>
> Without it, the implied value for "framework" property was "framework" or
> "static-framework" depending on whether you had dependency on
> mvn:org.apache.karaf.features/framework/VERSION/kar or
> mvn:org.apache.karaf.features/static/VERSION/kar in your POM. The
> discovered "framework" property was added as "startup feature" which was
> very confusing (mixing "framework" and "feature" concepts).
>
> I can't tell much about improvements for
> karaf-maven-plugin:features-generate-descriptor, because I always preferred
> manual creation of feature XMLs.
>
> However having dependency on existing karaf distro ("standard" apache-karaf
> or apache-karaf-minimal) is good idea! karaf-maven-plugin:assembly could
> search zip or tar.gz or tgz kind of dependencies (with "provided" or any
> other scope) and:
>   - unpack it
>   - check etc/profile.cfg
>
> etc/profile.cfg is (since 4.2.0) nicely written as (official
> apache-karaf-minimal distro):
>
> #
> # Profile generated by Karaf Assembly Builder
> #
>
> # Parent profiles
> attribute.parents = generated-startup generated-boot generated-installed
>
> # Attributes
> attribute.overlay = true
>
> # Feature XML repositories
> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
> repository.mvn\:org.apache.karaf.features/standard/4.2.0/xml/features =
> mvn:org.apache.karaf.features/standard/4.2.0/xml/features
> repository.mvn\:org.apache.karaf.features/spring/4.2.0/xml/features =
> mvn:org.apache.karaf.features/spring/4.2.0/xml/features
>
> # Features
> feature.framework = framework
> feature.jaas = jaas
> feature.shell = shell
> feature.feature = feature
> feature.ssh = ssh
> feature.bundle = bundle
> feature.config = config
> feature.log = log
>
> However, with complex distros it can look like this (my distro) - see all
> the blacklisted items and even special configuration options - these are
> all generated from pom.xml:
>
> #
> # Profile generated by Karaf Assembly Builder
> #
>
> # Parent profiles
> attribute.parents = generated-startup generated-boot generated-installed
>
> # Attributes
> attribute.overlay = true
>
> # Feature XML repositories
> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
> ...
> # Features
> feature.framework = framework
> feature.patch-management = patch-management
> ...
> feature.pax-jms-config = pax-jms-config
> feature.pax-jms-pool-narayana = pax-jms-pool-narayana
> feature.pax-jms-pool-transx = pax-jms-pool-transx
>
> # Bundles
> ...
> bundle.mvn\:org.bouncycastle/bcprov-jdk15on/1.60 =
> mvn:org.bouncycastle/bcprov-jdk15on/1.60
> bundle.mvn\:org.bouncycastle/bcpkix-jdk15on/1.60 =
> mvn:org.bouncycastle/bcpkix-jdk15on/1.60
>
> # Configuration properties for etc/config.properties
> config.karaf.delay.console = true
>
> # Blacklisted repositories
> blacklisted.repository.mvn\:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
> = mvn:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
> ...
>
> # Blacklisted features
> blacklisted.feature.httplite = httplite
> blacklisted.feature.jetty/[8,9) = jetty/[8,9)
> blacklisted.feature.pax-*jetty* = pax-*jetty*
> blacklisted.feature.cxf-*-jetty = cxf-*-jetty
> ...
>
> # Blacklisted bundles
> blacklisted.bundle.mvn\:org.ops4j.pax.cdi/pax-cdi-jetty-weld =
> mvn:org.ops4j.pax.cdi/pax-cdi-jetty-weld
>
> This etc/profile.cfg can be treated simply as it was added in
> karaf-maven-plugin:assembly configuration itself!
>
> <configuration>
>      <profilesUris>
>          <uri>path/to/profile.cfg</uri>
>      </profilesUris>
> </configuration>
>
> best regards
> Grzegorz Grzybek
>
> czw., 13 wrz 2018 o 13:51 Jean-Baptiste Onofré <[hidden email]> napisał(a):
>
>> Hi guys,
>>
>> Recently, we received a lot of questions around how to create Karaf
>> custom distribution based on karaf-maven-plugin, and how to use the
>> static profile to create "standalone/static" distribution.
>>
>> If the plugin works fine, it's not easy to understand some "details",
>> like the dependency scope impact, or providing the set of default
>> features repos and features. I already helped users (in private
>> communication) to fix their custom distributions.
>>
>> Obviously, we should simplify the way of creating custom distribution,
>> especially with the new tooling & feature we now provide around Docker.
>>
>> I would like to propose the following:
>>
>> 1. Set the default behavior of the assembly goal to create a custom
>> distribution based on standard. For the user, instead of providing
>> (again) all framework, standard, enterprise features repos and all
>> standard boot features (shell, ...), it will just specify the tar.gz/zip
>> base and his own features repo/repos (or the goal will use the same
>> version of the goal plugin itself). All the rest will be done by the
>> plugin for him. Use the karaf packaging as default to define this. At
>> the end of the day, the user pom.xml will look like:
>>
>> <project>
>>          <groupId>foo</groupId>
>>          <artifactId>bar</artifactId>
>>          <version>1.0-SNAPSHOT</version>
>>          <packaging>karaf</packaging>
>>
>>          <dependencies>
>>                  <dependency>
>>                          <groupId>org.apache.karaf</groupId>
>>                          <artifactId>apache-karaf</artifactId>
>>                          <version>4.2.1</version>
>>                          <type>tar.gz</type>
>>                  </dependency>
>>                  <dependency>
>>                          <groupId>foo</groupId>
>>                          <artifactId>my</artifactId>
>>                          <version>1.0-SNAPSHOT</version>
>>                          <classifier>features</classifier>
>>                          <type>xml</type>
>>                  </dependency>
>>          </dependencies>
>>
>>          <build>
>>                  <plugins>
>>                          <plugin>
>>                                  <groupId>org.apache.karaf.tooling</groupId>
>> <artifactId>karaf-maven-plugin</artifactId>
>> <extensions>true</extensions>
>> <inherited>true</inherited>
>> <configuration>
>> <bootFeatures>
>>          <feature>my</feature>
>> </bootFeatures>
>> <installedFeatures>
>>          <feature>my-other</feature>
>> </installedFeatures>
>> </configuration>
>>                          </plugin>
>>                  </plugins>
>>          </build>
>>
>> </project>
>>
>> The idea is to automatically execute install-kar + assembly for the
>> karaf packaging and let the user focus on its own resources (features,
>> config, ...) just providing the base Karaf archive.
>> The user will be able to use src/main/resources to provide any files in
>> etc, bin, or whatever in the resulting custom distribution.
>>
>> 2. Improve a bit the features XML generation
>> If the custom distribution is the highest priority, just after the
>> improvements on this area, I would like to improve the way of creating
>> features XML.
>> Now, to be honest, almost all of us write features repos XML by hand. It
>> gives us the maximum of flexibility. However, on the other hand, the
>> features XML and code contain should be sync.
>> I would like to improve the generate features MOJO, however leveraging
>> most of all functionalities around features (prerequisites, dependency
>> flag, inner features, ...).
>>
>> I have to dig a little bit around that, but if you want some ideas
>> already, please let me know.
>>
>> Thoughts ?
>>
>> Regards
>> JB
>> --
>> Jean-Baptiste Onofré
>> [hidden email]
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

fpapon
+1 !

regards

François Papon
[hidden email]

Le 13/09/2018 à 19:18, Jean-Baptiste Onofré a écrit :

> For the feature generation, I have two ideas that I would like to PoC:
>
> - use a CLI/interactive mode. For instance, if the user runs mvn
> karaf:feature-generate, the MOJO will ask some questions. The purpose
> is to "abstract" some concepts to simplify for the users.
> - additionally, we can imagine to have template (feature-template.xml
> or a yaml). The user can provide a template instead of answering
> questions (a kind of response file).
> - this could be used for bridge the gap with bndtools as well
>
> I would like to PoC this.
>
> Regards
> JB
>
> On 13/09/2018 14:18, Grzegorz Grzybek wrote:
>> Hello
>>
>> Thanks JBO for the idea. True - custom distro generation isn't that
>> intuitive as it could be. Personally I missed the flexibility, not
>> simplicity, that's why I added <framework>custom</framework> in
>> "[KARAF-5468] Cleaning up AssemblyMojo, Profiles and profile Builder".
>>
>> Without it, the implied value for "framework" property was
>> "framework" or
>> "static-framework" depending on whether you had dependency on
>> mvn:org.apache.karaf.features/framework/VERSION/kar or
>> mvn:org.apache.karaf.features/static/VERSION/kar in your POM. The
>> discovered "framework" property was added as "startup feature" which was
>> very confusing (mixing "framework" and "feature" concepts).
>>
>> I can't tell much about improvements for
>> karaf-maven-plugin:features-generate-descriptor, because I always
>> preferred
>> manual creation of feature XMLs.
>>
>> However having dependency on existing karaf distro ("standard"
>> apache-karaf
>> or apache-karaf-minimal) is good idea! karaf-maven-plugin:assembly could
>> search zip or tar.gz or tgz kind of dependencies (with "provided" or any
>> other scope) and:
>>   - unpack it
>>   - check etc/profile.cfg
>>
>> etc/profile.cfg is (since 4.2.0) nicely written as (official
>> apache-karaf-minimal distro):
>>
>> #
>> # Profile generated by Karaf Assembly Builder
>> #
>>
>> # Parent profiles
>> attribute.parents = generated-startup generated-boot generated-installed
>>
>> # Attributes
>> attribute.overlay = true
>>
>> # Feature XML repositories
>> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
>> repository.mvn\:org.apache.karaf.features/standard/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/standard/4.2.0/xml/features
>> repository.mvn\:org.apache.karaf.features/spring/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/spring/4.2.0/xml/features
>>
>> # Features
>> feature.framework = framework
>> feature.jaas = jaas
>> feature.shell = shell
>> feature.feature = feature
>> feature.ssh = ssh
>> feature.bundle = bundle
>> feature.config = config
>> feature.log = log
>>
>> However, with complex distros it can look like this (my distro) - see
>> all
>> the blacklisted items and even special configuration options - these are
>> all generated from pom.xml:
>>
>> #
>> # Profile generated by Karaf Assembly Builder
>> #
>>
>> # Parent profiles
>> attribute.parents = generated-startup generated-boot generated-installed
>>
>> # Attributes
>> attribute.overlay = true
>>
>> # Feature XML repositories
>> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
>> ...
>> # Features
>> feature.framework = framework
>> feature.patch-management = patch-management
>> ...
>> feature.pax-jms-config = pax-jms-config
>> feature.pax-jms-pool-narayana = pax-jms-pool-narayana
>> feature.pax-jms-pool-transx = pax-jms-pool-transx
>>
>> # Bundles
>> ...
>> bundle.mvn\:org.bouncycastle/bcprov-jdk15on/1.60 =
>> mvn:org.bouncycastle/bcprov-jdk15on/1.60
>> bundle.mvn\:org.bouncycastle/bcpkix-jdk15on/1.60 =
>> mvn:org.bouncycastle/bcpkix-jdk15on/1.60
>>
>> # Configuration properties for etc/config.properties
>> config.karaf.delay.console = true
>>
>> # Blacklisted repositories
>> blacklisted.repository.mvn\:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
>>
>> = mvn:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
>> ...
>>
>> # Blacklisted features
>> blacklisted.feature.httplite = httplite
>> blacklisted.feature.jetty/[8,9) = jetty/[8,9)
>> blacklisted.feature.pax-*jetty* = pax-*jetty*
>> blacklisted.feature.cxf-*-jetty = cxf-*-jetty
>> ...
>>
>> # Blacklisted bundles
>> blacklisted.bundle.mvn\:org.ops4j.pax.cdi/pax-cdi-jetty-weld =
>> mvn:org.ops4j.pax.cdi/pax-cdi-jetty-weld
>>
>> This etc/profile.cfg can be treated simply as it was added in
>> karaf-maven-plugin:assembly configuration itself!
>>
>> <configuration>
>>      <profilesUris>
>>          <uri>path/to/profile.cfg</uri>
>>      </profilesUris>
>> </configuration>
>>
>> best regards
>> Grzegorz Grzybek
>>
>> czw., 13 wrz 2018 o 13:51 Jean-Baptiste Onofré <[hidden email]>
>> napisał(a):
>>
>>> Hi guys,
>>>
>>> Recently, we received a lot of questions around how to create Karaf
>>> custom distribution based on karaf-maven-plugin, and how to use the
>>> static profile to create "standalone/static" distribution.
>>>
>>> If the plugin works fine, it's not easy to understand some "details",
>>> like the dependency scope impact, or providing the set of default
>>> features repos and features. I already helped users (in private
>>> communication) to fix their custom distributions.
>>>
>>> Obviously, we should simplify the way of creating custom distribution,
>>> especially with the new tooling & feature we now provide around Docker.
>>>
>>> I would like to propose the following:
>>>
>>> 1. Set the default behavior of the assembly goal to create a custom
>>> distribution based on standard. For the user, instead of providing
>>> (again) all framework, standard, enterprise features repos and all
>>> standard boot features (shell, ...), it will just specify the
>>> tar.gz/zip
>>> base and his own features repo/repos (or the goal will use the same
>>> version of the goal plugin itself). All the rest will be done by the
>>> plugin for him. Use the karaf packaging as default to define this. At
>>> the end of the day, the user pom.xml will look like:
>>>
>>> <project>
>>>          <groupId>foo</groupId>
>>>          <artifactId>bar</artifactId>
>>>          <version>1.0-SNAPSHOT</version>
>>>          <packaging>karaf</packaging>
>>>
>>>          <dependencies>
>>>                  <dependency>
>>>                          <groupId>org.apache.karaf</groupId>
>>>                          <artifactId>apache-karaf</artifactId>
>>>                          <version>4.2.1</version>
>>>                          <type>tar.gz</type>
>>>                  </dependency>
>>>                  <dependency>
>>>                          <groupId>foo</groupId>
>>>                          <artifactId>my</artifactId>
>>>                          <version>1.0-SNAPSHOT</version>
>>>                          <classifier>features</classifier>
>>>                          <type>xml</type>
>>>                  </dependency>
>>>          </dependencies>
>>>
>>>          <build>
>>>                  <plugins>
>>>                          <plugin>
>>>                                 
>>> <groupId>org.apache.karaf.tooling</groupId>
>>> <artifactId>karaf-maven-plugin</artifactId>
>>> <extensions>true</extensions>
>>> <inherited>true</inherited>
>>> <configuration>
>>> <bootFeatures>
>>>          <feature>my</feature>
>>> </bootFeatures>
>>> <installedFeatures>
>>>          <feature>my-other</feature>
>>> </installedFeatures>
>>> </configuration>
>>>                          </plugin>
>>>                  </plugins>
>>>          </build>
>>>
>>> </project>
>>>
>>> The idea is to automatically execute install-kar + assembly for the
>>> karaf packaging and let the user focus on its own resources (features,
>>> config, ...) just providing the base Karaf archive.
>>> The user will be able to use src/main/resources to provide any files in
>>> etc, bin, or whatever in the resulting custom distribution.
>>>
>>> 2. Improve a bit the features XML generation
>>> If the custom distribution is the highest priority, just after the
>>> improvements on this area, I would like to improve the way of creating
>>> features XML.
>>> Now, to be honest, almost all of us write features repos XML by
>>> hand. It
>>> gives us the maximum of flexibility. However, on the other hand, the
>>> features XML and code contain should be sync.
>>> I would like to improve the generate features MOJO, however leveraging
>>> most of all functionalities around features (prerequisites, dependency
>>> flag, inner features, ...).
>>>
>>> I have to dig a little bit around that, but if you want some ideas
>>> already, please let me know.
>>>
>>> Thoughts ?
>>>
>>> Regards
>>> JB
>>> --
>>> Jean-Baptiste Onofré
>>> [hidden email]
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>

François Papon
fpapon@apache.org
Yupiik - https://www.yupiik.com
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

fpapon
In reply to this post by jbonofre
For the template, we can share some templates on the Karaf website like
the feature XSD files ?

For example, a template to create a api rest feature based on Apache CXF.

regards,

François Papon
[hidden email]

Le 13/09/2018 à 19:18, Jean-Baptiste Onofré a écrit :

> For the feature generation, I have two ideas that I would like to PoC:
>
> - use a CLI/interactive mode. For instance, if the user runs mvn
> karaf:feature-generate, the MOJO will ask some questions. The purpose
> is to "abstract" some concepts to simplify for the users.
> - additionally, we can imagine to have template (feature-template.xml
> or a yaml). The user can provide a template instead of answering
> questions (a kind of response file).
> - this could be used for bridge the gap with bndtools as well
>
> I would like to PoC this.
>
> Regards
> JB
>
> On 13/09/2018 14:18, Grzegorz Grzybek wrote:
>> Hello
>>
>> Thanks JBO for the idea. True - custom distro generation isn't that
>> intuitive as it could be. Personally I missed the flexibility, not
>> simplicity, that's why I added <framework>custom</framework> in
>> "[KARAF-5468] Cleaning up AssemblyMojo, Profiles and profile Builder".
>>
>> Without it, the implied value for "framework" property was
>> "framework" or
>> "static-framework" depending on whether you had dependency on
>> mvn:org.apache.karaf.features/framework/VERSION/kar or
>> mvn:org.apache.karaf.features/static/VERSION/kar in your POM. The
>> discovered "framework" property was added as "startup feature" which was
>> very confusing (mixing "framework" and "feature" concepts).
>>
>> I can't tell much about improvements for
>> karaf-maven-plugin:features-generate-descriptor, because I always
>> preferred
>> manual creation of feature XMLs.
>>
>> However having dependency on existing karaf distro ("standard"
>> apache-karaf
>> or apache-karaf-minimal) is good idea! karaf-maven-plugin:assembly could
>> search zip or tar.gz or tgz kind of dependencies (with "provided" or any
>> other scope) and:
>>   - unpack it
>>   - check etc/profile.cfg
>>
>> etc/profile.cfg is (since 4.2.0) nicely written as (official
>> apache-karaf-minimal distro):
>>
>> #
>> # Profile generated by Karaf Assembly Builder
>> #
>>
>> # Parent profiles
>> attribute.parents = generated-startup generated-boot generated-installed
>>
>> # Attributes
>> attribute.overlay = true
>>
>> # Feature XML repositories
>> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
>> repository.mvn\:org.apache.karaf.features/standard/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/standard/4.2.0/xml/features
>> repository.mvn\:org.apache.karaf.features/spring/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/spring/4.2.0/xml/features
>>
>> # Features
>> feature.framework = framework
>> feature.jaas = jaas
>> feature.shell = shell
>> feature.feature = feature
>> feature.ssh = ssh
>> feature.bundle = bundle
>> feature.config = config
>> feature.log = log
>>
>> However, with complex distros it can look like this (my distro) - see
>> all
>> the blacklisted items and even special configuration options - these are
>> all generated from pom.xml:
>>
>> #
>> # Profile generated by Karaf Assembly Builder
>> #
>>
>> # Parent profiles
>> attribute.parents = generated-startup generated-boot generated-installed
>>
>> # Attributes
>> attribute.overlay = true
>>
>> # Feature XML repositories
>> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
>> ...
>> # Features
>> feature.framework = framework
>> feature.patch-management = patch-management
>> ...
>> feature.pax-jms-config = pax-jms-config
>> feature.pax-jms-pool-narayana = pax-jms-pool-narayana
>> feature.pax-jms-pool-transx = pax-jms-pool-transx
>>
>> # Bundles
>> ...
>> bundle.mvn\:org.bouncycastle/bcprov-jdk15on/1.60 =
>> mvn:org.bouncycastle/bcprov-jdk15on/1.60
>> bundle.mvn\:org.bouncycastle/bcpkix-jdk15on/1.60 =
>> mvn:org.bouncycastle/bcpkix-jdk15on/1.60
>>
>> # Configuration properties for etc/config.properties
>> config.karaf.delay.console = true
>>
>> # Blacklisted repositories
>> blacklisted.repository.mvn\:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
>>
>> = mvn:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
>> ...
>>
>> # Blacklisted features
>> blacklisted.feature.httplite = httplite
>> blacklisted.feature.jetty/[8,9) = jetty/[8,9)
>> blacklisted.feature.pax-*jetty* = pax-*jetty*
>> blacklisted.feature.cxf-*-jetty = cxf-*-jetty
>> ...
>>
>> # Blacklisted bundles
>> blacklisted.bundle.mvn\:org.ops4j.pax.cdi/pax-cdi-jetty-weld =
>> mvn:org.ops4j.pax.cdi/pax-cdi-jetty-weld
>>
>> This etc/profile.cfg can be treated simply as it was added in
>> karaf-maven-plugin:assembly configuration itself!
>>
>> <configuration>
>>      <profilesUris>
>>          <uri>path/to/profile.cfg</uri>
>>      </profilesUris>
>> </configuration>
>>
>> best regards
>> Grzegorz Grzybek
>>
>> czw., 13 wrz 2018 o 13:51 Jean-Baptiste Onofré <[hidden email]>
>> napisał(a):
>>
>>> Hi guys,
>>>
>>> Recently, we received a lot of questions around how to create Karaf
>>> custom distribution based on karaf-maven-plugin, and how to use the
>>> static profile to create "standalone/static" distribution.
>>>
>>> If the plugin works fine, it's not easy to understand some "details",
>>> like the dependency scope impact, or providing the set of default
>>> features repos and features. I already helped users (in private
>>> communication) to fix their custom distributions.
>>>
>>> Obviously, we should simplify the way of creating custom distribution,
>>> especially with the new tooling & feature we now provide around Docker.
>>>
>>> I would like to propose the following:
>>>
>>> 1. Set the default behavior of the assembly goal to create a custom
>>> distribution based on standard. For the user, instead of providing
>>> (again) all framework, standard, enterprise features repos and all
>>> standard boot features (shell, ...), it will just specify the
>>> tar.gz/zip
>>> base and his own features repo/repos (or the goal will use the same
>>> version of the goal plugin itself). All the rest will be done by the
>>> plugin for him. Use the karaf packaging as default to define this. At
>>> the end of the day, the user pom.xml will look like:
>>>
>>> <project>
>>>          <groupId>foo</groupId>
>>>          <artifactId>bar</artifactId>
>>>          <version>1.0-SNAPSHOT</version>
>>>          <packaging>karaf</packaging>
>>>
>>>          <dependencies>
>>>                  <dependency>
>>>                          <groupId>org.apache.karaf</groupId>
>>>                          <artifactId>apache-karaf</artifactId>
>>>                          <version>4.2.1</version>
>>>                          <type>tar.gz</type>
>>>                  </dependency>
>>>                  <dependency>
>>>                          <groupId>foo</groupId>
>>>                          <artifactId>my</artifactId>
>>>                          <version>1.0-SNAPSHOT</version>
>>>                          <classifier>features</classifier>
>>>                          <type>xml</type>
>>>                  </dependency>
>>>          </dependencies>
>>>
>>>          <build>
>>>                  <plugins>
>>>                          <plugin>
>>>                                 
>>> <groupId>org.apache.karaf.tooling</groupId>
>>> <artifactId>karaf-maven-plugin</artifactId>
>>> <extensions>true</extensions>
>>> <inherited>true</inherited>
>>> <configuration>
>>> <bootFeatures>
>>>          <feature>my</feature>
>>> </bootFeatures>
>>> <installedFeatures>
>>>          <feature>my-other</feature>
>>> </installedFeatures>
>>> </configuration>
>>>                          </plugin>
>>>                  </plugins>
>>>          </build>
>>>
>>> </project>
>>>
>>> The idea is to automatically execute install-kar + assembly for the
>>> karaf packaging and let the user focus on its own resources (features,
>>> config, ...) just providing the base Karaf archive.
>>> The user will be able to use src/main/resources to provide any files in
>>> etc, bin, or whatever in the resulting custom distribution.
>>>
>>> 2. Improve a bit the features XML generation
>>> If the custom distribution is the highest priority, just after the
>>> improvements on this area, I would like to improve the way of creating
>>> features XML.
>>> Now, to be honest, almost all of us write features repos XML by
>>> hand. It
>>> gives us the maximum of flexibility. However, on the other hand, the
>>> features XML and code contain should be sync.
>>> I would like to improve the generate features MOJO, however leveraging
>>> most of all functionalities around features (prerequisites, dependency
>>> flag, inner features, ...).
>>>
>>> I have to dig a little bit around that, but if you want some ideas
>>> already, please let me know.
>>>
>>> Thoughts ?
>>>
>>> Regards
>>> JB
>>> --
>>> Jean-Baptiste Onofré
>>> [hidden email]
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>

François Papon
fpapon@apache.org
Yupiik - https://www.yupiik.com
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

jbonofre
Good idea !

That makes sense.

Regards
JB

On 13/09/2018 17:27, Francois Papon wrote:

> For the template, we can share some templates on the Karaf website like
> the feature XSD files ?
>
> For example, a template to create a api rest feature based on Apache CXF.
>
> regards,
>
> François Papon
> [hidden email]
>
> Le 13/09/2018 à 19:18, Jean-Baptiste Onofré a écrit :
>> For the feature generation, I have two ideas that I would like to PoC:
>>
>> - use a CLI/interactive mode. For instance, if the user runs mvn
>> karaf:feature-generate, the MOJO will ask some questions. The purpose
>> is to "abstract" some concepts to simplify for the users.
>> - additionally, we can imagine to have template (feature-template.xml
>> or a yaml). The user can provide a template instead of answering
>> questions (a kind of response file).
>> - this could be used for bridge the gap with bndtools as well
>>
>> I would like to PoC this.
>>
>> Regards
>> JB
>>
>> On 13/09/2018 14:18, Grzegorz Grzybek wrote:
>>> Hello
>>>
>>> Thanks JBO for the idea. True - custom distro generation isn't that
>>> intuitive as it could be. Personally I missed the flexibility, not
>>> simplicity, that's why I added <framework>custom</framework> in
>>> "[KARAF-5468] Cleaning up AssemblyMojo, Profiles and profile Builder".
>>>
>>> Without it, the implied value for "framework" property was
>>> "framework" or
>>> "static-framework" depending on whether you had dependency on
>>> mvn:org.apache.karaf.features/framework/VERSION/kar or
>>> mvn:org.apache.karaf.features/static/VERSION/kar in your POM. The
>>> discovered "framework" property was added as "startup feature" which was
>>> very confusing (mixing "framework" and "feature" concepts).
>>>
>>> I can't tell much about improvements for
>>> karaf-maven-plugin:features-generate-descriptor, because I always
>>> preferred
>>> manual creation of feature XMLs.
>>>
>>> However having dependency on existing karaf distro ("standard"
>>> apache-karaf
>>> or apache-karaf-minimal) is good idea! karaf-maven-plugin:assembly could
>>> search zip or tar.gz or tgz kind of dependencies (with "provided" or any
>>> other scope) and:
>>>    - unpack it
>>>    - check etc/profile.cfg
>>>
>>> etc/profile.cfg is (since 4.2.0) nicely written as (official
>>> apache-karaf-minimal distro):
>>>
>>> #
>>> # Profile generated by Karaf Assembly Builder
>>> #
>>>
>>> # Parent profiles
>>> attribute.parents = generated-startup generated-boot generated-installed
>>>
>>> # Attributes
>>> attribute.overlay = true
>>>
>>> # Feature XML repositories
>>> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
>>> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
>>> repository.mvn\:org.apache.karaf.features/standard/4.2.0/xml/features =
>>> mvn:org.apache.karaf.features/standard/4.2.0/xml/features
>>> repository.mvn\:org.apache.karaf.features/spring/4.2.0/xml/features =
>>> mvn:org.apache.karaf.features/spring/4.2.0/xml/features
>>>
>>> # Features
>>> feature.framework = framework
>>> feature.jaas = jaas
>>> feature.shell = shell
>>> feature.feature = feature
>>> feature.ssh = ssh
>>> feature.bundle = bundle
>>> feature.config = config
>>> feature.log = log
>>>
>>> However, with complex distros it can look like this (my distro) - see
>>> all
>>> the blacklisted items and even special configuration options - these are
>>> all generated from pom.xml:
>>>
>>> #
>>> # Profile generated by Karaf Assembly Builder
>>> #
>>>
>>> # Parent profiles
>>> attribute.parents = generated-startup generated-boot generated-installed
>>>
>>> # Attributes
>>> attribute.overlay = true
>>>
>>> # Feature XML repositories
>>> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
>>> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
>>> ...
>>> # Features
>>> feature.framework = framework
>>> feature.patch-management = patch-management
>>> ...
>>> feature.pax-jms-config = pax-jms-config
>>> feature.pax-jms-pool-narayana = pax-jms-pool-narayana
>>> feature.pax-jms-pool-transx = pax-jms-pool-transx
>>>
>>> # Bundles
>>> ...
>>> bundle.mvn\:org.bouncycastle/bcprov-jdk15on/1.60 =
>>> mvn:org.bouncycastle/bcprov-jdk15on/1.60
>>> bundle.mvn\:org.bouncycastle/bcpkix-jdk15on/1.60 =
>>> mvn:org.bouncycastle/bcpkix-jdk15on/1.60
>>>
>>> # Configuration properties for etc/config.properties
>>> config.karaf.delay.console = true
>>>
>>> # Blacklisted repositories
>>> blacklisted.repository.mvn\:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
>>>
>>> = mvn:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
>>> ...
>>>
>>> # Blacklisted features
>>> blacklisted.feature.httplite = httplite
>>> blacklisted.feature.jetty/[8,9) = jetty/[8,9)
>>> blacklisted.feature.pax-*jetty* = pax-*jetty*
>>> blacklisted.feature.cxf-*-jetty = cxf-*-jetty
>>> ...
>>>
>>> # Blacklisted bundles
>>> blacklisted.bundle.mvn\:org.ops4j.pax.cdi/pax-cdi-jetty-weld =
>>> mvn:org.ops4j.pax.cdi/pax-cdi-jetty-weld
>>>
>>> This etc/profile.cfg can be treated simply as it was added in
>>> karaf-maven-plugin:assembly configuration itself!
>>>
>>> <configuration>
>>>       <profilesUris>
>>>           <uri>path/to/profile.cfg</uri>
>>>       </profilesUris>
>>> </configuration>
>>>
>>> best regards
>>> Grzegorz Grzybek
>>>
>>> czw., 13 wrz 2018 o 13:51 Jean-Baptiste Onofré <[hidden email]>
>>> napisał(a):
>>>
>>>> Hi guys,
>>>>
>>>> Recently, we received a lot of questions around how to create Karaf
>>>> custom distribution based on karaf-maven-plugin, and how to use the
>>>> static profile to create "standalone/static" distribution.
>>>>
>>>> If the plugin works fine, it's not easy to understand some "details",
>>>> like the dependency scope impact, or providing the set of default
>>>> features repos and features. I already helped users (in private
>>>> communication) to fix their custom distributions.
>>>>
>>>> Obviously, we should simplify the way of creating custom distribution,
>>>> especially with the new tooling & feature we now provide around Docker.
>>>>
>>>> I would like to propose the following:
>>>>
>>>> 1. Set the default behavior of the assembly goal to create a custom
>>>> distribution based on standard. For the user, instead of providing
>>>> (again) all framework, standard, enterprise features repos and all
>>>> standard boot features (shell, ...), it will just specify the
>>>> tar.gz/zip
>>>> base and his own features repo/repos (or the goal will use the same
>>>> version of the goal plugin itself). All the rest will be done by the
>>>> plugin for him. Use the karaf packaging as default to define this. At
>>>> the end of the day, the user pom.xml will look like:
>>>>
>>>> <project>
>>>>           <groupId>foo</groupId>
>>>>           <artifactId>bar</artifactId>
>>>>           <version>1.0-SNAPSHOT</version>
>>>>           <packaging>karaf</packaging>
>>>>
>>>>           <dependencies>
>>>>                   <dependency>
>>>>                           <groupId>org.apache.karaf</groupId>
>>>>                           <artifactId>apache-karaf</artifactId>
>>>>                           <version>4.2.1</version>
>>>>                           <type>tar.gz</type>
>>>>                   </dependency>
>>>>                   <dependency>
>>>>                           <groupId>foo</groupId>
>>>>                           <artifactId>my</artifactId>
>>>>                           <version>1.0-SNAPSHOT</version>
>>>>                           <classifier>features</classifier>
>>>>                           <type>xml</type>
>>>>                   </dependency>
>>>>           </dependencies>
>>>>
>>>>           <build>
>>>>                   <plugins>
>>>>                           <plugin>
>>>>                                  
>>>> <groupId>org.apache.karaf.tooling</groupId>
>>>> <artifactId>karaf-maven-plugin</artifactId>
>>>> <extensions>true</extensions>
>>>> <inherited>true</inherited>
>>>> <configuration>
>>>> <bootFeatures>
>>>>           <feature>my</feature>
>>>> </bootFeatures>
>>>> <installedFeatures>
>>>>           <feature>my-other</feature>
>>>> </installedFeatures>
>>>> </configuration>
>>>>                           </plugin>
>>>>                   </plugins>
>>>>           </build>
>>>>
>>>> </project>
>>>>
>>>> The idea is to automatically execute install-kar + assembly for the
>>>> karaf packaging and let the user focus on its own resources (features,
>>>> config, ...) just providing the base Karaf archive.
>>>> The user will be able to use src/main/resources to provide any files in
>>>> etc, bin, or whatever in the resulting custom distribution.
>>>>
>>>> 2. Improve a bit the features XML generation
>>>> If the custom distribution is the highest priority, just after the
>>>> improvements on this area, I would like to improve the way of creating
>>>> features XML.
>>>> Now, to be honest, almost all of us write features repos XML by
>>>> hand. It
>>>> gives us the maximum of flexibility. However, on the other hand, the
>>>> features XML and code contain should be sync.
>>>> I would like to improve the generate features MOJO, however leveraging
>>>> most of all functionalities around features (prerequisites, dependency
>>>> flag, inner features, ...).
>>>>
>>>> I have to dig a little bit around that, but if you want some ideas
>>>> already, please let me know.
>>>>
>>>> Thoughts ?
>>>>
>>>> Regards
>>>> JB
>>>> --
>>>> Jean-Baptiste Onofré
>>>> [hidden email]
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>>>>
>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

Freeman-2
In reply to this post by fpapon
Hi François,

I believe we can already do it to update/add/remove properties in configuration files.

You just need a src/main/karaf/assembly-property-edits.xml like

<property-edits xmlns="http://karaf.apache.org/tools/property-edits/1.0.0">
         <edits>
          <edit>
            <file>config.properties</file>
            <operation>put</operation>
            <key>karaf.framework</key>
            <value>equinox</value>
          </edit>
          <edit>
            <file>config.properties</file>
            <operation>extend</operation>
            <key>org.osgi.framework.system.capabilities</key>
            <value>my-magic-capability</value>
          </edit>
          <edit>
            <file>config.properties</file>
            <operation>remove</operation>
            <key>org.apache.karaf.security.providers</key>
          </edit>


        </edits>
      </property-edits>

You can get more details from [1]&[2]
[1]https://issues.apache.org/jira/browse/KARAF-3982
[2]https://issues.apache.org/jira/browse/KARAF-5868
Best Regards

-------------
Freeman(Yue) Fang

Red Hat, Inc.
FuseSource is now part of Red Hat



> On Sep 13, 2018, at 8:15 PM, Francois Papon <[hidden email]> wrote:
>
> Hi JB,
>
> I agree about this, we have to focus on the user friendly stuff and I
> have an additional point :
>
>  * Add a simple way to update/add properties in the default
>    configuration files of the standard distribution in the assembly
>    (like system.properties....)
>
> I'm not sure this is the correct discussion but I think we also have to
> work about helping user to upgrade the Karaf version of their custom
> distributions.
>
> regards,
>
> François Papon
> [hidden email]
>
> Le 13/09/2018 à 15:51, Jean-Baptiste Onofré a écrit :
>> Hi guys,
>>
>> Recently, we received a lot of questions around how to create Karaf
>> custom distribution based on karaf-maven-plugin, and how to use the
>> static profile to create "standalone/static" distribution.
>>
>> If the plugin works fine, it's not easy to understand some "details",
>> like the dependency scope impact, or providing the set of default
>> features repos and features. I already helped users (in private
>> communication) to fix their custom distributions.
>>
>> Obviously, we should simplify the way of creating custom distribution,
>> especially with the new tooling & feature we now provide around Docker.
>>
>> I would like to propose the following:
>>
>> 1. Set the default behavior of the assembly goal to create a custom
>> distribution based on standard. For the user, instead of providing
>> (again) all framework, standard, enterprise features repos and all
>> standard boot features (shell, ...), it will just specify the tar.gz/zip
>> base and his own features repo/repos (or the goal will use the same
>> version of the goal plugin itself). All the rest will be done by the
>> plugin for him. Use the karaf packaging as default to define this. At
>> the end of the day, the user pom.xml will look like:
>>
>> <project>
>> <groupId>foo</groupId>
>> <artifactId>bar</artifactId>
>> <version>1.0-SNAPSHOT</version>
>> <packaging>karaf</packaging>
>>
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.karaf</groupId>
>> <artifactId>apache-karaf</artifactId>
>> <version>4.2.1</version>
>> <type>tar.gz</type>
>> </dependency>
>> <dependency>
>> <groupId>foo</groupId>
>> <artifactId>my</artifactId>
>> <version>1.0-SNAPSHOT</version>
>> <classifier>features</classifier>
>> <type>xml</type>
>> </dependency>
>> </dependencies>
>>
>> <build>
>> <plugins>
>> <plugin>
>> <groupId>org.apache.karaf.tooling</groupId>
>> <artifactId>karaf-maven-plugin</artifactId>
>> <extensions>true</extensions>
>> <inherited>true</inherited>
>> <configuration>
>> <bootFeatures>
>> <feature>my</feature>
>> </bootFeatures>
>> <installedFeatures>
>> <feature>my-other</feature>
>> </installedFeatures>
>> </configuration>
>> </plugin>
>> </plugins>
>> </build>
>>
>> </project>
>>
>> The idea is to automatically execute install-kar + assembly for the
>> karaf packaging and let the user focus on its own resources (features,
>> config, ...) just providing the base Karaf archive.
>> The user will be able to use src/main/resources to provide any files in
>> etc, bin, or whatever in the resulting custom distribution.
>>
>> 2. Improve a bit the features XML generation
>> If the custom distribution is the highest priority, just after the
>> improvements on this area, I would like to improve the way of creating
>> features XML.
>> Now, to be honest, almost all of us write features repos XML by hand. It
>> gives us the maximum of flexibility. However, on the other hand, the
>> features XML and code contain should be sync.
>> I would like to improve the generate features MOJO, however leveraging
>> most of all functionalities around features (prerequisites, dependency
>> flag, inner features, ...).
>>
>> I have to dig a little bit around that, but if you want some ideas
>> already, please let me know.
>>
>> Thoughts ?
>>
>> Regards
>> JB
>

Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

fpapon
Hi Freeman,

Yes I know but we already have some user in the mailing asking for that.

I was thinking about including the configuration of property-edits
directly in the karaf-maven-plugin configuration.

Thoughts ?

Regards,

François Papon
[hidden email]

Le 14/09/2018 à 05:39, Freeman Fang a écrit :

> Hi François,
>
> I believe we can already do it to update/add/remove properties in configuration files.
>
> You just need a src/main/karaf/assembly-property-edits.xml like
>
> <property-edits xmlns="http://karaf.apache.org/tools/property-edits/1.0.0">
>          <edits>
>           <edit>
>             <file>config.properties</file>
>             <operation>put</operation>
>             <key>karaf.framework</key>
>             <value>equinox</value>
>           </edit>
>           <edit>
>             <file>config.properties</file>
>             <operation>extend</operation>
>             <key>org.osgi.framework.system.capabilities</key>
>             <value>my-magic-capability</value>
>           </edit>
>           <edit>
>             <file>config.properties</file>
>             <operation>remove</operation>
>             <key>org.apache.karaf.security.providers</key>
>           </edit>
>
>
>         </edits>
>       </property-edits>
>
> You can get more details from [1]&[2]
> [1]https://issues.apache.org/jira/browse/KARAF-3982
> [2]https://issues.apache.org/jira/browse/KARAF-5868
> Best Regards
>
> -------------
> Freeman(Yue) Fang
>
> Red Hat, Inc.
> FuseSource is now part of Red Hat
>
>
>
>> On Sep 13, 2018, at 8:15 PM, Francois Papon <[hidden email]> wrote:
>>
>> Hi JB,
>>
>> I agree about this, we have to focus on the user friendly stuff and I
>> have an additional point :
>>
>>  * Add a simple way to update/add properties in the default
>>    configuration files of the standard distribution in the assembly
>>    (like system.properties....)
>>
>> I'm not sure this is the correct discussion but I think we also have to
>> work about helping user to upgrade the Karaf version of their custom
>> distributions.
>>
>> regards,
>>
>> François Papon
>> [hidden email]
>>
>> Le 13/09/2018 à 15:51, Jean-Baptiste Onofré a écrit :
>>> Hi guys,
>>>
>>> Recently, we received a lot of questions around how to create Karaf
>>> custom distribution based on karaf-maven-plugin, and how to use the
>>> static profile to create "standalone/static" distribution.
>>>
>>> If the plugin works fine, it's not easy to understand some "details",
>>> like the dependency scope impact, or providing the set of default
>>> features repos and features. I already helped users (in private
>>> communication) to fix their custom distributions.
>>>
>>> Obviously, we should simplify the way of creating custom distribution,
>>> especially with the new tooling & feature we now provide around Docker.
>>>
>>> I would like to propose the following:
>>>
>>> 1. Set the default behavior of the assembly goal to create a custom
>>> distribution based on standard. For the user, instead of providing
>>> (again) all framework, standard, enterprise features repos and all
>>> standard boot features (shell, ...), it will just specify the tar.gz/zip
>>> base and his own features repo/repos (or the goal will use the same
>>> version of the goal plugin itself). All the rest will be done by the
>>> plugin for him. Use the karaf packaging as default to define this. At
>>> the end of the day, the user pom.xml will look like:
>>>
>>> <project>
>>> <groupId>foo</groupId>
>>> <artifactId>bar</artifactId>
>>> <version>1.0-SNAPSHOT</version>
>>> <packaging>karaf</packaging>
>>>
>>> <dependencies>
>>> <dependency>
>>> <groupId>org.apache.karaf</groupId>
>>> <artifactId>apache-karaf</artifactId>
>>> <version>4.2.1</version>
>>> <type>tar.gz</type>
>>> </dependency>
>>> <dependency>
>>> <groupId>foo</groupId>
>>> <artifactId>my</artifactId>
>>> <version>1.0-SNAPSHOT</version>
>>> <classifier>features</classifier>
>>> <type>xml</type>
>>> </dependency>
>>> </dependencies>
>>>
>>> <build>
>>> <plugins>
>>> <plugin>
>>> <groupId>org.apache.karaf.tooling</groupId>
>>> <artifactId>karaf-maven-plugin</artifactId>
>>> <extensions>true</extensions>
>>> <inherited>true</inherited>
>>> <configuration>
>>> <bootFeatures>
>>> <feature>my</feature>
>>> </bootFeatures>
>>> <installedFeatures>
>>> <feature>my-other</feature>
>>> </installedFeatures>
>>> </configuration>
>>> </plugin>
>>> </plugins>
>>> </build>
>>>
>>> </project>
>>>
>>> The idea is to automatically execute install-kar + assembly for the
>>> karaf packaging and let the user focus on its own resources (features,
>>> config, ...) just providing the base Karaf archive.
>>> The user will be able to use src/main/resources to provide any files in
>>> etc, bin, or whatever in the resulting custom distribution.
>>>
>>> 2. Improve a bit the features XML generation
>>> If the custom distribution is the highest priority, just after the
>>> improvements on this area, I would like to improve the way of creating
>>> features XML.
>>> Now, to be honest, almost all of us write features repos XML by hand. It
>>> gives us the maximum of flexibility. However, on the other hand, the
>>> features XML and code contain should be sync.
>>> I would like to improve the generate features MOJO, however leveraging
>>> most of all functionalities around features (prerequisites, dependency
>>> flag, inner features, ...).
>>>
>>> I have to dig a little bit around that, but if you want some ideas
>>> already, please let me know.
>>>
>>> Thoughts ?
>>>
>>> Regards
>>> JB
>


François Papon
fpapon@apache.org
Yupiik - https://www.yupiik.com
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

Freeman-2
yep, that  would be more neat!
-------------
Freeman(Yue) Fang

Red Hat, Inc.
FuseSource is now part of Red Hat



> On Sep 14, 2018, at 12:03 PM, Francois Papon <[hidden email]> wrote:
>
> Hi Freeman,
>
> Yes I know but we already have some user in the mailing asking for that.
>
> I was thinking about including the configuration of property-edits
> directly in the karaf-maven-plugin configuration.
>
> Thoughts ?
>
> Regards,
>
> François Papon
> [hidden email]
>
> Le 14/09/2018 à 05:39, Freeman Fang a écrit :
>> Hi François,
>>
>> I believe we can already do it to update/add/remove properties in configuration files.
>>
>> You just need a src/main/karaf/assembly-property-edits.xml like
>>
>> <property-edits xmlns="http://karaf.apache.org/tools/property-edits/1.0.0">
>>         <edits>
>>          <edit>
>>            <file>config.properties</file>
>>            <operation>put</operation>
>>            <key>karaf.framework</key>
>>            <value>equinox</value>
>>          </edit>
>>          <edit>
>>            <file>config.properties</file>
>>            <operation>extend</operation>
>>            <key>org.osgi.framework.system.capabilities</key>
>>            <value>my-magic-capability</value>
>>          </edit>
>>          <edit>
>>            <file>config.properties</file>
>>            <operation>remove</operation>
>>            <key>org.apache.karaf.security.providers</key>
>>          </edit>
>>
>>
>>        </edits>
>>      </property-edits>
>>
>> You can get more details from [1]&[2]
>> [1]https://issues.apache.org/jira/browse/KARAF-3982
>> [2]https://issues.apache.org/jira/browse/KARAF-5868
>> Best Regards
>>
>> -------------
>> Freeman(Yue) Fang
>>
>> Red Hat, Inc.
>> FuseSource is now part of Red Hat
>>
>>
>>
>>> On Sep 13, 2018, at 8:15 PM, Francois Papon <[hidden email]> wrote:
>>>
>>> Hi JB,
>>>
>>> I agree about this, we have to focus on the user friendly stuff and I
>>> have an additional point :
>>>
>>> * Add a simple way to update/add properties in the default
>>>   configuration files of the standard distribution in the assembly
>>>   (like system.properties....)
>>>
>>> I'm not sure this is the correct discussion but I think we also have to
>>> work about helping user to upgrade the Karaf version of their custom
>>> distributions.
>>>
>>> regards,
>>>
>>> François Papon
>>> [hidden email]
>>>
>>> Le 13/09/2018 à 15:51, Jean-Baptiste Onofré a écrit :
>>>> Hi guys,
>>>>
>>>> Recently, we received a lot of questions around how to create Karaf
>>>> custom distribution based on karaf-maven-plugin, and how to use the
>>>> static profile to create "standalone/static" distribution.
>>>>
>>>> If the plugin works fine, it's not easy to understand some "details",
>>>> like the dependency scope impact, or providing the set of default
>>>> features repos and features. I already helped users (in private
>>>> communication) to fix their custom distributions.
>>>>
>>>> Obviously, we should simplify the way of creating custom distribution,
>>>> especially with the new tooling & feature we now provide around Docker.
>>>>
>>>> I would like to propose the following:
>>>>
>>>> 1. Set the default behavior of the assembly goal to create a custom
>>>> distribution based on standard. For the user, instead of providing
>>>> (again) all framework, standard, enterprise features repos and all
>>>> standard boot features (shell, ...), it will just specify the tar.gz/zip
>>>> base and his own features repo/repos (or the goal will use the same
>>>> version of the goal plugin itself). All the rest will be done by the
>>>> plugin for him. Use the karaf packaging as default to define this. At
>>>> the end of the day, the user pom.xml will look like:
>>>>
>>>> <project>
>>>> <groupId>foo</groupId>
>>>> <artifactId>bar</artifactId>
>>>> <version>1.0-SNAPSHOT</version>
>>>> <packaging>karaf</packaging>
>>>>
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>org.apache.karaf</groupId>
>>>> <artifactId>apache-karaf</artifactId>
>>>> <version>4.2.1</version>
>>>> <type>tar.gz</type>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>foo</groupId>
>>>> <artifactId>my</artifactId>
>>>> <version>1.0-SNAPSHOT</version>
>>>> <classifier>features</classifier>
>>>> <type>xml</type>
>>>> </dependency>
>>>> </dependencies>
>>>>
>>>> <build>
>>>> <plugins>
>>>> <plugin>
>>>> <groupId>org.apache.karaf.tooling</groupId>
>>>> <artifactId>karaf-maven-plugin</artifactId>
>>>> <extensions>true</extensions>
>>>> <inherited>true</inherited>
>>>> <configuration>
>>>> <bootFeatures>
>>>> <feature>my</feature>
>>>> </bootFeatures>
>>>> <installedFeatures>
>>>> <feature>my-other</feature>
>>>> </installedFeatures>
>>>> </configuration>
>>>> </plugin>
>>>> </plugins>
>>>> </build>
>>>>
>>>> </project>
>>>>
>>>> The idea is to automatically execute install-kar + assembly for the
>>>> karaf packaging and let the user focus on its own resources (features,
>>>> config, ...) just providing the base Karaf archive.
>>>> The user will be able to use src/main/resources to provide any files in
>>>> etc, bin, or whatever in the resulting custom distribution.
>>>>
>>>> 2. Improve a bit the features XML generation
>>>> If the custom distribution is the highest priority, just after the
>>>> improvements on this area, I would like to improve the way of creating
>>>> features XML.
>>>> Now, to be honest, almost all of us write features repos XML by hand. It
>>>> gives us the maximum of flexibility. However, on the other hand, the
>>>> features XML and code contain should be sync.
>>>> I would like to improve the generate features MOJO, however leveraging
>>>> most of all functionalities around features (prerequisites, dependency
>>>> flag, inner features, ...).
>>>>
>>>> I have to dig a little bit around that, but if you want some ideas
>>>> already, please let me know.
>>>>
>>>> Thoughts ?
>>>>
>>>> Regards
>>>> JB
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

jbonofre
Agree, it's what I had in mind and part of the PoC.

Regards
JB

On 14/09/2018 07:13, Freeman Fang wrote:

> yep, that  would be more neat!
> -------------
> Freeman(Yue) Fang
>
> Red Hat, Inc.
> FuseSource is now part of Red Hat
>
>
>
>> On Sep 14, 2018, at 12:03 PM, Francois Papon <[hidden email]> wrote:
>>
>> Hi Freeman,
>>
>> Yes I know but we already have some user in the mailing asking for that.
>>
>> I was thinking about including the configuration of property-edits
>> directly in the karaf-maven-plugin configuration.
>>
>> Thoughts ?
>>
>> Regards,
>>
>> François Papon
>> [hidden email]
>>
>> Le 14/09/2018 à 05:39, Freeman Fang a écrit :
>>> Hi François,
>>>
>>> I believe we can already do it to update/add/remove properties in configuration files.
>>>
>>> You just need a src/main/karaf/assembly-property-edits.xml like
>>>
>>> <property-edits xmlns="http://karaf.apache.org/tools/property-edits/1.0.0">
>>>         <edits>
>>>          <edit>
>>>            <file>config.properties</file>
>>>            <operation>put</operation>
>>>            <key>karaf.framework</key>
>>>            <value>equinox</value>
>>>          </edit>
>>>          <edit>
>>>            <file>config.properties</file>
>>>            <operation>extend</operation>
>>>            <key>org.osgi.framework.system.capabilities</key>
>>>            <value>my-magic-capability</value>
>>>          </edit>
>>>          <edit>
>>>            <file>config.properties</file>
>>>            <operation>remove</operation>
>>>            <key>org.apache.karaf.security.providers</key>
>>>          </edit>
>>>
>>>
>>>        </edits>
>>>      </property-edits>
>>>
>>> You can get more details from [1]&[2]
>>> [1]https://issues.apache.org/jira/browse/KARAF-3982
>>> [2]https://issues.apache.org/jira/browse/KARAF-5868
>>> Best Regards
>>>
>>> -------------
>>> Freeman(Yue) Fang
>>>
>>> Red Hat, Inc.
>>> FuseSource is now part of Red Hat
>>>
>>>
>>>
>>>> On Sep 13, 2018, at 8:15 PM, Francois Papon <[hidden email]> wrote:
>>>>
>>>> Hi JB,
>>>>
>>>> I agree about this, we have to focus on the user friendly stuff and I
>>>> have an additional point :
>>>>
>>>> * Add a simple way to update/add properties in the default
>>>>   configuration files of the standard distribution in the assembly
>>>>   (like system.properties....)
>>>>
>>>> I'm not sure this is the correct discussion but I think we also have to
>>>> work about helping user to upgrade the Karaf version of their custom
>>>> distributions.
>>>>
>>>> regards,
>>>>
>>>> François Papon
>>>> [hidden email]
>>>>
>>>> Le 13/09/2018 à 15:51, Jean-Baptiste Onofré a écrit :
>>>>> Hi guys,
>>>>>
>>>>> Recently, we received a lot of questions around how to create Karaf
>>>>> custom distribution based on karaf-maven-plugin, and how to use the
>>>>> static profile to create "standalone/static" distribution.
>>>>>
>>>>> If the plugin works fine, it's not easy to understand some "details",
>>>>> like the dependency scope impact, or providing the set of default
>>>>> features repos and features. I already helped users (in private
>>>>> communication) to fix their custom distributions.
>>>>>
>>>>> Obviously, we should simplify the way of creating custom distribution,
>>>>> especially with the new tooling & feature we now provide around Docker.
>>>>>
>>>>> I would like to propose the following:
>>>>>
>>>>> 1. Set the default behavior of the assembly goal to create a custom
>>>>> distribution based on standard. For the user, instead of providing
>>>>> (again) all framework, standard, enterprise features repos and all
>>>>> standard boot features (shell, ...), it will just specify the tar.gz/zip
>>>>> base and his own features repo/repos (or the goal will use the same
>>>>> version of the goal plugin itself). All the rest will be done by the
>>>>> plugin for him. Use the karaf packaging as default to define this. At
>>>>> the end of the day, the user pom.xml will look like:
>>>>>
>>>>> <project>
>>>>> <groupId>foo</groupId>
>>>>> <artifactId>bar</artifactId>
>>>>> <version>1.0-SNAPSHOT</version>
>>>>> <packaging>karaf</packaging>
>>>>>
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>org.apache.karaf</groupId>
>>>>> <artifactId>apache-karaf</artifactId>
>>>>> <version>4.2.1</version>
>>>>> <type>tar.gz</type>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>foo</groupId>
>>>>> <artifactId>my</artifactId>
>>>>> <version>1.0-SNAPSHOT</version>
>>>>> <classifier>features</classifier>
>>>>> <type>xml</type>
>>>>> </dependency>
>>>>> </dependencies>
>>>>>
>>>>> <build>
>>>>> <plugins>
>>>>> <plugin>
>>>>> <groupId>org.apache.karaf.tooling</groupId>
>>>>> <artifactId>karaf-maven-plugin</artifactId>
>>>>> <extensions>true</extensions>
>>>>> <inherited>true</inherited>
>>>>> <configuration>
>>>>> <bootFeatures>
>>>>> <feature>my</feature>
>>>>> </bootFeatures>
>>>>> <installedFeatures>
>>>>> <feature>my-other</feature>
>>>>> </installedFeatures>
>>>>> </configuration>
>>>>> </plugin>
>>>>> </plugins>
>>>>> </build>
>>>>>
>>>>> </project>
>>>>>
>>>>> The idea is to automatically execute install-kar + assembly for the
>>>>> karaf packaging and let the user focus on its own resources (features,
>>>>> config, ...) just providing the base Karaf archive.
>>>>> The user will be able to use src/main/resources to provide any files in
>>>>> etc, bin, or whatever in the resulting custom distribution.
>>>>>
>>>>> 2. Improve a bit the features XML generation
>>>>> If the custom distribution is the highest priority, just after the
>>>>> improvements on this area, I would like to improve the way of creating
>>>>> features XML.
>>>>> Now, to be honest, almost all of us write features repos XML by hand. It
>>>>> gives us the maximum of flexibility. However, on the other hand, the
>>>>> features XML and code contain should be sync.
>>>>> I would like to improve the generate features MOJO, however leveraging
>>>>> most of all functionalities around features (prerequisites, dependency
>>>>> flag, inner features, ...).
>>>>>
>>>>> I have to dig a little bit around that, but if you want some ideas
>>>>> already, please let me know.
>>>>>
>>>>> Thoughts ?
>>>>>
>>>>> Regards
>>>>> JB
>>>
>>
>>
>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

luke
In reply to this post by Grzegorz Grzybek
Just as an side note - there is an inconsistency in naming of features and their location in repository which doesn’t make whole thing easier.

Given example of static assemblies we have two URIs:
org.apache.karaf.features/framework
org.apache.karaf.features/static
where first one contains "framework” feature but second delivers "static-framework”. Such small thing makes it harder to spot how whole thing works. I placed “framework-static” couple of times in my attempts and plugin didn’t protest.

If we could align these it would make users life a bit easier I guess.

Cheers,
Lukasz

> On 13 Sep 2018, at 14:18, Grzegorz Grzybek <[hidden email]> wrote:
>
> Hello
>
> Thanks JBO for the idea. True - custom distro generation isn't that
> intuitive as it could be. Personally I missed the flexibility, not
> simplicity, that's why I added <framework>custom</framework> in
> "[KARAF-5468] Cleaning up AssemblyMojo, Profiles and profile Builder".
>
> Without it, the implied value for "framework" property was "framework" or
> "static-framework" depending on whether you had dependency on
> mvn:org.apache.karaf.features/framework/VERSION/kar or
> mvn:org.apache.karaf.features/static/VERSION/kar in your POM. The
> discovered "framework" property was added as "startup feature" which was
> very confusing (mixing "framework" and "feature" concepts).
>
> I can't tell much about improvements for
> karaf-maven-plugin:features-generate-descriptor, because I always preferred
> manual creation of feature XMLs.
>
> However having dependency on existing karaf distro ("standard" apache-karaf
> or apache-karaf-minimal) is good idea! karaf-maven-plugin:assembly could
> search zip or tar.gz or tgz kind of dependencies (with "provided" or any
> other scope) and:
> - unpack it
> - check etc/profile.cfg
>
> etc/profile.cfg is (since 4.2.0) nicely written as (official
> apache-karaf-minimal distro):
>
> #
> # Profile generated by Karaf Assembly Builder
> #
>
> # Parent profiles
> attribute.parents = generated-startup generated-boot generated-installed
>
> # Attributes
> attribute.overlay = true
>
> # Feature XML repositories
> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
> repository.mvn\:org.apache.karaf.features/standard/4.2.0/xml/features =
> mvn:org.apache.karaf.features/standard/4.2.0/xml/features
> repository.mvn\:org.apache.karaf.features/spring/4.2.0/xml/features =
> mvn:org.apache.karaf.features/spring/4.2.0/xml/features
>
> # Features
> feature.framework = framework
> feature.jaas = jaas
> feature.shell = shell
> feature.feature = feature
> feature.ssh = ssh
> feature.bundle = bundle
> feature.config = config
> feature.log = log
>
> However, with complex distros it can look like this (my distro) - see all
> the blacklisted items and even special configuration options - these are
> all generated from pom.xml:
>
> #
> # Profile generated by Karaf Assembly Builder
> #
>
> # Parent profiles
> attribute.parents = generated-startup generated-boot generated-installed
>
> # Attributes
> attribute.overlay = true
>
> # Feature XML repositories
> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
> ...
> # Features
> feature.framework = framework
> feature.patch-management = patch-management
> ...
> feature.pax-jms-config = pax-jms-config
> feature.pax-jms-pool-narayana = pax-jms-pool-narayana
> feature.pax-jms-pool-transx = pax-jms-pool-transx
>
> # Bundles
> ...
> bundle.mvn\:org.bouncycastle/bcprov-jdk15on/1.60 =
> mvn:org.bouncycastle/bcprov-jdk15on/1.60
> bundle.mvn\:org.bouncycastle/bcpkix-jdk15on/1.60 =
> mvn:org.bouncycastle/bcpkix-jdk15on/1.60
>
> # Configuration properties for etc/config.properties
> config.karaf.delay.console = true
>
> # Blacklisted repositories
> blacklisted.repository.mvn\:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
> = mvn:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
> ...
>
> # Blacklisted features
> blacklisted.feature.httplite = httplite
> blacklisted.feature.jetty/[8,9) = jetty/[8,9)
> blacklisted.feature.pax-*jetty* = pax-*jetty*
> blacklisted.feature.cxf-*-jetty = cxf-*-jetty
> ...
>
> # Blacklisted bundles
> blacklisted.bundle.mvn\:org.ops4j.pax.cdi/pax-cdi-jetty-weld =
> mvn:org.ops4j.pax.cdi/pax-cdi-jetty-weld
>
> This etc/profile.cfg can be treated simply as it was added in
> karaf-maven-plugin:assembly configuration itself!
>
> <configuration>
>    <profilesUris>
>        <uri>path/to/profile.cfg</uri>
>    </profilesUris>
> </configuration>
>
> best regards
> Grzegorz Grzybek
>
> czw., 13 wrz 2018 o 13:51 Jean-Baptiste Onofré <[hidden email]> napisał(a):
>
>> Hi guys,
>>
>> Recently, we received a lot of questions around how to create Karaf
>> custom distribution based on karaf-maven-plugin, and how to use the
>> static profile to create "standalone/static" distribution.
>>
>> If the plugin works fine, it's not easy to understand some "details",
>> like the dependency scope impact, or providing the set of default
>> features repos and features. I already helped users (in private
>> communication) to fix their custom distributions.
>>
>> Obviously, we should simplify the way of creating custom distribution,
>> especially with the new tooling & feature we now provide around Docker.
>>
>> I would like to propose the following:
>>
>> 1. Set the default behavior of the assembly goal to create a custom
>> distribution based on standard. For the user, instead of providing
>> (again) all framework, standard, enterprise features repos and all
>> standard boot features (shell, ...), it will just specify the tar.gz/zip
>> base and his own features repo/repos (or the goal will use the same
>> version of the goal plugin itself). All the rest will be done by the
>> plugin for him. Use the karaf packaging as default to define this. At
>> the end of the day, the user pom.xml will look like:
>>
>> <project>
>>        <groupId>foo</groupId>
>>        <artifactId>bar</artifactId>
>>        <version>1.0-SNAPSHOT</version>
>>        <packaging>karaf</packaging>
>>
>>        <dependencies>
>>                <dependency>
>>                        <groupId>org.apache.karaf</groupId>
>>                        <artifactId>apache-karaf</artifactId>
>>                        <version>4.2.1</version>
>>                        <type>tar.gz</type>
>>                </dependency>
>>                <dependency>
>>                        <groupId>foo</groupId>
>>                        <artifactId>my</artifactId>
>>                        <version>1.0-SNAPSHOT</version>
>>                        <classifier>features</classifier>
>>                        <type>xml</type>
>>                </dependency>
>>        </dependencies>
>>
>>        <build>
>>                <plugins>
>>                        <plugin>
>>                                <groupId>org.apache.karaf.tooling</groupId>
>> <artifactId>karaf-maven-plugin</artifactId>
>> <extensions>true</extensions>
>> <inherited>true</inherited>
>> <configuration>
>> <bootFeatures>
>>        <feature>my</feature>
>> </bootFeatures>
>> <installedFeatures>
>>        <feature>my-other</feature>
>> </installedFeatures>
>> </configuration>
>>                        </plugin>
>>                </plugins>
>>        </build>
>>
>> </project>
>>
>> The idea is to automatically execute install-kar + assembly for the
>> karaf packaging and let the user focus on its own resources (features,
>> config, ...) just providing the base Karaf archive.
>> The user will be able to use src/main/resources to provide any files in
>> etc, bin, or whatever in the resulting custom distribution.
>>
>> 2. Improve a bit the features XML generation
>> If the custom distribution is the highest priority, just after the
>> improvements on this area, I would like to improve the way of creating
>> features XML.
>> Now, to be honest, almost all of us write features repos XML by hand. It
>> gives us the maximum of flexibility. However, on the other hand, the
>> features XML and code contain should be sync.
>> I would like to improve the generate features MOJO, however leveraging
>> most of all functionalities around features (prerequisites, dependency
>> flag, inner features, ...).
>>
>> I have to dig a little bit around that, but if you want some ideas
>> already, please let me know.
>>
>> Thoughts ?
>>
>> Regards
>> JB
>> --
>> Jean-Baptiste Onofré
>> [hidden email]
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>

Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Simplify the karaf-maven-plugin to easily create custom distribution

jbonofre
Hi Lukasz

Good point. In order to avoid to introduce a breaking change, I would
propose to create a feature alias for those URIs.

Regards
JB

On 17/09/2018 14:33, [hidden email] wrote:

> Just as an side note - there is an inconsistency in naming of features and their location in repository which doesn’t make whole thing easier.
>
> Given example of static assemblies we have two URIs:
> org.apache.karaf.features/framework
> org.apache.karaf.features/static
> where first one contains "framework” feature but second delivers "static-framework”. Such small thing makes it harder to spot how whole thing works. I placed “framework-static” couple of times in my attempts and plugin didn’t protest.
>
> If we could align these it would make users life a bit easier I guess.
>
> Cheers,
> Lukasz
>
>> On 13 Sep 2018, at 14:18, Grzegorz Grzybek <[hidden email]> wrote:
>>
>> Hello
>>
>> Thanks JBO for the idea. True - custom distro generation isn't that
>> intuitive as it could be. Personally I missed the flexibility, not
>> simplicity, that's why I added <framework>custom</framework> in
>> "[KARAF-5468] Cleaning up AssemblyMojo, Profiles and profile Builder".
>>
>> Without it, the implied value for "framework" property was "framework" or
>> "static-framework" depending on whether you had dependency on
>> mvn:org.apache.karaf.features/framework/VERSION/kar or
>> mvn:org.apache.karaf.features/static/VERSION/kar in your POM. The
>> discovered "framework" property was added as "startup feature" which was
>> very confusing (mixing "framework" and "feature" concepts).
>>
>> I can't tell much about improvements for
>> karaf-maven-plugin:features-generate-descriptor, because I always preferred
>> manual creation of feature XMLs.
>>
>> However having dependency on existing karaf distro ("standard" apache-karaf
>> or apache-karaf-minimal) is good idea! karaf-maven-plugin:assembly could
>> search zip or tar.gz or tgz kind of dependencies (with "provided" or any
>> other scope) and:
>> - unpack it
>> - check etc/profile.cfg
>>
>> etc/profile.cfg is (since 4.2.0) nicely written as (official
>> apache-karaf-minimal distro):
>>
>> #
>> # Profile generated by Karaf Assembly Builder
>> #
>>
>> # Parent profiles
>> attribute.parents = generated-startup generated-boot generated-installed
>>
>> # Attributes
>> attribute.overlay = true
>>
>> # Feature XML repositories
>> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
>> repository.mvn\:org.apache.karaf.features/standard/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/standard/4.2.0/xml/features
>> repository.mvn\:org.apache.karaf.features/spring/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/spring/4.2.0/xml/features
>>
>> # Features
>> feature.framework = framework
>> feature.jaas = jaas
>> feature.shell = shell
>> feature.feature = feature
>> feature.ssh = ssh
>> feature.bundle = bundle
>> feature.config = config
>> feature.log = log
>>
>> However, with complex distros it can look like this (my distro) - see all
>> the blacklisted items and even special configuration options - these are
>> all generated from pom.xml:
>>
>> #
>> # Profile generated by Karaf Assembly Builder
>> #
>>
>> # Parent profiles
>> attribute.parents = generated-startup generated-boot generated-installed
>>
>> # Attributes
>> attribute.overlay = true
>>
>> # Feature XML repositories
>> repository.mvn\:org.apache.karaf.features/framework/4.2.0/xml/features =
>> mvn:org.apache.karaf.features/framework/4.2.0/xml/features
>> ...
>> # Features
>> feature.framework = framework
>> feature.patch-management = patch-management
>> ...
>> feature.pax-jms-config = pax-jms-config
>> feature.pax-jms-pool-narayana = pax-jms-pool-narayana
>> feature.pax-jms-pool-transx = pax-jms-pool-transx
>>
>> # Bundles
>> ...
>> bundle.mvn\:org.bouncycastle/bcprov-jdk15on/1.60 =
>> mvn:org.bouncycastle/bcprov-jdk15on/1.60
>> bundle.mvn\:org.bouncycastle/bcpkix-jdk15on/1.60 =
>> mvn:org.bouncycastle/bcpkix-jdk15on/1.60
>>
>> # Configuration properties for etc/config.properties
>> config.karaf.delay.console = true
>>
>> # Blacklisted repositories
>> blacklisted.repository.mvn\:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
>> = mvn:org.ops4j.pax.cdi/pax-cdi-features/1.0.0.RC1/xml/features
>> ...
>>
>> # Blacklisted features
>> blacklisted.feature.httplite = httplite
>> blacklisted.feature.jetty/[8,9) = jetty/[8,9)
>> blacklisted.feature.pax-*jetty* = pax-*jetty*
>> blacklisted.feature.cxf-*-jetty = cxf-*-jetty
>> ...
>>
>> # Blacklisted bundles
>> blacklisted.bundle.mvn\:org.ops4j.pax.cdi/pax-cdi-jetty-weld =
>> mvn:org.ops4j.pax.cdi/pax-cdi-jetty-weld
>>
>> This etc/profile.cfg can be treated simply as it was added in
>> karaf-maven-plugin:assembly configuration itself!
>>
>> <configuration>
>>    <profilesUris>
>>        <uri>path/to/profile.cfg</uri>
>>    </profilesUris>
>> </configuration>
>>
>> best regards
>> Grzegorz Grzybek
>>
>> czw., 13 wrz 2018 o 13:51 Jean-Baptiste Onofré <[hidden email]> napisał(a):
>>
>>> Hi guys,
>>>
>>> Recently, we received a lot of questions around how to create Karaf
>>> custom distribution based on karaf-maven-plugin, and how to use the
>>> static profile to create "standalone/static" distribution.
>>>
>>> If the plugin works fine, it's not easy to understand some "details",
>>> like the dependency scope impact, or providing the set of default
>>> features repos and features. I already helped users (in private
>>> communication) to fix their custom distributions.
>>>
>>> Obviously, we should simplify the way of creating custom distribution,
>>> especially with the new tooling & feature we now provide around Docker.
>>>
>>> I would like to propose the following:
>>>
>>> 1. Set the default behavior of the assembly goal to create a custom
>>> distribution based on standard. For the user, instead of providing
>>> (again) all framework, standard, enterprise features repos and all
>>> standard boot features (shell, ...), it will just specify the tar.gz/zip
>>> base and his own features repo/repos (or the goal will use the same
>>> version of the goal plugin itself). All the rest will be done by the
>>> plugin for him. Use the karaf packaging as default to define this. At
>>> the end of the day, the user pom.xml will look like:
>>>
>>> <project>
>>>        <groupId>foo</groupId>
>>>        <artifactId>bar</artifactId>
>>>        <version>1.0-SNAPSHOT</version>
>>>        <packaging>karaf</packaging>
>>>
>>>        <dependencies>
>>>                <dependency>
>>>                        <groupId>org.apache.karaf</groupId>
>>>                        <artifactId>apache-karaf</artifactId>
>>>                        <version>4.2.1</version>
>>>                        <type>tar.gz</type>
>>>                </dependency>
>>>                <dependency>
>>>                        <groupId>foo</groupId>
>>>                        <artifactId>my</artifactId>
>>>                        <version>1.0-SNAPSHOT</version>
>>>                        <classifier>features</classifier>
>>>                        <type>xml</type>
>>>                </dependency>
>>>        </dependencies>
>>>
>>>        <build>
>>>                <plugins>
>>>                        <plugin>
>>>                                <groupId>org.apache.karaf.tooling</groupId>
>>> <artifactId>karaf-maven-plugin</artifactId>
>>> <extensions>true</extensions>
>>> <inherited>true</inherited>
>>> <configuration>
>>> <bootFeatures>
>>>        <feature>my</feature>
>>> </bootFeatures>
>>> <installedFeatures>
>>>        <feature>my-other</feature>
>>> </installedFeatures>
>>> </configuration>
>>>                        </plugin>
>>>                </plugins>
>>>        </build>
>>>
>>> </project>
>>>
>>> The idea is to automatically execute install-kar + assembly for the
>>> karaf packaging and let the user focus on its own resources (features,
>>> config, ...) just providing the base Karaf archive.
>>> The user will be able to use src/main/resources to provide any files in
>>> etc, bin, or whatever in the resulting custom distribution.
>>>
>>> 2. Improve a bit the features XML generation
>>> If the custom distribution is the highest priority, just after the
>>> improvements on this area, I would like to improve the way of creating
>>> features XML.
>>> Now, to be honest, almost all of us write features repos XML by hand. It
>>> gives us the maximum of flexibility. However, on the other hand, the
>>> features XML and code contain should be sync.
>>> I would like to improve the generate features MOJO, however leveraging
>>> most of all functionalities around features (prerequisites, dependency
>>> flag, inner features, ...).
>>>
>>> I have to dig a little bit around that, but if you want some ideas
>>> already, please let me know.
>>>
>>> Thoughts ?
>>>
>>> Regards
>>> JB
>>> --
>>> Jean-Baptiste Onofré
>>> [hidden email]
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com