Confused about feature dependency="true"

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Confused about feature dependency="true"

lechlukasz

Hello,

 

I’m quite confused about feature dependencies.  The documentation about provisioning doesn’t explain much how the dependency=”true” is expected to work. I’ve tried adding or removing it, but I experience unexpected behavior…

 

I have a core feature and other features that depend on it. The system is modular, and there are many production endpoints with diverse feature sets.

 

I’ll try to present oversimplified example, which hopefully will give a clue what could get wrong.

 

I have <core>, that is used by any other actor.

I have <service1-core> (API) that has 2 implementations, defined as features <service1-impl1> and <service1-impl2>.

I have <endpoint1> that uses only <core>

I have <endpoint2> that uses <core> and <service1-core>.

 

You can have only 1 implementation of service1. You can have one of 2 endpoints, or both.

 

The problem is, that when I install everything, I see in logs, many bundles defined in <core> or <service1-core> are started many times. As would the installation of another ‘consumer’ of that feature cause feature to be reinstalled. What is worse, I’ve expected once infinite loop of triggered restarts.

 

What I need is a stable way to define the dependency from B to A, telling that if B is going to be installed, and A is not present, A should be installed, but if A is already installed, it should be not re-installed neither re-started.

 

So if I say, install service1-impl, the service1-core will be installed and started. If I later say, install endpoint2, the service1-core will not be touched, because it is already installed and running.

 

Shoudn’t the dependency=”true” work that way? What can cause the common feature to be reinstalled many times by installing dependent features?

 

Best regards,

Lukasz Lech

 

Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

Guillaume Nodet-2
So first, bundles are only updated / refreshed / restarted if necessary.  If there's no change for a given bundle, it won't be touched at all.

The dependency="true" flag means that the artifact can be used by karaf to solve some constraints, but it won't be installed if not needed.
This is mainly useful for imported packages or services, not for the core of your feature.
For example you write a feature that needs  apache commons-collections.  If you don't add the dependency="true" flag on the bundle, it will always be installed, even if a newer version of commons-collections is required by another bundle, so you'll end up with 2 different versions of commons-collections in use.  If you put the dependency="true" flag on the commons-collections bundles, then the resolver deploy only the latest bundle.

The same idea holds if you put the flag on a feature: i.e. it will be used by the resolver, but if it is not needed, it may not be installed.

2017-12-08 10:27 GMT+01:00 Lukasz Lech <[hidden email]>:

Hello,

 

I’m quite confused about feature dependencies.  The documentation about provisioning doesn’t explain much how the dependency=”true” is expected to work. I’ve tried adding or removing it, but I experience unexpected behavior…

 

I have a core feature and other features that depend on it. The system is modular, and there are many production endpoints with diverse feature sets.

 

I’ll try to present oversimplified example, which hopefully will give a clue what could get wrong.

 

I have <core>, that is used by any other actor.

I have <service1-core> (API) that has 2 implementations, defined as features <service1-impl1> and <service1-impl2>.

I have <endpoint1> that uses only <core>

I have <endpoint2> that uses <core> and <service1-core>.

 

You can have only 1 implementation of service1. You can have one of 2 endpoints, or both.

 

The problem is, that when I install everything, I see in logs, many bundles defined in <core> or <service1-core> are started many times. As would the installation of another ‘consumer’ of that feature cause feature to be reinstalled. What is worse, I’ve expected once infinite loop of triggered restarts.

 

What I need is a stable way to define the dependency from B to A, telling that if B is going to be installed, and A is not present, A should be installed, but if A is already installed, it should be not re-installed neither re-started.

 

So if I say, install service1-impl, the service1-core will be installed and started. If I later say, install endpoint2, the service1-core will not be touched, because it is already installed and running.

 

Shoudn’t the dependency=”true” work that way? What can cause the common feature to be reinstalled many times by installing dependent features?

 

Best regards,

Lukasz Lech

 




--
------------------------
Guillaume Nodet

Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

lechlukasz
Does the same appply to features?

Because I've noticed an improvement, after marking the dependent features as
dependency="true":

    <feature dependency="true">service1-core</feture>

However, if I've done the same for all my features, including system ones
(jpa, transaction-api, transaction etc.) I've ended up with my bundles not
starting because the blueprint namespace handlers were not installed...

I've reached the point where the whole orchestration is very fragile. I need
often to stop Karaf after installing all features and start it again, so
that everything starts fine.



--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

Guillaume Nodet-2
It's all about constraints.
If something does not work when installed, it means that the constraints are not correctly expressed.
If you use the maven bundle plugin for example, it should generate service requirements for the various namespace handlers used.  Those will be used at resolution time, unless you use an old feature namespace.

For the dependency flag, a rough heuristic is that bundles/features from the project have dependency="false" while external ones have dependency="true".

Also, make sure to at least use the karaf maven plugin to validate the resolution of your features at build time.  At least, this will ensure that all constraints are solved somehow.

2017-12-08 11:38 GMT+01:00 lechlukasz <[hidden email]>:
Does the same appply to features?

Because I've noticed an improvement, after marking the dependent features as
dependency="true":

    <feature dependency="true">service1-core</feture>

However, if I've done the same for all my features, including system ones
(jpa, transaction-api, transaction etc.) I've ended up with my bundles not
starting because the blueprint namespace handlers were not installed...

I've reached the point where the whole orchestration is very fragile. I need
often to stop Karaf after installing all features and start it again, so
that everything starts fine.



--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html



--
------------------------
Guillaume Nodet

Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

lechlukasz
This post was updated on .
Ok thank you, all my features are verified by maven plugin.

So you say that no matter if I have features with dependency="true" or not,
if the feature is already installed, it will not be touched?

So the observed behavior, that after installing new feature the blueprint
context from bundles from dependent bundle are destroyed and re-created has
nothing to do with my feature descriptor and I should search for the cause
for that behavior somewhere else?

Have I understood you correctly?


// UPDATE

here is the example what is going wrong when I install a feature:

2017-12-08T12:26:40,689 | INFO  | pipe-feature:install myfeature | FeaturesServiceImpl              | 9 - org.apache.karaf.features.core - 4.1.2 | Adding features: my-feature/[118.0.0.SNAPSHOT,118.0.0.SNAPSHOT]
2017-12-08T12:26:47,958 | INFO  | features-1-thread-1 | FeaturesServiceImpl              | 9 - org.apache.karaf.features.core - 4.1.2 | Changes to perform:
2017-12-08T12:26:47,958 | INFO  | features-1-thread-1 | FeaturesServiceImpl              | 9 - org.apache.karaf.features.core - 4.1.2 |   Region: root
2017-12-08T12:26:47,958 | INFO  | features-1-thread-1 | FeaturesServiceImpl              | 9 - org.apache.karaf.features.core - 4.1.2 |     Bundles to install:

//OK

2017-12-08T12:26:47,974 | INFO  | features-1-thread-1 | FeaturesServiceImpl              | 9 - org.apache.karaf.features.core - 4.1.2 | Installing bundles:

// OK

2017-12-08T12:26:48,787 | INFO  | features-1-thread-1 | FeaturesServiceImpl              | 9 - org.apache.karaf.features.core - 4.1.2 | Stopping bundles:
2017-12-08T12:26:48,787 | INFO  | features-1-thread-1 | FeaturesServiceImpl              | 9 - org.apache.karaf.features.core - 4.1.2 |   org.apache.aries.jpa.javax.persistence_2.1/2.7.0.SNAPSHOT
2017-12-08T12:26:48,802 | INFO  | features-1-thread-1 | FeaturesServiceImpl              | 9 - org.apache.karaf.features.core - 4.1.2 |   org.eclipse.persistence.jpa/2.6.4.v20160829-44060b6
2017-12-08T12:26:48,802 | INFO  | features-1-thread-1 | FeaturesServiceImpl              | 9 - org.apache.karaf.features.core - 4.1.2 |   org.eclipse.persistence.core/2.6.4.v20160829-44060b6
2017-12-08T12:26:48,802 | INFO  | features-1-thread-1 | FeaturesServiceImpl              | 9 - org.apache.karaf.features.core - 4.1.2 |   org.apache.aries.jpa.eclipselink.adapter/2.6.1

// ERROR!

So something during the installation has caused FeaturesServiceImpl to stop the bundles, that are used by other features (which triggers their activators).

--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

Guillaume Nodet-2


2017-12-08 12:15 GMT+01:00 lechlukasz <[hidden email]>:
Ok thank you, all my features are verified by maven plugin.

So you say that no matter if I have features with dependency="true" or not,
if the feature is already installed, it will not be touched? 

So the observed behavior, that after installing new feature the blueprint
context from bundles from dependent bundle are destroyed and re-created has
nothing to do with my feature descriptor and I should search for the cause
for that behavior somewhere else?

Have I understood you correctly?


No, that's the simple case. What I said is that the bundles are not touched if they don't need to be changed.  So if they are refreshed, there's a reason.   The reason can be obtained using the following command:
  feature:install --simulate --verbose the-feature-to-install

It will list all the bundles that will be refreshed and the reason why.
From this list, find a bundle that you think should not be refreshed, it will point you to the reason why it is.  It may be because one of its imported package is refreshed too.  If so, you need to go up the tree and find a reason that you can actually fix. 
 



--
------------------------
Guillaume Nodet

Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

lechlukasz
This post was updated on .
I think I've found the offender.

The message says:

org.glassfish.jersey.containers.jersey-container-servlet-core/2.22.2 (Wired
to javax.persistence/2.1.1.v201509150925 which is being refreshed)

there's a line before :

 javax.persistence/2.1.1.v201509150925 (Should be wired to: org.apache.aries.jpa.container/2.7.0.SNAPSHOT (through [javax.persistence/2.1.1.v201509150925] osgi.wiring.package; filter:="(&(osgi.wiring.package=org.osgi.service.jpa)(version>=1.1.0))"))

The exact bundle is already installed in the system, in the exact the same version.  Why it need to be refreshed? how to find that out?

The jersey-karaf-feature descriptor (that I've found somewhere in the
internet, full of 'magic' numbers (start-levels) apparently needed to
install jax-rs) refers the persistence bundle:

<bundle
start-level="50">mvn:org.eclipse.persistence/javax.persistence/2.1.1</bundle>

Can this line cause the problems?


--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

Guillaume Nodet-2
You should have a line saying why javax.persistence/2.1.1.v201509150925 is being refreshed.
Can you paste the whole output ?

Guillaume

2017-12-08 13:53 GMT+01:00 lechlukasz <[hidden email]>:
I think I've found the offender.

The message says:

org.glassfish.jersey.containers.jersey-container-servlet-core/2.22.2 (Wired
to javax.persistence/2.1.1.v201509150925 which is being refreshed)

However, there's no indicator why it should be refreshed. The exact bundle
is already installed in the system, in the exact the same version.

The jersey-karaf-feature descriptor (that I've found somewhere in the
internet, full of 'magic' numbers (start-levels) apparently needed to
install jax-rs) refers the persistence bundle:

<bundle
start-level="50">mvn:org.eclipse.persistence/javax.persistence/2.1.1</bundle>

however, that feature is already installed, and is using the
2.1.1.v201509150925 version.

What can be the reason the bundle is wanted to be refreshed? how to find it
out?



--
------------------------
Guillaume Nodet

Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

lechlukasz
Bundles to install:
here are only my private bundles listed

  Bundles to refresh:
    com.eclipsesource.jaxrs.jaxrs-publisher/5.3.1 (Wired to
org.glassfish.jersey.containers.jersey-container-servlet-core/2.22.2 which
is being refreshed)
    javax.persistence/2.1.1.v201509150925 (Should be wired to:
org.apache.aries.jpa.container/2.7.0.SNAPSHOT (through
[javax.persistence/2.1.1.v201509150925] osgi.wiring.package;
filter:="(&(osgi.wiring.package=org.osgi.service.jpa)(version>=1.1.0))"))
    org.apache.aries.jpa.api/2.7.0.SNAPSHOT (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.apache.aries.jpa.blueprint/2.7.0.SNAPSHOT (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.apache.aries.jpa.container/2.7.0.SNAPSHOT (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.apache.aries.jpa.eclipselink.adapter/2.6.1 (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.apache.aries.jpa.javax.persistence_2.1/2.7.0.SNAPSHOT (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.apache.aries.jpa.support/2.7.0.SNAPSHOT (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.eclipse.persistence.core/2.6.4.v20160829-44060b6 (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.eclipse.persistence.jpa/2.6.4.v20160829-44060b6 (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.eclipse.persistence.moxy/2.6.4.v20160829-44060b6 (Wired to
org.eclipse.persistence.core/2.6.4.v20160829-44060b6 which is being
refreshed)
    org.glassfish.jersey.containers.jersey-container-servlet/2.22.2 (Wired
to org.glassfish.jersey.containers.jersey-container-servlet-core/2.22.2
which is being refreshed)
    org.glassfish.jersey.containers.jersey-container-servlet-core/2.22.2
(Wired to javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.ops4j.pax.logging.pax-logging-log4j2/1.10.1 (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)



--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

Guillaume Nodet-2
So your bundle is refreshed because it's wired to the glassfish jersey-container-servlet-core bundle, which itself is wired to javax.persistence bundle, refreshed because:
  Should be wired to: org.apache.aries.jpa.container/2.7.0.SNAPSHOT (through [javax.persistence/2.1.1.v201509150925] osgi.wiring.package; filter:="(&(osgi.wiring.package=org.osgi.service.jpa)(version>=1.1.0))"))

The javax.persistence bundle is wired to org.apache.aries.jpa.container/2.7.0.SNAPSHOT, which is .... a snapshot, so bound to be updated often.
In order to break this link, you need to use a released version of aries.jpa.container (well, never use snapshot in production...)

Guillaume

2017-12-08 16:02 GMT+01:00 lechlukasz <[hidden email]>:
Bundles to install:
here are only my private bundles listed

  Bundles to refresh:
    com.eclipsesource.jaxrs.jaxrs-publisher/5.3.1 (Wired to
org.glassfish.jersey.containers.jersey-container-servlet-core/2.22.2 which
is being refreshed)
    javax.persistence/2.1.1.v201509150925 (Should be wired to:
org.apache.aries.jpa.container/2.7.0.SNAPSHOT (through
[javax.persistence/2.1.1.v201509150925] osgi.wiring.package;
filter:="(&(osgi.wiring.package=org.osgi.service.jpa)(version>=1.1.0))"))
    org.apache.aries.jpa.api/2.7.0.SNAPSHOT (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.apache.aries.jpa.blueprint/2.7.0.SNAPSHOT (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.apache.aries.jpa.container/2.7.0.SNAPSHOT (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.apache.aries.jpa.eclipselink.adapter/2.6.1 (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.apache.aries.jpa.javax.persistence_2.1/2.7.0.SNAPSHOT (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.apache.aries.jpa.support/2.7.0.SNAPSHOT (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.eclipse.persistence.core/2.6.4.v20160829-44060b6 (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.eclipse.persistence.jpa/2.6.4.v20160829-44060b6 (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.eclipse.persistence.moxy/2.6.4.v20160829-44060b6 (Wired to
org.eclipse.persistence.core/2.6.4.v20160829-44060b6 which is being
refreshed)
    org.glassfish.jersey.containers.jersey-container-servlet/2.22.2 (Wired
to org.glassfish.jersey.containers.jersey-container-servlet-core/2.22.2
which is being refreshed)
    org.glassfish.jersey.containers.jersey-container-servlet-core/2.22.2
(Wired to javax.persistence/2.1.1.v201509150925 which is being refreshed)
    org.ops4j.pax.logging.pax-logging-log4j2/1.10.1 (Wired to
javax.persistence/2.1.1.v201509150925 which is being refreshed)






--
------------------------
Guillaume Nodet

Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

lechlukasz
OK so the code would not refresh if aries-jpa was not a snapshot?

Quite unfortunately, there is a HUGE (for us) bug in aries-jpa is fixed in
2.7.0 version, which is not released...

I'm able to do a 'fake' release to our local maven repository. I didn't
thought there's some logic in feature refreshing bound to that. Thanks for
explaining.



--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

Seth Leger
On 12/8/17 10:47 AM, lechlukasz wrote:
> I'm able to do a 'fake' release to our local maven repository. I didn't
> thought there's some logic in feature refreshing bound to that. Thanks for
> explaining.

Is this true? Does Karaf (or something inside Karaf) assume that
SNAPSHOT versions should automatically be refreshed when calculating
which bundles to refresh?

Seth Leger
The OpenNMS Group
Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

Guillaume Nodet-2
There's a checksum verification, so if the snapshot has changed, it will be updated whenever the FeaturesService resolver kicks in.
In short, if the snapshot has been rebuilt, it will be updated (not only refreshed).
This can be disabled though.

2017-12-11 19:49 GMT+01:00 Seth Leger <[hidden email]>:
On 12/8/17 10:47 AM, lechlukasz wrote:
> I'm able to do a 'fake' release to our local maven repository. I didn't
> thought there's some logic in feature refreshing bound to that. Thanks for
> explaining.

Is this true? Does Karaf (or something inside Karaf) assume that
SNAPSHOT versions should automatically be refreshed when calculating
which bundles to refresh?

Seth Leger
The OpenNMS Group



--
------------------------
Guillaume Nodet

Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

lechlukasz
So it doesn't look like my case. The update happens when the subsequent
features are batch-updated, there is no change of snapshot in the meantime.

If it can be disabled, how? How is that mechanism called and which part of
code is responsible for that? I'd like to take look on java sources to know
what can I expect and what could happen.



--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

Guillaume Nodet-2
Please raise a JIRA, explain which bundle is updated when you think it should not, and attach the full log, I'll try to have a quick look.

2017-12-13 15:51 GMT+01:00 lechlukasz <[hidden email]>:
So it doesn't look like my case. The update happens when the subsequent
features are batch-updated, there is no change of snapshot in the meantime.

If it can be disabled, how? How is that mechanism called and which part of
code is responsible for that? I'd like to take look on java sources to know
what can I expect and what could happen.



--
------------------------
Guillaume Nodet

Reply | Threaded
Open this post in threaded view
|

Re: Confused about feature dependency="true"

lechlukasz