Karaf console and Embed Mongo...

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Karaf console and Embed Mongo...

rkmoquin
This might be better to ask of the Embed Mongo team, but I am not entirely sure.  I've been experimenting with an Embedded Mongo running inside Karaf, which I've gotten to work successfully with one caveat that is driving me crazy trying to troubleshoot (I think I understand what's happening, but not exactly what makes it happen).  If I have a feature which installs a bundle that created and starts a Flapdoodle Embed Mongo instance, then the Karaf console will "hang" since I think the Embed Mongo code, kind of "hijacking" the Karaf JLine input stream (it attaches some input streams for the Mongo process.)  I did switch the outputstreams to use SLF4J which helped a bit, but there is one last small issue I can't figure out.  The reason I'm wondering if someone on this list might have some ideas is because if when I install that feature, I force kill the Karaf process and start it back up, then the Embed Mongo process ends up running in the background without "hijacking" the Karaf jline console.

I guess I'm wondering if there is a way to initially install my embed mongo feature and prevent it from grabbing the Karaf JLine stream if I understand how the feature startup is different than than when installing a feature (around the Karaf console input).  I hope what I'm asking actually makes sense to someone.  Basically, the feature command doesn't return control to the Karaf console after installing that feature due to what the Embedded mongo process does under the covers.... I've been debugging the embed mongo code to see if I can figure out how this conflict is happening.. it appears one of the threads that reads an inputstream from mongod is the culprit, but this is only a problem when installing the feature, not running karaf after the feature has been installed.....

Thanks for any advice!

Ryan
Reply | Threaded
Open this post in threaded view
|

Re: Karaf console and Embed Mongo...

Seth Leger
Hi Ryan,

You are probably running into an issue where some dependency of Mongo is
triggering a refresh of the system bundles inside Karaf. Mongo probably
isn't "hijacking" jline; it's just causing jline + karaf shell to
refresh which kills your client shell. There are several ways to do this
on Karaf 4.1, I documented one way to do it (with Apache MINA) in this bug:

https://issues.apache.org/jira/browse/KARAF-5384

If you use:

feature:install -t -v your-problematic-feature

then you should be able to track down which bundle is triggering the
refresh. Also, what version of Karaf are you using?

Seth Leger
The OpenNMS Group


On 1/21/18 12:23 PM, Ryan Moquin wrote:

> This might be better to ask of the Embed Mongo team, but I am not
> entirely sure.  I've been experimenting with an Embedded Mongo running
> inside Karaf, which I've gotten to work successfully with one caveat
> that is driving me crazy trying to troubleshoot (I think I understand
> what's happening, but not exactly what makes it happen).  If I have a
> feature which installs a bundle that created and starts a Flapdoodle
> Embed Mongo instance, then the Karaf console will "hang" since I think
> the Embed Mongo code, kind of "hijacking" the Karaf JLine input stream
> (it attaches some input streams for the Mongo process.)  I did switch
> the outputstreams to use SLF4J which helped a bit, but there is one last
> small issue I can't figure out.  The reason I'm wondering if someone on
> this list might have some ideas is because if when I install that
> feature, I force kill the Karaf process and start it back up, then the
> Embed Mongo process ends up running in the background without
> "hijacking" the Karaf jline console.
>
> I guess I'm wondering if there is a way to initially install my embed
> mongo feature and prevent it from grabbing the Karaf JLine stream if I
> understand how the feature startup is different than than when
> installing a feature (around the Karaf console input).  I hope what I'm
> asking actually makes sense to someone.  Basically, the feature command
> doesn't return control to the Karaf console after installing that
> feature due to what the Embedded mongo process does under the covers....
> I've been debugging the embed mongo code to see if I can figure out how
> this conflict is happening.. it appears one of the threads that reads an
> inputstream from mongod is the culprit, but this is only a problem when
> installing the feature, not running karaf after the feature has been
> installed.....
>
> Thanks for any advice!
>
> Ryan

Reply | Threaded
Open this post in threaded view
|

Re: Karaf console and Embed Mongo...

jbonofre
In reply to this post by rkmoquin
Hi Ryan,

It could be JLine, or simply your Activator doesn't give back hand. Can you
share your code ?

Actually, I have a "old" Mongo embedded instance on a private branch that I can
share with you.

Regards
JB

On 01/21/2018 06:23 PM, Ryan Moquin wrote:

> This might be better to ask of the Embed Mongo team, but I am not entirely
> sure.  I've been experimenting with an Embedded Mongo running inside Karaf,
> which I've gotten to work successfully with one caveat that is driving me crazy
> trying to troubleshoot (I think I understand what's happening, but not exactly
> what makes it happen).  If I have a feature which installs a bundle that created
> and starts a Flapdoodle Embed Mongo instance, then the Karaf console will "hang"
> since I think the Embed Mongo code, kind of "hijacking" the Karaf JLine input
> stream (it attaches some input streams for the Mongo process.)  I did switch the
> outputstreams to use SLF4J which helped a bit, but there is one last small issue
> I can't figure out.  The reason I'm wondering if someone on this list might have
> some ideas is because if when I install that feature, I force kill the Karaf
> process and start it back up, then the Embed Mongo process ends up running in
> the background without "hijacking" the Karaf jline console.
>
> I guess I'm wondering if there is a way to initially install my embed mongo
> feature and prevent it from grabbing the Karaf JLine stream if I understand how
> the feature startup is different than than when installing a feature (around the
> Karaf console input).  I hope what I'm asking actually makes sense to someone. 
> Basically, the feature command doesn't return control to the Karaf console after
> installing that feature due to what the Embedded mongo process does under the
> covers.... I've been debugging the embed mongo code to see if I can figure out
> how this conflict is happening.. it appears one of the threads that reads an
> inputstream from mongod is the culprit, but this is only a problem when
> installing the feature, not running karaf after the feature has been installed.....
>
> Thanks for any advice!
>
> Ryan

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Karaf console and Embed Mongo...

rkmoquin
In reply to this post by Seth Leger
Good point, I don't know why I completely forgot to verify that.  I could have SWORE when I switched the EmbedMongo config to use the SLF4J appender for output, the problem had disappeared, but then all of a sudden it came back (I really could be crazy).  I think because I thought I had fixed the issue, I am a little discombobulated around this one.  It probably is something getting refreshed under the covers, I'll feel dumb if that's the case since I was just troubleshooting some similar stuff recently. :)

Ryan

On Mon, Jan 22, 2018 at 12:25 AM Seth Leger <[hidden email]> wrote:
Hi Ryan,

You are probably running into an issue where some dependency of Mongo is
triggering a refresh of the system bundles inside Karaf. Mongo probably
isn't "hijacking" jline; it's just causing jline + karaf shell to
refresh which kills your client shell. There are several ways to do this
on Karaf 4.1, I documented one way to do it (with Apache MINA) in this bug:

https://issues.apache.org/jira/browse/KARAF-5384

If you use:

feature:install -t -v your-problematic-feature

then you should be able to track down which bundle is triggering the
refresh. Also, what version of Karaf are you using?

Seth Leger
The OpenNMS Group


On 1/21/18 12:23 PM, Ryan Moquin wrote:
> This might be better to ask of the Embed Mongo team, but I am not
> entirely sure.  I've been experimenting with an Embedded Mongo running
> inside Karaf, which I've gotten to work successfully with one caveat
> that is driving me crazy trying to troubleshoot (I think I understand
> what's happening, but not exactly what makes it happen).  If I have a
> feature which installs a bundle that created and starts a Flapdoodle
> Embed Mongo instance, then the Karaf console will "hang" since I think
> the Embed Mongo code, kind of "hijacking" the Karaf JLine input stream
> (it attaches some input streams for the Mongo process.)  I did switch
> the outputstreams to use SLF4J which helped a bit, but there is one last
> small issue I can't figure out.  The reason I'm wondering if someone on
> this list might have some ideas is because if when I install that
> feature, I force kill the Karaf process and start it back up, then the
> Embed Mongo process ends up running in the background without
> "hijacking" the Karaf jline console.
>
> I guess I'm wondering if there is a way to initially install my embed
> mongo feature and prevent it from grabbing the Karaf JLine stream if I
> understand how the feature startup is different than than when
> installing a feature (around the Karaf console input).  I hope what I'm
> asking actually makes sense to someone.  Basically, the feature command
> doesn't return control to the Karaf console after installing that
> feature due to what the Embedded mongo process does under the covers....
> I've been debugging the embed mongo code to see if I can figure out how
> this conflict is happening.. it appears one of the threads that reads an
> inputstream from mongod is the culprit, but this is only a problem when
> installing the feature, not running karaf after the feature has been
> installed.....
>
> Thanks for any advice!
>
> Ryan

Reply | Threaded
Open this post in threaded view
|

Re: Karaf console and Embed Mongo...

rkmoquin
In reply to this post by Seth Leger
Thanks Seth, I feel stupid for completely forgetting to check if the problem is because of the karaf bundles getting refreshed since that turned out to be the reason. In this case though, it's not Mina, it's because of JLine and the JNA library (which would be a good candidate to preinstall.) It stinks to see all these get refreshed because of a dependency getting added that happens to be an option dependency of a system bundle.  It seems like this happens quite a lot (I find the log4j2 bundle seems to end up getting refreshed a lot as well) and makes me feel there has to be some sort of intuitive way to prevent it, especially because it feels like it could in general hurt Karaf's adoption when used with an OOTB distribution (I can work around it with a custom distro.)  Out of curiousity, maybe at least a small enhancement which would at least prevent this issue from confusing people (even those who are aware of it but are asleep at the wheel that day) could be to show a warning in the console that some system bundles will be refreshed which can cause some weird stuff.  

In case interested, this is where the problems are (since mongo embed needs the com.sun.jna bundle):

    org.jline/3.4.0 (Should be wired to: com.sun.jna/4.5.1 (through [org.jline/3.4.0] osgi.wiring.package; filter:="(osgi.wiring.package=com.sun.jna)"; resolution:=optional))
    org.ops4j.pax.logging.pax-logging-log4j2/1.10.1 (Should be wired to: org.apache.commons.compress/1.10.0 (through [org.ops4j.pax.logging.pax-logging-log4j2/1.10.1] osgi.wiring.package; filter:="(osgi.wiring.package=org.apache.commons.compress.compressors)"; resolution:=optional))

Thanks again Seth, this was driving me nuts, I think my brain was telling me to ask here because I've seen this before (just forgot about it for some reason).  I'll make a change and am sure it will resolve.  Also, I moved to Karaf 4.1.4 since I started having these problems with 4.2.x and decided at that rate, might as well go to 4.1.4.  Maybe I'll just switch back to working with 4.2.x.  Choices. choices.

Ryan

On Mon, Jan 22, 2018 at 12:25 AM Seth Leger <[hidden email]> wrote:
Hi Ryan,

You are probably running into an issue where some dependency of Mongo is
triggering a refresh of the system bundles inside Karaf. Mongo probably
isn't "hijacking" jline; it's just causing jline + karaf shell to
refresh which kills your client shell. There are several ways to do this
on Karaf 4.1, I documented one way to do it (with Apache MINA) in this bug:

https://issues.apache.org/jira/browse/KARAF-5384

If you use:

feature:install -t -v your-problematic-feature

then you should be able to track down which bundle is triggering the
refresh. Also, what version of Karaf are you using?

Seth Leger
The OpenNMS Group


On 1/21/18 12:23 PM, Ryan Moquin wrote:
> This might be better to ask of the Embed Mongo team, but I am not
> entirely sure.  I've been experimenting with an Embedded Mongo running
> inside Karaf, which I've gotten to work successfully with one caveat
> that is driving me crazy trying to troubleshoot (I think I understand
> what's happening, but not exactly what makes it happen).  If I have a
> feature which installs a bundle that created and starts a Flapdoodle
> Embed Mongo instance, then the Karaf console will "hang" since I think
> the Embed Mongo code, kind of "hijacking" the Karaf JLine input stream
> (it attaches some input streams for the Mongo process.)  I did switch
> the outputstreams to use SLF4J which helped a bit, but there is one last
> small issue I can't figure out.  The reason I'm wondering if someone on
> this list might have some ideas is because if when I install that
> feature, I force kill the Karaf process and start it back up, then the
> Embed Mongo process ends up running in the background without
> "hijacking" the Karaf jline console.
>
> I guess I'm wondering if there is a way to initially install my embed
> mongo feature and prevent it from grabbing the Karaf JLine stream if I
> understand how the feature startup is different than than when
> installing a feature (around the Karaf console input).  I hope what I'm
> asking actually makes sense to someone.  Basically, the feature command
> doesn't return control to the Karaf console after installing that
> feature due to what the Embedded mongo process does under the covers....
> I've been debugging the embed mongo code to see if I can figure out how
> this conflict is happening.. it appears one of the threads that reads an
> inputstream from mongod is the culprit, but this is only a problem when
> installing the feature, not running karaf after the feature has been
> installed.....
>
> Thanks for any advice!
>
> Ryan

Reply | Threaded
Open this post in threaded view
|

Re: Karaf console and Embed Mongo...

jbonofre
By the way, you can always add such bundle in etc/startup.properties (or
corresponding boot features) to avoid the refresh.

I did that for Cassandra (used in Decanter) for instance.

Regards
JB

On 01/22/2018 04:13 PM, Ryan Moquin wrote:

> Thanks Seth, I feel stupid for completely forgetting to check if the problem is
> because of the karaf bundles getting refreshed since that turned out to be the
> reason. In this case though, it's not Mina, it's because of JLine and the JNA
> library (which would be a good candidate to preinstall.) It stinks to see all
> these get refreshed because of a dependency getting added that happens to be an
> option dependency of a system bundle.  It seems like this happens quite a lot (I
> find the log4j2 bundle seems to end up getting refreshed a lot as well) and
> makes me feel there has to be some sort of intuitive way to prevent it,
> especially because it feels like it could in general hurt Karaf's adoption when
> used with an OOTB distribution (I can work around it with a custom distro.)  Out
> of curiousity, maybe at least a small enhancement which would at least prevent
> this issue from confusing people (even those who are aware of it but are asleep
> at the wheel that day) could be to show a warning in the console that some
> system bundles will be refreshed which can cause some weird stuff.  
>
> In case interested, this is where the problems are (since mongo embed needs the
> com.sun.jna bundle):
>
>     org.jline/3.4.0 (Should be wired to: com.sun.jna/4.5.1 (through
> [org.jline/3.4.0] osgi.wiring.package;
> filter:="(osgi.wiring.package=com.sun.jna)"; resolution:=optional))
>     org.ops4j.pax.logging.pax-logging-log4j2/1.10.1 (Should be wired to:
> org.apache.commons.compress/1.10.0 (through
> [org.ops4j.pax.logging.pax-logging-log4j2/1.10.1] osgi.wiring.package;
> filter:="(osgi.wiring.package=org.apache.commons.compress.compressors)";
> resolution:=optional))
>
> Thanks again Seth, this was driving me nuts, I think my brain was telling me to
> ask here because I've seen this before (just forgot about it for some reason). 
> I'll make a change and am sure it will resolve.  Also, I moved to Karaf 4.1.4
> since I started having these problems with 4.2.x and decided at that rate, might
> as well go to 4.1.4.  Maybe I'll just switch back to working with 4.2.x. 
> Choices. choices.
>
> Ryan
>
> On Mon, Jan 22, 2018 at 12:25 AM Seth Leger <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Ryan,
>
>     You are probably running into an issue where some dependency of Mongo is
>     triggering a refresh of the system bundles inside Karaf. Mongo probably
>     isn't "hijacking" jline; it's just causing jline + karaf shell to
>     refresh which kills your client shell. There are several ways to do this
>     on Karaf 4.1, I documented one way to do it (with Apache MINA) in this bug:
>
>     https://issues.apache.org/jira/browse/KARAF-5384
>
>     If you use:
>
>     feature:install -t -v your-problematic-feature
>
>     then you should be able to track down which bundle is triggering the
>     refresh. Also, what version of Karaf are you using?
>
>     Seth Leger
>     The OpenNMS Group
>
>
>     On 1/21/18 12:23 PM, Ryan Moquin wrote:
>     > This might be better to ask of the Embed Mongo team, but I am not
>     > entirely sure.  I've been experimenting with an Embedded Mongo running
>     > inside Karaf, which I've gotten to work successfully with one caveat
>     > that is driving me crazy trying to troubleshoot (I think I understand
>     > what's happening, but not exactly what makes it happen).  If I have a
>     > feature which installs a bundle that created and starts a Flapdoodle
>     > Embed Mongo instance, then the Karaf console will "hang" since I think
>     > the Embed Mongo code, kind of "hijacking" the Karaf JLine input stream
>     > (it attaches some input streams for the Mongo process.)  I did switch
>     > the outputstreams to use SLF4J which helped a bit, but there is one last
>     > small issue I can't figure out.  The reason I'm wondering if someone on
>     > this list might have some ideas is because if when I install that
>     > feature, I force kill the Karaf process and start it back up, then the
>     > Embed Mongo process ends up running in the background without
>     > "hijacking" the Karaf jline console.
>     >
>     > I guess I'm wondering if there is a way to initially install my embed
>     > mongo feature and prevent it from grabbing the Karaf JLine stream if I
>     > understand how the feature startup is different than than when
>     > installing a feature (around the Karaf console input).  I hope what I'm
>     > asking actually makes sense to someone.  Basically, the feature command
>     > doesn't return control to the Karaf console after installing that
>     > feature due to what the Embedded mongo process does under the covers....
>     > I've been debugging the embed mongo code to see if I can figure out how
>     > this conflict is happening.. it appears one of the threads that reads an
>     > inputstream from mongod is the culprit, but this is only a problem when
>     > installing the feature, not running karaf after the feature has been
>     > installed.....
>     >
>     > Thanks for any advice!
>     >
>     > Ryan
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Karaf console and Embed Mongo...

rkmoquin
Thanks JB, I was just looking to see how to add that in the startup.properties without making one more file to worry about becoming out of sync with Karaf upgrades :)

On Mon, Jan 22, 2018 at 10:19 AM Jean-Baptiste Onofré <[hidden email]> wrote:
By the way, you can always add such bundle in etc/startup.properties (or
corresponding boot features) to avoid the refresh.

I did that for Cassandra (used in Decanter) for instance.

Regards
JB

On 01/22/2018 04:13 PM, Ryan Moquin wrote:
> Thanks Seth, I feel stupid for completely forgetting to check if the problem is
> because of the karaf bundles getting refreshed since that turned out to be the
> reason. In this case though, it's not Mina, it's because of JLine and the JNA
> library (which would be a good candidate to preinstall.) It stinks to see all
> these get refreshed because of a dependency getting added that happens to be an
> option dependency of a system bundle.  It seems like this happens quite a lot (I
> find the log4j2 bundle seems to end up getting refreshed a lot as well) and
> makes me feel there has to be some sort of intuitive way to prevent it,
> especially because it feels like it could in general hurt Karaf's adoption when
> used with an OOTB distribution (I can work around it with a custom distro.)  Out
> of curiousity, maybe at least a small enhancement which would at least prevent
> this issue from confusing people (even those who are aware of it but are asleep
> at the wheel that day) could be to show a warning in the console that some
> system bundles will be refreshed which can cause some weird stuff.  
>
> In case interested, this is where the problems are (since mongo embed needs the
> com.sun.jna bundle):
>
>     org.jline/3.4.0 (Should be wired to: com.sun.jna/4.5.1 (through
> [org.jline/3.4.0] osgi.wiring.package;
> filter:="(osgi.wiring.package=com.sun.jna)"; resolution:=optional))
>     org.ops4j.pax.logging.pax-logging-log4j2/1.10.1 (Should be wired to:
> org.apache.commons.compress/1.10.0 (through
> [org.ops4j.pax.logging.pax-logging-log4j2/1.10.1] osgi.wiring.package;
> filter:="(osgi.wiring.package=org.apache.commons.compress.compressors)";
> resolution:=optional))
>
> Thanks again Seth, this was driving me nuts, I think my brain was telling me to
> ask here because I've seen this before (just forgot about it for some reason). 
> I'll make a change and am sure it will resolve.  Also, I moved to Karaf 4.1.4
> since I started having these problems with 4.2.x and decided at that rate, might
> as well go to 4.1.4.  Maybe I'll just switch back to working with 4.2.x. 
> Choices. choices.
>
> Ryan
>
> On Mon, Jan 22, 2018 at 12:25 AM Seth Leger <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Ryan,
>
>     You are probably running into an issue where some dependency of Mongo is
>     triggering a refresh of the system bundles inside Karaf. Mongo probably
>     isn't "hijacking" jline; it's just causing jline + karaf shell to
>     refresh which kills your client shell. There are several ways to do this
>     on Karaf 4.1, I documented one way to do it (with Apache MINA) in this bug:
>
>     https://issues.apache.org/jira/browse/KARAF-5384
>
>     If you use:
>
>     feature:install -t -v your-problematic-feature
>
>     then you should be able to track down which bundle is triggering the
>     refresh. Also, what version of Karaf are you using?
>
>     Seth Leger
>     The OpenNMS Group
>
>
>     On 1/21/18 12:23 PM, Ryan Moquin wrote:
>     > This might be better to ask of the Embed Mongo team, but I am not
>     > entirely sure.  I've been experimenting with an Embedded Mongo running
>     > inside Karaf, which I've gotten to work successfully with one caveat
>     > that is driving me crazy trying to troubleshoot (I think I understand
>     > what's happening, but not exactly what makes it happen).  If I have a
>     > feature which installs a bundle that created and starts a Flapdoodle
>     > Embed Mongo instance, then the Karaf console will "hang" since I think
>     > the Embed Mongo code, kind of "hijacking" the Karaf JLine input stream
>     > (it attaches some input streams for the Mongo process.)  I did switch
>     > the outputstreams to use SLF4J which helped a bit, but there is one last
>     > small issue I can't figure out.  The reason I'm wondering if someone on
>     > this list might have some ideas is because if when I install that
>     > feature, I force kill the Karaf process and start it back up, then the
>     > Embed Mongo process ends up running in the background without
>     > "hijacking" the Karaf jline console.
>     >
>     > I guess I'm wondering if there is a way to initially install my embed
>     > mongo feature and prevent it from grabbing the Karaf JLine stream if I
>     > understand how the feature startup is different than than when
>     > installing a feature (around the Karaf console input).  I hope what I'm
>     > asking actually makes sense to someone.  Basically, the feature command
>     > doesn't return control to the Karaf console after installing that
>     > feature due to what the Embedded mongo process does under the covers....
>     > I've been debugging the embed mongo code to see if I can figure out how
>     > this conflict is happening.. it appears one of the threads that reads an
>     > inputstream from mongod is the culprit, but this is only a problem when
>     > installing the feature, not running karaf after the feature has been
>     > installed.....
>     >
>     > Thanks for any advice!
>     >
>     > Ryan
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Karaf console and Embed Mongo...

Seth Leger
In reply to this post by rkmoquin
Yep, I had the same problem with JNA. Glad you tracked it down.

https://issues.apache.org/jira/browse/KARAF-5251

Seth Leger
The OpenNMS Group


On 1/22/18 10:13 AM, Ryan Moquin wrote:

> Thanks Seth, I feel stupid for completely forgetting to check if the
> problem is because of the karaf bundles getting refreshed since that
> turned out to be the reason. In this case though, it's not Mina, it's
> because of JLine and the JNA library (which would be a good candidate to
> preinstall.) It stinks to see all these get refreshed because of a
> dependency getting added that happens to be an option dependency of a
> system bundle.  It seems like this happens quite a lot (I find the
> log4j2 bundle seems to end up getting refreshed a lot as well) and makes
> me feel there has to be some sort of intuitive way to prevent it,
> especially because it feels like it could in general hurt Karaf's
> adoption when used with an OOTB distribution (I can work around it with
> a custom distro.)  Out of curiousity, maybe at least a small enhancement
> which would at least prevent this issue from confusing people (even
> those who are aware of it but are asleep at the wheel that day) could be
> to show a warning in the console that some system bundles will be
> refreshed which can cause some weird stuff.  
>
> In case interested, this is where the problems are (since mongo embed
> needs the com.sun.jna bundle):
>
>     org.jline/3.4.0 (Should be wired to: com.sun.jna/4.5.1 (through
> [org.jline/3.4.0] osgi.wiring.package;
> filter:="(osgi.wiring.package=com.sun.jna)"; resolution:=optional))
>     org.ops4j.pax.logging.pax-logging-log4j2/1.10.1 (Should be wired to:
> org.apache.commons.compress/1.10.0 (through
> [org.ops4j.pax.logging.pax-logging-log4j2/1.10.1] osgi.wiring.package;
> filter:="(osgi.wiring.package=org.apache.commons.compress.compressors)";
> resolution:=optional))
>
> Thanks again Seth, this was driving me nuts, I think my brain was
> telling me to ask here because I've seen this before (just forgot about
> it for some reason).  I'll make a change and am sure it will resolve. 
> Also, I moved to Karaf 4.1.4 since I started having these problems with
> 4.2.x and decided at that rate, might as well go to 4.1.4.  Maybe I'll
> just switch back to working with 4.2.x.  Choices. choices.
>
> Ryan
>
> On Mon, Jan 22, 2018 at 12:25 AM Seth Leger <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Ryan,
>
>     You are probably running into an issue where some dependency of Mongo is
>     triggering a refresh of the system bundles inside Karaf. Mongo probably
>     isn't "hijacking" jline; it's just causing jline + karaf shell to
>     refresh which kills your client shell. There are several ways to do this
>     on Karaf 4.1, I documented one way to do it (with Apache MINA) in
>     this bug:
>
>     https://issues.apache.org/jira/browse/KARAF-5384
>
>     If you use:
>
>     feature:install -t -v your-problematic-feature
>
>     then you should be able to track down which bundle is triggering the
>     refresh. Also, what version of Karaf are you using?
>
>     Seth Leger
>     The OpenNMS Group
>
>
>     On 1/21/18 12:23 PM, Ryan Moquin wrote:
>     > This might be better to ask of the Embed Mongo team, but I am not
>     > entirely sure.  I've been experimenting with an Embedded Mongo running
>     > inside Karaf, which I've gotten to work successfully with one caveat
>     > that is driving me crazy trying to troubleshoot (I think I understand
>     > what's happening, but not exactly what makes it happen).  If I have a
>     > feature which installs a bundle that created and starts a Flapdoodle
>     > Embed Mongo instance, then the Karaf console will "hang" since I think
>     > the Embed Mongo code, kind of "hijacking" the Karaf JLine input stream
>     > (it attaches some input streams for the Mongo process.)  I did switch
>     > the outputstreams to use SLF4J which helped a bit, but there is
>     one last
>     > small issue I can't figure out.  The reason I'm wondering if
>     someone on
>     > this list might have some ideas is because if when I install that
>     > feature, I force kill the Karaf process and start it back up, then the
>     > Embed Mongo process ends up running in the background without
>     > "hijacking" the Karaf jline console.
>     >
>     > I guess I'm wondering if there is a way to initially install my embed
>     > mongo feature and prevent it from grabbing the Karaf JLine stream if I
>     > understand how the feature startup is different than than when
>     > installing a feature (around the Karaf console input).  I hope
>     what I'm
>     > asking actually makes sense to someone.  Basically, the feature
>     command
>     > doesn't return control to the Karaf console after installing that
>     > feature due to what the Embedded mongo process does under the
>     covers....
>     > I've been debugging the embed mongo code to see if I can figure
>     out how
>     > this conflict is happening.. it appears one of the threads that
>     reads an
>     > inputstream from mongod is the culprit, but this is only a problem
>     when
>     > installing the feature, not running karaf after the feature has been
>     > installed.....
>     >
>     > Thanks for any advice!
>     >
>     > Ryan
>

Reply | Threaded
Open this post in threaded view
|

Re: Karaf console and Embed Mongo...

jbonofre
Yes, but as said, I don't think it's good to define JNA by default in Karaf.

WDYT ?

Regards
JB

On 01/22/2018 04:36 PM, Seth Leger wrote:

> Yep, I had the same problem with JNA. Glad you tracked it down.
>
> https://issues.apache.org/jira/browse/KARAF-5251
>
> Seth Leger
> The OpenNMS Group
>
>
> On 1/22/18 10:13 AM, Ryan Moquin wrote:
>> Thanks Seth, I feel stupid for completely forgetting to check if the
>> problem is because of the karaf bundles getting refreshed since that
>> turned out to be the reason. In this case though, it's not Mina, it's
>> because of JLine and the JNA library (which would be a good candidate to
>> preinstall.) It stinks to see all these get refreshed because of a
>> dependency getting added that happens to be an option dependency of a
>> system bundle.  It seems like this happens quite a lot (I find the
>> log4j2 bundle seems to end up getting refreshed a lot as well) and makes
>> me feel there has to be some sort of intuitive way to prevent it,
>> especially because it feels like it could in general hurt Karaf's
>> adoption when used with an OOTB distribution (I can work around it with
>> a custom distro.)  Out of curiousity, maybe at least a small enhancement
>> which would at least prevent this issue from confusing people (even
>> those who are aware of it but are asleep at the wheel that day) could be
>> to show a warning in the console that some system bundles will be
>> refreshed which can cause some weird stuff.  
>>
>> In case interested, this is where the problems are (since mongo embed
>> needs the com.sun.jna bundle):
>>
>>     org.jline/3.4.0 (Should be wired to: com.sun.jna/4.5.1 (through
>> [org.jline/3.4.0] osgi.wiring.package;
>> filter:="(osgi.wiring.package=com.sun.jna)"; resolution:=optional))
>>     org.ops4j.pax.logging.pax-logging-log4j2/1.10.1 (Should be wired to:
>> org.apache.commons.compress/1.10.0 (through
>> [org.ops4j.pax.logging.pax-logging-log4j2/1.10.1] osgi.wiring.package;
>> filter:="(osgi.wiring.package=org.apache.commons.compress.compressors)";
>> resolution:=optional))
>>
>> Thanks again Seth, this was driving me nuts, I think my brain was
>> telling me to ask here because I've seen this before (just forgot about
>> it for some reason).  I'll make a change and am sure it will resolve. 
>> Also, I moved to Karaf 4.1.4 since I started having these problems with
>> 4.2.x and decided at that rate, might as well go to 4.1.4.  Maybe I'll
>> just switch back to working with 4.2.x.  Choices. choices.
>>
>> Ryan
>>
>> On Mon, Jan 22, 2018 at 12:25 AM Seth Leger <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>     Hi Ryan,
>>
>>     You are probably running into an issue where some dependency of Mongo is
>>     triggering a refresh of the system bundles inside Karaf. Mongo probably
>>     isn't "hijacking" jline; it's just causing jline + karaf shell to
>>     refresh which kills your client shell. There are several ways to do this
>>     on Karaf 4.1, I documented one way to do it (with Apache MINA) in
>>     this bug:
>>
>>     https://issues.apache.org/jira/browse/KARAF-5384
>>
>>     If you use:
>>
>>     feature:install -t -v your-problematic-feature
>>
>>     then you should be able to track down which bundle is triggering the
>>     refresh. Also, what version of Karaf are you using?
>>
>>     Seth Leger
>>     The OpenNMS Group
>>
>>
>>     On 1/21/18 12:23 PM, Ryan Moquin wrote:
>>     > This might be better to ask of the Embed Mongo team, but I am not
>>     > entirely sure.  I've been experimenting with an Embedded Mongo running
>>     > inside Karaf, which I've gotten to work successfully with one caveat
>>     > that is driving me crazy trying to troubleshoot (I think I understand
>>     > what's happening, but not exactly what makes it happen).  If I have a
>>     > feature which installs a bundle that created and starts a Flapdoodle
>>     > Embed Mongo instance, then the Karaf console will "hang" since I think
>>     > the Embed Mongo code, kind of "hijacking" the Karaf JLine input stream
>>     > (it attaches some input streams for the Mongo process.)  I did switch
>>     > the outputstreams to use SLF4J which helped a bit, but there is
>>     one last
>>     > small issue I can't figure out.  The reason I'm wondering if
>>     someone on
>>     > this list might have some ideas is because if when I install that
>>     > feature, I force kill the Karaf process and start it back up, then the
>>     > Embed Mongo process ends up running in the background without
>>     > "hijacking" the Karaf jline console.
>>     >
>>     > I guess I'm wondering if there is a way to initially install my embed
>>     > mongo feature and prevent it from grabbing the Karaf JLine stream if I
>>     > understand how the feature startup is different than than when
>>     > installing a feature (around the Karaf console input).  I hope
>>     what I'm
>>     > asking actually makes sense to someone.  Basically, the feature
>>     command
>>     > doesn't return control to the Karaf console after installing that
>>     > feature due to what the Embedded mongo process does under the
>>     covers....
>>     > I've been debugging the embed mongo code to see if I can figure
>>     out how
>>     > this conflict is happening.. it appears one of the threads that
>>     reads an
>>     > inputstream from mongod is the culprit, but this is only a problem
>>     when
>>     > installing the feature, not running karaf after the feature has been
>>     > installed.....
>>     >
>>     > Thanks for any advice!
>>     >
>>     > Ryan
>>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Karaf console and Embed Mongo...

rkmoquin
I kind of feel that it might be better to ask why it wouldn't be good to define it by default.  I normally would agree with you, but due to how many refreshes it causes without some manual intervention beforehand, I wonder what would be the downsides to including it by default?  Is there a particular disadvantage to it being in the distribution by default?  On the other hand, if there are a lot of libraries that will result in the safe kinds of problems that are solved by adding as a startup bundle, then it probably isn't worth it.  Since I particularly really love Karaf, I wonder if the inclusion of it could potentially alleviate some pains for newer users in the future.  But as I said, if there are more situations similar to this that wouldn't be solved by adding that jar, then probably isn't much point.

At the very least, I think for sanity's sake, it might be nice if one or more of these three alternatives are possible (I don't know if there are any negatives to any of these, I just hate to see anyone struggle with sort of problem in the future):

1.  Have the features service print a warning when a feature is being installed for the first time to say that Karaf will need restarted after the feature is installed, anything is better than the appearance that Karaf or the console is hanging (or other side effects.)
2.  Maybe provide an option on a feature to indicate that installation of the feature should not cause any existing bundles that optionally depend on the contained bundles to refresh?  This could cause a restart of Karaf to result in some different behavior at startup due to an optional dependency being satisfied, or it feels like that could be a side effect.  I'm honestly not a big proponent of adding gazillions of configuration parameters for things since it can be confusing (I still can't quite understand the use of dependency or prerequisite on a feature definition.)
3. Does it make any sense to only limit a refresh to only direct bundles?  Would that solve anything?  It doesn't feel like "transitive optionals" would ever make sense, such as if JLine is refreshed due to an optional dependency becoming available, is there any reason that bundles using JLine would really have to be refreshed?  If you ask Karaf to refresh a particular bundle using "bundle:refresh", it doesn't refresh all the bundles depending on the refreshed bundle, does it?

Those are my thoughts on it, hopefully they might be useful. :)

Ryan

On Mon, Jan 22, 2018 at 10:56 AM Jean-Baptiste Onofré <[hidden email]> wrote:
Yes, but as said, I don't think it's good to define JNA by default in Karaf.

WDYT ?

Regards
JB

On 01/22/2018 04:36 PM, Seth Leger wrote:
> Yep, I had the same problem with JNA. Glad you tracked it down.
>
> https://issues.apache.org/jira/browse/KARAF-5251
>
> Seth Leger
> The OpenNMS Group
>
>
> On 1/22/18 10:13 AM, Ryan Moquin wrote:
>> Thanks Seth, I feel stupid for completely forgetting to check if the
>> problem is because of the karaf bundles getting refreshed since that
>> turned out to be the reason. In this case though, it's not Mina, it's
>> because of JLine and the JNA library (which would be a good candidate to
>> preinstall.) It stinks to see all these get refreshed because of a
>> dependency getting added that happens to be an option dependency of a
>> system bundle.  It seems like this happens quite a lot (I find the
>> log4j2 bundle seems to end up getting refreshed a lot as well) and makes
>> me feel there has to be some sort of intuitive way to prevent it,
>> especially because it feels like it could in general hurt Karaf's
>> adoption when used with an OOTB distribution (I can work around it with
>> a custom distro.)  Out of curiousity, maybe at least a small enhancement
>> which would at least prevent this issue from confusing people (even
>> those who are aware of it but are asleep at the wheel that day) could be
>> to show a warning in the console that some system bundles will be
>> refreshed which can cause some weird stuff.  
>>
>> In case interested, this is where the problems are (since mongo embed
>> needs the com.sun.jna bundle):
>>
>>     org.jline/3.4.0 (Should be wired to: com.sun.jna/4.5.1 (through
>> [org.jline/3.4.0] osgi.wiring.package;
>> filter:="(osgi.wiring.package=com.sun.jna)"; resolution:=optional))
>>     org.ops4j.pax.logging.pax-logging-log4j2/1.10.1 (Should be wired to:
>> org.apache.commons.compress/1.10.0 (through
>> [org.ops4j.pax.logging.pax-logging-log4j2/1.10.1] osgi.wiring.package;
>> filter:="(osgi.wiring.package=org.apache.commons.compress.compressors)";
>> resolution:=optional))
>>
>> Thanks again Seth, this was driving me nuts, I think my brain was
>> telling me to ask here because I've seen this before (just forgot about
>> it for some reason).  I'll make a change and am sure it will resolve. 
>> Also, I moved to Karaf 4.1.4 since I started having these problems with
>> 4.2.x and decided at that rate, might as well go to 4.1.4.  Maybe I'll
>> just switch back to working with 4.2.x.  Choices. choices.
>>
>> Ryan
>>
>> On Mon, Jan 22, 2018 at 12:25 AM Seth Leger <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>     Hi Ryan,
>>
>>     You are probably running into an issue where some dependency of Mongo is
>>     triggering a refresh of the system bundles inside Karaf. Mongo probably
>>     isn't "hijacking" jline; it's just causing jline + karaf shell to
>>     refresh which kills your client shell. There are several ways to do this
>>     on Karaf 4.1, I documented one way to do it (with Apache MINA) in
>>     this bug:
>>
>>     https://issues.apache.org/jira/browse/KARAF-5384
>>
>>     If you use:
>>
>>     feature:install -t -v your-problematic-feature
>>
>>     then you should be able to track down which bundle is triggering the
>>     refresh. Also, what version of Karaf are you using?
>>
>>     Seth Leger
>>     The OpenNMS Group
>>
>>
>>     On 1/21/18 12:23 PM, Ryan Moquin wrote:
>>     > This might be better to ask of the Embed Mongo team, but I am not
>>     > entirely sure.  I've been experimenting with an Embedded Mongo running
>>     > inside Karaf, which I've gotten to work successfully with one caveat
>>     > that is driving me crazy trying to troubleshoot (I think I understand
>>     > what's happening, but not exactly what makes it happen).  If I have a
>>     > feature which installs a bundle that created and starts a Flapdoodle
>>     > Embed Mongo instance, then the Karaf console will "hang" since I think
>>     > the Embed Mongo code, kind of "hijacking" the Karaf JLine input stream
>>     > (it attaches some input streams for the Mongo process.)  I did switch
>>     > the outputstreams to use SLF4J which helped a bit, but there is
>>     one last
>>     > small issue I can't figure out.  The reason I'm wondering if
>>     someone on
>>     > this list might have some ideas is because if when I install that
>>     > feature, I force kill the Karaf process and start it back up, then the
>>     > Embed Mongo process ends up running in the background without
>>     > "hijacking" the Karaf jline console.
>>     >
>>     > I guess I'm wondering if there is a way to initially install my embed
>>     > mongo feature and prevent it from grabbing the Karaf JLine stream if I
>>     > understand how the feature startup is different than than when
>>     > installing a feature (around the Karaf console input).  I hope
>>     what I'm
>>     > asking actually makes sense to someone.  Basically, the feature
>>     command
>>     > doesn't return control to the Karaf console after installing that
>>     > feature due to what the Embedded mongo process does under the
>>     covers....
>>     > I've been debugging the embed mongo code to see if I can figure
>>     out how
>>     > this conflict is happening.. it appears one of the threads that
>>     reads an
>>     > inputstream from mongod is the culprit, but this is only a problem
>>     when
>>     > installing the feature, not running karaf after the feature has been
>>     > installed.....
>>     >
>>     > Thanks for any advice!
>>     >
>>     > Ryan
>>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com