Different behaviours when trying to run integration in devmode vs build/run container

[darnautovic]

Hi everyone,

I'm experiencing two issues when running project in devmode (Run button) vs Build / Run container

Issue #1: Camel jbang dependencies not being picked up in devmode**

Integration I have uses some dependancies in beans (created and registered in .java file in project) that I added to application.properties file within project
For example I've added two dependencies (camel.jbang.dependencies) to existing default ones:

camel:sql,org.postgresql:postgresql:42.6.0

so that org.apache.camel.processor.idempotent.jdbc.JdbcMessageIdRepository can be constructed :

import org.apache.camel.processor.idempotent.jdbc.JdbcMessageIdRepository;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.beans.factory.annotation.Autowired;
import javax.sql.DataSource;

public class Testidempotentrepository {

    @Autowired
    private DataSource dataSource;

    @Bean(name = "Testidempotentrepository")
    public IdempotentRepository createIdempotentRegistry() {
        return new JdbcMessageIdRepository(dataSource, "some-key");
    }
}

If I run this project in devmode I get error indicating that dependency is missing and jbang did not resolve required dependencies: /TestIntegrationIdempotentRepository.java:1: error: package org.apache.camel.processor.idempotent.jdbc does not exist import org.apache.camel.processor.idempotent.jdbc.JdbcMessageIdRepository;

while if I build and run this project, it works fine (well I get another error not related to this one, which I'll describe in Issue #2 below).

I must note that I discovered if i set jBangOptions in DevModeResource.java -> runProjectWithJBangOptions to --deps=org.apache.camel:camel-sql:4.1.0,org.postgresql:postgresql:42.6.0,org.apache.commons:commons-dbcp2:2.10.0 devmode runs fine and picks up all dependencies.

I'm a little bit puzzled why this different behaviour happens? Also since there is an jBangOptions param available in the controller, is there a way to send these options from the builder UI or this is not used?

Issue #2: Integration containing IdempotentConsumer and beans required to run it runs correctly in devmode while when built/run container error is thrown**

I'm trying to run simple integration that uses Idempotent Consumer which stores duplacate messages in IdempotentRepository (postgres db). Two beans are needed for this to work:

1. DataSource bean and
2. IdempotentRepository bean

IdempotentConsumer is then configured with IdempotentRepository. Idempotent repository bean example can be seen in Issue #1 (it's configured as Java file because I could not find a way to configure two dependant beans such we have in this case via builder or camel yaml specification):

Integration with defined bean:

- route:
    nodePrefixId: route-587
    id: route-fc37
    from:
      uri: kamelet:timer-source
      id: from-a244
      parameters:
        message: A repeating message
      steps:
        - log:
            message: 'Before idempotent consumer: ${body}'
            id: log-3d70
        - idempotentConsumer:
            expression:
              simple:
                expression: ${body}
                id: simple-15e0
            idempotentRepository: Testidempotentrepository
            id: idempotentConsumer-0929
            steps:
              - log:
                  message: THIS MESSAGE SHOULD APPEAR ONLY ONCE! ${body}
                  id: log-dba0
- beans:
    - constructors: {}
      name: DataSourceBean
      properties:
        driverClassName: org.postgresql.Driver
        url: jdbc:postgresql://localhost:5432/test_db
        username: postgres
        password: postgres
      type: org.apache.commons.dbcp2.BasicDataSource

If I run in devmode this integration (with added .java file above), it runs fine and idempotent consumer works as expected (stores messages in configured DB via DataSourceBean) *BUT if I build and run in container, I get an error stating that TestIntegrationIdempotentRepository is not registered in registry and cannot be found:

Exception in thread "main" org.apache.camel.FailedToCreateRouteException: Failed to create route route-fc37 at: >>> IdempotentConsumer[simple{${body}} -> [Log[THIS MESSAGE SHOULD APPEAR ONLY ONCE! ${body}]]] <<< in route: Route(route-fc37)[From[kamelet:timer-source?message=A repeat... because of No bean could be found in the registry for: Testidempotentrepository of type: org.apache.camel.spi.IdempotentRepository

This is even a bigger puzzle for me because it seems as if Testidempotentrepository is not registered, but I wonder why? Why this works in devmode but doesn't in build/run container configuration, shouldn't the the procedure and configuration be the same underneath the hood when deploying these integrations?

[davsclaus]

You class Testidempotentrepository lacks an import for IdempotentRepository

[davsclaus]

And when I fix that and run it in dev mode with 4.3 then it startup, but lacks connection to the database

Caused by: java.net.ConnectException: Connection refused

So when you then deploy (then jbang export) as camel-main and that runs a bit differently as it runs this as general maven based project with compile to JAR

[davsclaus]

Also I found a jbang export problem that this starts the database connection, which it should not do
https://issues.apache.org/jira/browse/CAMEL-20211

[davsclaus]

Okay so you are using spring annotations that only works with jbang (as jbang can simulate that). However karavan exports to camel-main. So you should use Camel annotations instead.

So your code would be

import org.apache.camel.Configuration;
import org.apache.camel.spi.IdempotentRepository;
import org.apache.camel.processor.idempotent.jdbc.JdbcMessageIdRepository;
import org.apache.camel.BeanInject;
import org.apache.camel.BindToRegistry;
import javax.sql.DataSource;

@Configuration
public class Testidempotentrepository {

    @BindToRegistry("Testidempotentrepository")
    public IdempotentRepository createIdempotentRegistry(@BeanInject DataSource dataSource) {
        return new JdbcMessageIdRepository(dataSource, "some-key");
    }
}

[davsclaus]

There is maybe a little bug with the field dependency injection so notice how its part of the method parameters.
I created a ticket:
https://issues.apache.org/jira/browse/CAMEL-20213

[darnautovic]

Hi @davsclaus !

Thanks for your replies and giving it a shot. Sorry for the non-functioning example (I missed import).

I wasn't aware that initiating a connection in jbang export is an issue, although I thought it as a strange behaviour, and it doesn't make sense, you are right.

Regarding using camel-main annotations - I think real issue there lies in bean processing/loading order - IdempotentRepository bean is processed before DataSource bean is available in Registry. I don't know if there is any way to ensure beans processing order in camel-jbang?

Also, in situation like this, when one bean is dependent to another, I think it would work if beans are ordered in yaml file - but the issue there is that I don't know how to define (is it even possible?) dependency using yaml? I see constructors are introduced, but I had no luck on creating a IdempotentRepository bean using yaml and getting DataSource picked up by DI. Any idea if this would somehow be possible?

[davsclaus]

Yeah there is dependency ordering if bean a -> b, and you refer to the bean via #bean:beanIdGoesHere

[davsclaus]

So yaml beans don't do magic dependency injection ala springs @Autowired so the JdbcMessageIdRepository takes 2 ctr parameters, and you can configure them in yaml with the new constructors feature.

https://camel.apache.org/components/next/others/yaml-dsl.html

[davsclaus]

stuff fixed in newer camel and karavan releases.