Apache camel - Testing integrations

As far Systems are became more complex, It turns out that Test Automation is a mandatory Software Engineering discipline that needs to be followed. It is specially true when talking about Integrations between systems.On such scenarios, there are some challenges

• Hard to isolate components - to test a single component, call to other components may need to be made in order to make the test possible, which makes the test process expansive
• How to simulate system failures -  How to test the implementation behaviour in case one of the system being integrated fails? It is gonna be complicated or at least very time consuming simulate this test scenario because we need to control components that we usually can't.
• Slow Build  - calls to external systems can be slow (slow connection, external system unavailability, etc). If your tests are build calling external systems, the build time might get slower over time as far you test coverage grows.

At the end of the day, these are things we need to workaround because tests would need to be done anyway. The good news is that It is possible to achieve certain level of test coverage even  on such scenarios. By using some DI techniques and a couple of frameworks I'm gonna make it happen.

I'm gonna use my last post as base. Actually I'll keep It as It is and create a different implementation applying modifications that will let It testable.

What is worth to test?

Looking to the implementation as It were, would be desirable test if the routing logic works as we expect on both cases (when it finishes successfully and when there is an error). To achieve that, I don't necessarily need  to rely on Amazon S3 or a different external component. I can "mock" them and then test the routing logic isolatedly.

How to do that?

In order to not depend on Amazon S3 on my tests, I need somehow  "replace" It only during the tests by "mocked" endpoints. By doing that, I'll be able to isolate what needs to be tested (the routing logic) and also control the input data, then simulate the behaviour I want.
First thing to do is to remove the hard coded references to S3 externalising them. The code will look as follows:

 @Component
class FileRouter extends RouteBuilder {
  @Autowired val sourceJsonIn: String = null
  @Autowired val targetJsonOut: String = null
  @Autowired var targetJsonError: String = null

  override def configure(): Unit = {
    from(sourceJsonIn).
      to("bean:fileProcessor").
      choice().
        when(header("status").isEqualTo("ok")).
          to(targetJsonOut).
        otherwise().
          to(targetJsonError)
  }

}

Here I'm using Camel Spring support for java. The FileRouter class will receive the endpoints from the spring context in runtime. In fact, these endpoints are now spring beans defined as follows:

trait S3Beans {
  @Bean
  def sourceJsonIn = "aws-s3://json-in?amazonS3Client=#client&maxMessagesPerPoll=15&delay=3000&region=sa-east-1"
  @Bean
  def targetJsonOut = "aws-s3://json-out?amazonS3Client=#client&region=sa-east-1"
  @Bean
  def targetJsonError = "aws-s3://json-error?amazonS3Client=#client&region=sa-east-1"
  @Bean
  def client = new AmazonS3Client(new BasicAWSCredentials("[use your credentials]", "[use your credentials]"))

}

There is also the FileProcessor that handles the file content. It is also defined as a spring bean as follows:

trait NoThirdPartBeans {
  @Bean def fileProcessor() = new FileProcessor
}

The S3 endpoints,  FileProcessor and FileRouter classes are ready to be added into the spring context. As far we are using spring support from camel, they will on be available on the camel context as well. It's being done as following:

@Configuration
@ComponentScan(Array("com.example"))
class MyApplicationContext extends CamelConfiguration with S3Beans with NoThirdPartBeans {}

Now the implementation is ready to be tested. In order to achieve to behaviour I want, I need to replace all endpoints set on S3Beans class by "mocked" endpoints. By doing that, I'll be able to "control" the external dependencies and then simulate different scenarios. To do that, I'll create a different "test context"but only replacing the beans I need to mock.

@Configuration
class TestApplicationContext extends SingleRouteCamelConfiguration with NoThirdPartBeans {
  @Bean override def route() = new FileRouter
  @Bean def sourceJsonIn = "direct:in"
  @Bean def targetJsonOut = "mock:success"
  @Bean def targetJsonError = "mock:error"
}

Direct is a camel component that works as a "memory" queue. It is gonna replace the S3 bucket where the files come from. Mock is another camel component that we can assert in runtime.  They are replacing the output S3 buckets. I can now check whether they receive messages or not.
Now It's time to create the test class. It' s gonna use the"test context" I just created and then validate different test scenarios. It's being done as follows:

@RunWith(classOf[CamelSpringJUnit4ClassRunner])
@ContextConfiguration(classes = Array(classOf[TestApplicationContext]))
class SimpleTest {
  @EndpointInject(uri =  "mock:success")
  val mockSuccess:MockEndpoint = null

  @EndpointInject(uri =  "mock:error")
  val mockError:MockEndpoint = null

  @Produce(uri = "direct:in")
  val template:ProducerTemplate = null

  @Test
  def shouldHitTheSuccesEndpoiny(): Unit ={
    val fileContent =  IOUtils.toString(getClass.getResourceAsStream("/json-file.json"))
    template.sendBody(fileContent)
    mockError.expectedMessageCount(0)
    mockSuccess.message(0).body().convertToString().isEqualTo(fileContent)
    mockSuccess.assertIsSatisfied()
    mockError.assertIsSatisfied()
  }

}

Conclusion

Real world integrations can be much more complex than the example I used. But they still need to be tested somehow. Systems like these without tests will became unmaintainable soon. The test approach i used fits well on case there is routing logic or between the components being integrated.

It is also  important to notice how the spring api made the implementation simpler and testable. As It was implemented before (without spring or any DI technique) would be very hard to achieve the result.
The working example can be found here.