Creating OSGI bundles and services from the ground up using Apache Karaf

This article describes the fundamental principles of the OSGi development model, using Apache Karaf or ServiceMix as the container. No software tools other than a text editor and Java JDK are used — my intention is to explain in detail how OSGi bundles are packaged, and how they interact with their container and one another. Of course most development of this kind is done with sophisticated IDE tools, but these just tend to obscure the low-level details.

The example I will use is a simple one, but a step up from "Hello, World" — it shows how one bundle can be exposed as a service with a well-defined API, and that service consumed by another, independent bundle.

About OSGi and its container concept

This section briefly describes the OSGi component model, and how to obtain and install the Apache Karaf or ServiceMix OSGi containers.

The OSGi concept

The OSGi development model for Java is based on loosely-coupled software components called bundles, deployed in a lightweight container. Each bundle consists of one or more Java packages, at least one of which contains an Activator, which forms the link between the bundle and the container.

The OSGi container is lightweight in the sense that it offers very few services, and has little runtime overhead. Developers familiar with J2EE will already have experience with servlet and EJB containers; the OSGi container is conceptually simpler, except that it does not attempt to offer the same range of business-oriented services that J2EE does.

What the OSGi container does offer, however, is a strict dependency management and versioning system. Each bundle that uses a service from a different bundle must declare that it has a dependency, and the container will not allow a bundle to start until the dependency is satisfied.

OSGi bundles are kept segregated by the use of independent classloaders; bundles can only interact by well-defined mechanisms exposed by the container. The simplest of these, and the one that will be used in this example, is for one of the bundles to expose a service that the other imports. The service is defined by a Java interface, which has to be known to both the importing and the exporting bundle.

OSGi containers

Many software products are now based on the OSGi model; this may, or may not, be apparent in use. A number of IDE tools, for example, use OSGi internally to provide a plug-in/extension mechanism. The Apache project maintains a general-purpose OSGi container called Felix; Felix can be used programatically, but for experimentation it is probably easiest to use one of the two Apache projects based on Felix — Karaf or ServiceMix. Both of these projects provide an interactive shell through which OSGi modules and services can be installed and managed. The main difference between the two is that ServiceMix bundles a bunch of other integration components — ActiveMQ message broker, the Camel routing engine, and some other things. In this example, I have shown Karaf being used, but it works with ServiceMix as well (with some changes to the logging configuration as, by default, you won't see any output).

Obtaining and setting up Karaf

Karaf may be obtained from its download page. The download bundles are platform-specific because Karaf uses native-code libraries to provide line editing functions that do not exist in pure Java.

Installation of Karaf is trivial — just unpack the distribution bundle into any convenient directory. You should then be able to start the Karaf shell by running

$ /path/to/karaf/bin/karaf

Naturally, you'll need to replace the /path/to part with the actual installation directory in this command (and in all other commands in this article).

With luck you'll see a welcome banner and a prompt:

karaf@root>

The name 'root' here does not indicate the OS root user, but the Karaf user; different users can be defined with specific security privileges over the container, but I'll just be using the 'root' user in this example.

Important Karaf shell commands

The Karaf shell exposes a large number of management commands, but only a few are required for these article. I'll describe them here, just so they're all in one place, rather than scattered around the text

osgi:list
Lists the installed bundles with their IDs and status. The ID will be used in many other commands to manage the bundle. The status will usually be one of Installed, Active, and Resolved. Be aware that Installed can be an unhealthy status — if you've attempted to start a bundle, but there are missing dependencies, then it will be shown as Installed. If you successfully start a bundle and then stop it, its status is Resolved. There are interim status codes — Starting, Stopping, etc — but their appearance for any length of time usually indicates some sort of fault.

osgi:install file:/path/to/my/bundle.jar
Installs the bundle whose JAR file is at the specified location. By default, installation does not attempt to start the bundle, or to resolve dependencies. If the installation succeeds, Karaf will report the ID of the new bundle.

osgi:start [id]
Attempts to start the bundle. That is, dependencies are resolved (if possible), and the bundle activator's start() method is called (more on activators later).

osgi:stop [id]
Attempts to stop the bundle. The bundle activator's start() method is called.

headers [id]
Shows details from the bundle's manifest (more on manifests below). In particular, it shows which packages are imported and exported, and failed dependencies are highlighted. headers is very helpful in diagnosing problems where the bundle won't start because of missing dependencies (as is find-class, described below).

osgi:uninstall [id]
Stops and uninstalls the specified bundle. In general, this does not stop or uninstall any bundles that depend on the one being uninstalled, although they might well fail to function, and will probably not restart once stopped themselves.

find-class [class_name]
Attempts to find the bundle that exposes a class with the specified name. This can be useful if you're seeing NoClassDefFound messages in the log, because you haven't imported the necessary packages in your bundle.

There are other important commands concerned with 'feature' management; a feature is a set of bundles that provide a particular facility. We won't need to install any new features in this example but it is possible, for example, to configure support for the Camel routing engine by installing it as a feature.

Anatomy of a bundle

At its simplest, a bundle consists of some Java packages and a manifest, bundled into a JAR. The manifest is provided in the usual META-INF/MANIFEST.MF file that most Java developers will know about. OSGi defines a few extra manifest entries specific to bundles, most particularly entries that define dependencies between bundles.

You can take any Java library in JAR form and make it 'OSGi-ready' simply by adding the relevant OSGi entries to its manifest. However, this will not allow the container to manage the bundle beyond installing and uninstalling it.

The essential OSGi headers, taken from the 'Tick' bundle that will be presented later, are:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: OSGi test: tick
Bundle-SymbolicName: net.kevinboone.OSGiTest.tick;singleton:=true
Bundle-Version: 1.0.2

In practice, the first two headers are boilerplate, and will appear at the start of every bundle manifest. The two names and the version identify this specific bundle; it is possible to install the same bundle more than once in the container, if the bundle versions are different. Bundles can, and probably should, import other bundles by their specific version numbers if multiple versions will be deployed. For the container to be able to start and stop the bundle, the developer must provide as a minimum an activator. The activator is a Java class that implements theBundleActivator interface provided by the OSGi framework. This interface has only two methods: start() and stop(), and it probably isn't difficult to guess when they get called. These methods can do pretty much anything, with the proviso that they don't block — if they take a long time to complete, they can interfere with the Karaf shell getting on with the next task. For long-running, service-based tasks, the start()method can start a new thread, as it does in the example in this article.

The container is made aware of the bundle activator by the following entry in the manifest:

Bundle-Activator: net.kevinboone.osgitest.tick.Activator

Introducing the example

The example in this article consists of two bundles: Tick and Tock. The Tick bundle starts a background thread that wakes up every five seconds. It then calls a method tick() on every class that has registered an interest in it. A class registers an interest by getting a reference to the TickService interface, and calling its addListener() method, passing a class that implements the TickListener interface. So the Tock bundle is a consumer of the service exposed by the Tick bundle.

An overview of the two bundles is shown in the UML diagram below.

Each bundle in this example consists of one Java package, although bundles need not be so limited in practice. As discussed above, each bundle has a activator that implements the BundleActivator interface, so the container can start or stop it.

Building, deploying, and starting the Tick service

Setting up the workspace

The Tick service is a bundle that consists of two interfaces, one class, and a manifest. The directory layout I have chosen for the source of this bundle (and the corresponding Tock bundle) looks like this:

src
  kevinboone
    osgitest
      tick
        Activator.java
        TickService.java
        TickListener.java
      tock
        Activator.java
etc
  MANIFEST_tick.MF  # Manifest for the Tick bundle
  MANIFEST_tock.MF  # Manifest for the Tock bundle

target
  tick
    classes    # compiled classes for the Tick bundle will go here
  tock
    classes    # compiled classes for the Tock bundle will go here

The complete source, with the described directory structure, is provided in the download bundle at the end of this article.

Coding the bundle

Here is the source of the Tick activator (and service implementation, as I've put these in the same class). I hope that with the comments it's reasonably clear what's going on. The important points to note are that the activator class implements the interface TickService, and provides the corresponding addListener and removeListener methods; and that it provides the essential start() and stop() methods that will be called by the container.

/*
Activator for Tick service

Tick is an OSGI bundle that "ticks" every few seconds, and notifies one or more 
listeners. This bundle exposes itself as an OSGi service, using the name of 
the interface TickService. TickService is implemented in this class, which 
would probably not be a good design decision in a substantial project,
because it means that the service implementation and API are _a fortiri_ in the same package.

The bundle manifest must export the package net.kevinboone.osgitest.tick

(c)2013 Kevin Boone
*/
package net.kevinboone.osgitest.tick;

import org.osgi.framework.*;
import java.util.*;

public class Activator implements BundleActivator, TickService
  {
  // stop is flag to indicate whether the timer thread should be
  //  running or not. 
  private boolean stop = false;

  // We maintain a list of all the clients of this service that have
  //  shown an interest in being ticked at
  private List<TickListener> listeners = new Vector<TickListener>();

  /** addListener() implements the method in the service interface
       TickService. It adds a client of the service to the list of 
       clients */
  @Override
  public void addListener (TickListener listener)
    {
    listeners.add (listener);
    }

  /** removeListener() implements the method in the service interface
       TickService. It removes a client of the service from the list of 
       clients */
  @Override
  public void removeListener (TickListener listener)
    {
    listeners.remove (listener);
    }

  /**
    start() method gets called when running osgi:start from the console
  */
  public void start(BundleContext bundleContext) throws Exception
    {
    System.out.println ("Tick bundle started");
    stop = false;
    // Start a thread that will run until the value of stop changes to
    //  false. The thread waits 5 seconds, and that calls the tick() method
    //  on every client registered with the service
    new Thread (new Runnable()
      {
      public void run()
        {
        while (!stop)
          {
          try
            {
            Thread.sleep(5000);
            System.out.println ("Tick!");
            for (TickListener listener : listeners)
              listener.tick();
            }
          catch (Exception e){}
          }
        }
      }).start();

    // Register TickService as the API for a service, and this class
    //  as the implementation.
    bundleContext.registerService(TickService.class.getName(),
      this, new Properties());
    }

  /**
    stop() method gets called when running osgi:start from the console
  */
  public void stop(BundleContext bundleContext) throws Exception
    {
    System.out.println("Tick bundle stopped");
    // Set the stop flag, so that the timer thread will finish on the
    //  next tick
    stop = true;
    }
  }

This section of code deserves special mention:

 bundleContext.registerService(TickService.class.getName(),
      this, new Properties());

Here this class registers itself ('this') as the provider of a service whose interface is TickService. Note that we're passing the interface by name, not by class. This is because the provider and the consumer of the service will likely be in different bundles, and be served by different class loaders. The framework will have to construct the interface reflectively from its name.

Coding the interfaces

There are two interfaces — an interface to the service's API (TickService) and an interface that must be implemented by any consumer of the service that wants to listen for tick events (TickListener). These interfaces are trivially simple, but shown here just for completeness.

TickService:

package net.kevinboone.osgitest.tick;

public interface TickService
  {
  public void addListener (TickListener listener);
  public void removeListener (TickListener listener);
  }

TickListener:

package net.kevinboone.osgitest.tick;

public interface TickListener
  {
  /** The tick() method is called on every registered listener every time
      the Tick service ticks */
  public void tick();
  }

Writing the manifest

Most of the manifest for the Tick service has already been presented, but here is the whole thing.

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: OSGi test: tick
Bundle-SymbolicName: net.kevinboone.OSGiTest.tick;singleton:=true
Bundle-Version: 1.0.2
Bundle-Activator: net.kevinboone.osgitest.tick.Activator
Bundle-Vendor: Kevin Boone 
Bundle-Localization: plugin
Import-Package: org.osgi.framework
Export-Package: net.kevinboone.osgitest.tick

The important entries not discussed so far are Import-Package and Export-Package. These control which bundles we will see, and which bits of this bundle other bundles will see. In this case I'm exporting the whole package. We need to import the framework package, to have access to interfaces like BundleActivator. It's vital to understand that the class search path seen at runtime in the container will not look much like that seen at compile time. It's perfectly possible — common even — for a bundle to compile perfectly well, and then fail at start time because classes that were available to the compiler are not available in the container. This is why Impot-Package is so important: apart from the basic JVM classes (java.lang, etc), every class required by an OSGi bundle must be imported from somewhere.

If you're not using clever build tools or an IDE, it can be quite difficult to work out which bundle exports a particular class. This is where the Karaf find-class and headers commands come in handy, along with a bit of trial-and-error.

Building the Tick bundle

To build the bundle at the command line:

$ javac -d target/tick/classes -classpath /path/to/karaf/lib/karaf.jar \\
  src/net/kevinboone/osgitest/tick/*.java 

$ jar cfm osgitest_tick.jar etc/MANIFEST_tick.MF  -C target/tick/classes .

The reference to karaf.jar provides the java compiler with definitions of the OSGi framework classes that the source uses. The jar command bundles up the classes and provides the specified manifest.

The final JAR file, osgitest_tick.jar should have a structure like this:

$ jar tf osgitest_tick.jar 
META-INF/MANIFEST.MF
net/kevinboone/osgitest/tick/TickListener.class
net/kevinboone/osgitest/tick/TickService.class
net/kevinboone/osgitest/tick/Activator$1.class
net/kevinboone/osgitest/tick/Activator.class

The $1 class here corresponds to the anonymous inner class used to create the implementation of the background thread in the Activator class.

Deploying the Tick bundle

Now all we need to do is start Karaf, and run at the console:

karaf@root> osgi:install file:/path/to/osgitest_tick.jar
Bundle ID: 102
karaf@root> osgi:start 102
Tick
Tick
...

Because the the bundle uses System.out.println for its output, then by default this output will just be seen in the Karaf console. Yes, it does mess up the look of the console a bit, but it's easier to see what's going on than by writing to a log file.

Note that the bundle ID — 102 in this case — will most likely be different from this; what matters is that you start the bundle whose ID is given by the shell.

As written, the Tick service ticks every five seconds. To stop it, you can just run

karaf@root> osgi:stop 102

This calls the bundle's stop() method, which sets the stop flag, which causes the background thread to stop. Note that stopping the bundle does not unload it from memory, or force its threads to stop running, although uninstalling it does. It's perfectly possible for a badly-behaved bundle to ignore the stop() call and carry on regardless.

Building, deploying, and starting the Tock bundle

The Tock service is a consumer of ticks. That is, it locates the Tick service, and calls its addListener method with itself as a reference. Thereafter, the tick() method will get called every five seconds, and the Tock bundle will output a message.

Here is the complete source of the Tock bundle. Again, I hope that with the comments it's reasonably clear how it works.

/*
Tock. This bundle finds registers itself with the Tick service,
and "tocks" every time there is a "tick".

Note that the bundle manifest muts import org.osgi.util.tracker to use the 
service tracker, and net.kevinboone.osgitest.tick to use the tick service
itself.

(c)2013 Kevin Boone
*/
package net.kevinboone.osgitest.tock;

import org.osgi.framework.*;
import org.osgi.util.tracker.*;
import net.kevinboone.osgitest.tick.*;

public class Activator implements BundleActivator, TickListener
  {
  /**
    start() method gets called when running osgi:start from the console
  */
  public void start(BundleContext bundleContext) throws Exception
    {
    System.out.println("Tock bundle started");
    // Get a service tracker for the Tick service, using its
    //  class name 
    ServiceTracker tracker = new ServiceTracker
      (bundleContext, TickService.class.getName(), null);
    // Start the tracker
    tracker.open();
    // Find the service's API from the tracker. If the tick service
    //  is not running, we get a null here 
    TickService tick = (TickService) tracker.getService();
    // Stop the tracker, since we're done with it
    tracker.close ();
    // If we got a reference to the tick service, then call its
    //  addListener method to register this class as a receiver of
    //  tick events. Otherwise, throw an exception
    if (tick != null)
      tick.addListener (this);
    else
      throw new Exception
        ("Can't start tock bundle, as tick service is not running");
    }

  /**
    stop() method gets called when running osgi:start from the console
  */
  public void stop(BundleContext bundleContext) throws Exception
    {
    System.out.println("Tock bundle stopped");
    ServiceTracker tracker = new ServiceTracker
      (bundleContext, TickService.class.getName(), null);
    tracker.open();
    TickService tick = (TickService) tracker.getService();
    tracker.close ();
    // If the Tick service is running, then tick will be non-null, and
    //  we can remove ourself from the list of listeners on the service
    if (tick != null)
      tick.removeListener (this);
    }

  /**
    tick() implements the tick() method in TickListener. Once we are
      registered with the Tick service, this method will get called on
      every tick 
  */
  @Override
  public void tick()
    {
    System.out.println ("tock");
    }
  }

Note that this class uses a ServiceTracker to get a reference to the interface exposed by the Tick service. If the Tick service is installed but not started then the interface will not be found. In that case, the Tock service must fail by throwing an exception; otherwise there is no way for the container, or the administrator, to know that initialization failed. Here is the manifest for the Tock service:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: OSGi test: tock
Bundle-SymbolicName: net.kevinboone.OSGiTest.tock
Bundle-Version: 1.0.2
Bundle-Activator: net.kevinboone.osgitest.tock.Activator
Bundle-Vendor: Kevin Boone 
Bundle-Localization: plugin
Import-Package: org.osgi.framework, org.osgi.util.tracker,  net.kevinboone.osgitest.tick

Notice that we import the package of the Tick service, and also org.osgi.util.tracker — this latter is required for the bundle to be able to load the ServiceTracker class at runtime.

Building the Tock service

We can easily build this service at the command line also:

javac -d target/tock/classes -classpath /path/to/karaf/lib/karaf.jar \\
   src/net/kevinboone/osgitest/tock/*.java \\
   src/net/kevinboone/osgitest/tick/Tick*.java

jar cfm osgitest_tock.jar etc/MANIFEST_tock.MF  -C target/tock/classes .

Notice that we must include in the javac invocation some reference to the interfaces defined by the Tick service. We can do that by specifying the source (as here), or by adding the location of the compiled classes to the classpath. However, including the source means that compiled versions of these classes will be added written to the target directory, and thus included automatically in the JAR. This avoid one extra file-copy step; in practice, in a real application your files are less like to be set out in such a way as to allow tricks like this to work.

Deploying the Tock bundle

The procedure is the same as for the Tick bundle, which should be started by this point.

karaf@root> osgi:install file:/path/to/osgitest_tock.jar
Bundle ID: 103
karaf@root> osgi:start 103
Tick
Tock
Tick
Tock
...

Now every time you see a tick from the Tick service, you should also see a tock from the Tock bundle. If you stop the Tick service, you should see both ticks and tocks stop.

Notice that the bundles are sufficiently loosely coupled that you can completely uninstall the Tick service without damaging the Tock service. The Tock service won't get any more ticks, but it won't crash or spit out errors — at least until you try to stop and start it again, at which point it will realize that there is no Tick service for it to consume from.

You should also find that if you uninstall Tick and Tock, and then try to install Tock alone, it should fail — the dependency on Tick listed in the manifest cannot be satisfied.

Summary

This has been a fairly low-level exposition of the OSGi framework and how to use it. In practice, in a substantial project you're likely to use an IDE or a build/dependency tool like Maven to manage bundles. But, as you can see, the OSGi architecture does not depend on any of these things; and, I hope, working at this low level makes it clearer what the tools are doing.

Java World Bugs

Friday, August 19, 2016

Karaf Beginner's Tutorial