Wednesday, March 05, 2014

Signing SOAP Messages - Generation of Enveloped XML Signatures

Digital signing is a widely used mechanism to make digital contents authentic. By producing a digital signature for some content, we can let another party capable of validating that content. It can provide a guarantee that, is not altered after we signed it, with this validation. With this sample I am to share how to generate the a signature for SOAP envelope. But of course this is valid for any other content signing as well.

Here, I will sign
  • The SOAP envelope itself
  • An attachment 
  • Place the signature inside SOAP header 
With the placement of signature inside the SOAP header which is also signed by the signature, this becomes a demonstration of enveloped signature.

I am using Apache Santuario library for signing. Following is the code segment I used. I have shared the complete sample here to to be downloaded.

public static void main(String unused[]) throws Exception {

        String keystoreType = "JKS";
        String keystoreFile = "src/main/resources/PushpalankaKeystore.jks";
        String keystorePass = "pushpalanka";
        String privateKeyAlias = "pushpalanka";
        String privateKeyPass = "pushpalanka";
        String certificateAlias = "pushpalanka";
        File signatureFile = new File("src/main/resources/signature.xml");
        Element element = null;
        String BaseURI = signatureFile.toURI().toURL().toString();
        //SOAP envelope to be signed
        File attachmentFile = new File("src/main/resources/sample.xml");

        //get the private key used to sign, from the keystore
        KeyStore ks = KeyStore.getInstance(keystoreType);
        FileInputStream fis = new FileInputStream(keystoreFile);
        ks.load(fis, keystorePass.toCharArray());
        PrivateKey privateKey =

                (PrivateKey) ks.getKey(privateKeyAlias, privateKeyPass.toCharArray());
        //create basic structure of signature
        javax.xml.parsers.DocumentBuilderFactory dbf =
        DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
        DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
        Document doc = dBuilder.parse(attachmentFile);
        XMLSignature sig =
                new XMLSignature(doc, BaseURI, XMLSignature.ALGO_ID_SIGNATURE_RSA_SHA1);

        //optional, but better
        element = doc.getDocumentElement();

            Transforms transforms = new Transforms(doc);
            //Sign the content of SOAP Envelope
            sig.addDocument("", transforms, Constants.ALGO_ID_DIGEST_SHA1);

            //Adding the attachment to be signed
            sig.addDocument("../resources/attachment.xml", transforms, Constants.ALGO_ID_DIGEST_SHA1);


        //Signing procedure
            X509Certificate cert =
                    (X509Certificate) ks.getCertificate(certificateAlias);

        //write signature to file
        FileOutputStream f = new FileOutputStream(signatureFile);
        XMLUtils.outputDOMc14nWithComments(doc, f);

At first it reads in the private key which is to be used in signing. To create a key pair for your own, this post  will be helpful. Then it has created the signature and added the SOAP message and the attachment as the documents to be signed. Finally it performs signing  and write the signed document to a file.

The signed SOAP message looks as follows.

<soap:Envelope xmlns:dsig="" xmlns:pj=""
        <pj:MessageHeader pj:version="1.0" soap:mustUnderstand="1">
                <pj:PartyId pj:type="ABCDE">FUN</pj:PartyId>
                <pj:PartyId pj:type="ABCDE">PARTY</pj:PartyId>
            <pj:ConversationId>FUN PARTY FUN 59c64t0087fg3kfs000003n9</pj:ConversationId>
                <pj:MessageId>FUN 59c64t0087fg3kfs000003n9</pj:MessageId>
        <pj:Via pj:id="59c64t0087fg3ki6000003na" pj:syncReply="False" pj:version="1.0"
                soap:actor="" soap:mustUnderstand="1">
        <ds:Signature xmlns:ds="">
                <ds:SignatureMethod Algorithm=""></ds:SignatureMethod>
                <ds:Reference URI="">
                    <ds:DigestMethod Algorithm=""></ds:DigestMethod>
                <ds:Reference URI="../resources/attachment.xml">
                        <ds:Transform Algorithm=""></ds:Transform>
                    <ds:DigestMethod Algorithm=""></ds:DigestMethod>
            <ds:SignatureValue>d0hBQLIvZ4fwUZlrsDLDZojvwK2DVaznrvSoA/JTjnS7XZ5oMplN9  THX4xzZap3+WhXwI2xMr3GKO................x7u+PQz1UepcbKY3BsO8jB3dxWN6r+F4qTyWa+xwOFxqLj546WX35f8zT4GLdiJI5oiYeo1YPLFFqTrwg==
   <ds:X509Certificate>                MIIDjTCCAnWgAwIBAgIEeotzFjANBgkqhkiG9w0BAQsFADB3MQswCQYDVQQGEwJMSzEQMA4GA1UE...............qXfD/eY+XeIDyMQocRqTpcJIm8OneZ8vbMNQrxsRInxq+DsG+C92b
        <pr:GetPriceResponse xmlns:pr="">

In a next post we will see how to verify this signature, so that we can guarantee signed documents are not changed (in other words guarantee that the integrity of the content is preserved) .


Wednesday, February 19, 2014

Latest ESB Performance Round

The results of latest round of ESB performance study is out now, which has been conducted by WSO2, comparing performance of 4 open source ESBs.  This is an open and repeatable performance study, where the used EC2 AMI is published as a public AMI that contains all the configured ESBs and execution logs along with system configurations.

Following graph summarizes the results where we can clearly see WSO2 ESB leading in performance, outperforming other ESBs.

Understanding ESB Performance

We can refer this blogpost(Understanding ESB Performnce) to understand more on the bench-marking ESBs and what to read from the results.

Is It Only Performance?

This graph is just about performance which is a critical aspect of an ESB. But there are more aspects to be considered on ESBs. If we want to know what added values are there in WSO2 ESB which has been evolving over 7 years, this blogpost(WSO2 ESB: Why Does It Matter) will be of use.

Thursday, February 06, 2014

WSO2 DSS - Batch Insert Sample (end to end)

WSO2 DSS wraps Data Services Layer and provides us with a simple GUI to define a Data Service with zero Java code. With this, a change to the data source is just a simple click away and no other party needs to be aware of this.

With this sample demonstration, we will see how to do a batch insert to a table. Batch insert is useful when you want to insert data in sequential manner. This also means that if at least one of the insertion query fails all the other queries ran so far in the batch will be rolled back as well. If one insertion in the batch fails means whole batch is failed.

This can be used if you are running the same query to insert data many times. With batch insert all the data will be sent in one call. So this reduce the number calls you have to call, to get the data inserted. 

This comes with one condition that,
The query should not be producing results back. (We will only be notified whether the query was successful or not.)

WSO2 Data Services Server - (current latest 3.1.1)

If we already have a data service running which is not sending back a result set , then it's just a matters of adding following property in service declaration.


Anyway I will be demonstrating the creation of the service from the scratch.

1. Create a service as follows going through the wizard,

2. Create the data source

3. Create the query - (This is an insert query. Also note the input mapping we have add as relevant to the query. To know more about input mapping and using validation refer the documentation.)

4. Create the operation - Select the query to be executed once the operation is called. By enabling return request status, we will be notified whether the operation was a success or not.

5. Try it! - When we list the services we will see this new service now. In the right we will have an option to try it.

Here we can see the option to try the service giving the input parameters. Here I have tried it two insertions in a batch.
Now if we go to XML view of the service it will be similar to following, which is saved in server as a .dbs file.

<data enableBatchRequests="true" name="BatchInsertSample">
   <config id="json">
      <property name="driverClassName">com.mysql.jdbc.Driver</property>
      <property name="url">jdbc:mysql://localhost:3306/json_array</property>
      <property name="username">root</property>
      <property name="password">root</property>
      <property name="minIdle">1</property>
      <property name="maxActive">10</property>
      <property name="validationQuery">SELECT 1</property>
   <query id="addFlightQuery" useConfig="json">
      <sql>insert into flights (flight_no, number_of_cases, created_by, description, trips) values (:flight_no,:number_of_cases,:created_by,:description,:trips)</sql>
      <param name="flight_no" ordinal="1" sqlType="BIGINT"/>
      <param name="number_of_cases" ordinal="2" sqlType="BIGINT"/>
      <param name="created_by" ordinal="3" sqlType="STRING"/>
      <param name="description" ordinal="4" sqlType="STRING"/>
      <param name="trips" ordinal="5" sqlType="BIGINT"/>
   <operation name="addFlight" returnRequestStatus="true">
      <call-query href="addFlightQuery">
         <with-param name="flight_no" query-param="flight_no"/>
         <with-param name="number_of_cases" query-param="number_of_cases"/>
         <with-param name="created_by" query-param="created_by"/>
         <with-param name="description" query-param="description"/>
         <with-param name="trips" query-param="trips"/>

If we hit on the service name in the list of services, we will be directed to Service Dashboard where we can see several other options for the service. It provides the option to generate an Axis2 client for the service. Once we get the client then it's a matter of calling the methods in the stub as follows.

private static BatchRequestSampleOldStub.AddFlight_type0 createFlight(int cases, String creator, String description, int trips) {

        BatchRequestSampleOldStub.AddFlight_type0 val = new BatchRequestSampleOldStub.AddFlight_type0();
        printFlightInfo(cases, creator, description, trips);
        return val;

    public static void main(String[] args) throws Exception {
        String epr = "http://localhost:9763" + "/services/BatchInsertSample";
        BatchRequestSampleOldStub stub = new BatchRequestSampleOldStub(epr);
        BatchRequestSampleOldStub.AddFlight_batch_req vals1 = new BatchRequestSampleOldStub.AddFlight_batch_req();

        vals1.addAddFlight(createFlight(1, "Pushpalanka", "test", 2));
        vals1.addAddFlight(createFlight(2, "Jayawardhana", "test", 2));
        vals1.addAddFlight(createFlight(3, "", "test", 2));
        try {
            System.out.println("Executing Add Flights..");
        } catch (Exception e) {
            System.out.println("Error in Add Flights!");

Complete client code can be found here.



Thursday, January 30, 2014

Invoking APIs using a Web App with OAuth2 and use of JWT - WSO2 API Manager

In this post I am to share my experience and understandings using WSO2 API Manager(API-M) for a very common and useful scenario in the industry. 

In brief following is the flow.

An API is exposed for app developers to be used under the control of API Manager (which adds access control for the API). Then app developers make their apps consuming those APIs. After development and testing is completed they make it available for end users at App store. The end users can then get registered in the store and use the apps with own credentials. The app will provide the desired services calling the APIs it has subscribed to.

The above scenario is well demonstrated in WSO2 API-M with the pizza shack example explained in the documentation at [1].

For clarity I will be including the steps in brief. For detailed steps we can refer documentation at [1]. 

API Developer Role

  • We deploy the back-end services related to 'pizza ordering' in WSO2-Application server or any other desired application server. (Download the code from API-M samples svn, build using Maven3 and deploy it in WSO2 AS. If we check the WADL, we can check the resources it exposes. Note the endpoint URL.)

  • Then we publishes these services as APIs in WSO2 API-M Publisher, so that they will be available in API-M Store (Login to API-M Publisher, in default pack https://localhost:9443/publisher and publish the APIs as guided in the sample doc. We should make sure the production endpoint URL  matches with what we observed in the first step). 

App Developer Role

  • Now here comes an App developer who wish to develop an App to order Pizza. He/she can register this App in store and get subscribed to these APIs that are required for development of application. So this APP developer will be consuming the services exposed by the APIs published by the previous developer. The code for the pizza ordering sample web app can be downloaded from svn as well.

  • At subscription he/she gets consumer secret and consumer key which are then used to request OAuth tokens to access the APIs (In this example we use user name and password which is required in grant type 'password'. There are several other possible grant types as well, if we don't want to send password).

Get the consumer key and secret from 'My Subscriptions'.


  • Developer embeds the consumer key and consumer secret into the Pizza ordering application (In most of the cases in web.xml).

At this point we can check how the token works with a simple curl command as follows. Provide the access token taken from above UI.
curl -k -H "Authorization: Bearer <access_token>" https://localhost:8245/pizzashack/menu/1.0.0

 which will return the menu details for pizza orering as follows,

[{"price":"13.99","icon":"/images/6.png","description":"Grilled white chicken, hickory-smoked bacon and fresh sliced onions in barbeque sauce","name":"BBQ Chicken Bacon"},{"price":"24.99","icon":"......................:"/images/5.png","description":"Rich and creamy blend of spinach and garlic Parmesan with Alfredo sauce","name":"Spinach Alfredo"},{"price":"15.99","icon":"/images/4.png","description":"Six cheese blend of mozzarella, Parmesan, Romano, Asiago and Fontina","name":"Tuscan Six Cheese"}]

Since we have already seen the access token we can use it here. But when an end user comes and tries to order pizzas he/she is not seen these. Also this token is related with USER_TYPE: APPLICATION which has more privileges than an end user, so we can't anyway let the user use it. So what is happening underneath is generating a separate token for end users, using the embedded consumer key/secret and end user entered credentials(if password grant type is used) which will be related with USER_TYPE: APPLICATION_USER.

End User

So here comes the end user who get registered in the App Store.
Then end users can use the application to order pizzas online, entering their credentials in the application at http://localhost/pizzashack.

The API-M sitting in the middle act as the Authorization server in the scenario, managing the usage of the exposed APIs.

So where does the JWT assertion comes into play?

JWT assertion is a format used to send the details of the end user who invoked the API. Just as a SAML assertion would carry user claims JWT also carries user claims in JSON notation. We can find more details on this at [2]. This is used to pass those details to the back-end service which might require them for monitoring or some other purpose. The default JWT token will be as follows.



Friday, January 17, 2014

How to send an HTML email in Java (Using Google SMTP Server)

In most of the business services sometimes there comes requirements to send notifications to users or administrators via email.

For example :
  • Confirming a user registration
  • Password reset via emails
Following code segments can be used to send these emails using Google SMTP server. Here I am sharing two ways to do it. 
  1. Using javax.mail.jar directly
  2. Using Apache commons email jar which wraps javax.mail 

Using javax.mail

try {
            Properties props = new Properties();
            props.put("", "");
            props.put("mail.smtp.auth", "true");
            props.put("mail.debug", "false");
            props.put("mail.smtp.ssl.enable", "true");

            Session session = Session.getInstance(props, new EmailAuth());
            Message msg = new MimeMessage(session);

            InternetAddress from = new InternetAddress("sendersEmailAddress", "Sender's name");

            InternetAddress toAddress = new InternetAddress("Receiver's email");

            msg.setRecipient(Message.RecipientType.TO, toAddress);

            msg.setContent(msg.setContent("<html>\n" +
                    "<body>\n" +
                    "\n" +
                    "<a href=\"\">\n" +
                    "This is a link</a>\n" +
                    "\n" +
                    "</body>\n" +
                    "</html>", "text/html");, "text/html");
        } catch (UnsupportedEncodingException ex) {

        } catch (MessagingException ex) {

    static class EmailAuth extends Authenticator {

        protected PasswordAuthentication getPasswordAuthentication() {

            return new PasswordAuthentication("sendersEmailAddress", "password");


Using Apache commons e-mail

        HtmlEmail email = new HtmlEmail();
        email.setAuthenticator(new DefaultAuthenticator("sendersEmailAddress", "password"));
        email.setFrom("Senders' email");
        email.setSubject("TestMail- Alternative message");
        email.setHtmlMsg("<html>\n" +
                "<body>\n" +
                "\n" +
                "<a href=\"\">\n" +
                "This is a link</a>\n" +
                "\n" +
                "</body>\n" +
        // set the alternative message
        email.setTextMsg("This is a link:");
Here with setTextMsg method we can set a plain text message to be shown at receiver's end if receiver is not supporting HTML content in emails.

If the gmail account used to send the email is 2-Step verification enabled and you used your usual password to send the email in code, you will get a similar error to the following.

javax.mail.AuthenticationFailedException: 534-5.7.9 Application-specific password required. Learn more at 534 5.7.9

The solution is to go to Account>Security>App passwords>Settings and generate a password for the app we are to run. 

[1] -
[2] -

Saturday, November 16, 2013

Install Liferay over a MySQL Database

With this post I will share how to get started with Liferay Portal including initial configurations and login.

Environment: Linux

Pre-requisites :

Let's download the Liferay pack from here.

I got the Community Edition of bundle with Tomcat.

Extract it to a folder of your choice, let's call the extracted folder LR_HOME. Ok, the resources are ready.. Let's go..


Login to MySQL. If you just installed following command will do.

mysql -uroot -p

Enter the password (default root).

Create database to be used for Liferay.

create database lportal;

Create user for Liferay.
create user 'lr_user'@'localhost' identified by 'user123';

Give access to the created database for this user.
grant all privileges on lportal.* to 'lr_user'@'localhost' with grant option;

Mysql work is over now. Let's go to Liferay.


Go to LR_HOME.
Inside LR_HOME create file with the name ''. This is an option provided by the Liferay architecture to override default configurations.
Copy the following content to the file.









Fill the values according to the created database.

Go inside the tomcat folder in LR_HOME and issue the following command to start it.

This will trigger the start and to see how it goes issue the following command to observe the logs.
tail -f logs/catalina.out

This will take some time at start up, since it is creating the databases. It will finally print server started in ?? ms and a new window will be automatically open in Firefox for 'http://localhost:8080/'. This is the Liferay Portal welcome page.

Login as the default admin '' with password 'test', this will get us through wizard and allow us to change the default password.

Now we are in the portal.....

Saturday, September 21, 2013

How to Write a Custom User Store Manager - WSO2 Identity Server 4.5.0

With this post I will be demonstrating writing a simple custom user store manager for WSO2 Carbon and specifically in WSO2 Identity Server 4.5.0 which is released recently. The Content is as follows,
  1. Use case
  2. Writing the custom User Store Manager
  3. Configuration in Identity Server
You can download the sample here.

Use Case

By default WSO2 Carbon has four implementations of User Store Managers as follows.
  • org.wso2.carbon.user.core.jdbc.JDBCUserStoreManager
  • org.wso2.carbon.user.core.ldap.ReadOnlyLDAPUserStoreManager
  • org.wso2.carbon.user.core.ldap.ReadWriteLDAPUserStoreManager
  • org.wso2.carbon.user.core.ldap.ActiveDirectoryLDAPUserStoreManager
Let's look at a scenario where a company has a simple user store where they have kept customer_id, customer_name and the password (For the moment let's not worry about salting etc. as purpose is to demonstrate getting a custom user store into action). So the company may want to keep this as it is, as there may be other services depending on this and still wanting to have identities managed. Obviously it's not a good practice to duplicate these sensitive data to another database to be used by the Identity Server as then the cost of securing both databases is high and can guide to conflicts. That is where a custom User Store Manager comes handy, with the high extendibility of Carbon platform.

So this is the scenario I am to demonstrate with only basic authentication.

We have the following user store which is currently in use at the company.
             PASSWORD VARCHAR(255) NOT NULL,


I have only two entries in user store. :) Now what we want is to let these already available users to be visible to Identity Server, nothing less, nothing more. So it's only the basic authentication that User Store Manager should support, according to this scenario.

Writing the custom User Store Manager

There are just 3 things to adhere when we writing the User Store Manager and the rest will be done for us.

  • Implement the 'org.wso2.carbon.user.api.UserStoreManager' interface
There are several other options to do this, by implementing 'org.wso2.carbon.user.core.UserStoreManager' interface or extending 'org.wso2.carbon.user.core.common.AbstractUserStoreManager' class, as appropriate. In this case as we are dealing with a JDBC User Store, best option is to extend the existing JDBCUserStoreManager class and override the methods as required.
CustomUserStoreManager extends JDBCUserStoreManager 

    public boolean doAuthenticate(String userName, Object credential) throws UserStoreException {

        if (CarbonConstants.REGISTRY_ANONNYMOUS_USERNAME.equals(userName)) {
            log.error("Anonymous user trying to login");
            return false;

        Connection dbConnection = null;
        ResultSet rs = null;
        PreparedStatement prepStmt = null;
        String sqlstmt = null;
        String password = (String) credential;
        boolean isAuthed = false;

        try {
            dbConnection = getDBConnection();
            sqlstmt = realmConfig.getUserStoreProperty(JDBCRealmConstants.SELECT_USER);

            prepStmt = dbConnection.prepareStatement(sqlstmt);
            prepStmt.setString(1, userName);

            rs = prepStmt.executeQuery();

            if ( {
                String storedPassword = rs.getString(2);
                if ((storedPassword != null) && (storedPassword.trim().equals(password))) {
                    isAuthed = true;
        } catch (SQLException e) {
            throw new UserStoreException("Authentication Failure. Using sql :" + sqlstmt);
        } finally {
            DatabaseUtil.closeAllConnections(dbConnection, rs, prepStmt);

        if (log.isDebugEnabled()) {
            log.debug("User " + userName + " login attempt. Login success :: " + isAuthed);

        return isAuthed;


  • Register Custom User Store Manager in OSGI framework
This is just simple step to make sure new custom user store manager is available through OSGI framework. With this step the configuration of new user store manager becomes so easy with the UI in later steps. We just need to place following class inside the project.

 * @scr.component name="" immediate=true
 * @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService"
 * cardinality="1..1" policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"
public class CustomUserStoreMgtDSComponent {
    private static Log log = LogFactory.getLog(CustomUserStoreMgtDSComponent.class);
    private static RealmService realmService;

    protected void activate(ComponentContext ctxt) {

        CustomUserStoreManager customUserStoreManager = new CustomUserStoreManager();
        ctxt.getBundleContext().registerService(UserStoreManager.class.getName(), customUserStoreManager, null);"CustomUserStoreManager bundle activated successfully..");

    protected void deactivate(ComponentContext ctxt) {
        if (log.isDebugEnabled()) {
            log.debug("Custom User Store Manager is deactivated ");

    protected void setRealmService(RealmService rlmService) {
          realmService = rlmService;

    protected void unsetRealmService(RealmService realmService) {
        realmService = null;

  • Define the Properties Required for the User Store Manager
There needs to be this method 'getDefaultProperties()' as follows. The required properties are mentioned in the class 'CustomUserStoreConstants'. In the downloaded sample it can be clearly seen how this is used.
    public org.wso2.carbon.user.api.Properties getDefaultUserStoreProperties(){
        Properties properties = new Properties();
                (new Property[CustomUserStoreConstants.CUSTOM_UM_MANDATORY_PROPERTIES.size()]));
                (new Property[CustomUserStoreConstants.CUSTOM_UM_OPTIONAL_PROPERTIES.size()]));
                (new Property[CustomUserStoreConstants.CUSTOM_UM_ADVANCED_PROPERTIES.size()]));
        return properties;

The advanced properties carries the required SQL statements for the user store, written according to the custom schema of our user store.
Now all set to go. You can build the project with your customization to the sample project or just use the jar in the target. Drop the jar inside CARBON_HOME/repository/components/dropins and drop mysql-connector-java-<>.jar inside CARBON_HOME/repository/components/lib. Start the server with ./ from CARBON_HOME/bin. In the start-up logs you will see following log printed.

INFO {} -  CustomUserStoreManager bundle activated successfully.

Configuration in Identity Server

In the management console try to add a new user store as follows.
 In the shown space we will see our custom user store manager given as an options to use as the implementation class, as we registered this before in OSGI framework. Select it and fill the properties according to the user store.

Also in the property space we will now see the properties we defined in the constants class as below.
If our schema changes at any time we can edit it here in dynamic manner. Once finished we will have to wait a moment and after refreshing we will see the newly added user store domain, here I have named it ''. 
So let's verify whether the user are there. Go to 'Users and Roles' and in Users table we will now see the users details who were there in the custom user store as below.

 If we check the roles these users are assigned to Internal/everyone role. Modify the role permission to have 'login' allowed. Now if any of the above two users tried to login with correct credentials they are allowed.
So we have successfully configured Identity Server to use our Custom User Store without much hassel.