Creating test jar in Maven

My project is moduled as common, server, web, dao

For test cases I am using Random beans

Since the model objects are used across all the modules, to have a randomizer setup for each of these module is duplication.

So I am adding this randomizer related classes and configuration only at common module, and referencing at other module for test scope

At common :

<project>

  ...

  <build>

    <plugins>

      ...

      <plugin>

        <groupId>org.apache.maven.plugins</groupId>

        <artifactId>maven-jar-plugin</artifactId>

        <version>3.0.2</version>

        <executions>

          <execution>

            <goals>

              <goal>test-jar</goal>

            </goals>

          </execution>

        </executions>

      </plugin>

      ...

    </plugins>

  </build>

  ...

</project>

At dao :

<project>
  ...
  <dependencies>
    <dependency>
      <groupId>com.foo</groupId>
      <artifactId>common</artifactId>
      <type>test-jar</type>
      <version>version</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
  ...
</project>

You can read more about this here

If you are building your project with maven.test.skip=true this test-jar is not generated. To overcome this use profile activation. i.e. at dao, add test-jar dependency only if the maven.test.skip != true

<profiles>

     <profile>

         <activation>

            <property>

               <name>maven.test.skip</name>

               <value>!true</value>

            </property>

         </activation>

         <dependencies>

            <dependency>

             <groupId>com.foo</groupId>

        <artifactId>common</artifactId>
        <type>test-jar</type>
        <version>version</version>
        <scope>test</scope>
        </dependency>

         </dependencies>

      </profile>

    </profiles>

Log mybatis sql query

To log SQL queries from a specific mapper in mybatis, add the mapper namespace to the logger level.

Eg :

If this is the mapper file,

<mapper namespace="customer">

Then to log statements that are defined in this mapper xml,
<logger name="customer">
        <level value="trace"/>
        <appender-ref ref="consoleAppender" />
</logger>

Reference :
http://www.mybatis.org/mybatis-3/logging.html

Use of profiles in Spring

In this blog I would have detailed about creating a transaction proxy. Now lets say you need to hook this proxy only test cases.
i.e. stageUtils has to be stageUtilsProxy wherever it is used only for the test cases.

Here is where profile come into picture. You can read more about it here

Add a overridden-context-for-test-profile.xml with testcase profile

<?xml version="1.0" encoding="UTF-8"?>



<beans xmlns="http://www.springframework.org/schema/beans"

    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

    xsi:schemaLocation="http://www.springframework.org/schema/beans

         http://www.springframework.org/schema/beans/spring-beans.xsd" profile="testcase">



     <bean id="testStageUtils" scope="prototype" class="com.foo.infra.util.StageUtils" />



    <bean id="stageUtils"

class="org.springframework.transaction.interceptor.TransactionProxyFactoryBean">

<property name="transactionManager" ref="transactionManager" />

<property name="target" ref="testStageUtils" />

<property name="transactionAttributes">

<props>

<prop key="*">PROPAGATION_REQUIRES_NEW</prop>

</props>

</property>

</bean>



</beans>

Add this xml to the test-application-context.xml

<?xml version="1.0" encoding="UTF-8"?>



<beans xmlns="http://www.springframework.org/schema/beans"

    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

    xmlns:tx="http://www.springframework.org/schema/tx"

    xsi:schemaLocation="http://www.springframework.org/schema/beans

         http://www.springframework.org/schema/beans/spring-beans.xsd

         http://www.springframework.org/schema/tx

         http://www.springframework.org/schema/tx/spring-tx.xsd">



    <import resource="classpath:application-context.xml"/>

    <import resource="classpath:overridden-context-for-test-profile.xml"/>

</beans>

Annotate your BaseIntegrationTest class with @ActiveProfile annotation

@RunWith(SpringJUnit4ClassRunner.class)

@ContextConfiguration(locations = {

        "classpath:test-application-context.xml"

        })

@TransactionConfiguration(transactionManager = "transactionManager", defaultRollback = true)

@Transactional

@ActiveProfiles(profiles="testcase")

public abstract class BaseIntegrationTest {

Adding maven dependencies from github

How to add dependencies from github into your maven project

https://jitpack.io/

Example :
If I have to add dependency from
https://github.com/cheekychinnu/PnlAggregator

Enter the repo as below and hit lookup

I know! So cool

@ContextConfiguration with class

If your spring application is a booty, you would simply go with an @SpringBootTest to achieve the similar result. But unfortunately, some of our obsolete projects are far behind this.

In one of my post, I would have briefed how to configure random beans. I need to test the same. i.e. I need the context information of this random bean to have specific test cases on top of them.


This is the configuration class and
This is the test case for the same.

AnnotationConfigContextLoader is the one that facilitates this

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes=PnlRandomizerConfiguration.class, loader = AnnotationConfigContextLoader.class)
public class PnlRandomizerTest {

I would encourage you read more about this here

Note that it cannot load both xml and class.
More about it here and here

Random Beans

There are many instances while writing test cases, where we wanted a baked object with some default values, which we then modify partially as per the need for the test.
Random beans exactly achieves this

I have a Pnl object which has PnlKey. For my test, I need test data with different key elements - date, book, account, etc - so that I can check if the aggregation on those levels works perfectly.

So for the test setup, I would want to create a lot of Pnl objects with only certain concerned attributes with values, say,
date - last 5 days
book - from 1 to 10
etc

public class Pnl {

      private PnlKey pnlKey;

}

public class PnlKey {

    private String custodianAccount;



    private Integer bookId;



    private Integer pnlSpn;



    private Date date;



    private Integer bundleId;



}

@Bean
public EnhancedRandom getEnhancedRandomForPnl() {
 Randomizer<Date> dateRandomizer = () -> {
  int max = dates.length;
  Random random = new Random();
  return dates[random.nextInt(max)];
 };
 Randomizer<Integer> bookRandomizer = () -> {
  int min = 1;
  int max = 10;
  Random random = new Random();
  return min + random.nextInt((max - min) + 1);
 };
 Randomizer<Integer> bundleRandomizer = () -> {
  int min = 11;
  int max = 20;
  Random random = new Random();
  return min + random.nextInt((max - min) + 1);
 };
 Randomizer<String> custodianAccountRandomizer = () -> {
  Random random = new Random();
  return custodianAccounts[random.nextInt(custodianAccounts.length)];
 };
 EnhancedRandom enhancedRandom = EnhancedRandomBuilder.aNewEnhancedRandomBuilder()
   .randomize(Date.class, dateRandomizer)
   .randomize(new FieldDefinition<>("bookId", Integer.class, PnlKey.class), bookRandomizer)
   .randomize(new FieldDefinition<>("bundleId", Integer.class, PnlKey.class), bundleRandomizer)
   .randomize(new FieldDefinition<>("custodianAccount", String.class, PnlKey.class),
     custodianAccountRandomizer)
   .build();
 return enhancedRandom;
}

You can check how this is configured here

And how it is accessed/tested here

Default parameterization of varchar to nvarchar in sql driver - Part 2

In this post I have described the default parameterization of varchar to nvarchar in JDBC driver and how to switch off that behavior. In the following post I am going to explain in detail about the checklist that you should do in order to set sendStringParametersAsUnicode to false.

Checklist (for sql server)

• Check the database level collation

SELECT name, collation_name
FROM sys.databases
WHERE name = 'xyz';  --where xyz is the database name

• There should be no nvarchar columns in your database

SELECT * FROM xyz.INFORMATION_SCHEMA.COLUMNS WHERE DATA_TYPE IN ('nvarchar', 'nchar');

• The collation set for the varchar columns in your database should be the same as the database level collation

-- to view the table, column, data-type and collation details
SELECT OBJECT_NAME(c.object_id) as 'OBJECT NAME',
SCHEMA_NAME(t.schema_id) as 'SCHEMA NAME',
t.name as 'TABLE NAME',
y.name as 'DATA TYPE', c.* FROM xyz.sys.columns c
JOIN xyz.sys.tables t on c.object_id = t.object_id
JOIN sys.types y ON y.system_type_id = c.system_type_id
where c.collation_name is NOT NULL

-- check if there are any columns with collation different from that of the database
SELECT SCHEMA_NAME(t.schema_id) as 'SCHEMA NAME',
t.name as 'TABLE NAME',
y.name as 'DATA TYPE', c.* FROM xyz.sys.columns c
JOIN xyz.sys.tables t on c.object_id = t.object_id
JOIN sys.types y ON y.system_type_id = c.system_type_id
where c.collation_name  NOT IN  (SELECT collation_name
FROM sys.databases
WHERE name = 'xyz')

 Why concerned with collation?

This section is going to be long and will cover how we arrived at the checklist above

What is collation?

A character set is a set of symbols and encodings. A collation is a set of rules for comparing characters in a character set. Let's make the distinction clear with an example of an imaginary character set.
Suppose that we have an alphabet with four letters: 'A', 'B', 'a', 'b'. We give each letter a number: 'A' = 0, 'B' = 1, 'a' = 2, 'b' = 3. The letter 'A' is a symbol, the number 0 is the encoding for 'A', and the combination of all four letters and their encodings is a character set.
Now, suppose that we want to compare two string values, 'A' and 'B'. The simplest way to do this is to look at the encodings: 0 for 'A' and 1 for 'B'. Because 0 is less than 1, we say 'A' is less than 'B'. Now, what we've just done is apply a collation to our character set. The collation is a set of rules (only one rule in this case): "compare the encodings." We call this simplest of all possible collations a binary collation.
But what if we want to say that the lowercase and uppercase letters are equivalent? Then we would have at least two rules: (1) treat the lowercase letters 'a' and 'b' as equivalent to 'A' and 'B'; (2) then compare the encodings. We call this a case-insensitive collation. It's a little more complex than a binary collation.
In real life, most character sets have many characters: not just 'A' and 'B' but whole alphabets, sometimes multiple alphabets or eastern writing systems with thousands of characters, along with many special symbols and punctuation marks. Also in real life, most collations have many rules: not just case insensitivity but also accent insensitivity (an "accent" is a mark attached to a character as in German 'ö') and multiple-character mappings (such as the rule that 'ö' = 'OE' in one of the two German collations).

The collation can be Accent sensitive (AS) Accent Insensitive (AI) Case Sensitive (CS) and Case Insensitive (CI)

The following should give you a pretty good idea about this.

create table xyz..csas (name varchar(10) COLLATE Latin1_General_CS_AS)
insert into xyz..csas values ('Aries')
insert into xyz..csas values ('aries')
insert into xyz..csas values (N'áries')
select * from xyz..csas where name = 'Aries' --Aries
select * from xyz..csas where name = 'aries' --aries
select * from xyz..csas where name = N'áries' --áries

create table xyz..csai (name varchar(10) COLLATE Latin1_General_CS_AI)
insert into xyz..csai values ('Aries')
insert into xyz..csai values ('aries')
insert into xyz..csai values (N'áries')
select * from xyz..csai where name = 'Aries'  --Aries
select * from xyz..csai where name = 'aries' --aries, áries
select * from xyz..csai where name = N'áries' --aries, áries

create table xyz..ciai (name varchar(10) COLLATE Latin1_General_CI_AI)
insert into xyz..ciai values ('Aries')
insert into xyz..ciai values ('aries')
insert into xyz..ciai values (N'áries')
select * from xyz..ciai where name = 'Aries' --Aries,aries,áries
select * from xyz..ciai where name = 'aries' --Aries,aries,áries
select * from xyz..ciai where name = N'áries' --Aries,aries,áries

create table xyz..cias (name varchar(10) COLLATE Latin1_General_CI_AS)
insert into xyz..cias values ('Aries')
insert into xyz..cias values ('aries')
insert into xyz..cias values (N'áries')
select * from xyz..cias where name = 'Aries' --Aries,aries
select * from xyz..cias where name = 'aries' --Aries,aries
select * from xyz..cias where name = N'áries' --áries

Let me iterate over the steps through which I concluded with the checklist.

I have written test cases in our application that asserts the above query with and without sendStringParametersAsUnicode set.

I was expecting it to succeed for dataSourceWithUniCodeEnabled and fail for dataSourceWithUniCodeDisabled. But no, it was succeeding for both.

so when I am passing á in the code with the unicode disabled, how did it the string go as unicode?

This is where the database level collation matters. It was working fine because the collation at database level was also Latin1_General_CS_AS 

Confusing right? Let us take a step back.

I am creating a table with Latin1_General_CS_AS collation and varchar column. By definition, I cannot insert unicode. but why does the following work?

create table xyz..egvarchar (name varchar(10) COLLATE Latin1_General_CS_AS)
insert into  xyz..egvarchar  values (N'áries')
select * from xyz..egvarchar where name = N'áries' -- gives áries as output

This is because defining collation allows certain characters to be allowed even in the varchar column. To know what are supported in a collation,

;WITH AllNumbers AS
(
SELECT 1 AS Number
UNION ALL
SELECT Number+1
FROM AllNumbers
WHERE Number+1<256
)
SELECT Number AS ASCII_Value,CHAR(Number) COLLATE Latin1_General_CS_AI ASCII_Char FROM AllNumbers

OPTION (MAXRECURSION 256);

so if I try to insert a non-latin in the same table

insert into  xyz..egvarchar  values (N'áяies')

it gets inserted but when I select.

áries

á?ies --> note the question mark here. Meaning it does not support that character.

If I repeat the same exercise with the column as nvarchar, the output would be

create table xyz..egnvarchar (name nvarchar(10) COLLATE Latin1_General_CS_AS)
insert into  xyz..egnvarchar  values (N'áries')
insert into  xyz..egnvarchar  values (N'áяies')
select * from xyz..egnvarchar

OUTPUT:

áries

áяies

This is why it is important to make sure there are no nvarchar column before turning unicode off. Now back to our problem, the reason why my tests did not fail with sendStringParametersAsUnicode set to false was because á was supported because of the collation. What collation? The collation at the level you have defined your datasource.

<property name="url" value="jdbc:jtds:sqlserver://dataserver_name/xyz" />

 In the above example, it is the collation set to the database xyz. This is very important. Hence the checklist 3) collation of the columns in your database should be the same as the database level collation. To validate again, we created another database abc with collation Cyrillic_General_CI_AS Created 2 tables -one with Cyrillic_General_CI_AS collation and another with Latin1_General_CI_AS

create table abc..cyrillic (name varchar(10) COLLATE Cyrillic_General_CI_AS)
insert into abc..cyrillic values (N'Aяies')
select * from abc..cyrillic where name = N'Aяies'

create table abc..latin (name varchar(10) COLLATE Latin1_General_CI_AS)
insert into abc..latin values (N'áries')
select * from abc..latin where name = N'áries'

The table with collation set to Cyrillic will return results irrespective of the property sendStringParameterAsUnicode The table with collation set to Latin will return result only when sendStringParameterAsUnicode is set to true Meaning, the default behavior of the sql driver to consider whether something is unicode/not depends on the collation of the datasource. Note that it is very important at what level the url property is set. It should be set at the database level since you have done the pre-check against your database.

For Eg:

jdbc:jtds:sqlserver://dataserver_name/xyz -> right
jdbc:jtds:sqlserver://dataserver_name -> wrong (collation at dataserver and the database might be different)

 Code

<bean id="cyrillicDataSourceWithUniCodeDisabled" class="org.apache.tomcat.jdbc.pool.DataSource">
<property name="driverClassName" value="net.sourceforge.jtds.jdbc.Driver" />
<property name="url" value="jdbc:jtds:sqlserver://dataserver_name/abc" />
<property name="initialSize" value="2" />
<property name="maxActive"
value="<some_value>" />
<property name="maxWait"
value="<some_value>" />
<property name="removeAbandonedTimeout"
value="<some_value>" />
        <property name="minEvictableIdleTimeMillis" value="<some_value>"/>
        <property name="timeBetweenEvictionRunsMillis" value="<some_value>"/>
<property name="testOnBorrow" value="true" />
<property name="validationQuery">
<value>SELECT 1</value>
</property>
<property name="connectionProperties" value="sendStringParametersAsUnicode=false;"/>
</bean>
<bean id="cyrillicDataSourceWithUniCodeEnabled" class="org.apache.tomcat.jdbc.pool.DataSource">
<property name="driverClassName" value="net.sourceforge.jtds.jdbc.Driver" />
<property name="url" value="jdbc:jtds:sqlserver://dataserver_name/abc" />
<property name="initialSize" value="2" />
<property name="maxActive"
value="<some_value>" />
<property name="maxWait"
value="<some_value>" />
<property name="removeAbandonedTimeout"
value="<some_value>" />
        <property name="minEvictableIdleTimeMillis" value="<some_value>"/>
        <property name="timeBetweenEvictionRunsMillis" value="<some_value>"/>
<property name="testOnBorrow" value="true" />
<property name="validationQuery">
<value>SELECT 1</value>
</property>
</bean>

 If you change the url of the datasource injected from
<property name="url" value="jdbc:jtds:sqlserver://dataserver_name/abc" >
    to
<property name="url" value="jdbc:jtds:sqlserver://dataserver_name/" />
 the dataserver's collation will be taken which is latin, only the test with unicode enabled will succeed here.

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;

import java.util.List;

import javax.sql.DataSource;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.core.RowMapper;

public class CyrillicDatasourceUnicodeTest extends BaseDAOTest {

    private static final String CYRILLIC_ACCENT_CASE = "A" + "\u044F" + "ies";

    private static final String LATIN_ACCENT_CASE = "\u00E1" + "ries";

    @Autowired
    private DataSource cyrillicDataSourceWithUniCodeEnabled;

    @Autowired
    private DataSource cyrillicDataSourceWithUniCodeDisabled;

    @org.junit.Test
    public void testForCyrillicForUnicodeEnabled() {
        JdbcTemplate jdbcTemplate = new JdbcTemplate(cyrillicDataSourceWithUniCodeEnabled);
        String sql = "select name from abc..cyrillic where name = ?";
        RowMapper rowMapper = (RowMapper) (rs, rowNum) -> {
            return rs.getString(1);
        };

        List results = jdbcTemplate.query(sql, rowMapper, CYRILLIC_ACCENT_CASE);
        assertNotNull(results);
        System.out.println(results);
        assertEquals(1, results.size());
        assertEquals(CYRILLIC_ACCENT_CASE, results.get(0));

    }

    @org.junit.Test
    public void testForCyrillicForUnicodeDisabled() {
        JdbcTemplate jdbcTemplate = new JdbcTemplate(cyrillicDataSourceWithUniCodeDisabled);
        String sql = "select name from abc..cyrillic where name = ?";
        RowMapper rowMapper = (RowMapper) (rs, rowNum) -> {
            return rs.getString(1);
        };
        List results = jdbcTemplate.query(sql, rowMapper, CYRILLIC_ACCENT_CASE);
        assertNotNull(results);
        System.out.println(results);
        assertEquals(
                "supposed to match the result regardless of the sendStringParametersAsUnicode 
property because the datasource is also the same collation as the column",
                1, results.size());
        assertEquals(CYRILLIC_ACCENT_CASE, results.get(0));
    }

    @org.junit.Test
    public void testForLatinForUnicodeEnabled() {
        JdbcTemplate jdbcTemplate = new JdbcTemplate(cyrillicDataSourceWithUniCodeEnabled);
        String sql = "select name from abc..latin where name = ?";
        RowMapper rowMapper = (RowMapper) (rs, rowNum) -> {
            return rs.getString(1);
        };

        List results = jdbcTemplate.query(sql, rowMapper, LATIN_ACCENT_CASE);
        assertNotNull(results);
        System.out.println(results);
        assertEquals(1, results.size());
        assertEquals(LATIN_ACCENT_CASE, results.get(0));

    }

    @org.junit.Test
    public void testForLatinForUnicodeDisabled() {
        JdbcTemplate jdbcTemplate = new JdbcTemplate(cyrillicDataSourceWithUniCodeDisabled);
        String sql = "select name from abc..latin where name = ?";
        RowMapper rowMapper = (RowMapper) (rs, rowNum) -> {
            return rs.getString(1);
        };
        List results = jdbcTemplate.query(sql, rowMapper, LATIN_ACCENT_CASE);
        assertNotNull(results);
        System.out.println(results);
        assertEquals(
                "No results will come because unicode is disabled and the collation of the column (latin) 
is different from that of the datasource (cyrillic)",
                0, results.size());
    }

}

ReflectionTestUtils to tweak private fields for unit tests

I was writing an integration test for create in bulk mode which inserts data in bulk if the number of records is over 100.

Now just in order to test this bulk insert feature I do not want to load 100 records. So I need to tweak this batchSize set to 100 to 1

public class DaoImpl {
private static Integer batchSize = 100;

@Autowired
private Dao dao;

public void test()
{
// get the default size
Integer defaultBatchSize = (Integer) ReflectionTestUtils.getField(dao, "batchSize");

// tweak the data
Integer batchSize = 1;
ReflectionTestUtils.setField(dao, "batchSize", batchSize, Integer.class);
assertEquals(batchSize, ReflectionTestUtils.getField(dao, "batchSize"));

// proceed with test

// revert it back to the default size
ReflectionTestUtils.setField(dao, "batchSize", defaultBatchSize, Integer.class);
}

ReflectionTestUtils.setField will internally take care of making your field accessible.

Use of TransactionProxyFactoryBean in Springs

Generally while writing integration test, we annotate with @TransactionConfiguration and set defaultRollback to true. The rollback helps when you do not want to explicitly delete the test data after every test case.

In a project I was working on, I happened to write an integration test for bulk insert. Bulk insert to the table is done by dumping the records in the stage table. There is a commons library in the firm that is used inorder to create this stage table.

I observed that the rollback was not happening only with the bulk insert test cases. At first I thought - we are creating stage table. which means there is a create statement executed. And since DDL statements are implicitly committed, we will not be able to rollback. But later I found that in sql server, DDLs are not implicitly committed

So when I actually went through the stageUtils source code, I found the cause was connection.commit() it was invoking and not the fact the statement was DDL, that is not causing the rollback.

How to fix it?

1) stageUtils is not something the project owns. Hence there is no way I can modify that.
2) For my transaction to rollback, I need to treat this stage table creation as a new transaction. The way you can achieve it, is by annotation stageUtils with @Transactional(propagation = Propagation.REQUIRES_NEW)

But note that I cannot do 2) because of 1)

Here is where TransactionProxyFactoryBean comes for rescue. This enabling creating proxy around the stageUtils and define required transaction attributes.

<bean id="stageUtils" class="com.foo.infra.util.StageUtils"
    scope="prototype"/>
<bean id="stageUtilsProxy"
    class="org.springframework.transaction.interceptor.TransactionProxyFactoryBean">
    <property name="transactionManager" ref="someTransactionManager" />
    <property name="target" ref="stageUtils" />
    <property name="transactionAttributes">
        <props>
            <prop key="*">PROPAGATION_REQUIRES_NEW</prop>
        </props>
    </property>
</bean>
<bean id="myDAO"
    class="com.foo.myproject.dao.someDAOImpl">
        <!-- BEFORE -->
        <!-- <property name="stageUtils" ref="stageUtils" /> -->
        <!-- AFTER -->
        <property name="stageUtils" ref="stageUtilsProxy" />
</bean>

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
@TransactionConfiguration(transactionManager = "someTransactionManager", defaultRollback = true)
@Transactional
public abstract class BaseIntegrationDAOTest

Propagation requires new will force the stageUtils to start a new transaction and hence it won't disturb your transaction rollback

Mockito - Rule and Runner

While using mockito for unit test, you initialize using either

@RunWith(MockitoJUnitRunner.class)

or

@Before
public void setUp() {
   MockitoAnnotations.initMocks(this);
}

If you are getting NPE for the mocked objects at the class you are testing, then you have missed either of the above. MockitoJUnitRunner automatically provides you with a initMocks() It also provides automatic validation of framework usage.

There might be cases where you are already using runner, say, SpringJunit4ClassRunner. In such cases you cannot have 2 runners. In such scenarios, we can go with

- the second approach using initMocks.
- With JUnit 4.12, we can have multiple runners operate on the same test class by means of JUnit Rules. Mockito already provides us MockitoRule

Note that when you are trying to write Parameterized tests, always use runner for Parameterized and rule for Mockito. Quoting this reference

The reason is that the Runner is responsible for reporting the number of tests, and Parameterized manipulates the number of tests based on the number of test methods and the number of parameterized inputs, so it's really important for Parameterized to be a part of the Runner process. By contrast, the use of a Mockito runner or rule is simply to encapsulate the @Before and @After methods that initialize Mockito annotations and validate Mockito usage, which can be done very easily as a @Rule that works adjacent to other @Rule instances--to the point that the MockitoJUnitRunner is very nearly deprecated.

Default parameterization of varchar to nvarchar in sql driver

We came across a strange case where there were high logical reads for some queries coming from Spring batch. This was a query executed from Spring batch's  SimpleJobRepository where it tries to create JobInstance using the jobInstanceDao

This is the query which we have no control over.

When it goes to db, the type is nvarchar and not varchar

DECLARE @P0 nvarchar(4000) = 'someJobTable', @P1 nvarchar(4000) = 'someJobKey'
SELECT JOB_INSTANCE_ID, JOB_NAME from SOME_JOB_INSTANCE where JOB_NAME =  @P0  and JOB_KEY =  @P1

This has ~1900 logical reads

While the same query with type varchar took only 4 logical reads

DECLARE @P0 varchar(100) = 'someJobTable', @P1 varchar(32) = 'someJobKey'
SELECT JOB_INSTANCE_ID, JOB_NAME from SOME_JOB_INSTANCE where JOB_NAME =  @P0  and JOB_KEY =  @P1

On digging further we analyzed that the call from
org.springframework.jdbc.core.JdbcTemplate.query(PreparedStatementCreator, PreparedStatementSetter, ResultSetExtractor<T>)
to
net.sourceforge.jtds.jdbc.JtdsPreparedStatement.setParameter(int, Object, int, int, int)
where the connection.getUseUnicode() is coming as true and hence the type nvarchar

Note that this causes high logical reads even for an indexed column. nvarchar are double-byte and even those indexed varchar column gets converted to nvarchar for comparison since the query parameter in the where clause is coming over as nvarchar.

To suppress this behaviour, you need to set sendStringParametersAsUnicode to false in the connection properties of the datasource used by the spring batch. You can choose to set this property in all of the datasources used in your application. But that depends on the usecase. If you are supporting internationalization, then you might have to analyze where to switch this property on/off.

<batch:job-repository id="jobRepository"
table-prefix="SOME"
data-source="someDataSource" isolation-level-for-create="READ_UNCOMMITTED"
transaction-manager="someTransactionManager" />

<bean id="someDataSource" class="org.apache.tomcat.jdbc.pool.DataSource">
<property name="driverClassName" value="net.sourceforge.jtds.jdbc.Driver" />
<property name="url" value="jdbc:jtds:sqlserver://someserver/somedatabase" />
        <property name="connectionProperties" value="sendStringParametersAsUnicode=false;"/>
</bean>

Reference :
http://www.digitalsanctuary.com/tech-blog/java/mssql-jtds-nvarchar-and-slow-indexes.html
https://stackoverflow.com/questions/4717520/sql-server-uses-high-cpu-when-searching-inside-nvarchar-strings

https://blogs.msdn.microsoft.com/sqlcat/2010/04/05/character-data-type-conversion-when-using-sql-server-jdbc-drivers/

Overriding Spring Context Initializers

Consider the scenario where you need to override certain properties across the integration tests

User class with name and hobby

PropertySources :

NameHobbyPropertySource

public class NameHobbyPropertySource extends PropertySource<properties> {

 static Properties source = new Properties();
 
 public NameHobbyPropertySource(String name) {
  super(name, source);
  source.setProperty("name", "chinnu");
  source.setProperty("hobby", "cycling");
 }

 @Override
 public Object getProperty(String key) {
  return getSource().get(key);
 }
}

HobbyOnlyPropertySource

public class HobbyOnlyPropertySource extends PropertySource<properties> {

 static Properties source = new Properties();
 
 public HobbyOnlyPropertySource(String name) {
  super(name, source);
  source.setProperty("hobby", "running");
 }

 @Override
 public Object getProperty(String key) {
  return getSource().get(key);
 }
}

And this is your default context initializer for your application (that uses NameHobby property source)

public class MainApplicationContextInitializer implements ApplicationContextInitializer<configurableapplicationcontext> {

 public void initialize(ConfigurableApplicationContext applicationContext) {
  System.out.println("Main initializer called");
  NameHobbyPropertySource nameHobbyPropertySource = new NameHobbyPropertySource("name-hobby");
  applicationContext.getEnvironment().getPropertySources().addFirst(nameHobbyPropertySource);
 }

}

Say your BaseIntegrationTest is configured with initializer that has NameHobbyPropertySource

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(initializers = { MainApplicationContextInitializer.class
  }, locations = { "classpath:application-context.xml" })
public abstract class BaseApplicationTest {
 
}

but you want to override this for only one test class.
i.e. your special test case should run with overridden hobby from HobbyOnlyPropertySource. You can achieve this using inheritInitializers as follows,

public class SubSetApplicationContextInitializer implements
  ApplicationContextInitializer<configurableapplicationcontext> {

 public void initialize(ConfigurableApplicationContext applicationContext) {
  System.out.println("Subset initializer called");
  HobbyOnlyPropertySource hobbyOnlyPropertySource = new HobbyOnlyPropertySource(
    "hobby");
  applicationContext.getEnvironment().getPropertySources()
    .addFirst(hobbyOnlyPropertySource);
 }

}

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(initializers = { SubSetApplicationContextInitializer.class }, inheritInitializers = true)
public class ApplicationTest extends BaseApplicationTest {

but this will not work because, By default the initializers of the derived class gets called and then the base class's initializers are called.

To override this behaviour, BOTH the initializer should implement Ordered interface and define an order.

Whichever one wants to run later should be given higher order. You can also try @Order annotation for the same.

 Lets say MainApplicationContextInitializer is from third party lib, and it doesn't implement or annotated with Order, then this wouldn't work. So have a wrapper over the third party initializer and annotate it with @Order.

@Order(0)
public class WrapperMainApplicationContextInitializer extends MainApplicationContextInitializer{

}

And so the BaseApplicationTest should now use this wrapper,

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(initializers = { WrapperMainApplicationContextInitializer.class
  }, locations = { "classpath:application-context.xml" })
public abstract class BaseApplicationTest {
 
}

and your subset application context initializer should also be ordered

public class SubSetApplicationContextInitializer implements
  ApplicationContextInitializer<configurableapplicationcontext>, Ordered {

 public void initialize(ConfigurableApplicationContext applicationContext) {
  System.out.println("Subset initializer called");
  HobbyOnlyPropertySource hobbyOnlyPropertySource = new HobbyOnlyPropertySource(
    "hobby");
  applicationContext.getEnvironment().getPropertySources()
    .addFirst(hobbyOnlyPropertySource);
 }

 public int getOrder() {
  return 1;
 }
}

Refer :

• MainApplicationContextInitializer (that is not ordered)
• WrapperMainApplicationContextInitializer that makes the MainApplicationContextInitializer ordered
• SubSetApplicationContextInitializer that overrides certain properties
• BaseApplicationTest & ApplicationTest -> Test case that confirms that the overriding of properties succeeded 

NullComparator

I will demonstrate the use of this comparator in the scenario below.

Let us consider some entity called Allocation

There are many types of allocations.
ALevelAllocation
BLevelAllocation
etc

All of them have month.

if month is null -> Global allocation
if month is not null -> Monthly allocation

So let us introduce an interface

public interface MonthLevelAllocationMarker {
    String getMonth();
}

class ALevelAllocation implements MonthLevelAllocationMarker{

class BLevelAllocation implements MonthLevelAllocationMarker{

This way when I need to write any custom comparator to sort the allocations, I need to write only one Comparator for type MonthLevelAllocationMarker

import org.apache.commons.collections.comparators.NullComparator;



public class MonthFirstAllocationSorter implements Comparator<MonthLevelAllocationMarker>, Serializable

{

    private static final long serialVersionUID = 1L;



    @Override

    public int compare(MonthLevelAllocationMarker a, MonthLevelAllocationMarker b)

    {

        boolean nullsAreHighForGlobalMonth = true;

        NullComparator nullComparator = new NullComparator(nullsAreHighForGlobalMonth);

        return nullComparator.compare(a.getMonth(), b.getMonth());

    }

}

I am using NullComparator to sort the nulls are high. Meaning Monthly allocations first and global allocation last.

Usage :

List<ALevelAllocation> allocations;

Collections.sort(allocations, new MonthFirstAllocationSorter());

Equals and HashCode Builder - Why not to use them

Many times I have seen HashCodeBuilder and EqualsBuilder getting used in the Models to write hashCode() and equals() method. In SDEs like eclipse, help is just one click away. But for many reasons, one might not resort to this option because you have to keep updating your equals and hashCode methods when you are adding new variables to the class.

It is important to note that these builders use reflection to access the variables.

It uses AccessibleObject.setAccessible to gain access to private fields. This means that it will throw a security exception if run under a security manager, if the permissions are not set up correctly. It is also not as efficient as testing explicitly.
Transient members will be not be tested, as they are likely derived fields, and not part of the value of the Object.
Static fields will not be tested. Superclass fields will be included.

The important point to notice here is that the superclass fields will be included.

I know it is too tempting to use these builders especially when you want to ignore certain fields for equals().

For instance, you would not want to compare a enriched field such as "Account" you would want to restrict the equals() to only "accountID"

When you are over-using this utility with ignoreFields, you run into the following problem.

import org.apache.commons.lang.builder.EqualsBuilder;
import org.apache.commons.lang.builder.HashCodeBuilder;
public class BaseClass {
     
    public static String[] ignoreFieldsInEquals = new String[] {"comments"};
     
    private String id;
    private String comments;
     
    public BaseClass()
    {
        super();
    }
     
    public BaseClass(String id, String comments) {
        super();
        this.id = id;
        this.comments = comments;
    }
    public String getId() {
        return id;
    }
    public void setId(String id) {
        this.id = id;
    }
    public String getComments() {
        return comments;
    }
    public void setComments(String comments) {
        this.comments = comments;
    }
    @Override
    public int hashCode() {
        return HashCodeBuilder.reflectionHashCode(this, ignoreFieldsInEquals);
    }
    @Override
    public boolean equals(Object obj) {
        return EqualsBuilder.reflectionEquals(this, obj, ignoreFieldsInEquals);
    }
}

package com.foo;
 
import org.apache.commons.lang.builder.EqualsBuilder;
import org.apache.commons.lang.builder.HashCodeBuilder;
 
public class SubClass extends BaseClass {
    private static String[] ignoreFieldsInEquals = new String[] {"alias"};
     
    private String name;
    private String alias;
     
    public SubClass() {
        super();
    }
    public SubClass(String id, String comments,String name, String alias) {
        super(id, comments);
        this.name = name;
        this.alias = alias;
    }
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public String getAlias() {
        return alias;
    }
    public void setAlias(String alias) {
        this.alias = alias;
    }
     
    @Override
    public int hashCode() {
        return super.hashCode() + HashCodeBuilder.reflectionHashCode(this, ignoreFieldsInEquals);
    }
 
    @Override
    public boolean equals(Object obj) {
        return super.equals(obj) && EqualsBuilder.reflectionEquals(this, obj, ignoreFieldsInEquals);
    }
}

import static org.junit.Assert.*;
import org.junit.Test;
public class ReflectionBuilderTest {
     
    @Test
    public void test()
    {
        String id1 = "TE1";
                 
        SubClass sub1 = new SubClass(id1, "random comment 1", "Bruce Wayne", "batsy");
        SubClass sub2 = new SubClass(id1, "random comment 2", "Bruce Wayne", "batman");
         
        assertEquals(sub1, sub2); //fails
    }
}

In SubClass, we are expecting super.equals() to ignore comments from BaseClass and the equals to ignore alias but it will not work since it uses reflection.
so EqualsBuilder.reflectionEquals() already read your alias and returned false. One way to fix this would be, to pass the base class's ignore field as well in the SubClass's equals.

BaseClass :

private static String[] ignoreFieldsInEquals = new String[] {"comments"};
 
@Override
public int hashCode() {
    return HashCodeBuilder.reflectionHashCode(this, getFieldsToIgnoreInEquals());
}
@Override
public boolean equals(Object obj) {
    return EqualsBuilder.reflectionEquals(this, obj, getFieldsToIgnoreInEquals());
}
protected String[] getFieldsToIgnoreInEquals() {
    return ignoreFieldsInEquals;
}

SubClass:

private static String[] ignoreFieldsInEquals = new String[] {"alias"};
 
protected String[] getFieldsToIgnoreInEquals()
{
    return Stream.of(ignoreFieldsInEquals, super.getFieldsToIgnoreInEquals()).flatMap(Stream::of).toArray(String[]::new);
}
 
@Override
public int hashCode() {
    return  HashCodeBuilder.reflectionHashCode(this, getFieldsToIgnoreInEquals());
}
@Override
public boolean equals(Object obj) {
    return EqualsBuilder.reflectionEquals(this, obj, getFieldsToIgnoreInEquals());
}

But even this would not work when the sub class has to choose what BaseClass's variables to ignore and what not to ignore.

For example, there are many sub classes for the BaseClass and one of them decides that comments are also important for them. Usually such need does not arrive for equals() rather for something like "isValueEqualsForPersist"

The ideal solution would be a careful engineering about where to put what. In this case, for example, maybe the comments should not even be in the BaseClass.

But this post is more about how much reflection can screw it up. For Example I am working on a 10 year old project where Equals and HashCode builders are used and there are sub classes, where one of them needs a super class fields in equalsForPersist and the other does not. Now refactoring the whole thing and restructuring the classes is a big pain in the ass, which could have been avoided by careful designing.

Generics - T extends Class & Interface

Say you have a Transaction object

public abstract class Transaction {
Long transactionId

And you have various transactions,

MoneyTransaction
RewardPointTransaction
CouponTransaction
etc

There are some of these transactions that are  Transferable and some are not

public interface Transferable {
    Long getDestinationAccount();
}

I need to write a common assertion while writing test case.

public void assertTransferableTransactions

takes in lists

expectedTransactions
actualTransactions

I would need transactionId to group the inputs, which is in type Transaction
And the methods to be in tested that are in Transferable

So I could have my assertion like

public static <T extends Transaction & Transferable> void assertTransferrableTransactions(
            List<T> expectedTransactions, List<T> expectedTransactions)

You can have just this method in a test util and avoid the code duplicate throughout all the Transaction tests that are transferable

Note that in
T extends A & B
class name should come first and interfaces follow.
i.e.
<T extends Transferable & Transaction> will give compilation error

Junit : Mocking a method of spy

I came across a test case failure in my project. The method mocked using spy was something like

public class Service {
 public String foo()
 {
  return "dummy";
 }
}

And in the test cases

private Service spyService;

@Test
public void test()
{
 String test = "test";
 when(spyService.foo()).thenReturn(test);
}

We did some changes in the Service to cache the reference values during initialization and use the map to get the values.

public class Service {
 private Map cache;
 ...
 public String foo()
 {
  return cache.get("someKey");
 }
}

And now the test cases started throwing NPE at cache.get() in Service's foo

Why is this? When you use spy, the method actually gets called. That's why before adding the cache, it was working fine.

So when you are using when on spy, call doReturn first. This will prevent the method being mocked from getting executed.

doReturn(test).when(spyService).foo();

Read this for more details.
Be careful when using spy in junit tests.

Cordova Push Plugin

I had an opportunity to work on an mobile application built on cordova. My hackathon project was about building a Notification Service and in order to integrate it to various channels, one of them being Mobile app, I worked on adding push plugin to the cordova app. For those who are new to cordova, it is "code in one language and deploy in many platforms" i.e. Platform independent. My JavaScript knowledge ranges from zero to none. But lucky for me, this change didn't involve much of JavaScript.

There were a few interesting review on exposing "too"much information on the notification payload. What is too much information depends on your business platform. So our initial thoughts of adding action buttons to the notification and hooking url to the action button on callbacks were all discarded due to the security risks. What I am going to outline here, is

- To get started with Cordova
- To send a push notification from FCM - Firebase Cloud Messaging and
- Receive it via Cordova Push Plugin

Installation

Follow the installation steps in the official Cordova documentation here

• Install Node.js
• Ìnstall GIT
• Install Cordova - (I have tried the entire setup on windows)

Create a sample application

Since we are only going to try push notification, we don't need any fancy application here.

cordova create hello com.example.hello HelloWorld

Push plugin does not work in browser platform, So lets directly add android platform.

cd hello/

cordova platform add android

You can check the prerequisite for the building the platform.

cordova requirements

You would have to install Android SDK if you have not already. Follow this documentation. You would have to add java home, android home and path variables (both platform tools and tools folders)

export ANDROID_HOME=~/Android/Sdk

export PATH=$PATH:~/npm-global/bin

export PATH=$PATH:$ANDROID_HOME/platform-tools:$ANDROID_HOME/tools

While installing Android SDK, if you face the issue where the gradle is not found under tools table.
http://stackoverflow.com/a/43027871 just download the tools lib and paste it in the location

And while installing emulator make sure the image has Google Play Service - any image greater than 4.2 is supposed to have Google Play Service

To check if your setup is fine, build and run it in emulator.

cordova build 

cordova run --emulator

Make sure your emulator is running before issuing the command. Boot up the "Hello World" application in the emulator app screen.

How does Push Notification work?

I cannot explain it better than this. Briefly,

There are Push Notification Providers such as FCM - Firebase Cloud Messaging (previously GCM - Google Cloud Messaging) APNS - Apple Push Notification Service

For these providers to send notification to your devices, 

• Your device should register to providers. This will give your device an unique device registration key.
• This registration key is sent to the server
• Server asks the Notification Provider to send message to this registration key

This concept will be clear by the end of this post.

Setting up FCM

From this step you will need 

- Sender ID

- Server Key

1. Go to FCM
2. Go to Project settings
   
3. Navigate to Cloud Messaging tab
   

GCM has been migrated to FCM. now if you already have your project setup done at GCM and the API key is already generated, then this setup will work just fine (Google has provided backward compatibility) But if you have not setup API Key yet, then you would have to migrate to FCM. Note that the Legacy Server Key in the screenshot is the one generated in GCM.

Sender ID in FCM is equivalent to the Project Number in GCM

Server Key in FCM is equivalent to the API Key in GCM

Let me briefly explain how the setup is in GCM. You can totally skip it if you are starting new with FCM

1. Go to GCM
2.  Project Number, the one followed by # is your sender ID
   
3. API Key - created specifically for GCM, is your Server Key. Navigate to API Manager

Navigate to the Credentials in Left Panel. That API Key is your server key

At this point you should have Sender ID and Registration Key

Setting up Push Plugin in the Cordova App

cordova plugin add phonegap-plugin-push --variable SENDER_ID="SENDER_ID_HERE"

Navigate to hello/www/index.js

var app = {
// Application Constructor
initialize: function() {
console.log("device readyState")
 document.addEventListener("deviceready", this.onDeviceReady, false);
},

onDeviceReady: function() {
 var push = PushNotification.init({ "android": {"senderID": "88338672842"},
  "browser": {
  "pushServiceURL": 'https://push.api.phonegap.com/v1/push'
  },
  "ios": {"alert": "true", "badge": "true", "sound": "true"}, "windows": {} }
  );
 push.on('registration', function(data) {
  console.log("registration:");
  console.log(data.registrationId);
 });
 
 push.on('notification', function(data) {
  //console.log(data.title);
  console.log(data.message);
  // data.message,
  // data.title,
  // data.count,
  // data.sound,
  // data.image,
  // data.additionalData
 });

 push.on('error', function(e) {
  console.log("error");
  console.log(e.message);
 });
},
};
app.initialize();

During the application initialization, we are hooking the registration callback to deviceReady event.

Ideally you will send the registration Id to your server (to registration your user login with the registration key, which will be used further for sending push notification via providers by the server)

And then there is a notification callback - what should I do when the user taps on the notification or if the notification is received when the application is foreground. And there is an error handling callback as well.

Now rebuild the application.

cordova build android
cordova emulate android

Note that your emulator should already be running (Android SDK -> Tools -> Android -> AVD Manager). Navigate to the Hello World app.

We need the registration key that I printed in the registration callback. How to view the console log statements?

Navigate to your android_home/platform-tools.

adb logcat browser:V

Sending Push Notification

Now that we have the registration key, all we need to send message to this key. You can do it via python like detailed here

Or you can use a restful client and use the API here

I used PostMan

• Add headers
• Authorization is the sender id (format : key=sender_id)
• set content-type to JSON
  
• Send the request body
• To - the registration key you got from the app
  

You will receive notification like:

you can check the console for the on Notification callback when you receive the notification while the app is on foreground

If you want the callback to work on the message tap, you need to set "content-available" in the notification payload.

And there are many more things to work on - logo, sounds, message stacking, etc. I would highly recommend you to  go through this documentation and decide what you want.

And note that you might have to do a lot of special handling depending on the device on which the app is installed - Android/IOS. You can make use of the device plugin to determine the the device type and also send this information to the server when you are registering the device key. This will help the server send different payload (as per android/ios) when they are using the provider to send push notifications.

Also, as the documentation mentions, you might notice there are double events when the content-available is set to 1. This post talks in detail about how to avoid that. Highlighting one particular comment.

if (data.additionalData.foreground) {

  /**

   * This block is reached when a push notification is received when the app is in foreground.

   * Depending on the design of your app, you might want to save the incoming data to the localstorage here.

   * That's not really what your question is about, but I thought mentioning it would give a better picture.

   */

}

else {

  if (data.additionalData.coldstart) {

    /**

     * This block is reached when a push notification is tapped on *AND*

     * the app is closed (ie. Not even in background).

     */

  }

  else {

    /**

     * The following block is reached in the following two scenarios:

     * 1. A push notification is tapped on *AND* the app is in background.

     * 2. A push notification with "content-available=1" is received while the app is in background. (iOS only)

     *

     * Therefore, for iOS devices, both of the above two scenarios can occur for an individual push notification.

     *

     * For Android, it's always the first scenario as the other one is not applicable.

     */



    // Here we check for the existence of the incoming data in the localstorage and, with that, we determine

    // which scenario of the two possible is happening.

    var pastPushSavedID = window.localStorage.getItem("pastPushSavedID");



    if (pastPushSavedID) {

      // `pastPushSavedID` is found in localStorage and thus this is scenario 1 mentioned above.

    }

    else {

      // `pastPushSavedID` is `null` and thus this is scenario 2 mentioned above.

    }

  }

}

A little bonus point. Try playing around with notification callback when
- app is foreground
- app is in background and running
- app is shut

Note that in all of these cases you are sending payload with content-available set

When I tried the callback worked for
- foreground
- app is shut (coldstart set to true)

But did not work for
- app is background and running

I found out I was not the only one. But I still do not have a solution for this. Post on the comments if  you face this as well.