Thursday, June 29, 2017

Mockito - Rule and Runner

While using mockito for unit test, you initialize using either

@RunWith(MockitoJUnitRunner.class)

or

@Before
public void setUp() {
   MockitoAnnotations.initMocks(this);
}

If you are getting NPE for the mocked objects at the class you are testing, then you have missed either of the above. MockitoJUnitRunner automatically provides you with a initMocks() It also provides automatic validation of framework usage.

There might be cases where you are already using runner, say, SpringJunit4ClassRunner. In such cases you cannot have 2 runners. In such scenarios, we can go with

- the second approach using initMocks.
- With JUnit 4.12, we can have multiple runners operate on the same test class by means of JUnit Rules. Mockito already provides us MockitoRule

Note that when you are trying to write Parameterized tests, always use runner for Parameterized and rule for Mockito. Quoting this reference

The reason is that the Runner is responsible for reporting the number of tests, and Parameterized manipulates the number of tests based on the number of test methods and the number of parameterized inputs, so it's really important for Parameterized to be a part of the Runner process. By contrast, the use of a Mockito runner or rule is simply to encapsulate the @Before and @After methods that initialize Mockito annotations and validate Mockito usage, which can be done very easily as a @Rule that works adjacent to other @Rule instances--to the point that the MockitoJUnitRunner is very nearly deprecated.

Default parameterization of varchar to nvarchar in sql driver

We came across a strange case where there were high logical reads for some queries coming from Spring batch. This was a query executed from Spring batch's  SimpleJobRepository where it tries to create JobInstance using the jobInstanceDao

This is the query which we have no control over.

When it goes to db, the type is nvarchar and not varchar

DECLARE @P0 nvarchar(4000) = 'someJobTable', @P1 nvarchar(4000) = 'someJobKey'
SELECT JOB_INSTANCE_ID, JOB_NAME from SOME_JOB_INSTANCE where JOB_NAME =  @P0  and JOB_KEY =  @P1

This has ~1900 logical reads

While the same query with type varchar took only 4 logical reads

DECLARE @P0 varchar(100) = 'someJobTable', @P1 varchar(32) = 'someJobKey'
SELECT JOB_INSTANCE_ID, JOB_NAME from SOME_JOB_INSTANCE where JOB_NAME =  @P0  and JOB_KEY =  @P1

On digging further we analyzed that the call from
org.springframework.jdbc.core.JdbcTemplate.query(PreparedStatementCreator, PreparedStatementSetter, ResultSetExtractor<T>)
to
net.sourceforge.jtds.jdbc.JtdsPreparedStatement.setParameter(int, Object, int, int, int)
where the connection.getUseUnicode() is coming as true and hence the type nvarchar

Note that this causes high logical reads even for an indexed column. nvarchar are double-byte and even those indexed varchar column gets converted to nvarchar for comparison since the query parameter in the where clause is coming over as nvarchar.

To suppress this behaviour, you need to set sendStringParametersAsUnicode to false in the connection properties of the datasource used by the spring batch. You can choose to set this property in all of the datasources used in your application. But that depends on the usecase. If you are supporting internationalization, then you might have to analyze where to switch this property on/off.

<batch:job-repository id="jobRepository"
table-prefix="SOME"
data-source="someDataSource" isolation-level-for-create="READ_UNCOMMITTED"
transaction-manager="someTransactionManager" />

<bean id="someDataSource" class="org.apache.tomcat.jdbc.pool.DataSource">
<property name="driverClassName" value="net.sourceforge.jtds.jdbc.Driver" />
<property name="url" value="jdbc:jtds:sqlserver://someserver/somedatabase" />
        <property name="connectionProperties" value="sendStringParametersAsUnicode=false;"/>
</bean>

Reference :
http://www.digitalsanctuary.com/tech-blog/java/mssql-jtds-nvarchar-and-slow-indexes.html
https://stackoverflow.com/questions/4717520/sql-server-uses-high-cpu-when-searching-inside-nvarchar-strings

Friday, June 23, 2017

Overriding Spring Context Initializers

Consider the scenario where you need to override certain properties across the integration tests

User class with name and hobby

PropertySources :

NameHobbyPropertySource
public class NameHobbyPropertySource extends PropertySource<properties> {

 static Properties source = new Properties();
 
 public NameHobbyPropertySource(String name) {
  super(name, source);
  source.setProperty("name", "chinnu");
  source.setProperty("hobby", "cycling");
 }

 @Override
 public Object getProperty(String key) {
  return getSource().get(key);
 }
}

HobbyOnlyPropertySource
public class HobbyOnlyPropertySource extends PropertySource<properties> {

 static Properties source = new Properties();
 
 public HobbyOnlyPropertySource(String name) {
  super(name, source);
  source.setProperty("hobby", "running");
 }

 @Override
 public Object getProperty(String key) {
  return getSource().get(key);
 }
}

And this is your default context initializer for your application (that uses NameHobby property source)

public class MainApplicationContextInitializer implements ApplicationContextInitializer<configurableapplicationcontext> {

 public void initialize(ConfigurableApplicationContext applicationContext) {
  System.out.println("Main initializer called");
  NameHobbyPropertySource nameHobbyPropertySource = new NameHobbyPropertySource("name-hobby");
  applicationContext.getEnvironment().getPropertySources().addFirst(nameHobbyPropertySource);
 }

}
Say your BaseIntegrationTest is configured with initializer that has NameHobbyPropertySource

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(initializers = { MainApplicationContextInitializer.class
  }, locations = { "classpath:application-context.xml" })
public abstract class BaseApplicationTest {
 
}

but you want to override this for only one test class.
i.e. your special test case should run with overridden hobby from HobbyOnlyPropertySource. You can achieve this using inheritInitializers as follows,

public class SubSetApplicationContextInitializer implements
  ApplicationContextInitializer<configurableapplicationcontext> {

 public void initialize(ConfigurableApplicationContext applicationContext) {
  System.out.println("Subset initializer called");
  HobbyOnlyPropertySource hobbyOnlyPropertySource = new HobbyOnlyPropertySource(
    "hobby");
  applicationContext.getEnvironment().getPropertySources()
    .addFirst(hobbyOnlyPropertySource);
 }

}

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(initializers = { SubSetApplicationContextInitializer.class }, inheritInitializers = true)
public class ApplicationTest extends BaseApplicationTest {

but this will not work because, By default the initializers of the derived class gets called and then the base class's initializers are called.

To override this behaviour, BOTH the initializer should implement Ordered interface and define an order.

Whichever one wants to run later should be given higher order. You can also try @Order annotation for the same.

 Lets say MainApplicationContextInitializer is from third party lib, and it doesn't implement or annotated with Order, then this wouldn't work. So have a wrapper over the third party initializer and annotate it with @Order.

@Order(0)
public class WrapperMainApplicationContextInitializer extends MainApplicationContextInitializer{

}
And so the BaseApplicationTest should now use this wrapper,
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(initializers = { WrapperMainApplicationContextInitializer.class
  }, locations = { "classpath:application-context.xml" })
public abstract class BaseApplicationTest {
 
}

and your subset application context initializer should also be ordered
public class SubSetApplicationContextInitializer implements
  ApplicationContextInitializer<configurableapplicationcontext>, Ordered {

 public void initialize(ConfigurableApplicationContext applicationContext) {
  System.out.println("Subset initializer called");
  HobbyOnlyPropertySource hobbyOnlyPropertySource = new HobbyOnlyPropertySource(
    "hobby");
  applicationContext.getEnvironment().getPropertySources()
    .addFirst(hobbyOnlyPropertySource);
 }

 public int getOrder() {
  return 1;
 }
}

Refer :

Monday, June 19, 2017

NullComparator

I will demonstrate the use of this comparator in the scenario below.

Let us consider some entity called Allocation

There are many types of allocations.
ALevelAllocation
BLevelAllocation
etc

All of them have month.

if month is null -> Global allocation
if month is not null -> Monthly allocation

So let us introduce an interface

public interface MonthLevelAllocationMarker {
    String getMonth();
}

class ALevelAllocation implements MonthLevelAllocationMarker{

class BLevelAllocation implements MonthLevelAllocationMarker{

This way when I need to write any custom comparator to sort the allocations, I need to write only one Comparator for type MonthLevelAllocationMarker

import org.apache.commons.collections.comparators.NullComparator;



public class MonthFirstAllocationSorter implements Comparator<MonthLevelAllocationMarker>, Serializable

{

    private static final long serialVersionUID = 1L;



    @Override

    public int compare(MonthLevelAllocationMarker a, MonthLevelAllocationMarker b)

    {

        boolean nullsAreHighForGlobalMonth = true;

        NullComparator nullComparator = new NullComparator(nullsAreHighForGlobalMonth);

        return nullComparator.compare(a.getMonth(), b.getMonth());

    }

}



I am using NullComparator to sort the nulls are high. Meaning Monthly allocations first and global allocation last.

Usage :

List<ALevelAllocation> allocations;

Collections.sort(allocations, new MonthFirstAllocationSorter());

Saturday, June 17, 2017

Equals and HashCode Builder - Why not to use them

Many times I have seen HashCodeBuilder and EqualsBuilder getting used in the Models to write hashCode() and equals() method. In SDEs like eclipse, help is just one click away. But for many reasons, one might not resort to this option because you have to keep updating your equals and hashCode methods when you are adding new variables to the class.

It is important to note that these builders use reflection to access the variables.

It uses AccessibleObject.setAccessible to gain access to private fields. This means that it will throw a security exception if run under a security manager, if the permissions are not set up correctly. It is also not as efficient as testing explicitly.
Transient members will be not be tested, as they are likely derived fields, and not part of the value of the Object.
Static fields will not be tested. Superclass fields will be included.

The important point to notice here is that the superclass fields will be included.

I know it is too tempting to use these builders especially when you want to ignore certain fields for equals().

For instance, you would not want to compare a enriched field such as "Account" you would want to restrict the equals() to only "accountID"

When you are over-using this utility with ignoreFields, you run into the following problem.


import org.apache.commons.lang.builder.EqualsBuilder;
import org.apache.commons.lang.builder.HashCodeBuilder;
public class BaseClass {
     
    public static String[] ignoreFieldsInEquals = new String[] {"comments"};
     
    private String id;
    private String comments;
     
    public BaseClass()
    {
        super();
    }
     
    public BaseClass(String id, String comments) {
        super();
        this.id = id;
        this.comments = comments;
    }
    public String getId() {
        return id;
    }
    public void setId(String id) {
        this.id = id;
    }
    public String getComments() {
        return comments;
    }
    public void setComments(String comments) {
        this.comments = comments;
    }
    @Override
    public int hashCode() {
        return HashCodeBuilder.reflectionHashCode(this, ignoreFieldsInEquals);
    }
    @Override
    public boolean equals(Object obj) {
        return EqualsBuilder.reflectionEquals(this, obj, ignoreFieldsInEquals);
    }
}
package com.foo;
 
import org.apache.commons.lang.builder.EqualsBuilder;
import org.apache.commons.lang.builder.HashCodeBuilder;
 
public class SubClass extends BaseClass {
    private static String[] ignoreFieldsInEquals = new String[] {"alias"};
     
    private String name;
    private String alias;
     
    public SubClass() {
        super();
    }
    public SubClass(String id, String comments,String name, String alias) {
        super(id, comments);
        this.name = name;
        this.alias = alias;
    }
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public String getAlias() {
        return alias;
    }
    public void setAlias(String alias) {
        this.alias = alias;
    }
     
    @Override
    public int hashCode() {
        return super.hashCode() + HashCodeBuilder.reflectionHashCode(this, ignoreFieldsInEquals);
    }
 
    @Override
    public boolean equals(Object obj) {
        return super.equals(obj) && EqualsBuilder.reflectionEquals(this, obj, ignoreFieldsInEquals);
    }
}
import static org.junit.Assert.*;
import org.junit.Test;
public class ReflectionBuilderTest {
     
    @Test
    public void test()
    {
        String id1 = "TE1";
                 
        SubClass sub1 = new SubClass(id1, "random comment 1", "Bruce Wayne", "batsy");
        SubClass sub2 = new SubClass(id1, "random comment 2", "Bruce Wayne", "batman");
         
        assertEquals(sub1, sub2); //fails
    }
}

In SubClass, we are expecting super.equals() to ignore comments from BaseClass and the equals to ignore alias but it will not work since it uses reflection.
so EqualsBuilder.reflectionEquals() already read your alias and returned false. One way to fix this would be, to pass the base class's ignore field as well in the SubClass's equals.

BaseClass :
private static String[] ignoreFieldsInEquals = new String[] {"comments"};
 
@Override
public int hashCode() {
    return HashCodeBuilder.reflectionHashCode(this, getFieldsToIgnoreInEquals());
}
@Override
public boolean equals(Object obj) {
    return EqualsBuilder.reflectionEquals(this, obj, getFieldsToIgnoreInEquals());
}
protected String[] getFieldsToIgnoreInEquals() {
    return ignoreFieldsInEquals;
}

SubClass:
private static String[] ignoreFieldsInEquals = new String[] {"alias"};
 
protected String[] getFieldsToIgnoreInEquals()
{
    return Stream.of(ignoreFieldsInEquals, super.getFieldsToIgnoreInEquals()).flatMap(Stream::of).toArray(String[]::new);
}
 
@Override
public int hashCode() {
    return  HashCodeBuilder.reflectionHashCode(this, getFieldsToIgnoreInEquals());
}
@Override
public boolean equals(Object obj) {
    return EqualsBuilder.reflectionEquals(this, obj, getFieldsToIgnoreInEquals());
}

But even this would not work when the sub class has to choose what BaseClass's variables to ignore and what not to ignore.

For example, there are many sub classes for the BaseClass and one of them decides that comments are also important for them. Usually such need does not arrive for equals() rather for something like "isValueEqualsForPersist"

The ideal solution would be a careful engineering about where to put what. In this case, for example, maybe the comments should not even be in the BaseClass.

But this post is more about how much reflection can screw it up. For Example I am working on a 10 year old project where Equals and HashCode builders are used and there are sub classes, where one of them needs a super class fields in equalsForPersist and the other does not. Now refactoring the whole thing and restructuring the classes is a big pain in the ass, which could have been avoided by careful designing.

Thursday, June 15, 2017

Generics - T extends Class & Interface

Say you have a Transaction object

public abstract class Transaction {
Long transactionId


And you have various transactions,

MoneyTransaction
RewardPointTransaction
CouponTransaction
etc

There are some of these transactions that are  Transferable and some are not

public interface Transferable {
    Long getDestinationAccount();
}

I need to write a common assertion while writing test case.

public void assertTransferableTransactions

takes in lists

expectedTransactions
actualTransactions

I would need transactionId to group the inputs, which is in type Transaction
And the methods to be in tested that are in Transferable

So I could have my assertion like

public static <T extends Transaction & Transferable> void assertTransferrableTransactions(
            List<T> expectedTransactions, List<T> expectedTransactions)

You can have just this method in a test util and avoid the code duplicate throughout all the Transaction tests that are transferable

Note that in
T extends A & B
class name should come first and interfaces follow.
i.e.
<T extends Transferable & Transaction> will give compilation error