Apache Velocity — Build dynamic Json request

Problem Statement —

Currently in software engineering, it’s always a challenge to build dynamic Json both for development and testing an application. Building dynamic Json using pojo is not only required lot of java codes but also hard to maintain in current challenging environment. For development if there is any change then we have to wait for deployment cycle to publish to production. I am testing an application which requires to take input as Json response from one system and then convert it to another form of Json request by taking some Json fields with conditions(includes loop, if else condition, math round-off, date conversion, escaping special characters, trimming null/blank/empty string values). There is even a case where I have to take input from multiple sources to build the required Json template. Initially, I was planning to continue with pojo but then I notice that to build Json from multiple sources require multiple classes and also I would require to write java codes to convert it to the required Json.

Solution —

As you can see in the problem statement, in this case, I have to write all the pojo classes and also java codes to convert it to the required Json. I am looking for some solution where I can eliminate the creation of pojo classes. Adding to this, I am looking for template which is not only easy to maintain but also require less time to build and test my application. I find out that Apache Velocity Engine and Apache Velocity-tools are something which can solve my purpose. Velocity-Tools has lot of built-in classes (Example — MathTool, EscapeTool, StringUtils, DateTool, JsonTool etc.) . There are lot more tools are available. I have used only these tools to solve my purpose.

As part of this article, I am not going into detail about Apache Velocity. Please find this link to have a detailed understanding about Apache Velocity Engine and Apache Velocity tools — link .

My approach of validating Json -

  1. Get the response from source1 as String.
  2. Feed the response to template and transform the input Json string using velocity vm to expected Json
  3. Compare actual and expected Json if all the values are matching

Implementation —

To include Apache Velocity in your project you have to include these dependencies to your project —

<dependency>
<groupId>org.apache.velocity</groupId>
<artifactId>velocity-engine-core</artifactId>
<version>2.3</version>
</dependency>
<dependency>
<groupId>org.apache.velocity.tools</groupId>
<artifactId>velocity-tools-generic</artifactId>
<version>3.1</version>
</dependency>
</dependencies>

To initialize velocity engine in your java you can refer below snippet —

public String buildTemplate(String requestJson) throws ParseException {
VelocityEngine velocityEngine = new VelocityEngine();
velocityEngine.setProperty(RuntimeConstants.RESOURCE_LOADER, "classpath");
velocityEngine.setProperty("classpath.resource.loader.class", "org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader");
velocityEngine.setProperty(RuntimeConstants.EVENTHANDLER_INCLUDE, IncludeRelativePath.class.getName());
velocityEngine.init();
Map<String, Object> map = new HashMap<>();
map.put("format", "yyyy-MM-dd'T'HH:mm:SS.ssss");
map.put("timezone", TimeZone.getTimeZone("UTC"));
Map<String, Object> properties = new HashMap<>();
properties.put("engine", velocityEngine);

Template template = velocityEngine.getTemplate("templates/index.vm");
DateTool dateTool = new DateTool();
dateTool.configure(map);
VelocityContext velocityContext = new VelocityContext();
velocityContext.put("root", requestJson);
velocityContext.put("esc", new EscapeTool());
velocityContext.put("date", dateTool);
velocityContext.put("math", new MathTool());

StringWriter stringWriter = new StringWriter();
template.merge(velocityContext, stringWriter);
return new JSONParser().parse(stringWriter.toString()).toString();
}

This returns new transformed Json as string.

Request Json —

{
"dummy": "dummyJson",
"entry": {
"employeeId": 12345,
"firstName": "Anindya",
"middleName": "Sundar",
"lastName": "Roy",
"address": "address 1 \n address 2 \n address 3 \t address 4",
"country": "India",
"percentage": 85.45672534,
"startDate": "2021-04-06T07:18:53.596+0000"
}
}

Please look into the sample VMs for macros.

###################################################################
##
## Common user define macros should be defined after this line
##
###################################################################
#set($baseUrl = "localhost")
## Velocity date format
#macro(dateFormat $value $fromat)$date.format($fromat, $value)#end
## Escape java addition
#macro(escapeText $value)$esc.java($value)#end

Json body template —

#parse("commonTemplate.vm") ## parsing macro template
#set($employeeId = $root.entry.employeeId)
#set($firstName = $root.entry.firstName)
#set($lastName = $root.entry.lastName)
#set($address = $root.entry.address)
#set($country = $root.entry.country)
#set($percentage = $root.entry.percentage)
#set($startDate = $root.entry.startDate)
###################################################################
##
## Json body creation section
##
###################################################################
{
"entityId": "https://$baseUrl/employee/v1.0/$employeeId"
"summary": {
"entityId": "https://$baseUrl/employee/v1.0/$employeeId"
"firstName": "$firstName"
"lastName": "$lastName"
"address": "#escapeText($address)"
"country": "$country"
"percentage" : $math.roundTo(2, $percentage)
"startDate": "#dateFormat($date, "yyyy-MM-dd")"
}
}
#############################END###################################

Result Json after parsing input to template —

{
"summary": {
"firstName": "Anindya",
"lastName": "Roy",
"country": "India",
"address": "address 1 \n address 2 \n address 3 \t address 4",
"percentage": 85.46,
"entityId": "https:\/\/localhost\/employee\/v1.0\/12345",
"startDate": "2021-04-09"
},
"entityId": "https:\/\/localhost\/employee\/v1.0\/12345"
}

Note : If you look at both the request and response Json then you can see I’m not transforming all the values to form the result. I can pick whatever values I want by setting in the variable and transform to result Json. Adding to this, you can see math round-off , escaping character and date conversion can be done in template itself. Even you can use other velocity tools but we need to add unique key for each tools in java before parsing it to template. You can add multiple inputs as well but again you have to define unique keys to add in velocity context.

Few Points to Remember -

  1. Always keep your macros(In VTL methods are mentioned as macros) in separate file. This will be helpful if you have need to parse those in separate template. There would be a case where you have to use macros across different templates.If you keep your content and macros in the same file and if you parse then calling template content will also be displayed in the called template. To avoid this, you need to keep content and macros in separate file.
  2. Keep your main content as clean as possible . This is help in tracking the mapping objects with design document.
  3. Keep indentation of velocity content. Since we know that Json contains arrays and objects, if we indent properly it will help in debugging as well.
  4. If you format your velocity content then all the indentation will go away. I have observed this in my editor. Due to that, I had to rework on indentation again.
  5. Common macros and constants can be kept in base template. You can parse multiple templates within a template.
  6. If you are planning to use DateTool or ComparisonDateTool then you need to define the date format in Map and configure it in the velocityContext. Post that you will be able to parse date using DateTool or ComparisonDateTool.
  7. You don’t need to add any comma after each object of Json in template. Modern Json-parser(I have used JsonSimple) automatically adds comma after each object. Otherwise, you have to add comma after each objects. Adding to this, if you have conditional if in foreach statement then it will be very difficult to deal with.

Please let me know if you find this article helpful. Kindly let me know if I miss anything while transforming Json and if there is any area of improvement which can be required to make this transformation better.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store