JSON Viewtool documentation for the dotCMS Content Management System

The JSONTool allows you to call local or remote json-based RESTful web services and parse the results for inclusion in a velocity template, page or widget. The following examples show how to use the JSONTool and JSONObjects with JSON in dotCMS.

• Methods
• Limitations
• Retrieve and Parse External JSON ($json.fetch and $json.post)
  • Usage
  • Example
  • Parsing Arrays and Loops
  • Limiting JSON Results
  • Handling Alternate JSON Array Formats
  • Perform a JSON Post ($json.post)
• Create JSON Objects ($json.generate)
  • Usage
  • Example
    
  • Parse Internal JSON Files
• Toolbox.xml Configuration

Methods

By default, the JSONTool is mapped in the Velocity Context with the variable name $json:

• $json == com.dotmarketing.viewtools.JSONTool

The following methods are supported:

| generate | $json.generate( Object ) | Generate JSON objects from Java objects, strings, or maps.

Limitations

Keep mind that any use of the $json.fetch("https://remoteUrl.com/") is a synchronous call and is only going to be as reliable and as responsive as the endpoint https://remoteUrl.com/ that the $json.fetch is calling. If that remote endpoint has a latency of a few seconds, it means that every dotCMS page that calls that endpoint is going to hang for a few seconds before the page can begin to render. On a loaded site, or a site that does a $json.fetch call on every page (say in a header or footer) this can block all the receiving threads in tomcat and take a server down.

How to deal with Synchronous Remote Calls:

1. Use the BlockCache to cache the $json.fetch and the results If you want to make the request server side, that is fine, but wrap the call and rendered results in a #dotCache block. This is especially important when making remote calls. Something like:

   #dotcache("currentMortgageRates",3600) 
   #set($myJson = $json.fetch("https://remoteUrl.com/"))
   <div> <b>$myJson.type</b> : $myJson.rate % </div>
   #end
   

   which will cache it the result for an hour and will not take a site down if the remote endpoint gets slow. For more information on block caching, see: https://dotcms.com/docs/latest/tag-based-caching-block-cach

2. Make remote $json calls client side, in javascript dotCMS blocks a receiver thread when a $json.fetch call is made and waiting for a response. This means a visitor will have to wait until the remote response is returned before they will see a page rendered. If a site is designed to make the same request client side, using asynchronous javascript, dotcms will render the page fully and no threads in dotCMS will be blocked on dotCMS internally - only the visitors browser will hang trying to render the async request. This prevents a remote API problem from taking a dotCMS server down.

3. Create a cron that imports real time data into dotCMS You can create a vtl script file that that calls a remote service and stores that value in dotCMS as a piece of content - and then use this content on your pages. This pattern is robust - the content can be updated asynchronous and your server will survive if there are any issues with the remote API. See this forum thread as an example:

https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/dotcms/fjFoWwyGj2E/z0v-HVCUAwAJ

1. Final note - it is never recommended to make $json.fetch calls to a local dotCMS server This is a inefficient pattern, where 1 request begets 2 or 3 internal requests to fulfill. It is fragile as domain names and ports that dotCMS run on can change. Many times, an API that was calling an internal domain http://www.customerdomain.com/api/... changes to https://customerdomain.com/api which can cause all remote calls to fail. Also a dotCMS server running behind load balancers or a NATed firewall might not be able resolve an external domain name as an internal resource - and calling localhost/api might fail as the ports that dotCMS server listens on (internally) might change. Regardless, it is never recommended to use $json.fetch to call internal dotCMS APIs as it has proven too brittle and inefficient.

Retrieve and Parsing External JSON ($json.fetch)

The $json.fetch(), $json.put(), $json.post() methods both retrieve and parse JSON from a URL you supply. The methods return a JSON object whose properties can be traversed via the . (dot) operator.

These two methods behave very similar in most respects. For example, both methods allow you to retrieve JSON data from an external web API, both allow you to send headers with the request, and both return a traversible JSON object. There are 2 main differences between the fetch() and post() methods:

• The fetch() method uses an HTTP GET to submit the request, while the post() method uses an HTTP POST.
• Since it uses a POST instead of a GET, the post() method allows you to send data with the request.

Usage

The $json.fetch() method calls HTTP GET and can be called in any of the following ways:

$json.fetch( URL )
$json.fetch( URL, Timeout )
$json.fetch( URL, Headers )
$json.fetch( URL, Timeout, headerMap )

The $json.post() method calls HTTP POST and can be called in any of the following ways:

$json.post( URL, headerMap, rawData )
$json.post( URL, Timeout, headerMap, rawData )
$json.post( URL, headerMap, paramMap )
$json.post( URL, Timeout, headerMap, paramMap )

The $json.put() method calls HTTP PUT and can be called in any of the following ways:

$json.put( URL, headerMap, rawData )
$json.put( URL, Timeout, headerMap, rawData )
$json.put( URL, headerMap, paramMap )
$json.put( URL, Timeout, headerMap, paramMap )

Timeout Parameter

The timeout specifies the total time to wait to both make the connection and perform the read. The Import Tool will automatically kill the connection after the specified time, regardless of what state the import is in (even if it is in the middle of a read). Therefore the timeout should be greater than the time it takes for both the connection and read of the JSON resource.

Note:

• The timeout is specified in milliseconds (not seconds).
• If no timeout value is specified, the default timeout is used.
  • The default value may be changed by modifying the URL_CONNECTION_TIMEOUT property in the dotmarketing-config.properties file.
  • Important: It is strongly recommended that all changes to the dotmarketing-config.properties file be made through a properties extension file.

Headers Parameter

You may pass headers to be sent with the JSON request. This may be necessary, for example, to perform a fetch from a server that restricts access to certain types of clients (such as regular browsers) by checking the User Agent header.

The headers must be in a map, with each header to be passed consisting of both a header label and a header value.

The following code demonstrates how to generate and use the headers map:

#set($headers = {})
#set($dummy = $headers.put("User-Agent", "Mozilla/5.0"))
#set($results = $json.fetch("http://www.omdbapi.com/?t=Battlestar%20Galactica&y=&plot=short&r=json", $headers))

Form Parameters

You may pass form params to be sent with the request. This may be necessary, for example, to POST data to a server.

The form params must be in a map, with each param to be passed consisting of both a string key and a string value.

The following code demonstrates how to generate and use the param map:

#set($headers = {})
#set($params = {})
#set($dummy = $params.put("firstName", "Will"))
#set($dummy = $params.put("lastName", "Ezell"))
#set($results = $json.put("https://demo.dotcms.com/testing",$headers, $params))

Raw Data

You can send raw json data as well - just pass the json in as a fully formed json string

#set($headers = {})
#set($data = "{'firstName':'Will','lastName','Ezell'}")
#set($results = $json.put("https://demo.dotcms.com/testing", $headers, $data))

Example

The following example fetches a JSON object from an external site and displays different properties of the object.

#set($myjson = $json.fetch("http://www.omdbapi.com/?t=Battlestar%20Galactica&y=&plot=short&r=json"))

<p>The title is: $myjson.Title</p>
<p>The year is: $myjson.Year</p>
<p>The rating is: $myjson.Rated</p>

The results of the code above are:

The title is: Battlestar Galactica
The year is: 2004–2009
The rating is: TV-14

Parsing Arrays and Loops

If the JSON property is an array, you can loop over the array using Velocity's #foreach directive:

#set($myjson = $json.fetch("http://imdbapi.poromenos.org/json/?name=battlestar%"))

#foreach($show in $myjson.shows)
    <p>$show.name ($show.year)</p>
#end

The code above lists every version of the Battlestar Galactica television show, including the year it was broadcast.

Limiting JSON Results

To limit the number of results returned in the JSON query, use Velocity $foreach.count and #break directives to exit the #foreach loop as soon as the count is met. The following example limits the previous code to only display 3 results:

#set($myjson = $json.fetch("http://imdbapi.poromenos.org/json/?name=battlestar%"))

#foreach($show in $myjson.shows)
   #if( $foreach.count > 3 )
      #break
   #end
   <p>$show.name ($show.year)</p>
#end

Handling Alternate JSON Array Formats

The JSONTool handles JSON arrays in certain specific formats automatically; however JSON arrays can be formatted in different ways, and for some JSON array formats, additional processing may be required to be able to iterate through the array results. If you can not iterate over a JSON array using the standard loop syntax (above), you must first read the JSON array and create a JSON object from it, and then iterate over the JSON object, as in the following example:

#set($jsonData = $import.read("http://imdbapi.poromenos.org/json/?name=battlestar%"))
#set($jsonData = "{data:${jsonData}}")
#set($jsonObject = $json.generate($jsonData))
#foreach($show in $jsonObject.data)
   <p>$show.name ($show.year)</p>
#end

Passing Data to a Post

The format and contents of the Data object passed with the post() method must match the requirements of the API/URL the post is being sent to. The Data object is created in the same way as the Headers object used in both the fetch() and post() methods.

#set($data    = $contents.getEmptyMap())
#set($dummy   = $data.put("Query", "+contentType:news"))
#set($results = $json.post($url, $data))

Create JSON Objects ($json.generate)

The $json.generate() method generates JSONObjects from Java Objects,Strings or Maps. The JSON object values can be populated with static or dynamic content from your dotCMS instance.

Usage

$json.generate( Object )

Example

The following code generates a JSONObject from a java.collections.Map, demonstrating how to serve a JSON object from your site.

#set($mymap = $contents.getEmptyMap())
#set($dummy = $mymap.put("one","value 1"))
#set($dummy = $mymap.put("two","value 2"))
#set($dummy = $mymap.put("three","value 3"))
#set($myjson = $json.generate($mymap))

<p>$myjson</p>

<p>The value of key "one" is: $myjson.one</p>

The code generates the following results:

{"two":"value 2","one":"value 1","three":"value 3"}
The value of key "one" is: value 1

Parse Internal JSON Files

The following example code will allow you to parse JSON files that are stored on the dotCMS file system. Since the #include directive is only aware of files on the system and not files in dotCMS's virtual file system, use $webapi.getAssetPath to convert the virtual path to the system path, and then $json.generate to convert the file contents into a variable.

#define($file)#include($webapi.getAssetPath("/your/dotCMS/file/path/data.json", $host.identifier))#end
#set($data = $json.generate($file.toString()))  

-Thanks to Leonardo Bell for providing this example :-)

Toolbox.xml Configuration

The following example shows how the JSONTool is mapped in the toolbox-xml file:

    <tool>
        <key>json</key>
        <scope>application</scope>
        <class>com.dotcms.rendering.velocity.viewtools.JSONTool</class>
    </tool>