Configuring an External Elasticsearch Server - Documentation topics on: cluster,elasticsearch,elastic search,external,.

Configuring an External Elasticsearch Server

The ElasticSearch feature in dotCMS can be configured in several ways. In addition to the embedded ElasticSearch server included in the dotCMS distribution, dotCMS Enterprise edition can also be manually configured to work with an external ElasticSearch server. For more information on manual ElasticSearch configuration, please see the Cluster Configuration documentation.


Important Notes:

  • External ElasticSearch servers are only supported in dotCMS Enterprise editions.
  • External ElasticSearch servers are only supported in ElasticSearch Unicast mode.
  • Use of an external ElasticSearch server is an advanced feature, and is not recommended for most customers.
    • Use of an external ElasticSearch server should only be performed by administrators fully comfortable managing an ElasticSearch server/cluster.
    • dotCMS automatically manages the embedded ElasticSearch server, but does not manage the external server. When using an external ElasticSearch server, you must perform these management functions yourself to ensure proper operation of the external server.

For more information on configuring an external ElasticSearch server, please view the following sections:

Configuration Options

You may configure ElasticSearch in dotCMS in the following different ways:

Embedded ElasticSearch Server Only

By default, dotCMS will use only the ElasticSearch server embedded in the dotCMS distribution. The dotCMS distribution comes configured this way, so to use this configuration you do not need to make any configuration changes.

Combine Embedded ES Server with an External ES Server as Master

The following steps guide you to set up an ElasticSearch cluster which uses the external ElasticSearch server as the master, keeping the data on the external server only. This configures the dotCMS embedded ElasticSearch cluster as a node with no master and no data, and the external ElasticSearch cluster as a node with both master and data.

1. Download the Correct ElasticSearch Version from the ElasticSearch site

2. Extract the Downloaded File

3. Edit the elasticsearch.yml file

Add the following properties to the elasticsearch.yml file in the elasticsearch /config folder (replacing <EXTERNAL-SERVER-CLUSTER-NAME>, <EXTERNAL-SERVER-HOSTNAME> and <EXTERNAL-SERVER-IP-ADDRESS> with appropriate values).

cluster.name: <EXTERNAL-SERVER-CLUSTER-NAME>
node.name: <EXTERNAL-SERVER-HOSTNAME>
node.master: true
node.data: true
index.number_of_shards: 1
index.number_of_replicas: 0
bootstrap.mlockall: true
network.host: <EXTERNAL-SERVER-IP-ADDRESS>
transport.tcp.port: 9300
http.port: 9200
http.enabled: true
discovery.zen.ping.multicast.enabled: false
script.native.related.type: 'com.dotcms.elasticsearch.script.RelationshipSortOrderScriptFactory'

Important:

  • The number of replicas and shards in this sample configuration are intended only for a single dotCMS server and a single external ElasticSearch server.
    • More complex architectures (multiple dotCMS or external servers) should configure replicas and shards appropriate to the architecture.

4. Change the ElasticSearch script to start the server with at least 2GB memory

Add the following property to beginning of the elasticsearch file in the elasticsearch /bin folder:

ES_HEAP_SIZE=2g 

5. Copy the dotCMS Packages from your dotCMS installation to your Elasticsearch server

Copy the dotCMS packages .jar file from the /dotserver/tomcat-X.x.xx/webapps/ROOT/WEB-INF/lib/ directory in your dotCMS installation to the $elasticsearch_home/lib/ directory on the ElasticSearch data servers.

The dotcms packages .jar filename will vary by release, and will be of the form dotcms_{version}_*.jar, where {version} is replaced by the dotCMS version number. For example, the dotCMS packages .jar file for the dotCMS 3.3.1 release is named dotcms_3.3.1_2521e19.jar.

6. Start the external ElasticSearch server

From the ElasticSearch installation folder, run the bin/elasticsearch command.

Note: Make sure the external ElasticSearch server is up and running before starting dotCMS (below).

7. Verify that the ElasticSearch server is up and running correctly

Run the following command (replacing <EXTERNAL-SERVER-IP-ADDRESS> with the appropriate value):

curl -XGET http://<EXTERNAL-SERVER-IP-ADDRESS>:9200/_cluster/health

The output of the command should look similar to the following:

{"cluster_name":"<EXTERNAL-SERVER-CLUSTER-NAME>","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}

8. Apply your dotCMS Enterprise License

External ElasticSearch servers are only supported with dotCMS Enterprise editions, so you must apply an Enterprise license before you can continue with the ElasticSearch configuration.

  • Start dotCMS.
  • Make sure the server starts without problems.
  • Login to the dotCMS backend as an administrator.
  • Apply the license to dotCMS.
    • For more information on how to apply a license, please see the License Management documentation.
  • Shut down dotCMS.

9. Configure ElasticSearch in dotCMS

Change the ElasticSearch properties in the dotcms-config-cluster.properties file. Note: It is strongly recommended that all changes to this file be done via a static configuration plugin.

Set the following properties (replacing <DOTCMS-IP-ADDRESS> and <DOTCMS-CLUSTER-NAME> with appropriate values).

CLUSTER_AUTOWIRE=false
DIST_INDEXATION_ENABLED=true
CACHE_FORCE_IPV4=true
CACHE_PROTOCOL=tcp
CACHE_BINDPORT=7800
CACHE_BINDADDRESS=<DOTCMS-IP-ADDRESS>
CACHE_TCP_INITIAL_HOSTS=<DOTCMS-IP-ADDRESS>[7800]
es.cluster.name=<DOTCMS-CLUSTER-NAME>
es.network.host=<DOTCMS-IP-ADDRESS>
es.transport.tcp.port=9309
es.node.data=false
es.node.master=false
es.discovery.zen.ping.unicast.hosts=<DOTCMS-IP-ADDRESS>:9300
es.http.enabled=false

Important:

  • Make sure that none of the properties are duplicated, or the server will not start.
  • This configuration disables autowire (automatic cluster configuration).
    • External ElasticSearch is not currently compatible with autowire.
  • On some systems, you may need to override the default values of the es.path.work and es.path.data properties for the configuration to work without errors. To override these properties, add the following lines to the dotcms-config-cluster-ext.properties file, to remove the default values:
    es.path.work=
    es.path.data=
    

For more information on configuring ElasticSearch properties in dotCMS, please see the Cluster Configuration documentation.

10. Start dotCMS

11. Reindex your dotCMS ElasticSearch Index

  • Login to the dotCMS backend.
  • Navigate to System -> Maintenance and select the Index tab.
  • In the Reindex: field, select Rebuild Whole Index.
  • Press the Reindex button.
  • Watch the log on the external ElasticSearch server.
    • You should see the external ElasticSearch server log updating and adding content.

12. Verify ElasticSearch Operation

  • Login into the dotCMS backend.
  • Navigate to System -> Maintenance and select the Index tab.
  • Click on the green box located in the Health column of the Index. Health column on the Index tab
  • Verify the Store Size and Document Counts for both servers
    • For the External Server, both values should be appropriate size for a full index.
    • For the Embedded Server, both values should be 0. The Index Store Popup

Combine Embedded and External ES Servers, with Both Servers as Master Nodes with Data

The following steps guide you to set up an ElasticSearch cluster using both the embedded dotCMS ElasticSearch Cluster and an external ElasticSearch Cluster as master nodes with data. This configuration duplicates the data on both the embedded and external servers.

1. Download the Correct ElasticSearch Version from the ElasticSearch site

2. Extract the Downloaded File

3. Edit the elasticsearch.yml file

Add the following properties to the elasticsearch.yml file in the elasticsearch /config folder (replacing <EXTERNAL-SERVER-CLUSTER-NAME>, <EXTERNAL-SERVER-HOSTNAME> and <EXTERNAL-SERVER-IP-ADDRESS> with appropriate values).

cluster.name: <EXTERNAL-SERVER-CLUSTER-NAME>
node.name: <EXTERNAL-SERVER-HOSTNAME>
node.master: true
node.data: true
node.max_local_storage_nodes: 1
index.number_of_shards: 1
index.number_of_replicas: 1
bootstrap.mlockall: true
network.host: <EXTERNAL-SERVER-IP-ADDRESS>
http.port: 9200
http.enabled: true
discovery.zen.ping.multicast.enabled: false
transport.tcp.port: 9300
script.native.related.type: 'com.dotcms.elasticsearch.script.RelationshipSortOrderScriptFactory'

Important:

  • The number of replicas and shards in this sample configuration are intended only for a single dotCMS server and a single external ElasticSearch server.
    • More complex architectures (multiple dotCMS or external servers) should configure replicas and shards appropriate to the architecture.

4. Change the ElasticSearch script to start the server with at least 2GB memory

Add the following property to beginning of the elasticsearch file in the elasticsearch /bin folder:

ES_HEAP_SIZE=2g 

5. Copy the dotCMS Packages from your dotCMS installation to your Elasticsearch server

Copy the dotCMS packages .jar file from the /dotserver/tomcat-X.x.xx/webapps/ROOT/WEB-INF/lib/ directory in your dotCMS installation to the $elasticsearch_home/lib/ directory on the ElasticSearch data servers.

The dotcms packages .jar filename will vary by release, and will be of the form dotcms_{version}_*.jar, where {version} is replaced by the dotCMS version number. For example, the dotCMS packages .jar file for the dotCMS 3.3.1 release is named dotcms_3.3.1_2521e19.jar.

6. Start the external ElasticSearch server

From the ElasticSearch installation folder, run the bin/elasticsearch command.

Note: Make sure the external ElasticSearch server is up and running before starting dotCMS (below).

7. Verify that the ElasticSearch server is up and running correctly

Run the following command (replacing <EXTERNAL-SERVER-IP-ADDRESS> with the appropriate value):

curl -XGET http://<EXTERNAL-SERVER-IP-ADDRESS>:9200/_cluster/health

The output of the command should look similar to the following:

{"cluster_name":"<EXTERNAL_SERVER-CLUSTER-NAME>","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}

8. Apply your dotCMS License

External ElasticSearch servers are only supported with dotCMS Enterprise editions, so you must apply en Enterprise license before you can continue with the ElasticSearch configuration.

  • Start dotCMS with the default configuration.
    • Make sure the server starts without problems.
  • Login to the dotCMS backend as an administrator.
  • Apply the license to dotCMS.
    • For more information on how to apply a license, please see the License Management documentation.
  • Shut down dotCMS.

9. Configure ElasticSearch in dotCMS

Change the ElasticSearch properties in the dotcms-config-cluster.properties file. Note: It is strongly recommended that all changes to this file be done via a static configuration plugin.

Set the following properties (replacing <DOTCMS-IP-ADDRESS> and <DOTCMS-CLUSTER-NAME> with appropriate values).

CLUSTER_AUTOWIRE=false
DIST_INDEXATION_ENABLED=true
CACHE_FORCE_IPV4=true
CACHE_PROTOCOL=tcp
CACHE_BINDPORT=7800
CACHE_BINDADDRESS=<DOTCMS-IP-ADDRESS>
CACHE_TCP_INITIAL_HOSTS=<DOTCMS-IP-ADDRESS>[7800]
es.cluster.name=<DOTCMS-CLUSTER-NAME>
es.network.host=<DOTCMS-IP-ADDRESS>
es.transport.tcp.port=9309
es.node.data=true
es.node.master=true
es.discovery.zen.ping.unicast.hosts=<DOTCMS-IP-ADDRESS>:9300
es.http.enabled=false

Important:

  • Make sure that none of the properties are duplicated, or the server will not start.
  • This configuration disables autowire (automatic cluster configuration).
    • External ElasticSearch is not currently compatible with autowire.
  • On some systems, you may need to override the default values of the es.path.work and es.path.data properties for the configuration to work without errors. To override these properties, add the following lines to the dotcms-config-cluster-ext.properties file, to remove the default values:
    es.path.work=
    es.path.data=
    

For more information on configuring ElasticSearch properties in dotCMS, please see the Cluster Configuration documentation.

10. Restart dotCMS

11. Reindex your dotCMS ElasticSearch Index

  • Login to the dotCMS backend.
  • Navigate to System -> Maintenance and select the Index tab.
  • In the Reindex: field, select Rebuild Whole Index.
  • Press the Reindex button.
  • Watch the log on the external ElasticSearch server.
    • You should see the external ElasticSearch server log updating and adding content.

12. Verify ElasticSearch Operation

  • Login into the dotCMS backend.
  • Navigate to System -> Maintenance and select the Index tab.
  • Click on the green box that is located on the Health column of the Index. Health column on the Index tab
  • Verify that the Store Size and Document Counts are appropriate size for a full index. The Index Store Popup

 
For more information on ElasticSearch configuration, please see the documentation on Managing Site Indexes with ElasticSearch.

Troubleshooting

IndexMissingException Errors

The first time you start dotCMS after configuring a new external ElasticSearch server you may see errors with the message “org.elasticsearch.indices.IndexMissingException”. This is completely normal, since the new ElasticSearch server does not yet have a valid index for your dotCMS installation. To eliminate these errors, rebuild the ElasticSearch index to create a new index on your external server.

Debug Logging

When troubleshooting issues with your external ElasticSearch server configuration, it may be helpful to increase the logging level for ElasticSearch to debug. To do this, add the following to the Loggers section of the Log4j configuration file:

<Logger name="org.elasticsearch" level="debug" additivity="false"> 
    <AppenderRef ref="generic"/> 
</Logger> 

Important: Setting the log level to debug may generate large numbers of log messages and cause significant performance degredation on highly active servers. Therefore it is strongly recommended that you disable debug logging on production servers.