What are Long-Term-Supported (LTS) Releases?

The dotCMS Long-Term Supported (LTS) release process was started in December 2020 to help us improve both the reliability of our products and our level of support to all our customers. It's been a clear success, with customers adopting LTS releases seeing a fairly dramatic reduction in both the number of support tickets logged and support call resolution times. In fact, the process has been so successful that even customers who haven’t adopted LTS releases have seen an improvement in service levels.

In this blog, we're going to discuss what an LTS release is, explain why we implemented them, and show you the benefits they offer - including some data to back that up.

What is an LTS Release?

Let’s start with the basics - what is an LTS release?

A long-term supported release is a version of software that is designed to receive long-term support and maintenance for several years after its release. We provide dotCMS users with a stable and reliable version, so that they can focus on giving their customers amazing digital experiences without worrying about compatibility issues or unexpected changes in functionality.

LTS Releases vs Agile Releases

dotCMS has two different kinds of releases: Agile and Long-Term-Supported (LTS). Agile releases are regular releases that are part of our CI/CD process to deliver both fixes and new features continually. Every 6-9 months, dotCMS will designate a previous Agile release as an LTS release. Once a release has been designated as an LTS release, we perform continual updates to fix known issues, improve the reliability, and support the release for longer..

LTS releases are built to maximize stability and reliability, and since they continually have new fixes, the reliability keeps improving over time. They receive regular patch releases (without the need for a full upgrade), and they're supported for 18-24 months from the date the release was designated as an LTS release. However, they won’t have the latest features and enhancements; the newest LTS release may be anywhere from 2 to 8 months behind the features in the latest Agile release.

Agile releases contain all of the latest fixes, new features, and enhancements. We deliver a new Agile release monthly, each with improvements and new features. However, they’re only supported for 12 months after release, and since they contain new features, they may have new undiscovered issues. Also, if you need a fix for an Agile release, you’ll need to upgrade to a more recent Agile release that contains the fix for that issue instead of patching your release.

Here's a summary of the differences between Agile and LTS releases:

* LTS patch releases may occasionally include small, low-risk enhancements

Which Release Should You Choose?

You can choose to run on either an Agile or LTS release, depending on your needs and your plans.

If you’d like to use a brand new feature now, or want to take advantage of all the latest new features and enhancements as they’re released, you’ll want to be on an Agile Release.

If you place a higher priority on consistency and reliability, or you’d like to reduce your frequency of upgrades, you’ll want to be on an LTS release.

Evergreen Environments

For dotCMS Cloud customers, there’s also a third option, combining the best of both worlds: you can run your main environments on an LTS version, while maintaining a separate “evergreen” environment continually updated to the latest Agile release.

This enables you to receive all the advantages of LTS releases for your day-to-day work, but your development team can continually move forward with the latest Agile releases and features. So, when the time comes to upgrade to a newer LTS release, you’ve already had a chance to integrate the latest new features, and the majority of your data migration has already been performed.

Why We Started Implementing LTS Releases

When we created the LTS release process, we had one simple goal in mind: to provide a higher level of support to customers, without an increase in cost. And that’s exactly what LTS releases have accomplished.

Customers on Agile releases still receive the same level of support as they always have, including full helpdesk support with all the same terms and SLAs as before, and the possibility that hotfixes will be delivered for critical issues.

In addition, customers that move to LTS releases may receive an even higher level of support, including all of the following:

Benefits of LTS Releases

Improved Patches and Upgrades

One benefit we'd like to highlight is the improvement LTS releases offer for patches and upgrades.

dotCMS is a platform, and every single dotCMS customer implements their sites in different ways, using different features of dotCMS, writing their own custom code, and integrating different third party applications and libraries. Because of this, no two upgrades are the same.

So, although many upgrades go relatively smoothly, upgrading from one dotCMS version to another always requires some work to ensure that your specific implementation works properly with the new version. This difference in implementations impacts hotfixes as well; even the best tested hotfixes can’t be tested against every possible implementation detail for all dotCMS versions it may apply to.

Thankfully, the LTS process enables us to improve both the patch and upgrade processes for customers on LTS releases.

Improved Patch Process

The LTS patch process has been streamlined to enable us to deliver fast, reliable, and more frequent patches for LTS releases.

Eliminates hotfixes

Since fixes are delivered as part of a new full patch release, and patch releases are delivered regularly, it virtually eliminates the need for hotfixes for LTS releases.. This also simplifies system configuration, and eliminates potential conflicts between different hotfixes.

Low-risk changes

We don’t include all possible fixes in LTS patch releases. Instead, we choose which fixes to install, based on both the impact of the issue and on the likelihood that customers will experience them. As part of this decision process, we ensure that only fixes with low impact and low risk to existing systems are included; if a fix has the potential to cause new issues, it will intentionally be excluded.

Increased testing

Since the patch applies to only a single version, and only low-impact fixes are included, we can perform much more extensive testing. In other words, instead of wide testing, we can now test deep.

No data migration

In addition, we exclude any fixes that would require data migration. This means you can install patch releases without changing any of your data..

Simple installation

Because data migration and other changes aren't needed, a patch install can be performed with a simple “in-place” upgrade for binary installs, and containerized instances can be upgraded by simply changing the version label and restarting the container.

Minimal or no downtime

All of this means that if you have a clustered instance, you can install a patch release with no downtime; each node can be upgraded and restarted separately without your environment going offline. And even for non-clustered instances, the downtime is typically ten minutes or less.

The following compares the traditional hotfix process with the new LTS patch release process:

* LTS Patch releases are automatically applied for all dotCMS Cloud customers on LTS releases.

The most important difference is that the hotfix model is necessarily reactive; fixes are only generated and installed after a problem is encountered (and even then only for critical issues). But LTS patch releases are proactive; fixes are continually integrated, and patch releases are automatically applied for all dotCMS Cloud customers.

Improved Upgrade Process

Another benefit for customers on LTS releases is that the process of performing a full upgrade between two different LTS releases is significantly improved over upgrades between major Agile releases.

Since there are many more Agile releases, when you upgrade between two Agile releases, there may be only a handful of customers who have upgraded between those two specific releases. So, it's less likely that another customer has performed the same upgrade with the same features that your implementation uses.

However, when you upgrade between LTS releases, there's almost always a large number of customers who have upgraded between those exact two releases, so the upgrade path is well known and well tested. In addition, since most customers will upgrade to a new LTS release after some patch releases have already been released for the later LTS release, it's less likely that you'll run into any issues during the upgrade.

Ultimately, this all means that upgrades between LTS versions are both faster and lower risk.

LTS Release Limitations

LTS releases do have some limitations compared to Agile releases.

Receive latest features and enhancements less frequently

As a general rule, new features and enhancements are not included in LTS patch releases. So, if you want a new feature or enhancement, you may need to upgrade to a newer LTS version - or possibly even an Agile release - to have access to that feature.

Not all fixes are included

Just as with hotfixes, there is no guarantee that any specific fix will be included in a given LTS release. While we make every effort to include fixes for issues which affect customers, we prioritize those that have the highest impact, and affect the most customers.

In addition, since the most important goal of LTS releases is to provide stability and reliability, high impact fixes (those which have the potential to cause new issues) are intentionally excluded. This also means that sometimes even if we plan to include a fix, we may need to leave it out if, while backporting the fix, we find that it presents a risk that we didn't anticipate.

No fixed schedule

Although we'll typically deliver a new patch release for each LTS version every 6-12 weeks, there is no guaranteed schedule. The schedule is determined by a combination of need and resources, and at times releases may not happen for several months. Longer patch release cycles are especially likely toward the end of an LTS version's lifecycle (when it's both very diverged from the latest Agile code, and also already very stable).

Progress

So, now that we’ve discussed the benefits, we’d like to show you how much progress we’ve made so far. As of the time of this writing:

Our currently supported LTS releases are:

• 21.06 LTS

• 22.03 LTS

• 23.01 LTS

81% of dotCMS Cloud customers are on LTS releases

About half of these are currently on the 22.03 LTS release, and many of these are planning for migration to 23.01 LTS.

In the last year of LTS releases, over 130 fixes have been included

The vast majority of these fixes were delivered to customers before they ever experienced the issues themselves.

So, LTS releases have been widely adopted, and a large number of fixes have been delivered to customers - far more than would have been possible with the traditional hotfix process.

Results

It’s easy to say that something will work better, but the only thing that really matters is - does it actually work?

And the answer is that, although our goal was to be able to improve service levels for customers who adopted LTS releases, the process has been so successful that we’ve been able to improve support for all of our customers - even those on Agile releases.

So, how do we know that? As an engineering-driven company, we believe that when it comes to showing that something works, it’s hard to beat looking at the numbers.

Number of support requests reduced

If the LTS release process is really improving the stability and reliability, then we should see a reduced number of support requests. And that's exactly what we've seen. And this has helped customers on both Agile and LTS release, because the fewer support requests we receive, the more time our support staff has to help all of our customers, regardless of what releases they’re running.

For the past two years, the number of support requests we received each month has been within a consistent range. But since the implementation of the LTS process, the number of requests has dropped to historically low levels.

One important thing to note is that these numbers are the total support requests for all customers (on both Agile and LTS releases). But the number of support requests just from customers on LTS releases has dropped by almost 50%. In other words, customers who’ve moved to LTS releases are now submitting half as many support requests as they did before they moved to an LTS release.

Support tickets resolved faster

If the LTS release process is improving support, the other place we expect to see a change is in the time it takes us to resolve the submitted support requests to the customers' satisfaction. So, let's take a look at that.

Just over a year after starting the LTS process, we’re resolving requests more than twice as fast as we ever did in the past.

Greatly improved response times

Let’s take a quick look at response times to round out the numbers. The LTS release process and other improvements in our support processes now enable us to respond to support requests much faster than in the past. Initial response times (the time it takes us to respond after a request is first submitted) has been cut by over 50%, and the time it takes to respond to each and every customer comment has dropped by almost 30%.

LTS Releases: A Game Changer for dotCMS Customers

The Long-Term-Supported (LTS) release process has successfully allowed us to improve stability and reliability, almost double release support periods, and simplify both patching and upgrades for customers on LTS releases. In addition, it's enabled us to improve the support we provide to all of our customers.

As an engineer I tend to understate things, but it’s fairly obvious from the data that the results have been dramatic. We’re excited about how much we’ve been able to help our customers, and looking forward to sharing with you more great changes as we keep moving forward!