Agile Development with FirstSpirit: External Sync

Published on 24 May 2019

Modern IT projects use agile methods to easily react to changes in the marketplace. To make agile workflows work with FirstSpirit, External Sync can be used. This feature allows one to manage the proprietary data of the CMS with a different Version Control System, thus enabling the use of distributed and parallel development of features. In this article we will show a typical FirstSpirit development setup with its problems, how we can we solve this, and how long it took us in the past to deliver the solution.

Dr. Max Möllers

CEO and Project Management

Scenario

classic deployment pipeline

Keeping things simple, let us assume FirstSpirit is used to generate a single website with static HTML + JS + CSS. An integration with a shop, portal, SPA, or multi-site setups would work similarly, but would just bloat the explanation.

In this setup, there are 3 FirstSpirit servers: D(evelopment), Q(uality Assurance), und P(roduction). These are used as follows:

  • On D, the developers create new templates or hotfixes.

  • On Q, editors can review and sign-off new features.

  • On P, the actual content is managed. All the templates here are the final versions. This set of data is regularly deployed to the live website.

Changes to the template thus move from D over to Q over to P. Small changes can be migrated as single packages, but larger changes require to update the whole template store. Developers thus need to coordinate development efforts, which leads to updates in intervals, e.g., every 4 weeks. Especially hotfixes are a problem in this setup, as you either have to develop them directly on P or you have to rollback Q or D to the state of P to have the proper environment.

All those issues are no longer of concern with External Sync.

External Sync

Git flow model

External Sync allows you to take the FirstSpirit objects, such as templates, from their proprietary repository and export them to folder structure with flat files. It can also import from that folder structure back in to FirstSpirit. For this, we can use a flexible command line tool, that controls the server via the public API: FirstSpirit Dev Tools.

An export, e.g., of the FirstSpirit TemplateStore, consist of a folder with all page templates, section templates, etc. These in turn are made of human-readable XML files and can thus be easily manipulated.

You can now setup a version control system on this folder structure using a tool like Git and carry out your own branching strategy that fits your project. For the remainder of the article, we will assume that the developers use the Git Flow.

This means that our P environment should reflect the "master" branch (turquoise), D reflects the "develop" branch (yellow) and Q will be used to review changes of the "release candidate" branch (green). So how does that work in detail?

Take a new feature that was developed on D and is now ready for review. The developer uses External Sync to export the latest set of templates to the Git-controlled folder, updates the Git, and pushes the update into the shared Git repository.

fs-cli -h fs-d.customer.com -port 4242 -c HTTP -p "DevProject" -u userA -pwd mypwd export
git commit && git push

The editors want to test the new feature. So we point the "release candidate" branch to the latest commit on the "develop" branch. We can then fetch the latest update on the Q environment using:

git pull 
fs-cli -h fs-q.customer.com -port 4242 -c HTTP -p "DevProject" -u userA -pwd mypwd import

The editors now see the updates and can review and sign them off. After sign-off, we can use the same process to push the changes to P.

We now use External Sync, but still rely on our old development process of D->Q->P. Let us change that.

Distributed Development

Deployment pipeline FirstSpirit

A big advantage of our new setup is that developers can now work on different features in parallel without getting in each others way. Every developer installs a local FirstSpirit server and for each issue assigned to him, he creates a feature branch in the Git (marked pink in the Git Flow figure). After developing this feature, he merges it to the "develop" branch (which is "located" on the D environment). The feature can now be tested on its own or together with several others. We "just" have to update the Q environment. Similarly, after the sign-off changes are marked ready for the P environment by pushing to the "master" branch. This gives everybody way better control over the development and a better overview of what environment has which state of development. Additionally, the developers can work on their local server which usually is faster, especially if the server sits in a remote location.

The only drawback for this distributed develipment ist that you need a licence for each local server. Fortunately, the e-Spirit licence model (currently, as of June 2019) consider these additional local servers different from a "full" server licence that you would need when going from a D+P to a D+Q+P setup. The costs are more comparable to an additional user rather than a full server. So far we had a positive experience with the sales department in this regard and the additional costs have not been an issue in our projects. This is obviously subject to any changes from the e-Spirit sales department.

Hotfix

Another advantage is that the developers can easily react to issues on the production environment. If something arises, the developer can recreate the P environment by pulling the "master" branch to this local server. He can identify the issue and create a hotfix (red in the Git flow figure). This is applied to the "master" branch / the P environment and will also be merged into the "develop" branch so it will persist for future updates.

This means we are always able to release and do not have to wait for the end of the sprint or milestone for the fix to be live.

Continuous Integration

Many of the aforementioned manual steps on the command-line can be automated. For example, it is very convenient if the state of D / Q always reflects the "develop" / "release candidate" branch of the git. For this, you can use a Continuous Integration Tool such as Jenkins or Bamboo and set it up to wait for changes in the Git:

As soon as a developer pushes his feature to "develop", the changes are checked out to the D server and imported using fs-cli. This works the same for "release candidate" and Q.

You could also setup it up the same for P but our experience has been that you would want to prepare this synchronisation using Jenkins or Bamboo, but you should trigger it manually. Otherwise, an intern that has access to Git could accidentally push his experiments to the "master" branch and have those in production instantly, potentially making the work of all the editors impossible.

External Compilers

Another scenario that can be improved using External Sync is when you use external preprocessors, such as SCSS for creating your CSS.

The editor can still specify the layout using SCSS, but he can now import the updated CSS directly into FirstSpirit using fs-cli. Similarly, a minifier for the own JavaScript development can be integrated without having to manually import the latest version of the JS files into the SiteArchitect.

Making it work in your project

We saw that using External Sync can speed up the development process, give more control over different software versions, and allows for a lot of automation. But how long does it take to get to this state?

In our experience, the first steps are very fast. You only need the command line tool and a Git Repository. On an afternoon you should be able to figure out the documentation and create the necessary scripts to easily import and export changes.

The next step is to get together with the dev team and figure out how they want to collaborate and what workflows should be used when updating code. Usually, the dev team already uses some process like the here mentioned Git Flow for other development tasks, e.g., developing FirstSpirit modules. That means there is not a lot to explain and adapt as template development works very similar. Lastly, you want to have a Continuous Integration chain, which, depending on the level of sophistication, can take several days to work properly.

All in all, after about 2 weeks of actual work, you should have a massively improved development process that pays for itself in no time. For reference, our customers never regretted changing to work this way.

facebook icon twitter icon xing icon linkedin icon
© 2019 aboutcontent GmbH