Cloud Build & FirstSpirit Dev Tools

Published on 6 Apr 2020

We suggested automating the development cycle in our article "Agile Development with FirstSpirit: External Sync". Take a look on how that works with Google Cloud Build!

Peter Zeidler

Research and Development

Initial Situation

Even Automation has several levels of maturity. We will take a look at a company that had a central FirstSpirit server and all developers created templates on their own local FirstSpirit servers. These templates were initially pushed to the central FS server as package using Content-Transport. The first improvement was to use the FirstSpirit DevTools for the update.

Detailed Goal

The goal now is to automate all the required manual steps when updating templates. As soon as devleoper commits a template change to the central, these updates should be pushed to corresponding FirstSpirit project., e.g., a commit to the "master"-branch should appear in the QA project.

Updating the production project needs to be prepared similarly, but we'd recommend a (single) manual step to actually trigger the update due to safety concerns.

Steps

An important component of the Google Cloud Build are the "cloud builders", these are prepared containers that can be adapted and can run on a schedule or trigger.

The first step is to create "pom.xlm" in which we can direct Maven to call the FirstSpirit Dev Tools.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>de.aboutcontent.blogpost</groupId>
	<artifactId>website-project</artifactId>
	<version>0.0.1</version>
</project>

We now extend our "pom.xml" with a profile that imports the templates into the target project using external sync.

<profile>
	<id>import</id>
	<build>
		<plugins>
			<plugin>
				<groupId>org.codehaus.mojo</groupId>
				<artifactId>exec-maven-plugin</artifactId>
				<version>1.6.0</version>
				<executions>
					<execution>
						<goals>
							<goal>exec</goal>
						</goals>
					</execution>
				</executions>
				<configuration>
					<longModulepath>false</longModulepath>
					<executable>bin/fs-cli.sh</executable>
					<arguments>
						<argument>-u</argument>
						<argument>${firstspirit.user}</argument>
						<argument>-pwd</argument>
						<argument>${firstspirit.pass}</argument>
						<argument>-h</argument>
						<argument>${firstspirit.host}</argument>
						<argument>-port</argument>
						<argument>${firstspirit.port}</argument>
						<argument>-c</argument>
						<argument>${firstspirit.mode}</argument>
						<argument>-p</argument>
						<argument>${firstspirit.project}</argument>
						<argument>-sd</argument>
						<argument>src</argument>
						<argument>import</argument>
					</arguments>
				</configuration>
			</plugin>
		</plugins>
	</build>
</profile>

In the snippet above we used several arguments for calling the update. This way we do not have to store credentials in a central repository AND we can use the same profile for several projects and FS servers. The actual arguments are stored in a "settings.xml" such das this one for a local server:

<profile>
	<id>firstspirit-localhost</id>
	<properties>
		<firstspirit.host>localhost</firstspirit.host>
		<firstspirit.port>8000</firstspirit.port>
		<firstspirit.mode>HTTP</firstspirit.mode>
		<firstspirit.user>Admin</firstspirit.user>
		<firstspirit.pass>Admin</firstspirit.pass>
		<firstspirit.project>Demoprojekt</firstspirit.project>
	</properties>
</profile>

Before we can try this out, we have to create the correct file structure. Our project folder should look like this:

.
├── bin
│   ├── fs-cli.cmd
│   └── fs-cli.sh
├── conf
│   └── log4j.properties
├── lib
│   ├── fs-access.jar
│   └── fsdevtools-cli-2.5.6.jar
├── pom.xml
└── src

We can now test the import steps on our local FirstSpirit server. We just execute the following command from the shell:

mvn exec:exec -Pfirstspirit-localhost -Pimport

Assuming this works, we now extend the "pom.xml" again. Now we use the same maven plugin to download the "fs-access.jar"

<profile>
	<id>get-fs-access-jar</id>
	<build>
		<plugins>
			<plugin>
				<groupId>org.codehaus.mojo</groupId>
				<artifactId>exec-maven-plugin</artifactId>
				<version>1.6.0</version>
				<executions>
					<execution>
						<goals>
							<goal>exec</goal>
						</goals>
					</execution>
				</executions>
				<configuration>
					<longModulepath>false</longModulepath>
					<executable>wget</executable>
					<arguments>
						<argument>--user</argument>
						<argument>${artifactory.user}</argument>
						<argument>--password</argument>
						<argument>${artifactory.pass}</argument>
						<argument>http://artifactory/de/espirit/firstspirit/fs-access/${firstspirit.version}/fs-access-${firstspirit.version}.jar</argument>
						<argument>-O</argument>
						<argument>lib/fs-access.jar</argument>
					</arguments>
				</configuration>
			</plugin>
		</plugins>
	</build>
</profile>

Using the following command, we can now put the "fs-access.jar" in the correct lib folder:

mvn exec:exec -Pfirstspirit-localhost -Pget-fs-access-jar

We now reached the first target: We can control the FirstSpirit Dev Tools using Maven.

Google Cloud Buckets and Build

We now move the data that we do not want in our repository into the cloud. For this, we create a file bucket in the respective Google Cloud project.

This means we create two folders: "ci-maven" for the "settings.xml" and "fs-cli" for all the data of the FirstSpirit Dev Tools.

Next, we need to define the tasks that should happen after a trigger in the "cloudbuild.yaml" file. This file is also stored in the source repository to be accessible when an update got triggered.

The file only consists of tasks in the "steps" section, here are the first four copy steps;

  - name: gcr.io/cloud-builders/gsutil
    args: ['cp', 'gs://artifacts.ac-shout.appspot.com/fs-cli/bin/fs-cli.sh', 'bin/fs-cli.sh']
  - name: gcr.io/cloud-builders/gsutil
    args: ['cp', 'gs://artifacts.ac-shout.appspot.com/fs-cli/lib/fsdevtools-cli-2.5.6.jar', 'lib/fsdevtools-cli-2.5.6.jar']
  - name: gcr.io/cloud-builders/gsutil
    args: ['cp', 'gs://artifacts.ac-shout.appspot.com/fs-cli/conf/log4j.properties', 'conf/log4j.properties']
  - name: gcr.io/cloud-builders/gsutil
    args: ['cp', 'gs://artifacts.ac-shout.appspot.com/ci-maven/settings.xml', 'settings.xml']

After copying the data, we execute our first maven task that downloads the "fs-access.jar" from the artifactory.

  - name: 'gcr.io/cloud-builders/mvn'
    args: ['--settings', '/workspace/settings.xml', 'exec:exec', '-Pget-fs-access-jar', '-Pfirstspirit-prod']

We now use the existing "ubuntu" container to make our shell script executable.

  - name: 'ubuntu'
    args: ['chmod', '755', '/workspace/bin/fs-cli.sh']

In the final step, we can update the FirstSpirit project. Instead of directly specifying the target project, we use "${_PROJECT_PROFILE}" which is set during runtime by the trigger and thus points to the correct project.

The full "cloudbuild.yaml" should look like this now:

steps:
  # copy fs-cli
  - name: gcr.io/cloud-builders/gsutil
    args: ['cp', 'gs://artifacts.ac-shout.appspot.com/fs-cli/bin/fs-cli.sh', 'bin/fs-cli.sh']
  - name: gcr.io/cloud-builders/gsutil
    args: ['cp', 'gs://artifacts.ac-shout.appspot.com/fs-cli/lib/fsdevtools-cli-2.5.6.jar', 'lib/fsdevtools-cli-2.5.6.jar']
  - name: gcr.io/cloud-builders/gsutil
    args: ['cp', 'gs://artifacts.ac-shout.appspot.com/fs-cli/conf/log4j.properties', 'conf/log4j.properties']
  - name: gcr.io/cloud-builders/gsutil
    args: ['cp', 'gs://artifacts.ac-shout.appspot.com/ci-maven/settings.xml', 'settings.xml']
  # get fs-access.jar
  - name: 'gcr.io/cloud-builders/mvn'
    args: ['--settings', '/workspace/settings.xml', 'exec:exec', '-Pget-fs-access-jar', '-Pfirstspirit-prod']
  # make fs-cli executable
  - name: 'ubuntu'
    args: ['chmod', '755', '/workspace/bin/fs-cli.sh']
  # import to firstspirit
  - name: 'gcr.io/cloud-builders/mvn'
    args: ['--settings', '/workspace/settings.xml', 'exec:exec', '-Pimport', '-P${_PROJECT_PROFILE}']

With everything prepared we can now create the Trigger that will call the scripts and maven profiles:

The important settings are as follows:

Google Cloud Build - Trigger settings
Google Cloud Build

We can specifiy different conditions when to run our scripts: We could trigger the update on a commit to the branch, e.g., on commit to the "dev"-branch update the central FS DEV server. Another option is to use tags as triggers. This is often combined with a regex to have more flexibility, i.e., any new tag starting with "RELEASE_CANDIDATE" is pushed to the QA server.

For "Build configuration" we have to specify what file contains the commands to be run, here it is "/cloudbuild.yaml".

Finally, we create the variable to be set by the trigger "_PROJECT_PROFILE" and set it to the corresponding project name.

Conclusion

In this article, we showed how to use Google Cloud Build to conveniently and automatically keep the FirstSpirit projects in-sync.

Our target environments are now always updated with the latest template and content in a timely fashion.

Building on the automatic updates, we can create end-to-end tests, to ensure a smoothly running system.

facebook icon twitter icon xing icon linkedin icon
© 2019 aboutcontent GmbH