Skip to content

Commit 35534cc

Browse files
committed
first bit of java native image on jx
1 parent 6661174 commit 35534cc

10 files changed

+505
-5
lines changed
+121
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
title: Jenkins X - Java Native Image Prod
2+
description: Creating a Java Native Image application and run it as Production with Jenkins X - Introduction - 1/8
3+
hero: Introduction - 1/8
4+
5+
# Introduction
6+
7+
## Stack
8+
9+
* **Google Cloud Platform(GCP)**: while you can do evyrthing required with other providers, I've chosen GCP
10+
* **Kubernetes**: in particular, **GKE**
11+
* **Jenkins X**: CI/CD
12+
* **Helm**: packaging our Kubernetes application
13+
* managed by Jenkins X (_to a degree_)
14+
* **Google Cloud SQL**(MySQL): our data storage
15+
* **HashiCorp Vault**: secrets storage
16+
* **Quarkus**: our Java framework
17+
* Spring Data JPA for ORM
18+
* Spring Web for the REST API
19+
* **Java 11**
20+
* **GraalVM**: compiler/runtime to create a native executable of our Java code
21+
22+
## What We Will Do
23+
24+
The outline of the steps to take is below. Each has its own page, so if you feel you have.
25+
26+
* Create Google Cloud SQL (MySql flavor) as datasource (we're on GCP afterall)
27+
* Create Quarkus application
28+
* Import the application into Jenkins X
29+
* Change the Build to Native Image (with GraalVM)
30+
* Retrieve application secrets (such as Database username/password) from HashiCorp Vault
31+
* Productionalize our Pipeline
32+
* Static Code Analysis with SonarQube/SonarCloud
33+
* Dependency Vulnerability scan with Sonatype's OSS Index
34+
* Integration Tests
35+
* Productionalize our Applications
36+
* Monitoring with Prometheus & Grafana
37+
* Tracing with OpenTracing & Jaeger
38+
* Manage our logs with Sentry.io
39+
40+
## Pre-requisites
41+
42+
The pre-requisites are a Kubernetes Cluster with Jenkins X installed, including Haschicorp Vault integration. The guide assumes you use GKE, we will create our MySQL database there, but should be reproducable on other Kubernetes services where Jenkins X supports Hashicorp Vault (currently GKE and AWS's EKS).
43+
44+
If you want to focus on a stable production ready cluster, I can also recommend to use [CloudBees' distribution of Jenkins X](https://docs.cloudbees.com/docs/cloudbees-jenkins-x-distribution/latest/). Don't worry, this is also free with no caveats, but has a slower release candence to focus more on stability than the OSS mainline does.
45+
46+
* GKE Cluster:
47+
* [GKE via Terraform](https://joostvdg.github.io/kubernetes/distributions/gke-terraform/)
48+
* [GKE via Gcloud](https://cloud.google.com/kubernetes-engine/docs/quickstart)
49+
* [GKE via Jenkins X's Terraform module](https://jenkins-x.io/docs/getting-started/)
50+
* EKS Cluster:
51+
* [EKS via Jenkins X's Terraform module](https://registry.terraform.io/modules/jenkins-x/eks-jx/aws/0.2.1)
52+
* [EKS via EKSCTL](https://eksctl.io/)
53+
* [EKS](https://aws.amazon.com/blogs/startups/from-zero-to-eks-with-terraform-and-helm/)
54+
* Jenkins X:
55+
* [Jenkins X Getting Started on GKE Guide](https://jenkins-x.io/docs/getting-started/)
56+
* [CloudBees Jenkins X Distribution](https://docs.cloudbees.com/docs/cloudbees-jenkins-x-distribution/latest/)
57+
* [Youtube video with installation and maintenance guidance](https://www.youtube.com/watch?v=rQlP_3iXvRE)
58+
59+
## Why Quarkus
60+
61+
Before we start, I'd like to make the case, why I chose to use Quarkus for this.
62+
63+
Wanting to build a Native Image with Java 11 is part of the reason, we'll dive into that next.
64+
65+
Quarkus has seen an tremendous amount of updates since its inception.
66+
It is a really active framework, which does not require you to forget everything you've learned in other Java frameworks such as Spring and Spring Boot.
67+
68+
It comes out of the same part from RedHat that is involved with OpenShift - RedHat's Kubernetes distribution.
69+
This ensures the framework is created with running Java on Kubernetes in mind.
70+
Jenkins X starts from Kubernetes, so this makes it a natural fit.
71+
72+
Next, the capabilities for making a Native Image and work done to ensure you - the developer - do not have to worry (too much) about how to get from a Spring application to a Native Image is staggering. This makes the Native Image experience pleasant and involve little to no debugging.
73+
74+
## Why Native Image
75+
76+
Great that Quarkus helps with making a Native Image. What is a Native Image?
77+
In short, its makes your Java code into a runnable executable build for a specific environment.
78+
79+
You might wonder, what is wrong with using a runnable Jar - such as Spring Boot - or using a JVM?
80+
Nothing in and on itself. However, there are cases where having a long running process with a slow start-up time hurts you.
81+
82+
In a Cloud Native world, including Kubernetes, this is far more likely than in traditional - read, VM's - environments. With the advent of creating many smaller services that may or may not be stateless, and should be capable of scaling horizontally from 0 to infinity, different characteristics are required.
83+
84+
Some of these characterics:
85+
86+
* minimal resource use as we pay per usage (to a degree)
87+
* fast startup time
88+
* perform as expected on startup (JVM needs to warm up)
89+
90+
A Native Image performs better on the above metrics than a classic Java application with a JVM.
91+
Next to that, when you have a fixed runtime, the benefit of Java's "build once, run everywhere" is not as useful. When you always run your application in the same container in similar Kubernetes environments, a Native Image is perfectly fine.
92+
93+
Now, wether a Native Image performs better for your application depends on your application and its usage. The Native Image is no silver bullet. So it is still on you to do load and performance tests to ensure you're not degrading your performance for no reason!
94+
95+
## Resources
96+
97+
* https://quarkus.io/guides/writing-native-applications-tips
98+
* https://quarkus.io/guides/building-native-image
99+
* https://cloud.google.com/community/tutorials/run-spring-petclinic-on-app-engine-cloudsql
100+
* https://github.com/GoogleCloudPlatform/community/tree/master/tutorials/run-spring-petclinic-on-app-engine-cloudsql/spring-petclinic/src/main/resources
101+
* https://github.com/GoogleCloudPlatform/google-cloud-spanner-hibernate/blob/master/google-cloud-spanner-hibernate-samples/quarkus-jpa-sample
102+
* https://medium.com/@hantsy/kickstart-your-first-quarkus-application-cde54f469973
103+
* https://developers.redhat.com/blog/2020/04/10/migrating-a-spring-boot-microservices-application-to-quarkus/
104+
* https://www.baeldung.com/rest-assured-header-cookie-parameter
105+
* https://jenkins-x.io/docs/reference/pipeline-syntax-reference/#containerOptions
106+
* https://openliberty.io/blog/2020/04/09/microprofile-3-3-open-liberty-20004.html#gra
107+
* https://openliberty.io/docs/ref/general/#metrics-catalog.html
108+
* https://grafana.com/grafana/dashboards/4701
109+
* https://phauer.com/2017/dont-use-in-memory-databases-tests-h2/
110+
* https://github.com/quarkusio/quarkus/tree/master/integration-tests
111+
* https://hub.docker.com/r/postman/newman
112+
* https://github.com/postmanlabs/newman
113+
* https://learning.postman.com/docs/postman/launching-postman/introduction/
114+
* https://jenkins-x.io/docs/guides/using-jx/pipelines/envvars/
115+
* https://github.com/quarkusio/quarkus/tree/master/integration-tests/flyway/
116+
* https://quarkus.io/guides/flyway
117+
* https://rhuanrocha.net/2019/03/17/how-to-microprofile-opentracing-with-jaeger/
118+
* https://medium.com/jaegertracing/microprofile-tracing-in-supersonic-subatomic-quarkus-43020f89a753
119+
* https://github.com/opentracing-contrib/java-jdbc
120+
* https://quarkus.io/guides/opentracing
121+
* https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine?hl=en_US
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
title: Jenkins X - Java Native Image Prod
2+
description: Creating a Java Native Image application and run it as Production with Jenkins X - Cloud SQL - 2/8
3+
hero: Cloud SQL - 2/8
4+
5+
# Google Cloud SQL
6+
7+
## Requirements
8+
9+
## UI
10+
11+
## Gcloud
12+
13+
## Terraform
14+
15+
## How To Connect To The Database
16+
17+
* https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine?hl=en_US
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
title: Jenkins X - Java Native Image Prod
2+
description: Creating a Java Native Image application and run it as Production with Jenkins X - Quarkus - 3/8
3+
hero: Quarkus - 3/8
4+
5+
# Create Quarkus Application
6+
7+
There are several ways you can create a Quarkus application.
8+
9+
You can create one by going to [code.quarkus.io](https://code.quarkus.io/), fill in your details and select your dependencies - this is an API call, so automatable. You can start with an maven archetype and add Quarkus details, or you can start from one of the [Quarkus Quickstarts](https://github.com/quarkusio/quarkus-quickstarts).
10+
11+
In this guide, we start with a Quarkus Quickstart (Spring Data JPA to be exact) and modify this to suit our needs.
12+
13+
## Fork, Clone, or Copy
14+
15+
We're going to start from Quarkus' `spring-data-jpa-quickstart`, I leave it up to you how you get the code in your own repository. You can fork it, clone it and copy it or whatever floats your boat.
16+
17+
You can find the quickstart [here](https://github.com/quarkusio/quarkus-quickstarts/tree/master/spring-data-jpa-quickstart), and for reference, the Quarkus Guide that comes with it, [here](https://quarkus.io/guides/spring-data-jpa).
18+
19+
## Update Dependencies
20+
21+
We're going to modify the application, so lets dive into the `pom.xml` and make our changes.
22+
23+
First, we will use MySQL as our RDBMS so we drop the `quarkus-jdbc-postgresql` dependency.
24+
25+
Next, we add our other spring dependencies and the `quarkus-jdbc-mysql` for MySQL.
26+
27+
```xml
28+
<dependency>
29+
<groupId>io.quarkus</groupId>
30+
<artifactId>quarkus-spring-web</artifactId>
31+
</dependency>
32+
<dependency>
33+
<groupId>io.quarkus</groupId>
34+
<artifactId>quarkus-spring-di</artifactId>
35+
</dependency>
36+
<dependency>
37+
<groupId>io.quarkus</groupId>
38+
<artifactId>quarkus-jdbc-mysql</artifactId>
39+
</dependency>
40+
```
41+
42+
## Transform Resource to Controller
43+
44+
Now that we're using Spring Web, we are going to change our Resource - FruitResource - to a Spring Web Controller.
45+
46+
Replace this annotation:
47+
48+
```java
49+
@Path("/fruits")
50+
```
51+
52+
With this.
53+
54+
```java
55+
@RestController
56+
@RequestMapping(value = "/fruits")
57+
```
58+
59+
Replace all `@PathParams` with Spring's `@PathVariable`'s.
60+
Mind you, these require the name of the variable as a parameter.
61+
62+
For example:
63+
64+
```java
65+
@POST
66+
@Path("/name/{name}/color/{color}")
67+
@Produces("application/json")
68+
public Fruit create(@PathParam String name, @PathParam String color) {}
69+
```
70+
71+
Becomes:
72+
73+
```java
74+
@PostMapping("/name/{name}/color/{color}")
75+
public Fruit create(@PathVariable(value = "name") String name, @PathVariable(value = "color") String color) {
76+
```
77+
78+
Then, replace the Http method annotations for the methods:
79+
80+
* `@GET` with `@GetMapping`
81+
* `@DELETE` with `@DeleteMapping`
82+
* `@POST` with `@PostMapping`
83+
* `@PUT` with `@PutMapping`
84+
85+
!!! note
86+
Spring's annotation includes the path, so you can collapse the `@PATH` into the Http method annotation.
87+
88+
For example:
89+
90+
```java
91+
@GET
92+
@Path("/color/{color}")
93+
@Produces("application/json")
94+
```
95+
96+
Becomes:
97+
98+
```java
99+
@GetMapping("/color/{color}")
100+
```
101+
102+
??? example "FruitResource.java"
103+
104+
```java
105+
@RestController
106+
@RequestMapping(value = "/fruits")
107+
public class FruitResource {
108+
109+
private final FruitRepository fruitRepository;
110+
111+
public FruitResource(FruitRepository fruitRepository) {
112+
this.fruitRepository = fruitRepository;
113+
}
114+
115+
@GetMapping("/")
116+
public List<Fruit> findAll() {
117+
...
118+
}
119+
120+
@DeleteMapping("{id}")
121+
public ResponseEntity<Long> delete(@PathVariable(value = "id") long id) {
122+
...
123+
}
124+
125+
@PostMapping("/name/{name}/color/{color}")
126+
public Fruit create(@PathVariable(value = "name") String name, @PathVariable(value = "color") String color) {
127+
...
128+
}
129+
130+
@PutMapping("/id/{id}/color/{color}")
131+
public Fruit changeColor(@PathVariable(value = "id") Long id, @PathVariable(value = "color") String color) {
132+
...
133+
}
134+
135+
@GetMapping("/color/{color}")
136+
public List<Fruit> findByColor(@PathVariable(value = "color") String color) {
137+
...
138+
}
139+
}
140+
```
141+
142+
## Update Application Properties
143+
144+
Let's update the applications properties, an initial configuration for MySQL.
145+
146+
For the username and password, we use environment variables wich we will address later - when we import the aplication into Jenkins X.
147+
148+
The JDBC URL looks a bit weird, but this has to do with [how Google Cloud SQL can be accessed via Kubernetes](https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine?hl=en_US). This is enough for now, we'll come back for more, don't worry.
149+
150+
```properties
151+
quarkus.datasource.db-kind=mysql
152+
quarkus.datasource.jdbc.url=jdbc:mysql://127.0.0.1:3306/fruits
153+
quarkus.datasource.jdbc.max-size=8
154+
quarkus.datasource.jdbc.min-size=2
155+
156+
quarkus.datasource.username=${GOOGLE_SQL_USER}
157+
quarkus.datasource.password=${GOOGLE_SQL_PASS}
158+
```
159+
160+
## Unit Testing
161+
162+
In order to build our application, we now need a MySQL database as we have unit tests, testing our FruitResource - as we should! Locally, we can address this by running MySQL as a Docker container. Unfortunately, when building applications in Jenkins X, we don't have access to Docker - you could, but in Kubernetes this is a big no-no. So, for now, we'll spin up an H2 database in MySQL mode to avoid the issue, [but we should probably come back to that later](https://phauer.com/2017/dont-use-in-memory-databases-tests-h2/).
163+
164+
This was in part inspired by [@hantsy's post on creating your first Quarkus application](https://medium.com/@hantsy/kickstart-your-first-quarkus-application-cde54f469973) on Medium, definitely worth a read in general.
165+
166+
In order to use the H2 database for our unit tests, we have to make three changes:
167+
168+
1. add the H2 test dependency
169+
1. create a `application.properties` file for tests, in `src/test/resources`
170+
1. annotate our test class with `@QuarkusTestResource(H2DatabaseTestResource.class)`, so Quarkus spins up the H2 database
171+
172+
173+
```xml
174+
<dependency>
175+
<groupId>io.quarkus</groupId>
176+
<artifactId>io.quarkus:quarkus-test-h2</artifactId>
177+
<scope>test</scope>
178+
</dependency>
179+
```
180+
181+
!!! example "src/test/java/../FruitResourceTest.java"
182+
183+
```java
184+
@QuarkusTestResource(H2DatabaseTestResource.class)
185+
@QuarkusTest
186+
class FruitResourceTest {
187+
...
188+
}
189+
```
190+
191+
!!! example "src/test/resoureces/application.properties"
192+
193+
```properties
194+
quarkus.datasource.url=jdbc:h2:tcp://localhost/mem:test
195+
quarkus.datasource.driver=org.h2.Driver
196+
quarkus.hibernate-orm.database.generation = drop-and-create
197+
quarkus.hibernate-orm.log.sql=true
198+
```
199+
200+
## Replace jsonb with jackson
201+
202+
Spring depends on `Jackson` for marshalling JSON to and from Java Objects.
203+
It makes sense to make our application depend on the same libary to reduce potential conflicts.
204+
205+
Remove the `quarkus-resteasy-jsonb` dependency:
206+
207+
```xml
208+
<dependency>
209+
<groupId>io.quarkus</groupId>
210+
<artifactId>quarkus-resteasy-jsonb</artifactId>
211+
</dependency>
212+
```
213+
214+
And add `quarkus-resteasy-jackson`.
215+
216+
```xml
217+
<dependency>
218+
<groupId>io.quarkus</groupId>
219+
<artifactId>quarkus-resteasy-jackson</artifactId>
220+
</dependency>
221+
```
222+
223+
## Next Steps
224+
225+
Running `mvn clean test` should result in a succesful build, with two tests testing most of our application.
226+
227+
This means we're ready to go to the next step, importing the application into Jenkins X!

0 commit comments

Comments
 (0)