LM
/
nwl-ci
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

doc/nwl-ci: added update src rev part, branch seeding and nightly trigger 

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
This commit is contained in:
Marc Mattmüller 2023-09-04 12:29:51 +02:00
parent 2a56bbd0e8
commit 3ff3dc9435
1 changed files with 250 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

250
doc/src/setup/nwl-ci.rst
View File

@ -1929,6 +1929,256 @@ Some side information about the directory structure of the JENKINS_HOME. The tre
+- workspace (working directory for the version control system)
Forward the Seeded Branch as well for the Pipelines
####################################################
Currently the pipelines that are created by the seed job are taken from the branch main. This shall change as we have
the seed job parameter ``SEEDING_BRANCH``. Its value needs to be used within the pipelines to seed. Therefore the
parameter needs to bind into the seed file. The changes are seen in the following diff:
.. code-block::
diff --git a/jobs/Jenkinsfile_Seed b/jobs/Jenkinsfile_Seed
index c3bb7cf..9cfb6f6 100644
--- a/jobs/Jenkinsfile_Seed
+++ b/jobs/Jenkinsfile_Seed
@@ -1,4 +1,6 @@
pipelineJob('nwl') {
+ def branchToUse = binding.variables.get("SEEDING_BRANCH")
+
displayName('1. NWL CI Pipeline')
description('CI pipeline for the NetModule Wireless Linux')
authorization {
@@ -24,7 +26,7 @@ pipelineJob('nwl') {
cpsScm {
scm {
git {
- branches('main')
+ branches(branchToUse)
extensions {
cloneOptions {
depth(1)
@@ -42,6 +44,8 @@ pipelineJob('nwl') {
}
pipelineJob('nwl-target') {
+ def branchToUse = binding.variables.get("SEEDING_BRANCH")
+
displayName('2. NWL Yocto Build')
description('CI pipeline building a NetModule Wireless Linux Yocto Target')
authorization {
@@ -58,7 +62,7 @@ pipelineJob('nwl-target') {
cpsScm {
scm {
git {
- branches('main')
+ branches(branchToUse)
extensions {
cloneOptions {
depth(1)
@@ -77,6 +81,8 @@ pipelineJob('nwl-target') {
pipelineJob('nwl-update-src-rev') {
+ def branchToUse = binding.variables.get("SEEDING_BRANCH")
+
displayName('0. Update Source Revisions')
description('CI pipeline updating the source revisions for the NetModule Wireless Linux')
authorization {
@@ -93,7 +99,7 @@ pipelineJob('nwl-update-src-rev') {
cpsScm {
scm {
git {
- branches('main')
+ branches(branchToUse)
extensions {
cloneOptions {
depth(1)
Extract Update Source Revision Part Into Job
############################################
Preliminary Information
***********************
As we have already seen in NetModule's OEM Linux, some recipes refer to a specific source revision of a repository which
is independent from the OEM Linux Yocto project. Now there are cases where the project moves forward but the recipe
holds still this specific source revision but should be at the latest state. Therefore a workflow was created using
devtool to check whether there are new revisions available. Now, it would be possible to update the recipe source
revision automatically by using these information. This is very helpful in nightly builds to check if head is still
working as expected. At a release we want to fix those revisions which is possible too by automatically commit the
revisions found.
This behavior can be reflected with a Jenkins job.
Seeding an additional Job
*************************
The following diff shows you the changes for seeding a new job (where the job is a skeleton).
.. code-block::
diff --git a/jobs/Jenkinsfile_Seed b/jobs/Jenkinsfile_Seed
index 45ec4e4..d1f3f48 100644
--- a/jobs/Jenkinsfile_Seed
+++ b/jobs/Jenkinsfile_Seed
@@ -65,3 +65,38 @@ pipelineJob('nwl-target') {
}
}
}
+
+
+pipelineJob('nwl-update-src-rev') {
+ def branchToUse = binding.variables.get("SEEDING_BRANCH")
+
+ displayName('0. Update Source Revisions')
+ description('CI pipeline updating the source revisions for the NetModule Wireless Linux')
+ authorization {
+ permissionAll('anonymous')
+ }
+ authenticationToken('buildNwlToken')
+ logRotator {
+ artifactDaysToKeep(7)
+ artifactNumToKeep(7)
+ daysToKeep(7)
+ numToKeep(7)
+ }
+ definition {
+ cpsScm {
+ scm {
+ git {
+ branches(branchToUse)
+ extensions {
+ cloneOptions {
+ depth(1)
+ }
+ }
+ remote {
+ credentials('nmgit_credentials')
+ url('ssh://git@bitbucket.gad.local:7999/nm-nsp/nwl-ci.git')
+ }
+ }
+ scriptPath('jobs/Jenkinsfile_UpdateSrcRevisions')
+ }
+ }
+ }
+}
diff --git a/jobs/Jenkinsfile_UpdateSrcRevisions b/jobs/Jenkinsfile_UpdateSrcRevisions
new file mode 100644
index 0000000..000bbf9
--- /dev/null
+++ b/jobs/Jenkinsfile_UpdateSrcRevisions
@@ -0,0 +1,41 @@
+// Loading code requires a NODE context
+// But we want the code accessible outside the node Context
+// So declare common (object created by the LOAD operation) outside the Node block.
+def common
+
+// declarative pipeline
+pipeline {
+ agent any
+
+ parameters {
+ string(name: 'BUILD_BRANCH', defaultValue: 'main', description: 'Enter the branch of the NWL to build (default = main), will skip deployment if not main')
+ booleanParam(name: 'CLEAN_BUILD', defaultValue: false, description: 'do a clean build, i.e. remove the yocto directory and start from scratch')
+ }
+
+ options {
+ timeout(time: 1, unit: 'HOURS')
+ buildDiscarder(
+ logRotator(numToKeepStr: '10')
+ )
+ }
+
+ stages {
+
+ stage('Prepare') {
+ steps {
+ script {
+ println "ToDo: prepare stage"
+ }
+ }
+ }
+
+ stage('Update') {
+ steps {
+ script {
+ println "ToDo: Update stage"
+ }
+ }
+ }
+
+ } // stages
+}
.. caution::
With the current setup the job seems to be sed but another seeding is triggered due to SCM change. See the following
subsection about how to solve this issue.
Adapt initial Seed Job not triggering on SCM change
===================================================
The initial seed job within the Jenkins instance needs to have a ``disableRemotePoll()`` so that the seed job is not
triggered on SCM change:
.. code-block::
diff --git a/jenkins-ci/jobs/seed.groovy b/jenkins-ci/jobs/seed.groovy
index 033eacf..277785c 100644
--- a/jenkins-ci/jobs/seed.groovy
+++ b/jenkins-ci/jobs/seed.groovy
@@ -58,6 +58,7 @@ freeStyleJob('nwl-seed-job') {
credentials('admin_credentials')
}
extensions {
+ disableRemotePoll()
wipeWorkspace()
}
}
With this change we create a new version of the NWL docker instance:
.. code-block::
# on your local machine:
DOCKER_BUILDKIT=1 ./build.sh nwl 0.3.2
docker save nwl-env-ci:latest | bzip2 | pv | ssh user@10.115.101.98 docker load
# on the build server:
docker image tag nwl-env-ci:latest nwl-env-ci:0.3.2
./manage.sh --name=nwl_0_3_1 destroy
./manage.sh --image=nwl-env-ci:0.3.2 --branch=main --name=nwl_0_3_2 --platform=nwl --config=/home/user/work/ci/config/config.xml --revision=0.3.2 --maintainer=TeamCHBE create
Nightly Trigger Integration
###########################
With the Job DSL plugin version 1.77 the option ``triggers`` is deprecated and will be removed soon (reason: it caused
problems, see `ticket <https://issues.jenkins.io/browse/JENKINS-53775>`_). A replacement is available as stated in
the `migration note <https://github.com/jenkinsci/job-dsl-plugin/wiki/Migration#migrating-to-177>`_. According to this
note, the nightly trigger uses the ``pipelineTriggers`` as follows:
.. code-block::
diff --git a/jobs/Jenkinsfile_Seed b/jobs/Jenkinsfile_Seed
index d1f3f48..c3bb7cf 100644
--- a/jobs/Jenkinsfile_Seed
+++ b/jobs/Jenkinsfile_Seed
@@ -11,6 +11,15 @@ pipelineJob('nwl') {
daysToKeep(7)
numToKeep(7)
}
+ properties {
+ pipelineTriggers {
+ triggers {
+ cron {
+ spec('H H(5-6) * * 1-5')
+ }
+ }
+ }
+ }
definition {
cpsScm {
scm {
.. |nwlCiChain| image:: ./media/nwl-ci-jenkins-dashboard.png
:width: 700px