Commit Graph

178 Commits

Author SHA1 Message Date
Marc Mattmüller 42f9c9eb4d jobs/Common: replaced the the variable for generating ci-artifacts
declaring COREOS_IMAGE_EXTRACLASSES += "coreos-image-ci" let
bitbake generate a file *.ci-artifacts containing the files of the
build to archive/deploy.
originally it was agreed to declare
NWL_IMAGE_EXTRACLASSES += "nwl-image-ci" accordingly for NWL but
this part was never implemented. Thus, the replacement with this
commit

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-18 11:38:23 +02:00
Marc Mattmüller c08549263b jobs/BuildAll,Common: added time measurement for building targets
the nightly builds the target in a loop. To have directly a job
log with an overview about the consumed build times, the nightly
prints the measured time after each target build.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-12 15:47:36 +02:00
Marc Mattmüller aa1e58f58c docker: integrate nginx credentials for docker-compose instance
The ansible instance is set up using ansible. The nginx credentials
were missing which let the sync of the sstate-cache mirror fail.
Thus, adapting the environment by adding the credentials.
Including the update of the documentation

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-12 14:38:48 +02:00
Marc Mattmüller 180fd4c3b3 jobs/buildAll: removed useless statement about cleaning repo
there is no repository to clean in the nightly pipeline.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-12 12:49:46 +02:00
Marc Mattmüller 6c83d053c8 jobs/Common,BuildAll: integrated Abort handling
if the nightly job is aborted any downstream job are aborted and
no further build target job is triggered. I case of an exception
the job is handled as failed with the note of exception but no
longer an error is thrown.
To get the handling properly done, the nightly triggers downstream
jobs with propagate flag set to false. The result handling is done
with <job>.getResult() and catching exceptions.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-12 11:55:38 +02:00
Marc Mattmüller 652739e0f0 jobs: added job parameter to force the source revision update
With the latest version where the nightly branch is forwarded to
the build job, the source revision update is no longer necessary.
But in case of a manual trigger of the build job it makes sense
to keep the functionality. Thus, an additional parameter is
introduced to the build target job. With it the nightly job and
the common files are updated too.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-11 15:49:16 +02:00
Marc Mattmüller 6277bc44bb jobs/updateSrcRev: fixed typo in PR/MR description
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-11 13:39:45 +02:00
Marc Mattmüller 8561fe8df1 doc/nwl-ci: updated with latest fixed, changes and findings
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-11 13:39:19 +02:00
Marc Mattmüller 00aa79a537 jobs/updateSrcRev: fixed java isse with german umlaut and variable name
german umlauts caused an issue and let the job crash. Thus the
umlaut is escaped with the unicode pattern. Additionally there was
a residual of an old variable name due to a refactoring.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-11 11:59:19 +02:00
Marc Mattmüller b93c05ff47 jobs/updateSrcRev: removed ./ from archiving artifact
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-11 11:32:29 +02:00
Marc Mattmüller 05a5786bb3 jobs/updateSrcRev: replaced ä by ue
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-11 11:19:39 +02:00
Marc Mattmüller b40c884da2 jobs/buildAll: build clean per default, added stage updating src rev
the update source revision job is important for a successful nightly
build- Thus the nightly job triggers this job and uses the returned
branch for the build target job.
With this we are able to pass the nightly without already merge a
pull request for new source revisions.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-11 10:45:20 +02:00
Marc Mattmüller c8b22f8cf5 jobs/updateSrcRev,common: added artifact with branch to use
In the nightly job there might be changes about some source
revisions and thus a new branch is created including a pull
request. The created branch is written to a file and archived
as artifact. This makes it possible to use this branch in a
nightly build for building the targets.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-11 10:26:30 +02:00
Marc Mattmüller d384814fb3 doc/researchAnsible: fixed some typos
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-05 14:58:55 +02:00
Marc Mattmüller 2a85c3ae25 doc/nwl-ci: added documentation about creating PR for src rev update
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-05 14:58:22 +02:00
Marc Mattmüller 73baa394f1 jobs/updateSrcRev: added creating a PR, set clean build as default
Implemented creating a pull request using REST API after pushing
the changes (new source revisions) to a nightly branch.
Additionally set the default value of the clean build parameter
to true. This is necessary to have per default a proper operation
of this job

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-05 14:16:39 +02:00
Marc Mattmüller 335bdead05 jobs/updateSrcRev: released commit and push part after the src rev update
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 13:39:18 +02:00
Marc Mattmüller 93eb7c12fa jobs/updateSrcRev: just make git reset HEAD after updating src rev
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 13:20:51 +02:00
Marc Mattmüller 86f82a6411 jobs/updateSrcRev: do not update src revision at clone step
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 12:56:42 +02:00
Marc Mattmüller c4d49cdc0e jobs/updateSrcRev: added cloning of the repository (was missing)
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 12:50:08 +02:00
Marc Mattmüller 3ff3dc9435 doc/nwl-ci: added update src rev part, branch seeding and nightly trigger
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 12:29:51 +02:00
Marc Mattmüller 2a56bbd0e8 jobs/updateSrcRevisions: added functionality to the skeleton
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 12:17:38 +02:00
Marc Mattmüller c79814899b jobs/common: update the yocto target (env.CI_IMAGE)
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 10:25:29 +02:00
Marc Mattmüller e0ae8a99e0 jobs/seed: moved environment getter into pipeline statement
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 10:02:19 +02:00
Marc Mattmüller bd63e87719 jobs/seed: use variable for branch and use environment syntax
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 09:41:58 +02:00
Marc Mattmüller a8e655a0d8 jobs/seed: use the parameter of the seeded branch for the seede jobs
The branch for the jobs to seed was set to main but we might want
to seed the branch develop. Thus the parameter of the seed job
replaces the hardcoded main.
I am not sure if this solution works but wihtout trying we won't
know :-D

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 09:26:49 +02:00
Marc Mattmüller 6779ef5846 jobs/seed: added nightly trigger with replaced pipelinetriggers
since job dsl plugin 1.77 the option 'triggers' is deprecated and
will be removed (causes problems:
https://issues.jenkins.io/browse/JENKINS-53775), see
https://github.com/jenkinsci/job-dsl-plugin/wiki/Migration#migrating-to-177
Thus the option triggers was replaced

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 09:07:34 +02:00
Marc Mattmüller 1840ec74ea docker/nwl-cicd.yaml: disable SCM poll for seed job
when seeding a branch other than the default one the poll
detects a SCM change and triggers right after a new seed job
which might revert the jobs seeded by the previous job run.
With this behavior it won't be possible to seed a specific
and run the intentionaly seeded jobs.
Thus, this commit disables SCM poll.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-09-04 08:38:40 +02:00
Marc Mattmüller be603724d2 jobs/common: refactored gitCheckout() for using more flexible
the function gitCheckout() holds the following steps:
- git clone unless done
- checking out a branch/tag/commit including rebase and updating
  submodules
- launching the update source revision script
- printing the submodule status
- printing the last 3 commits of the git history

But so far you could not use the steps separately. Thus, this
function was refactored by extracting some steps into own
functions so that they can be used easily.
Example: the job workspace is the cloned git repository and you
want to update the source revision hashes ;-P

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-29 14:40:31 +02:00
Marc Mattmüller 54161e2b5c jobs: added seed for new job UpdateSrcRevisions (incl. job skeleton)
A request was to have the update source revision part in a separate
job as in OEM Linux. This commit seeds a new job currently with
an empty skeleton.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-29 14:27:18 +02:00
Marc Mattmüller a138da021e jobs/buildAll: fixed build yocto job name and added clean-up of repo
with all the changes we have had regarding seed job integration in
the HAC instance the job name changed. Thus, fixing the name when
triggering the build job.
Additionally the script updating the source revisions leaves some
unstaged changes which is resolved by adding the clean-up of the
repository after building all targets.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-29 09:01:16 +02:00
Marc Mattmüller 705aa2c1ad jobs/buildTarget: added post step cleaning repository
the post step cleans the residuals from the update of the source
revisions.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-29 08:56:32 +02:00
Marc Mattmüller 246c859a35 jobs/common: added cleanup function restoring unstaged changes
with the script updating the source revisions some unstaged
changes reside in the repository. A following job will be unable
to succeed as the clone will fail. Thus, adding this function
which now can be used in the specific jobs.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-29 08:53:51 +02:00
Marc Mattmüller 3364307149 doc/nwl-ci: updated sections about integrating seed job in NWL HAC instance
On the ansible instance we saw that the seed job concept was working.
To be as felxible as possible as well on the HAC instance this
concept shall be integrated into this instance. Therefore this commit
holds all the steps so that the yocto build pipeline is starting to
build.
NOTE: currently the NWL project is in progress and the build is not
succeeding.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-28 18:22:42 +02:00
Marc Mattmüller fede53b9f5 Revert "jobs/seed: set git credentials back to admin_credentials"
This reverts commit c0086b4210.

There was a path mismatch for the ssh key. Thus this commit can be
reverted.
2023-08-28 17:31:55 +02:00
Marc Mattmüller c0086b4210 jobs/seed: set git credentials back to admin_credentials
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-28 16:55:18 +02:00
Marc Mattmüller 2efd3fdfa0 docker/nwl-cicd.yaml: added wipeWorkspace to seed job
withtout wiping the workspace seeding does not work correctly.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-28 16:32:40 +02:00
Marc Mattmüller 236f4b95d5 jobs: replaced admin_credentials with nmgit_credentials for git access
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-28 16:31:49 +02:00
Marc Mattmüller c582eb64d2 jobs/seed: set git credentials to admin_credentials and remove doc job
On the HAC instance, we currently do not want the documentation
job of the work of Marc Mattmueller, thus this job is removed.
Additionally the git credential ID differs between HAC- and ansible
instance. Therefore we set on this branch the credentials of the
HAC instance.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-28 14:01:35 +02:00
Marc Mattmüller f2a6d85321 jobs/seed: let the seeded jobs run anonymous
the reason is simple: There are currently still two AD's available
for HAC and Netmodule, i.e. the LDAP connection works only for
HAC employees.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-28 13:29:11 +02:00
Marc Mattmüller 7937b4bac7 doc/researchAnsible: updated due to enhancement of adding variable target
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-21 16:50:36 +02:00
Marc Mattmüller d2ce20787d playbooks: updated hosts with target variable
the playbooks shall only run on the hosts provided by the variable
target.
NOTE: the default target is set within the job templates

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-21 16:34:44 +02:00
Marc Mattmüller a3583e2b77 docker/plugins.txt: added plugin generic webhook trigger to list
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-21 15:47:57 +02:00
Marc Mattmüller a3edca0cb5 jobs/seed: removed the number in the description of the doc job
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-21 15:36:57 +02:00
Marc Mattmüller f4aeb3c3dd jobs/seed: adapted jenkins files and integrated split of nightly and yocto build
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-21 15:35:05 +02:00
Marc Mattmüller e645a68eac jobs/Common: added temporary source update tasks (sub-repositories)
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-21 11:58:12 +02:00
Marc Mattmüller 0afedfec06 jobs/Common: remove split of archiving built artifacts and manifests
The manifests will be added to the *.ci-artifacts file and thus
no longer need to be archived separately.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-21 11:58:00 +02:00
Marc Mattmüller 5768393db7 jobs/buildAll: released all legacy netmodule targets in nightly
The function isMachineSane() held so far a guard that only the
machine cn9130-cf-pro was built. According to the latest update
the legacy parts are now integrated and thus can be released to
be built.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-21 11:32:06 +02:00
Marc Mattmüller 64c24ccfcb doc/nwl-ci: added setup of hash equivalence server
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-21 11:28:21 +02:00
Marc Mattmüller 5313b9e4f1 jobs/Common: removed the duplicated file type for the temporary file name
Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>
2023-08-21 10:04:25 +02:00