ASF Build and Supported Services

Introduction

Here is a summary of The ASF's 'build' (CI/CD) and related services, those hosted at The ASF and those supported externally, directly or via integration.

Note that 'supported' means that Infra provides the service or provides access to the service. It does not mean that a particular service is considered an official 'release channel'.

• Jenkins
• Buildbot
• Apache Gump
• GitHub Actions
• Artifactory
• Nexus
• Nightlies
• DockerHub
• Gradle Enterprise
• Sonarcloud

Jenkins¶

Infra operates a Cloudbees Core cluster comprising a single Operations Center and several Controllers. These comprise a shared Controller ci-builds instance which many projects share, and some individual project Controllers (listed on the main Operations Center Controllers page).

Access control

Jenkins is LDAP enabled and so all ASF Committers have login access. Project level access is then applied at a Controller level for the project Controllers and at a Folder level for the shared ci-builds shared instance. If your project has a Folder here, you already have access there to edit/create builds. If you are a project without a Folder then an Infra Jira ticket will get you the Folder you need.

Integrations

Jenkins has integrations with GitHub Actions (GHA), nightlies.apache.org, and Nexus for snapshot deployments. More information on this will appear here soon.

More information

The Infra Confluence wiki space has more information about Jenkins.

Buildbot¶

ASF Infra runs an instance of the open source Buildbot software. Buildbot runs on a single controller and hosts many Ubuntu and Windows nodes, or 'workers' in current Buildbot terminology.

Adding/configuring Buildbot jobs

Projects can add/edit/remove their configuration files via Subversion or Git.

• Subversion
• Git/GitHub

Use the standard naming $projectname.py for your config file. You may place multiple build jobs in this file.

Once committed, changes should be picked up automatically. If this does not happen, please contact Infrastructure.

Access control

Log in to ci2.apache.org with your LDAP creds. Use $username@apache.org for the user and your LDAP password.

Integrations

Buildbot integrates with nightlies.apache.org and Nexus for snapshot deployments.

IRC: Currently irc integration is disabled. We have an open Jira ticket tracking this.

More Information

The Infra Confluence wiki has more about Buildbot.

Apache Gump¶

The Apache Gump project runs this instance. Projects are welcome to ask them directly for access.

Gump is a cross-project continuous integration server. It is different from the "usual" CI servers in that it expects the individual project builds to succeed; its purpose is to check the integration of a project with the latest code rather than with a fixed version of the project's dependencies. If you want a more traditional nightly build server, Gump is not for you.

Use Gump if you want to know when a change in your dependencies breaks your project or when your changes have broken other projects.

Gump is written in Python and supports several build tools (including shell scripts, GNU make, Ant, Maven and NAnt) and version control systems (svn, CVS, git, bzr, hg, darcs and Perforce). The Apache installation of Gump builds many ASF projects and their dependencies.

Gump started in the Java part of the Foundation but also builds projects like APR, HTTPd and log4net.

GitHub Actions¶

The ASF supports and recommends the use of GitHub Actions (GHA).

Integrations

Infra makes use of an 'allow' list to allow Marketplace actions for your workflows. File an Infra Jira ticket if you need to have one added to the list.

GHA integrates with nightlies.apache.org, Dockerhub, Artifactory, Jenkins and more via credentials supplied as GitHub Secrets.

Access control

All Committers have access to GHA and their workflows via commit using their LDAP credentials.

More information

In addition to the official GitHub documentation, Infra has placed some notes on a Confluence wiki page.

Artifactory¶

The folks at Jfrog provide us an instance of Artifactory for all ASF projects to use. Projects are free to publish debs, rpms, Helm Charts and more. Use a Jira ticket to ask Infra to set up the project's initial repository type.

Access Control

Access is via LDAP credentials. Infra needs to set up the project's initial repository/group access.

Integrations

Many possibilities here for upload/download via the use of a PAT token.

More information

A Confluence wiki page will soon contain some more Artifactory information.

Nexus¶

The ASF has a Nexus instance at repository.apache.org , maintained by the Maven community in conjunction with people from Sonatype.

Access control

The instance has committer-only access to push to staging and to snapshots via their LDAP credentials, and promotion from staging to release. Once released, the artifacts get synced over to Maven Central.

Snapshots and staged artifacts do not get synced.

Integrations

Pushing snapshots can be done via GHA, Buildbot and Jenkins.

Experimental signing and push to Staging (in readiness for a manual promotion to release) is being tested.

More information

See Publishing Maven artifacts.

Nightlies¶

Infra runs a server at nightlies.apache.org where projects can store various build output such as snapshot builds, versioned website documentation, artifacts (jars, etc.), and apidocs. Jenkins, Buildbot and GitHub Actions all integrate with nightlies. Committers also have PUT access via their LDAP credentials.

DockerHub¶

The ASF has an 'apache' account at DockerHub for all projects to use.

Access control

Committers need to sign up for a personal account, then create a Jira ticket asking Infra to set up their access. The ticket should state

• dockerhub_id : asf_id
• project name
• repository name(s)
• team name

Infra adds DockerHub user IDs to a project 'team', which has read/write access to the project repositories.

Integrations

The ASF has tokens/credentials in GHA, Jenkins and Buildbot for projects to use when pushing to Dockerhub using a role account.

More information

• GitHub Actions to DockerHub
• Jenkins and Dockerhub

Gradle Enterprise¶

Gradle is a suite of acceleration and analytics technologies for CI/CD systems to help projects identify and analyze trends while optimizing build resources. The result is faster builds, with fewer failures, The ASF instance of Gradle enterprise is at ge.apache.org.

More information is available at the Gradle page.

Sonarcloud¶

The ASF has an 'apache' account at sonarcloud.io where projects can have their code analyzed.

Access control

Committers must log in to Sonarcloud with their GitHub ID. In addition you must have your ASF account and your GitHub accounts linked so that you then appear as a member of the ASF organization on GitHub. That is how Sonarcloud identifies you as belonging to the 'apache' organization on their system.

Integrations

The ASF has auth tokens available under a role account for use via GHA and Jenkins.

More information

SonarCloud for ASF projects