forgejo/services/actions/workflows.go
Mathieu Fenniak c434b963b4 feat: implement "concurrency" block in Forgejo Actions at the workflow level (#9434)
Currently references a pre-release version of `code.forgejo.org/forgejo/runner/v11`, pending release of https://code.forgejo.org/forgejo/runner/pulls/1026.

Fixes #5914.

This PR is quite large, but it can be reviewed commit-by-commit in relatively small, logical chunks.

Adds support for workflows with a `concurrency` block, and submembers `group` and `cancel-in-progress`.  For example:
```
on:
  workflow_dispatch:
jobs:
  rust-checks:
    runs-on: debian-latest
    steps:
      - run: sleep 300
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: false
```

The concurrency block effectively ends up with four supported behaviors that users will want to choose from:
- Backwards compatibility / default -- if omitted completely, the existing Forgejo behavior will be implemented.  That behavior is that push and pull request synchronize events will cancel all previous runs on the same repository, branch, and workflow.
- Unlimited concurrency -- if the `cancel-in-progress` value is set to `false` and no `group` is provided, then the previously described Forgejo behavior will be disabled and an unlimited number of workflows can be executed simultaneously (to the maximum supported by the Forgejo Runner capacity).
- Queue-behind -- if a `group` is provided and `cancel-in-progress: false` is set, then every new action run with in the same repository with the same group value will be queued behind previous workflow runs, allowing only one workflow to execute at a time in the group, but allowing all workflows to finish naturally.
- Cancel-in-progress -- if a `group` is provided and `cancel-in-progress: true` is set, then every new action run with in the same repository with the same group value will cause previously queued or running runs to be cancelled, allowing only one workflow to execute at a time in the group, but preferring execution of the most recent workflow.

Both the `group` and `cancel-in-progress` values can access values from the `github`, `inputs` and `vars` context for dynamic behavior.

## Checklist

The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).

### Tests

- I added test coverage for Go changes...
  - [x] in their respective `*_test.go` for unit tests.
  - [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
  - [ ] in `web_src/js/*.test.js` if it can be unit tested.
  - [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).

### Documentation

- [x] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
  - https://codeberg.org/forgejo/docs/pulls/1513
- [ ] I did not document these changes and I do not expect someone else to do it.

### Release notes

- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.

<!--start release-notes-assistant-->

## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
  - [PR](https://codeberg.org/forgejo/forgejo/pulls/9434): <!--number 9434 --><!--line 0 --><!--description aW1wbGVtZW50ICJjb25jdXJyZW5jeSIgYmxvY2sgaW4gRm9yZ2VqbyBBY3Rpb25zIGF0IHRoZSB3b3JrZmxvdyBsZXZlbA==-->implement "concurrency" block in Forgejo Actions at the workflow level<!--description-->
<!--end release-notes-assistant-->

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9434
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
2025-10-03 18:43:02 +02:00

234 lines
6.3 KiB
Go

// Copyright The Forgejo Authors.
// SPDX-License-Identifier: MIT
package actions
import (
"bytes"
"context"
"errors"
"fmt"
"strconv"
actions_model "forgejo.org/models/actions"
"forgejo.org/models/perm"
"forgejo.org/models/perm/access"
repo_model "forgejo.org/models/repo"
"forgejo.org/models/user"
"forgejo.org/modules/actions"
"forgejo.org/modules/git"
"forgejo.org/modules/json"
"forgejo.org/modules/setting"
"forgejo.org/modules/structs"
"forgejo.org/modules/util"
"forgejo.org/modules/webhook"
"forgejo.org/services/convert"
"code.forgejo.org/forgejo/runner/v11/act/jobparser"
act_model "code.forgejo.org/forgejo/runner/v11/act/model"
)
type InputRequiredErr struct {
Name string
}
func (err InputRequiredErr) Error() string {
return fmt.Sprintf("input required for '%s'", err.Name)
}
func IsInputRequiredErr(err error) bool {
_, ok := err.(InputRequiredErr)
return ok
}
type Workflow struct {
WorkflowID string
Ref string
Commit *git.Commit
GitEntry *git.TreeEntry
}
type InputValueGetter func(key string) string
func (entry *Workflow) Dispatch(ctx context.Context, inputGetter InputValueGetter, repo *repo_model.Repository, doer *user.User) (r *actions_model.ActionRun, j []string, err error) {
content, err := actions.GetContentFromEntry(entry.GitEntry)
if err != nil {
return nil, nil, err
}
wf, err := act_model.ReadWorkflow(bytes.NewReader(content), false)
if err != nil {
return nil, nil, err
}
fullWorkflowID := ".forgejo/workflows/" + entry.WorkflowID
title := wf.Name
if len(title) < 1 {
title = fullWorkflowID
}
inputs := make(map[string]string)
inputsAny := make(map[string]any)
if workflowDispatch := wf.WorkflowDispatchConfig(); workflowDispatch != nil {
for key, input := range workflowDispatch.Inputs {
val := inputGetter(key)
if len(val) == 0 {
val = input.Default
if len(val) == 0 {
if input.Required {
name := input.Description
if len(name) == 0 {
name = key
}
return nil, nil, InputRequiredErr{Name: name}
}
continue
}
} else if input.Type == "boolean" {
// Since "boolean" inputs are rendered as a checkbox in html, the value inside the form is "on"
val = strconv.FormatBool(val == "on")
}
inputs[key] = val
inputsAny[key] = val
}
}
if int64(len(inputs)) > setting.Actions.LimitDispatchInputs {
return nil, nil, errors.New("to many inputs")
}
jobNames := util.KeysOfMap(wf.Jobs)
payload := &structs.WorkflowDispatchPayload{
Inputs: inputs,
Ref: entry.Ref,
Repository: convert.ToRepo(ctx, repo, access.Permission{AccessMode: perm.AccessModeNone}),
Sender: convert.ToUser(ctx, doer, nil),
Workflow: fullWorkflowID,
}
p, err := json.Marshal(payload)
if err != nil {
return nil, nil, err
}
notifications, err := wf.Notifications()
if err != nil {
return nil, nil, err
}
run := &actions_model.ActionRun{
Title: title,
RepoID: repo.ID,
Repo: repo,
OwnerID: repo.OwnerID,
WorkflowID: entry.WorkflowID,
TriggerUserID: doer.ID,
TriggerUser: doer,
Ref: entry.Ref,
CommitSHA: entry.Commit.ID.String(),
Event: webhook.HookEventWorkflowDispatch,
EventPayload: string(p),
TriggerEvent: string(webhook.HookEventWorkflowDispatch),
Status: actions_model.StatusWaiting,
NotifyEmail: notifications,
}
vars, err := actions_model.GetVariablesOfRun(ctx, run)
if err != nil {
return nil, nil, err
}
err = ConfigureActionRunConcurrency(wf, run, vars, inputsAny)
if err != nil {
return nil, nil, err
}
if run.ConcurrencyType == actions_model.CancelInProgress {
if err := CancelPreviousWithConcurrencyGroup(
ctx,
run.RepoID,
run.ConcurrencyGroup,
); err != nil {
return nil, nil, err
}
}
jobs, err := jobParser(content, jobparser.WithVars(vars))
if err != nil {
return nil, nil, err
}
return run, jobNames, actions_model.InsertRun(ctx, run, jobs)
}
func GetWorkflowFromCommit(gitRepo *git.Repository, ref, workflowID string) (*Workflow, error) {
ref, err := gitRepo.ExpandRef(ref)
if err != nil {
return nil, err
}
commit, err := gitRepo.GetCommit(ref)
if err != nil {
return nil, err
}
entries, err := actions.ListWorkflows(commit)
if err != nil {
return nil, err
}
var workflowEntry *git.TreeEntry
for _, entry := range entries {
if entry.Name() == workflowID {
workflowEntry = entry
break
}
}
if workflowEntry == nil {
return nil, errors.New("workflow not found")
}
return &Workflow{
WorkflowID: workflowID,
Ref: ref,
Commit: commit,
GitEntry: workflowEntry,
}, nil
}
// Sets the ConcurrencyGroup & ConcurrencyType on the provided ActionRun based upon the Workflow's `concurrency` data,
// or appropriate defaults if not present.
func ConfigureActionRunConcurrency(workflow *act_model.Workflow, run *actions_model.ActionRun, vars map[string]string, inputs map[string]any) error {
concurrencyGroup, cancelInProgress, err := jobparser.EvaluateWorkflowConcurrency(
workflow.RawConcurrency, generateGiteaContextForRun(run), vars, inputs)
if err != nil {
return fmt.Errorf("unable to evaluate workflow `concurrency` block: %w", err)
}
if concurrencyGroup != "" {
run.SetConcurrencyGroup(concurrencyGroup)
} else {
run.SetDefaultConcurrencyGroup()
}
if cancelInProgress == nil {
// Maintain compatible behavior from before concurrency groups were implemented -- if `cancel-in-progress`
// isn't defined in the workflow, cancel on push & PR sync events.
if run.Event == webhook.HookEventPush || run.Event == webhook.HookEventPullRequestSync {
run.ConcurrencyType = actions_model.CancelInProgress
} else {
run.ConcurrencyType = actions_model.UnlimitedConcurrency
}
} else if *cancelInProgress {
run.ConcurrencyType = actions_model.CancelInProgress
} else if concurrencyGroup == "" {
// A workflow has explicitly listed `cancel-in-progress: false`, but has *not* provided a concurrency group. In
// this case we want to trigger a different concurrency behavior -- we won't cancel in-progress builds (we were
// asked not to), we won't queue behind other builds (we weren't given a concurrency group so it's reasonable to
// assume the user doesn't want a concurrency limit).
run.ConcurrencyType = actions_model.UnlimitedConcurrency
} else {
run.ConcurrencyType = actions_model.QueueBehind
}
return nil
}