Commit Graph

902 Commits (forgejo)

Author SHA1 Message Date
Earl Warren e66f0e153b
Merge branch 'rebase-forgejo-f3' into wip-forgejo 2023-09-25 21:58:15 +02:00
Earl Warren c340e22197
[F3] Forgejo driver and CLI
user, topic, project, label, milestone, repository, pull_request,
release, asset, comment, reaction, review providers

Signed-off-by: Earl Warren <>

Preserve file size when creating attachments

Introduced in c6f5029708

repoList.LoadAttributes has a ctx argument now

Rename `repo.GetOwner` to `repo.LoadOwner`


upgrade to the latest gof3

(cherry picked from commit c770713656)

[F3] ID remapping logic is in place, remove workaround

(cherry picked from commit d0fee30167)

[F3] it is experimental, do not enable by default

(cherry picked from commit de325b21d0)
(cherry picked from commit 547e7b3c40)
(cherry picked from commit 820df3a56b)
(cherry picked from commit eaba87689b)
(cherry picked from commit 1b86896b3b)
(cherry picked from commit 0046aac1c6)
(cherry picked from commit f14220df8f)
(cherry picked from commit 559b731001)
(cherry picked from commit 801f7d600d)
(cherry picked from commit 6aa76e9bcf)
(cherry picked from commit a8757dcb07)

[F3] promote F3 users to matching OAuth2 users on first sign-in

(cherry picked from commit bd7fef7496)
(cherry picked from commit 07412698e8)
(cherry picked from commit d143e5b2a3)

[F3] upgrade to gof3 50a6e740ac04

Add new methods GetIDString() & SetIDString() & ToFormatInterface()
Change the prototype of the fixture function

(cherry picked from commit d7b263ff8b)
(cherry picked from commit b3eaf2249d)
(cherry picked from commit d492ddd9bb)

[F3] add GetLocalMatchingRemote with a default implementation

(cherry picked from commit 0a22015039)
(cherry picked from commit f1310c38fb)
(cherry picked from commit deb68552f2)

[F3] GetLocalMatchingRemote for user

(cherry picked from commit e73cb837f5)
(cherry picked from commit a24bc0b85e)
(cherry picked from commit 846a522ecc)

[F3] GetAdminUser now has a ctx argument

(cherry picked from commit 37357a92af)
(cherry picked from commit 660bc1673c)
(cherry picked from commit 72d692a767)

[F3] introduce UserTypeF3

To avoid conflicts should UserTypeRemoteUser be used differently by Gitea

(cherry picked from commit 6de2701bb3)

[F3] user.Put: idempotency

(cherry picked from commit 821e38573c)
(cherry picked from commit f7638f5414)

[F3] upgrade to urfave v2

(cherry picked from commit cc3dbdfd1d)

[F3] update gof3

(cherry picked from commit 2eee960751)

[F3] move f3 under forgejo-cli

* simplify the tests by re-using the forgejo-cli helpers to capture
  the output
* unify CmdF3 to be structured in the same way CmdActions is

(cherry picked from commit 4c9fe58b74)

[F3] replace f3 with forgejo-cli f3

(cherry picked from commit 7ba7ceef1b)

[F3] s/ListOptions/Paginator/

[F3] user: add unit tests

[F3] user comparison of F3 managed users is on content

[F3] issue: add unit tests

[F3] gof3 now has one more argument to Put()

[F3] re-use gof3 unit tests for the driver

(cherry picked from commit af7ee6200c)

	because of some code removed in forgejo-development, trivial
	context conflict resolution

[F3] more idempotent tests (#1275)

Co-authored-by: Loïc Dachary <>
Co-committed-by: Loïc Dachary <>

[F3] tests: do SQL update if nothing changes

[F3] tests comment idempotence

[F3] tests milestone idempotence

[F3] tests pull_request idempotence

[F3] tests release idempotence

[F3] tests asset idempotence

[F3] tests project idempotence

[F3] tests review idempotence

(cherry picked from commit 91038bb4e8)
(cherry picked from commit a7d2a65214)
(cherry picked from commit 59a17e5a34)

[F3] sub command of forgejo-cli

(cherry picked from commit 4d098e9b83)

[F3] implement --quiet, --debug, --verbose

(cherry picked from commit 82e2e17b45)

[F3] fix off by one error when importing repositories

(cherry picked from commit 31689b1397)

[F3] upgrade gof3

(cherry picked from commit 87b8cfe5a1)

[F3] set the logger for all drivers

The logger is set for the local Forgejo driver only. Even when --debug
is specified, the other drivers do not display debug
information. Use the gof3 context to set the logger for all of them at

(cherry picked from commit 8aa7de8ba0)

[F3] the closed date of an issue may be nil

(cherry picked from commit 93d3eaf0b5)

[F3] update gof3 to support system users

there now is a workaround to hardcode system users when they are not
supported by the API

(cherry picked from commit 915484daa7)
(cherry picked from commit b47ac73b8a)

[F3] upgrade gof3 with a version that can deal with system users

When they are missing from what the API returns, it will replace the
missing user with the Ghost user instead of leaving it be a null pointer.

(cherry picked from commit 9eeeab7f8e)

[F3] tests do not need a running Forgejo server

(cherry picked from commit b2b9749ac9)

[F3] upgrade gof3 to correctly fetch PRs from forks

(cherry picked from commit d2448de302)

[F3] upgrade gof3 to resolve incorrect object ID mappings

(cherry picked from commit af8c9de89f)

[F3] mirroring a repository needs --mirror instead of --bare

(cherry picked from commit 9941131920)

[F3] PR create the pull/N/head from the original PR head

It is incorrect to assume they are identical because:

* the IDs of a PR may be remapped and pull/N/head will become
* the head of a remote fork is a branch named after the fork

(cherry picked from commit 9c220bf23e)

[F3] gof3 upgrade with non overlapping PR references

(cherry picked from commit f1b2f82e7e)

[F3] refactor issue insertion to preserve the creation date

issues_model.NewIssue() is convenient but will override the creation
date. Update the database instead.

(cherry picked from commit 729f4be1e4)

[F3] gof3 upgrade to get performance improvements

(cherry picked from commit 996ac35d4d)
(cherry picked from commit 906e9eb3f5)
2023-09-25 20:46:18 +02:00
Rychart Redwerkz 6d4f25799f
[BRANDING] Rebrand dump log
(cherry picked from commit b92a65a23f)
(cherry picked from commit eaeb9f183f)
(cherry picked from commit cdbd0ebe5e)
(cherry picked from commit 8aaa4e3f4d)
(cherry picked from commit 6d7d3f048e)
(cherry picked from commit 1b6a2971ad)
(cherry picked from commit 4f7f35043d)
(cherry picked from commit 495a4351b1)
(cherry picked from commit 12d89b03a9)
(cherry picked from commit 476d49ff11)
(cherry picked from commit d6e95828c3)
(cherry picked from commit d043e8d69d)
(cherry picked from commit b060273442)
(cherry picked from commit 20437285c5)
(cherry picked from commit 0bc9eee5f3)
(cherry picked from commit b9f843776f)
(cherry picked from commit e6d64de500)
(cherry picked from commit 5ce2ff2f8b)
(cherry picked from commit 659677cf88)
(cherry picked from commit 648a5d944e)
(cherry picked from commit a77c8be704)
(cherry picked from commit 012cb57046)
(cherry picked from commit 2da63c976e)
(cherry picked from commit 792ce8b1b7)
(cherry picked from commit a4b3244712)
(cherry picked from commit 9ffc3571f1)
(cherry picked from commit 6aadfd0120)
(cherry picked from commit bdfc590b56)
2023-09-25 17:23:25 +02:00
Loïc Dachary a01f5c3475
FORGEJO_* environment variables are set to the corresponding GITEA_*
variable when the cli starts. This approach is intended to minimize
the conflicts on rebase. All occurences of GITEA_* are left untouched
in the codebase and they are only changed to FORGEJO_* if exposed to
the user.

(cherry picked from commit e466f9d10e)
(cherry picked from commit e33e95931b)
(cherry picked from commit 2cfc6519b7)
(cherry picked from commit af8864373a)
(cherry picked from commit a0550ff339)
(cherry picked from commit 24dc0a5191)
(cherry picked from commit e255eea2b4)
(cherry picked from commit 0c4f5afa7a)
(cherry picked from commit 42fce708d0)
(cherry picked from commit e7278c3c22)
(cherry picked from commit 0fb9ed7e0e)
(cherry picked from commit a98308aa4d)
(cherry picked from commit b8695fcbe0)
(cherry picked from commit 4aee8719f5)
(cherry picked from commit 1c503c1ba7)
(cherry picked from commit cf1ed8551e)
(cherry picked from commit c52459b088)
(cherry picked from commit 92cac277b2)
(cherry picked from commit 2c744f1118)
(cherry picked from commit 989a98a8c5)
(cherry picked from commit 9cf7052bd4)
(cherry picked from commit a13c9667e0)

(cherry picked from commit a9c3cf060d)
(cherry picked from commit eb6d904b23)
(cherry picked from commit fd4878e487)
(cherry picked from commit cc2cab9241)
(cherry picked from commit be523c63db)
(cherry picked from commit 254d1a2f9a)
(cherry picked from commit 1eecf4200e)
2023-09-25 17:23:25 +02:00
TomZ dd6fbbf332
[BRANDING] cosmetic s/Gitea/Forgejo/ in logs, messages, etc.
As the docs of codeberg refer to the strings printed by the Forgejo
ssh servers, this is user-facing and is nice to update to the new
product name.

(cherry picked from commit 103991d73f)
(cherry picked from commit 2a0d3f85f1)
(cherry picked from commit eb2b4ce388)
(cherry picked from commit 0998b51716)

[BRANDING] forgejo log message

(cherry picked from commit d51a046ebe)
(cherry picked from commit d66e1c7b6e)
(cherry picked from commit b5bffe4ce8)
(cherry picked from commit 3fa776d856)
(cherry picked from commit 18d064f472)
(cherry picked from commit c95094e355)
(cherry picked from commit 5784290bc4)
(cherry picked from commit aee336886b)
(cherry picked from commit ec2f60b516)
(cherry picked from commit 7af742a284)
(cherry picked from commit f279e2a264)
(cherry picked from commit fd38cfb14e)
(cherry picked from commit 64c8226618)
(cherry picked from commit b546fb2304)
(cherry picked from commit ad10202177)
(cherry picked from commit c89cab9c2b)
(cherry picked from commit 9579322ec2)
(cherry picked from commit 16b44ad18d)
(cherry picked from commit 2571ff703b)
(cherry picked from commit ad61d9ce9b)
(cherry picked from commit 9b2c45d4d3)
(cherry picked from commit ed01b79a59)
(cherry picked from commit d040b66427)
(cherry picked from commit ffe0bbea48)
(cherry picked from commit 4c1b2c409b)
(cherry picked from commit 3d8338ed10)
(cherry picked from commit a92f044ea9)

[BRANDING] link to instead of

(cherry picked from commit 3efafd0e08)
(cherry picked from commit 148185e34b)
(cherry picked from commit 834e264698)
(cherry picked from commit e72fa6eb1e)

[BRANDING] link to instead of

Fix the link that was 404.

(cherry picked from commit ae515d7258)
(cherry picked from commit facc2367f0)
(cherry picked from commit 25784b9f21)
(cherry picked from commit 2efc6138d9)
(cherry picked from commit b9d0871631)
(cherry picked from commit f0446e51b9)
(cherry picked from commit 1638aa67fb)
(cherry picked from commit 290db6a018)
(cherry picked from commit 89b87cf542)
(cherry picked from commit 656ed94962)
(cherry picked from commit 036f879f96)
(cherry picked from commit 69eea35f81)
(cherry picked from commit b72e3f4a92)
(cherry picked from commit af606b8574)
(cherry picked from commit 7e47f8135c)
(cherry picked from commit 0e5218cc53)
(cherry picked from commit 7c2a20a528)
(cherry picked from commit 4e94006363)
(cherry picked from commit e47cdfc43f)
(cherry picked from commit 1dcb3e1da4)
(cherry picked from commit 67367c4e0f)
(cherry picked from commit 252087d1ff)
(cherry picked from commit f5977a43e5)

(cherry picked from commit 594938eb15)
(cherry picked from commit 0257d038a7)
(cherry picked from commit 72821dd140)

[BRANDING] s/gitea/forgejo/ in HTML placeholders

Replaced Gitea branding with Forgejo for input placeholders

Closes: #686

(cherry picked from commit 6160d37ca9)
(cherry picked from commit df61138c7e)
(cherry picked from commit 1f30566c3f)
(cherry picked from commit 539bb825f5)
(cherry picked from commit bee0f66c86)
(cherry picked from commit 60ad005c95)
(cherry picked from commit 282e26222e)
(cherry picked from commit f9ca551f3d)
(cherry picked from commit b2e04b04c3)
(cherry picked from commit c8f395a03c)
(cherry picked from commit 0d58ce49ae)
(cherry picked from commit c602ddf91e)
(cherry picked from commit 029e37271e)
(cherry picked from commit fdaa96b3cc)
(cherry picked from commit 515d99e27d)
(cherry picked from commit da73274ba1)
(cherry picked from commit ce90b696a0)
(cherry picked from commit b6bf98763b)
(cherry picked from commit 5b380d22d7)

[BRANDING] How to start a runner: URL to Actions admin documentation

(cherry picked from commit da91799e6f)
(cherry picked from commit 28231663b6)
(cherry picked from commit 533a90345b)
(cherry picked from commit 6a0e4e55dd)
(cherry picked from commit f47cd611c6)
(cherry picked from commit 001264b784)
(cherry picked from commit e4099e9bb9)
(cherry picked from commit 3a1885649f)
(cherry picked from commit c42802c710)
(cherry picked from commit a611ce8d6d)
(cherry picked from commit a3d7d10a80)
(cherry picked from commit 52adde671f)
(cherry picked from commit c9a3820fef)
(cherry picked from commit dce40997c9)
(cherry picked from commit 312a6b92f3)

[BRANDING] package templates & links

- Change Gitea to Forgejo where necessary.
- Point all documentation to Forgejo's documentation.
- Resolves #992

(cherry picked from commit d0b78a6ede)
(cherry picked from commit e2382f30ba)
(cherry picked from commit c41cf05a33)
(cherry picked from commit 797e598ae7)
(cherry picked from commit 970031a1c2)
(cherry picked from commit 0c1180e2e1)

(cherry picked from commit 42ac9ff2ab)
(cherry picked from commit e390000bce)
(cherry picked from commit 56a437b29b)


[BRANDING] s/Gitea/Forgejo/ in user visible help & comments

- Modify the README of the docker directory to point to the relevant
docker files and documentation for Forgejo.

(cherry picked from commit aca6371215)
(cherry picked from commit 0ba96b1bc4)
(cherry picked from commit 5c8e6b53f1)

(cherry picked from commit b3121c8004)
(cherry picked from commit 607f870416)
(cherry picked from commit 191d96afe4)

[BRANDING] healthcheck/check.go

(cherry picked from commit d703a236ce)
(cherry picked from commit d84ce3ff20)
(cherry picked from commit 2dbb844606)
(cherry picked from commit 14d3ae7e3a)

[BRANDING] s/Gitea/Forgejo/g in CLI output

(cherry picked from commit 7543c126bb)
(cherry picked from commit b66f422fc3)
(cherry picked from commit a81e4e46f3)

[BRANDING] Gitea->Forgejo in mailer code

(cherry picked from commit b91afea4ff)
(cherry picked from commit 5d7428167c)
(cherry picked from commit ed8101ba6c)

[BRANDING] use 'Forgejo' for Discord, Packagist, and Slack webhooks


(cherry picked from commit 7dc3a05f5b)
(cherry picked from commit 133f2fc6cc)

[BRANDING] cmd/manager.go

(cherry picked from commit d1dba2c79d)

[BRANDING] pyproject.toml

(cherry picked from commit 7e8c868db2)
(cherry picked from commit 2395995c8b)
2023-09-25 17:22:35 +02:00
Earl Warren 588e5d552f
[CLI] implement forgejo-cli
(cherry picked from commit 2555e315f7)
(cherry picked from commit 51b9c9092e)

[CLI] implement forgejo-cli (squash) support initDB

(cherry picked from commit 5c31ae602a)
(cherry picked from commit bbf76489a7)

	because of d0dbe52e76
	upgrade to
(cherry picked from commit b6c1bcc008)

[CLI] implement forgejo-cli actions

(cherry picked from commit 08be2b226e)
(cherry picked from commit b6cfa88c6e)
(cherry picked from commit 59704200de)

[CLI] implement forgejo-cli actions generate-secret

(cherry picked from commit 6f7905c8ec)
(cherry picked from commit e085d6d273)

[CLI] implement forgejo-cli actions generate-secret (squash) NoInit

(cherry picked from commit 962c944eb2)

[CLI] implement forgejo-cli actions register

(cherry picked from commit 2f95143000)
(cherry picked from commit 42f2f8731e)

[CLI] implement forgejo-cli actions register (squash) no private

Do not go through the private API, directly modify the database

(cherry picked from commit 1ba7c0d39d)

[CLI] implement forgejo-cli actions

(cherry picked from commit 6f7905c8ec)
(cherry picked from commit e085d6d273)

[CLI] implement forgejo-cli actions generate-secret (squash) NoInit

(cherry picked from commit 962c944eb2)
(cherry picked from commit 4c121ef022)

(cherry picked from commit 36997a48e3)

[CLI] implement forgejo-cli actions (squash) restore --version

(cherry picked from commit 9739eb52d8)

[CI] implement forgejo-cli (squash) the actions subcommand needs config

(cherry picked from commit def638475122a26082ab3835842c84cd03839154)

(cherry picked from commit a1758a3910)
(cherry picked from commit 935fa650c7)
(cherry picked from commit cd21026bc9)
(cherry picked from commit 1700b8973a)
(cherry picked from commit 1def42a379)
(cherry picked from commit 839d97521d)
(cherry picked from commit fd8c13be6b)
2023-09-25 15:24:38 +02:00
JakobDev c548dde205
More refactoring of `db.DefaultContext` (#27083)
Next step of #27065
2023-09-15 06:13:19 +00:00
JakobDev 76659b1114
Reduce usage of `db.DefaultContext` (#27073)
Part of #27065

This reduces the usage of `db.DefaultContext`. I think I've got enough
files for the first PR. When this is merged, I will continue working on

Considering how many files this PR affect, I hope it won't take to long
to merge, so I don't end up in the merge conflict hell.


Co-authored-by: wxiaoguang <>
2023-09-14 17:09:32 +00:00
MomentQYC 598465eb43
fix: typo (#27009) 2023-09-11 09:30:18 +00:00
JonRB 99a5595408
clarify aspects of the dump command (#26887)
clarify aspects of the dump command

Possibly closes #26862


Co-authored-by: delvh <>
2023-09-03 14:44:01 -04:00
mainboarder c533991519
Expanded minimum RSA Keylength to 3072 (#26604)
German Federal Office for Information Security requests in its technical
guideline BSI TR-02102-1 RSA Keylength not shorter than 3000bits
starting 2024, in the year 2023 3000bits as a recommendation. Gitea
should request longer RSA Keys by default in favor of security and drop
old clients which do not support longer keys.
- Page 19, Table 1.2


Co-authored-by: wxiaoguang <>
2023-08-28 00:53:16 +00:00
wxiaoguang ed1be4ca68
Handle base64 decoding correctly to avoid panic (#26483)
Fix the panic if the "base64 secret" is too long.
2023-08-14 10:30:16 +00:00
wxiaoguang 82ea557dd3
Fix stderr usages (#26477) 2023-08-13 20:49:30 +08:00
wxiaoguang c28e29fd94
Refactor tests (#26464)
1. Give the global variable clear names
2. Use generic parameter for `onGiteaRun`
2023-08-12 16:30:16 +00:00
wxiaoguang f42cd4561b
Call git.InitSimple for runRepoSyncReleases (#26396)
Fix #26394

Otherwise, the git module is not initialized and it doesn't respect the
"timeout" config in app.ini
2023-08-10 22:42:48 +00:00
wxiaoguang d92b4cd093
Fix incorrect CLI exit code and duplicate error message (#26346)
Follow the CLI refactoring, and add tests.
2023-08-05 23:36:45 +08:00
wxiaoguang 4f513474dc
Improve CLI and messages (#26341)
Follow the CLI refactoring

1. Remove the "checkCommandFlags" helper
2. Unify the web startup message, make them have consistent names as `./gitea help`
3. Fine tune some other messages (see the diff)
2023-08-05 13:24:49 +00:00
Yarden Shoham d36ddfe26c
Fix CLI allowing creation of access tokens with existing name (#26071)
We are now:
- Making sure there is no existing access token with the same name
- Making sure the given scopes are valid (we already did this before but
now we have a message)

The logic is mostly taken from
a12a5f3652/routers/api/v1/user/app.go (L101-L123)

Closes #26044

Signed-off-by: Yarden Shoham <>
2023-07-25 23:06:11 +00:00
wxiaoguang 1ce51a55e3
Improve "gitea doctor" sub-command and fix "help" commands (#26072)
Replace #21790

And close #25965 by the way (it needs a separate fix for 1.20)

Major changes:

1. Move "gitea convert" to "gitea doctor conver". The old "gitea doctor"
still works as a hidden sub-command (to avoid breaking)
2. Do not write "doctor.log" by default, it's not useful in most cases
and causes bugs like 25965
3. Improve documents
4. Fix the "help" commands. Before, the "./gitea doctor" can't show the
sub-command help correctly (regression of the last cli/v2 refactoring)

After this PR:

./gitea help # show all sub-commands for the app
./gitea doctor # show the sub-commands for the "doctor"
./gitea doctor help # show the sub-commands for the "doctor", as above
2023-07-25 14:38:27 +00:00
wxiaoguang 478f36a538
Add missing default value for some Bool cli flags (#26082)
In #25959 I forgot to add default values to some Bool flags (which were
BoolT in cli/v1, BoolT means default to be true)

This PR adds the default "Value" for them.

./cmd/manager_logging.go:							}, cli.BoolTFlag{
./cmd/manager_logging.go-								Name:  "rotate, r",
./cmd/manager_logging.go-								Usage: "Rotate logs",
./cmd/manager_logging.go:							}, cli.BoolTFlag{
./cmd/manager_logging.go-								Name:  "daily, d",
./cmd/manager_logging.go-								Usage: "Rotate logs daily",
./cmd/manager_logging.go:							}, cli.BoolTFlag{
./cmd/manager_logging.go-								Name:  "compress, z",
./cmd/manager_logging.go-								Usage: "Compress rotated logs",
./cmd/admin.go:		cli.BoolTFlag{
./cmd/admin.go-			Name:  "force-smtps",
./cmd/admin.go-			Usage: "SMTPS is always used on port 465. Set this to force SMTPS on other ports.",
./cmd/admin.go:		cli.BoolTFlag{
./cmd/admin.go-			Name:  "skip-verify",
./cmd/admin.go-			Usage: "Skip TLS verify.",
./cmd/admin.go:		cli.BoolTFlag{
./cmd/admin.go-			Name:  "disable-helo",
./cmd/admin.go-			Usage: "Disable SMTP helo.",
./cmd/admin.go:		cli.BoolTFlag{
./cmd/admin.go-			Name:  "skip-local-2fa",
./cmd/admin.go-			Usage: "Skip 2FA to log on.",
./cmd/admin.go:		cli.BoolTFlag{
./cmd/admin.go-			Name:  "active",
./cmd/admin.go-			Usage: "This Authentication Source is Activated.",
2023-07-24 06:49:18 +00:00
delvh f3d41c61eb
Remove `db.DefaultContext` in `routers/` and `cmd/` (#26076)
Now, the only remaining usages of `models.db.DefaultContext` are in
  - `modules`
  - `models`
  - `services`
2023-07-23 23:47:27 -04:00
wxiaoguang 52fb936773
Serve pre-defined files in "public", add "security.txt", add CORS header for ".well-known" (#25974)
Replace #25892

Close  #21942
Close  #25464

Major changes:

1. Serve "robots.txt" and ".well-known/security.txt" in the "public"
custom path
* All files in "public/.well-known" can be served, just like
3. Add a test for ".well-known/security.txt"
4. Simplify the "FileHandlerFunc" logic, now the paths are consistent so
the code can be simpler
5. Add CORS header for ".well-known" endpoints
6. Add logs to tell users they should move some of their legacy custom
public files

2023/07/19 13:00:37 cmd/web.go:178:serveInstalled() [E] Found legacy public asset "img" in CustomPath. Please move it to /work/gitea/custom/public/assets/img
2023/07/19 13:00:37 cmd/web.go:182:serveInstalled() [E] Found legacy public asset "robots.txt" in CustomPath. Please move it to /work/gitea/custom/public/robots.txt
This PR is not breaking.


Co-authored-by: silverwind <>
Co-authored-by: Giteabot <>
2023-07-21 12:14:20 +00:00
wxiaoguang d0dbe52e76
Refactor to use urfave/cli/v2 (#25959)
Replace #10912

And there are many new tests to cover the CLI behavior

There were some concerns about the "option order in hook scripts"
it's not a problem now. Because the hook script uses `/gitea hook
--config=/app.ini pre-receive` format. The "config" is a global option,
it can appear anywhere.


## ⚠️ BREAKING ⚠️

This PR does it best to avoid breaking anything. The major changes are:

* `gitea` itself won't accept web's options: `--install-port` / `--pid`
/ `--port` / `--quiet` / `--verbose` .... They are `web` sub-command's
    * Use `./gitea web --pid ....` instead
* `./gitea` can still run the `web` sub-command as shorthand, with
default options
* The sub-command's options must follow the sub-command
* Before: `./gitea --sub-opt subcmd` might equal to `./gitea subcmd
--sub-opt` (well, might not ...)
    * After: only `./gitea subcmd --sub-opt` could be used
    * The global options like `--config` are not affected
2023-07-21 17:28:19 +08:00
harryzcy 0f9f6567bb
Bump to v5 (#25975)
Bumping `` from v4 to v5.

`` v5 is bringing some breaking changes:

- standard `Valid()` method on claims is removed. It's replaced by
`ClaimsValidator` interface implementing `Validator()` method instead,
which is called after standard validation. Gitea doesn't seem to be
using this logic.
- `jwt.Token` has a field `Valid`, so it's checked in `ParseToken`
function in `services/auth/source/oauth2/token.go`


Co-authored-by: Giteabot <>
2023-07-19 09:57:10 +00:00
silverwind 88f835192d
Replace `interface{}` with `any` (#25686)
Result of running `perl -p -i -e 's#interface\{\}#any#g' **/*` and `make fmt`.

Basically the same [as golang did](2580d0e08d).
2023-07-04 18:36:08 +00:00
wxiaoguang b4d0036fc3
Do not prepare oauth2 config if it is not enabled, do not write config in some sub-commands (#25567)


And some sub-commands like "generate" / "docs", they do not need to use
the ini config
2023-06-28 23:30:06 +02:00
wxiaoguang 65d3e1161b
Fix sub-command log level (#25537)
More fix for #24981

* #24981

Close #22361

* #22361

There were many patches for Gitea's sub-commands to satisfy the facts:

* Some sub-commands shouldn't output any log, otherwise the git protocol
would be broken
* Sometimes the users want to see "verbose" or "quiet" outputs

That's a longstanding problem, and very fragile. This PR is only a quick
patch for the problem.

In the future, the sub-command system should be refactored to a clear


Other changes:

* Use `ReplaceAllWriters` to replace
`RemoveAllWriters().AddWriters(writer)`, then it's an atomic operation.
* Remove unnecessary `syncLevelInternal` calls, because
`AddWriters/addWritersInternal` already calls it.

Co-authored-by: Giteabot <>
2023-06-28 08:02:06 +02:00
wxiaoguang 2cdf260f42
Refactor path & config system (#25330)
# The problem

There were many "path tricks":

* By default, Gitea uses its program directory as its work path
* Gitea tries to use the "work path" to guess its "custom path" and
"custom conf (app.ini)"
* Users might want to use other directories as work path
* The non-default work path should be passed to Gitea by GITEA_WORK_DIR
or "--work-path"
* But some Gitea processes are started without these values
    * The "serv" process started by OpenSSH server
    * The CLI sub-commands started by site admin
* The paths are guessed by SetCustomPathAndConf again and again
* The default values of "work path / custom path / custom conf" can be
changed when compiling

# The solution

* Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use
test code to cover its behaviors.
* When Gitea's web server runs, write the WORK_PATH to "app.ini", this
value must be the most correct one, because if this value is not right,
users would find that the web UI doesn't work and then they should be
able to fix it.
* Then all other sub-commands can use the WORK_PATH in app.ini to
initialize their paths.
* By the way, when Gitea starts for git protocol, it shouldn't output
any log, otherwise the git protocol gets broken and client blocks

The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path
> env var GITEA_WORK_DIR > builtin default

The "app.ini" searching order is: cmd arg --config > cmd arg "work path
/ custom path" > env var "work path / custom path" > builtin default


If your instance's "work path / custom path / custom conf" doesn't meet
the requirements (eg: work path must be absolute), Gitea will report a
fatal error and exit. You need to set these values according to the
error log.


Close #24818
Close #24222
Close #21606
Close #21498
Close #25107
Close #24981
Maybe close #24503

Replace #23301
Replace #22754

And maybe more
2023-06-21 13:50:26 +08:00
wxiaoguang df5cf5ddbd
Avoid polluting config file when "save" (#25395)
That's a longstanding INI package problem: the "MustXxx" calls change
the option values, and the following "Save" will save a lot of garbage
options into the user's config file.

Ideally we should refactor the INI package to a clear solution, but it's
a huge work.

A clear workaround is what this PR does: when "Save", load a clear INI
instance and save it.

Partially fix #25377, the "install" page needs more fine tunes.
2023-06-21 10:31:40 +08:00
wxiaoguang 4e2f1ee58d
Refactor web package and context package (#25298)
1. The "web" package shouldn't depends on "modules/context" package,
instead, let each "web context" register themselves to the "web"
2. The old Init/Free doesn't make sense, so simplify it
* The ctx in "Init(ctx)" is never used, and shouldn't be used that way
* The "Free" is never called and shouldn't be called because the SSPI
instance is shared


Co-authored-by: Giteabot <>
2023-06-18 09:59:09 +02:00
Lunny Xiao d6dd6d641b
Fix all possible setting error related storages and added some tests (#23911)
Follow up #22405

Fix #20703 

This PR rewrites storage configuration read sequences with some breaks
and tests. It becomes more strict than before and also fixed some
inherit problems.

- Move storage's MinioConfig struct into setting, so after the
configuration loading, the values will be stored into the struct but not
still on some section.
- All storages configurations should be stored on one section,
configuration items cannot be overrided by multiple sections. The
prioioty of configuration is `[attachment]` > `[storage.attachments]` |
`[storage.customized]` > `[storage]` > `default`
- For extra override configuration items, currently are `SERVE_DIRECT`,
`MINIO_BASE_PATH`, `MINIO_BUCKET`, which could be configured in another
section. The prioioty of the override configuration is `[attachment]` >
`[storage.attachments]` > `default`.
- Add more tests for storages configurations.
- Update the storage documentations.


Co-authored-by: wxiaoguang <>
2023-06-14 11:42:38 +08:00
Lunny Xiao f9cfd6ce5b
Use the type RefName for all the needed places and fix pull mirror sync bugs (#24634)
This PR replaces all string refName as a type `git.RefName` to make the
code more maintainable.

Fix #15367
Replaces #23070 
It also fixed a bug that tags are not sync because `git remote --prune
origin` will not remove local tags if remote removed.

We in fact should use `git fetch --prune --tags origin` but not `git
remote update origin` to do the sync.

Some answer from ChatGPT as ref.

> If the git fetch --prune --tags command is not working as expected,
there could be a few reasons why. Here are a few things to check:
>Make sure that you have the latest version of Git installed on your
system. You can check the version by running git --version in your
terminal. If you have an outdated version, try updating Git and see if
that resolves the issue.
>Check that your Git repository is properly configured to track the
remote repository's tags. You can check this by running git config
--get-all remote.origin.fetch and verifying that it includes
+refs/tags/*:refs/tags/*. If it does not, you can add it by running git
config --add remote.origin.fetch "+refs/tags/*:refs/tags/*".
>Verify that the tags you are trying to prune actually exist on the
remote repository. You can do this by running git ls-remote --tags
origin to list all the tags on the remote repository.
>Check if any local tags have been created that match the names of tags
on the remote repository. If so, these local tags may be preventing the
git fetch --prune --tags command from working properly. You can delete
local tags using the git tag -d command.


Co-authored-by: delvh <>
2023-05-26 01:04:48 +00:00
wxiaoguang abcf5a7b5e
Fix install page context, make the install page tests really test (#24858)
Fix #24856

Rename "context.contextKey" to "context.WebContextKey", this context is
for web context only. But the Context itself is not renamed, otherwise
it would cause a lot of changes (if we really want to rename it, there
could be a separate PR).

The old test code doesn't really test, the "install page" gets broken
not only one time, so use new test code to make sure the "install page"
could work.
2023-05-23 09:29:15 +08:00
Brecht Van Lommel 3588edbb08
Add gitea manager reload-templates command (#24843)
This can be useful to update custom templates in production mode, when
they are updated frequently and a full Gitea restart each time is
2023-05-22 23:51:40 +08:00
wxiaoguang 4647660776
Rewrite logger system (#24726)
## ⚠️ Breaking

The `log.<mode>.<logger>` style config has been dropped. If you used it,
please check the new config manual & app.example.ini to make your
instance output logs as expected.

Although many legacy options still work, it's encouraged to upgrade to
the new options.

The SMTP logger is deleted because SMTP is not suitable to collect logs.

If you have manually configured Gitea log options, please confirm the
logger system works as expected after upgrading.

## Description

Close #12082 and maybe more log-related issues, resolve some related
FIXMEs in old code (which seems unfixable before)

Just like rewriting queue #24505 : make code maintainable, clear legacy
bugs, and add the ability to support more writers (eg: JSON, structured

There is a new document (with examples): ``

This PR is safer than the queue rewriting, because it's just for
logging, it won't break other logic.

## The old problems

The logging system is quite old and difficult to maintain:
* Unclear concepts: Logger, NamedLogger, MultiChannelledLogger,
SubLogger, EventLogger, WriterLogger etc
* Some code is diffuclt to konw whether it is right:
`log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs
* The old system heavily depends on ini config system, it's difficult to
create new logger for different purpose, and it's very fragile.
* The "color" trick is difficult to use and read, many colors are
unnecessary, and in the future structured log could help
* It's difficult to add other log formats, eg: JSON format
* The log outputer doesn't have full control of its goroutine, it's
difficult to make outputer have advanced behaviors
* The logs could be lost in some cases: eg: no Fatal error when using
* Config options are passed by JSON, which is quite fragile.
* INI package makes the KEY in `[log]` section visible in `[log.sub1]`
and `[log.sub1.subA]`, this behavior is quite fragile and would cause
more unclear problems, and there is no strong requirement to support
`log.<mode>.<logger>` syntax.

## The new design

See `logger.go` for documents.

## Screenshot







* [x] add some new tests
* [x] fix some tests
* [x] test some sub-commands (manually ....)


Co-authored-by: Jason Song <>
Co-authored-by: delvh <>
Co-authored-by: Giteabot <>
2023-05-21 22:35:11 +00:00
Jason Song a7773d28d2
Support migrating storage for actions log via command line (#24679)
Close #24677


Co-authored-by: Giteabot <>
2023-05-12 13:30:28 +00:00
wxiaoguang 6f9c278559
Rewrite queue (#24505)
# ⚠️ Breaking

Many deprecated queue config options are removed (actually, they should
have been removed in 1.18/1.19).

If you see the fatal message when starting Gitea: "Please update your
app.ini to remove deprecated config options", please follow the error
messages to remove these options from your app.ini.


2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options

Many options in `[queue]` are are dropped, including:
from app.ini.

# The problem

The old queue package has some legacy problems:

* complexity: I doubt few people could tell how it works.
* maintainability: Too many channels and mutex/cond are mixed together,
too many different structs/interfaces depends each other.
* stability: due to the complexity & maintainability, sometimes there
are strange bugs and difficult to debug, and some code doesn't have test
(indeed some code is difficult to test because a lot of things are mixed
* general applicability: although it is called "queue", its behavior is
not a well-known queue.
* scalability: it doesn't seem easy to make it work with a cluster
without breaking its behaviors.

It came from some very old code to "avoid breaking", however, its
technical debt is too heavy now. It's a good time to introduce a better
"queue" package.

# The new queue package

It keeps using old config and concept as much as possible.

* It only contains two major kinds of concepts:
    * The "base queue": channel, levelqueue, redis
* They have the same abstraction, the same interface, and they are
tested by the same testing code.
* The "WokerPoolQueue", it uses the "base queue" to provide "worker
pool" function, calls the "handler" to process the data in the base
* The new code doesn't do "PushBack"
* Think about a queue with many workers, the "PushBack" can't guarantee
the order for re-queued unhandled items, so in new code it just does
"normal push"
* The new code doesn't do "pause/resume"
* The "pause/resume" was designed to handle some handler's failure: eg:
document indexer (elasticsearch) is down
* If a queue is paused for long time, either the producers blocks or the
new items are dropped.
* The new code doesn't do such "pause/resume" trick, it's not a common
queue's behavior and it doesn't help much.
* If there are unhandled items, the "push" function just blocks for a
few seconds and then re-queue them and retry.
* The new code doesn't do "worker booster"
* Gitea's queue's handlers are light functions, the cost is only the
go-routine, so it doesn't make sense to "boost" them.
* The new code only use "max worker number" to limit the concurrent
* The new "Push" never blocks forever
* Instead of creating more and more blocking goroutines, return an error
is more friendly to the server and to the end user.

There are more details in code comments: eg: the "Flush" problem, the
strange "code.index" hanging problem, the "immediate" queue problem.

Almost ready for review.


* [x] add some necessary comments during review
* [x] add some more tests if necessary
* [x] update documents and config options
* [x] test max worker / active worker
* [x] re-run the CI tasks to see whether any test is flaky
* [x] improve the `handleOldLengthConfiguration` to provide more
friendly messages
* [x] fine tune default config values (eg: length?)

## Code coverage:

2023-05-08 19:49:59 +08:00
Lunny Xiao 377a0a20f0
Merge setting.InitXXX into one function with options (#24389)
This PR will merge 3 Init functions on setting packages as 1 and
introduce an options struct.
2023-05-04 11:55:35 +08:00
KN4CK3R bf999e4069
Add Debian package registry (#24426)
Co-authored-by: @awkwardbunny

This PR adds a Debian package registry.
You can follow [this
tutorial]( to build
a *.deb package for testing.
Source packages are not supported at the moment and I did not find
documentation of the architecture "all" and how these packages should be


Part of #20751.

Revised copy of #22854.


Co-authored-by: Brian Hong <>
Co-authored-by: techknowlogick <>
Co-authored-by: Giteabot <>
2023-05-02 12:31:35 -04:00
Yarden Shoham c0ddec8a2a
Revert "Add Debian package registry" (#24412)
Reverts go-gitea/gitea#22854
2023-04-28 18:06:41 -04:00
KN4CK3R bf77e2163b
Add Debian package registry (#22854)
Co-authored-by: @awkwardbunny

This PR adds a Debian package registry. You can follow [this
tutorial]( to build
a *.deb package for testing. Source packages are not supported at the
moment and I did not find documentation of the architecture "all" and
how these packages should be treated.


Co-authored-by: Brian Hong <>
Co-authored-by: techknowlogick <>
2023-04-28 17:51:36 -04:00
Lunny Xiao 5cf7da63ee
Refactor config provider (#24245)
This PR introduces more abstract about `ConfigProvider` and hides more `ini` references.


Co-authored-by: delvh <>
2023-04-25 23:06:39 +08:00
KN4CK3R f1173d6879
Use more specific test methods (#24265)
Co-authored-by: silverwind <>
Co-authored-by: Giteabot <>
2023-04-22 17:56:27 -04:00
techknowlogick 4014200021
add CLI command to register runner tokens (#23762)
This is a CLI command to generate new tokens for the runners to register



Co-authored-by: delvh <>
2023-04-17 13:07:13 -04:00
Lunny Xiao cda84bec87
Support converting varchar to nvarchar for mssql database (#24105) 2023-04-17 21:22:10 +08:00
wxiaoguang 50a72e7a83
Use a general approach to access custom/static/builtin assets (#24022)
The idea is to use a Layered Asset File-system (modules/assetfs/layered.go)

For example: when there are 2 layers: "custom", "builtin", when access
to asset "my/page.tmpl", the Layered Asset File-system will first try to
use "custom" assets, if not found, then use "builtin" assets.

This approach will hugely simplify a lot of code, make them testable.

Other changes:

* Simplify the AssetsHandlerFunc code
* Simplify the `gitea embedded` sub-command code


Co-authored-by: Jason Song <>
Co-authored-by: Lunny Xiao <>
2023-04-12 18:16:45 +08:00
Fredrik Eriksson cb1536471b
Add --quiet option to gitea dump (#22969)
Fixes: #19687

The --quiet options to gitea dump silences informational and less
important messages, but will still log warnings and errors to console.
Very useful in combination with cron backups and '-f -'.

Since --verbose and --quiet are incompatible with each other I made it
an error to specify both. To get the error message to be printed to
stderr I had to make this test after the NewServices()-call, which is
why there are three new blocks of code instead of two.
2023-04-10 09:46:23 -04:00
wxiaoguang f4538791f5
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687)
# Why this PR comes

At first, I'd like to help users like #23636 (there are a lot)

The unclear "Internal Server Error" is quite anonying, scare users,
frustrate contributors, nobody knows what happens.

So, it's always good to provide meaningful messages to end users (of
course, do not leak sensitive information).

When I started working on the "response message to end users", I found
that the related code has a lot of technical debt. A lot of copy&paste
code, unclear fields and usages.

So I think it's good to make everything clear.

# Tech Backgrounds

Gitea has many sub-commands, some are used by admins, some are used by
SSH servers or Git Hooks. Many sub-commands use "internal API" to
communicate with Gitea web server.

Before, Gitea server always use `StatusCode + Json "err" field` to
return messages.

* The CLI sub-commands: they expect to show all error related messages
to site admin
* The Serv/Hook sub-commands (for git clients): they could only show
safe messages to end users, the error log could only be recorded by
"SSHLog" to Gitea web server.

In the old design, it assumes that:

* If the StatusCode is 500 (in some functions), then the "err" field is
error log, shouldn't be exposed to git client.
* If the StatusCode is 40x, then the "err" field could be exposed. And
some functions always read the "err" no matter what the StatusCode is.

The old code is not strict, and it's difficult to distinguish the
messages clearly and then output them correctly.

# This PR

To help to remove duplicate code and make everything clear, this PR
introduces `ResponseExtra` and `requestJSONResp`.

* `ResponseExtra` is a struct which contains "extra" information of a
internal API response, including StatusCode, UserMsg, Error
* `requestJSONResp` is a generic function which can be used for all
cases to help to simplify the calls.
* Remove all `map["err"]`, always use `private.Response{Err}` to
construct error messages.
* User messages and error messages are separated clearly, the `fail` and
`handleCliResponseExtra` will output correct messages.
* Replace all `Internal Server Error` messages with meaningful (still
safe) messages.

This PR saves more than 300 lines, while makes the git client messages
more clear.

Many gitea-serv/git-hook related essential functions are covered by


Co-authored-by: delvh <>
2023-03-29 14:32:26 +08:00
wxiaoguang 5727056ea1
Make minio package support legacy MD5 checksum (#23768)
A feedback from discord:

Some storages like:


They do not support "x-amz-checksum-algorithm" header

But minio recently uses that header with CRC32C by default. So we have
to tell minio to use legacy MD5 checksum.

I guess this needs to be backported because IIRC we 1.19 and 1.20 are
using similar minio package.

The minio package code for SendContentMD5 looks like this:


<img width="755" alt="image"

2023-03-28 11:10:24 -04:00
Zettat123 df411819eb
Check LFS/Packages settings in dump and doctor command (#23631)
Close #23622 

As described in the issue, disabling the LFS/Package settings will cause
errors when running `gitea dump` or `gitea doctor`. We need to check the
settings and the related operations should be skipped if the settings
are disabled.

Co-authored-by: Lunny Xiao <>
2023-03-23 20:30:28 +08:00