- Databases are one of the most important parts of Forgejo, every
interaction uses the database in one way or another. Therefore, it is
important to maintain the database and recognize when the server is not
doing well with the database. There already is the option to log *every*
SQL query along with its execution time, but monitoring becomes
impractical for larger instances and takes up unnecessary storage in the
logs.
- Add a QoL enhancement that allows instance administrators to specify a
threshold value beyond which query execution time is logged as a warning
in the xorm logger. The default value is a conservative five seconds to
avoid this becoming a source of spam in the logs.
- The use case for this patch is that with an instance the size of
Codeberg, monitoring SQL logs is not very fruitful and most of them are
uninteresting. Recently, in the context of persistent deadlock issues
(https://codeberg.org/forgejo/forgejo/issues/220), I have noticed that
certain queries hold locks on tables like comment and issue for several
seconds. This patch helps to identify which queries these are and when
they happen.
- Added unit test.
(cherry picked from commit 9cf501f1af4cd870221cef6af489618785b71186)
---------
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
Fix#28843
This PR will bypass the pushUpdateTag to database failure when
syncAllTags. An error log will be recorded.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fix#29175
Replace #29207
This PR makes some improvements to the `issue_comment` workflow trigger
event.
1. Fix the bug that pull requests cannot trigger `issue_comment`
workflows
2. Previously the `issue_comment` event only supported the `created`
activity type. This PR adds support for the missing `edited` and
`deleted` activity types.
3. Some events (including `issue_comment`, `issues`, etc. ) only trigger
workflows that belong to the workflow file on the default branch. This
PR introduces the `IsDefaultBranchWorkflow` function to check for these
events.
Fixes#29101
Related #29298
Discard all read data to prevent misinterpreting existing data. Some
discard calls were missing in error cases.
---------
Co-authored-by: yp05327 <576951401@qq.com>
Fixes the reason why #29101 is hard to replicate.
Related #29297
Create a repo with a file with minimum size 4097 bytes (I use 10000) and
execute the following code:
```go
gitRepo, err := gitrepo.OpenRepository(db.DefaultContext, <repo>)
assert.NoError(t, err)
commit, err := gitRepo.GetCommit(<sha>)
assert.NoError(t, err)
entry, err := commit.GetTreeEntryByPath(<file>)
assert.NoError(t, err)
b := entry.Blob()
// Create a reader
r, err := b.DataAsync()
assert.NoError(t, err)
defer r.Close()
// Create a second reader
r2, err := b.DataAsync()
assert.NoError(t, err) // Should be no error but is ErrNotExist
defer r2.Close()
```
The problem is the check in `CatFileBatch`:
79217ea63c/modules/git/repo_base_nogogit.go (L81-L87)
`Buffered() > 0` is used to check if there is a "operation" in progress
at the moment. This is a problem because we can't control the internal
buffer in the `bufio.Reader`. The code above demonstrates a sequence
which initiates an operation for which the code thinks there is no
active processing. The second call to `DataAsync()` therefore reuses the
existing instances instead of creating a new batch reader.
Follow #29024
Major changes:
* refactor validLinksPattern to fullURLPattern and add comments, now it
accepts "protocol:" prefix
* rename `IsLink*` to `IsFullURL*`, and remove unnecessray "mailto:"
check
* fix some comments (by the way)
* rename EmojiShortCodeRegex -> emojiShortCodeRegex (by the way)
Fix#29166
Add support for the following activity types of `pull_request`
- assigned
- unassigned
- review_requested
- review_request_removed
- milestoned
- demilestoned
Follow #29165.
* Introduce JSONTemplate to help to render JSON templates
* Introduce JSEscapeSafe for templates. Now only use `{{ ... |
JSEscape}}` instead of `{{ ... | JSEscape | Safe}}`
* Simplify "UserLocationMapURL" useage
Old code is not consistent for generating & decoding the JWT secrets.
Now, the callers only need to use 2 consistent functions:
NewJwtSecretWithBase64 and DecodeJwtSecretBase64
And remove a non-common function Base64FixedDecode from util.go
Clarify when "string" should be used (and be escaped), and when
"template.HTML" should be used (no need to escape)
And help PRs like #29059 , to render the error messages correctly.
Introduce a new function checkGitVersionCompatibility, when the git
version can't be used by Gitea, tell the end users to downgrade or
upgrade. The refactored functions are related to make the code easier to
test.
And simplify the comments for "safe.directory"
---------
Co-authored-by: delvh <dev.lh@web.de>
With this option, it is possible to require a linear commit history with
the following benefits over the next best option `Rebase+fast-forward`:
The original commits continue existing, with the original signatures
continuing to stay valid instead of being rewritten, there is no merge
commit, and reverting commits becomes easier.
Closes#24906
Replace #28849. Thanks to @yp05327 for the looking into the problem.
Fix#28840
The old behavior of newSignatureFromCommitline is not right. The new
parseSignatureFromCommitLine:
1. never fails
2. only accept one format (if there is any other, it could be easily added)
And add some tests.
Try to improve #28949
1. Make `ctx.Data["ShowOutdatedComments"] = true` by default: it brings
consistent user experience, and sometimes the "outdated (source
changed)" comments are still valuable.
2. Show a friendly message if the comment won't show, then the end users
won't fell that "the comment disappears" (it is the special case when
`ShowOutdatedComments = false`)
Fix for gitea putting everything into one request without batching and
sending it to Elasticsearch for indexing as issued in #28117
This issue occured in large repositories while Gitea tries to
index the code using ElasticSearch.
I've applied necessary changes that takes batch length from below config
(app.ini)
```
[queue.code_indexer]
BATCH_LENGTH=<length_int>
```
and batches all requests to Elasticsearch in chunks as configured in the
above config
Resolves https://github.com/go-gitea/gitea/issues/28704
Example of an entry in the generated `APKINDEX` file:
```
C:Q1xCO3H9LTTEbhKt9G1alSC87I56c=
P:hello
V:2.12-r1
A:x86_64
T:The GNU Hello program produces a familiar, friendly greeting
U:https://www.gnu.org/software/hello/
L:GPL-3.0-or-later
S:15403
I:36864
o:hello
m:
t:1705934118
D:so:libc.musl-x86_64.so.1
p:cmd:hello=2.12-r1
i:foobar=1.0 !baz
k:42
```
the `i:` and `k:` entries are new.
---------
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Fixes#28660
Fixes an admin api bug related to `user.LoginSource`
Fixed `/user/emails` response not identical to GitHub api
This PR unifies the user update methods. The goal is to keep the logic
only at one place (having audit logs in mind). For example, do the
password checks only in one method not everywhere a password is updated.
After that PR is merged, the user creation should be next.
In #28691, schedule plans will be deleted when a repo's actions unit is
disabled. But when the unit is enabled, the schedule plans won't be
created again.
This PR fixes the bug. The schedule plans will be created again when the
actions unit is re-enabled
Renames it to `ENABLED` to be consistent with other settings and
deprecates it.
I believe this change is necessary because other setting groups such as
`attachment`, `cors`, `mailer`, etc. have an `ENABLED` setting, but
`oauth2` is the only one with an `ENABLE` setting, which could cause
confusion for users.
This is no longer a breaking change because `ENABLE` has been set as
deprecated and as an alias to `ENABLED`.
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.
## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:
Firstly, there are temporary repositories which do not change:
```go
git.OpenRepository(ctx, diskPath)
```
Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:
```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```
Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.
## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.
---------
Co-authored-by: delvh <dev.lh@web.de>
The `ToUTF8*` functions were stripping BOM, while BOM is actually valid
in UTF8, so the stripping must be optional depending on use case. This
does:
- Add a options struct to all `ToUTF8*` functions, that by default will
strip BOM to preserve existing behaviour
- Remove `ToUTF8` function, it was dead code
- Rename `ToUTF8WithErr` to `ToUTF8`
- Preserve BOM in Monaco Editor
- Remove a unnecessary newline in the textarea value. Browsers did
ignore it, it seems but it's better not to rely on this behaviour.
Fixes: https://github.com/go-gitea/gitea/issues/28743
Related: https://github.com/go-gitea/gitea/issues/6716 which seems to
have once introduced a mechanism that strips and re-adds the BOM, but
from what I can tell, this mechanism was removed at some point after
that PR.
Fix#22066
# Purpose
This PR fix the releases will be deleted when mirror repository sync the
tags.
# The problem
In the previous implementation of #19125. All releases record in
databases of one mirror repository will be deleted before sync.
Ref:
https://github.com/go-gitea/gitea/pull/19125/files#diff-2aa04998a791c30e5a02b49a97c07fcd93d50e8b31640ce2ddb1afeebf605d02R481
# The Pros
This PR introduced a new method which will load all releases from
databases and all tags on git data into memory. And detect which tags
needs to be inserted, which tags need to be updated or deleted. Only
tags releases(IsTag=true) which are not included in git data will be
deleted, only tags which sha1 changed will be updated. So it will not
delete any real releases include drafts.
# The Cons
The drawback is the memory usage will be higher than before if there are
many tags on this repository. This PR defined a special release struct
to reduce columns loaded from database to memory.
This should fix https://github.com/go-gitea/gitea/issues/28927
Technically older versions of Git would support this flag as well, but
per https://github.com/go-gitea/gitea/pull/28466 that's the version
where using it (object-format=sha256) left "experimental" state.
`sha1` is (currently) the default, so older clients should be unaffected
in either case.
Signed-off-by: jolheiser <john.olheiser@gmail.com>
When LFS hooks are present in gitea-repositories, operations like git
push for creating a pull request fail. These repositories are not meant
to include LFS files or git push them, that is handled separately. And
so they should not have LFS hooks.
Installing git-lfs on some systems (like Debian Linux) will
automatically set up /etc/gitconfig to create LFS hooks in repositories.
For most git commands in Gitea this is not a problem, either because
they run on a temporary clone or the git command does not create LFS
hooks.
But one case where this happens is git archive for creating repository
archives. To fix that, add a GIT_CONFIG_NOSYSTEM=1 to disable using the
system configuration for that command.
According to a comment, GIT_CONFIG_NOSYSTEM is not used for all git
commands because the system configuration can be intentionally set up
for Gitea to use.
Resolves#19810, #21148
