1
0
mirror of https://github.com/go-gitea/gitea.git synced 2024-12-04 14:46:57 -05:00
Commit Graph

29 Commits

Author SHA1 Message Date
wxiaoguang
6f9c278559
Rewrite queue (#24505)
# ⚠️ Breaking

Many deprecated queue config options are removed (actually, they should
have been removed in 1.18/1.19).

If you see the fatal message when starting Gitea: "Please update your
app.ini to remove deprecated config options", please follow the error
messages to remove these options from your app.ini.

Example:

```
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options
```

Many options in `[queue]` are are dropped, including:
`WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`,
`BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed
from app.ini.

# The problem

The old queue package has some legacy problems:

* complexity: I doubt few people could tell how it works.
* maintainability: Too many channels and mutex/cond are mixed together,
too many different structs/interfaces depends each other.
* stability: due to the complexity & maintainability, sometimes there
are strange bugs and difficult to debug, and some code doesn't have test
(indeed some code is difficult to test because a lot of things are mixed
together).
* general applicability: although it is called "queue", its behavior is
not a well-known queue.
* scalability: it doesn't seem easy to make it work with a cluster
without breaking its behaviors.

It came from some very old code to "avoid breaking", however, its
technical debt is too heavy now. It's a good time to introduce a better
"queue" package.

# The new queue package

It keeps using old config and concept as much as possible.

* It only contains two major kinds of concepts:
    * The "base queue": channel, levelqueue, redis
* They have the same abstraction, the same interface, and they are
tested by the same testing code.
* The "WokerPoolQueue", it uses the "base queue" to provide "worker
pool" function, calls the "handler" to process the data in the base
queue.
* The new code doesn't do "PushBack"
* Think about a queue with many workers, the "PushBack" can't guarantee
the order for re-queued unhandled items, so in new code it just does
"normal push"
* The new code doesn't do "pause/resume"
* The "pause/resume" was designed to handle some handler's failure: eg:
document indexer (elasticsearch) is down
* If a queue is paused for long time, either the producers blocks or the
new items are dropped.
* The new code doesn't do such "pause/resume" trick, it's not a common
queue's behavior and it doesn't help much.
* If there are unhandled items, the "push" function just blocks for a
few seconds and then re-queue them and retry.
* The new code doesn't do "worker booster"
* Gitea's queue's handlers are light functions, the cost is only the
go-routine, so it doesn't make sense to "boost" them.
* The new code only use "max worker number" to limit the concurrent
workers.
* The new "Push" never blocks forever
* Instead of creating more and more blocking goroutines, return an error
is more friendly to the server and to the end user.

There are more details in code comments: eg: the "Flush" problem, the
strange "code.index" hanging problem, the "immediate" queue problem.

Almost ready for review.

TODO:

* [x] add some necessary comments during review
* [x] add some more tests if necessary
* [x] update documents and config options
* [x] test max worker / active worker
* [x] re-run the CI tasks to see whether any test is flaky
* [x] improve the `handleOldLengthConfiguration` to provide more
friendly messages
* [x] fine tune default config values (eg: length?)

## Code coverage:

![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
2023-05-08 19:49:59 +08:00
wxiaoguang
75c62054a6
Improve some modal action buttons (#24289)
Follow #24097 and #24285

And add a devtest page for modal action button testing.
http://localhost:3000/devtest/fomantic-modal

Now the `modal_actions_confirm.tmpl` could support: green / blue /
yellow positive buttons, the negative button is "secondary".

ps: this PR is only a small improvement, there are still a lot of
buttons not having proper colors. In the future these buttons could be
improved by this approach.

These buttons could also be improved according to the conclusion of
#24285 in the future.



![image](https://user-images.githubusercontent.com/2114189/233847773-a6d6b29b-7b5c-490e-8425-40dfd0ad2529.png)


And add GitHub-like single danger button (context:
https://github.com/go-gitea/gitea/issues/24285#issuecomment-1519100312)


![image](https://user-images.githubusercontent.com/2114189/233891566-055d7611-894d-4d5a-baf5-f6369180bf8d.png)

---------

Co-authored-by: silverwind <me@silverwind.io>
2023-04-24 07:08:59 -04:00
Hester Gong
44e0cfa96e
Change to vertical navbar layout for secondary navbar for repo/user/admin settings (#24246)
Co-Author: @wxiaoguang 

This is the first step of #24229. And this PR will only includes html
changes, and followed by other PRs that fine tune css and change to
submenus.

After:

Admin Level

<img width="1400" alt="Screen Shot 2023-04-21 at 10 07 16"
src="https://user-images.githubusercontent.com/17645053/233523870-f848b61d-056a-4b41-9760-a9a49fea1fe8.png">

User Level

<img width="1422" alt="Screen Shot 2023-04-21 at 10 07 23"
src="https://user-images.githubusercontent.com/17645053/233523878-979adb20-a657-43d9-99a6-ad414010c0ef.png">

Repo Level

<img width="1404" alt="Screen Shot 2023-04-21 at 10 07 07"
src="https://user-images.githubusercontent.com/17645053/233523863-337440bd-c03a-4dfd-87fa-cef40300bfe0.png">

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-04-23 18:21:21 +08:00
Hester Gong
476a043a5f
Refactor delete_modal_actions template and use it for project column related actions (#24097)
Co-Author: @wxiaoguang 

This PR is to fix
https://github.com/go-gitea/gitea/issues/23318#issuecomment-1506275446 .
The way to fix this in this PR is to use `delete_modal_actions.tmpl`
here both to fix this issue and keep ui consistency (as suggested by
[TODO
here](4299c3b7db/templates/projects/view.tmpl (L161)))

And this PR also refactors `delete_modal_actions.tmpl` and its related
styles, and use the template for more modal actions:

1. Added template attributes:
* locale
* ModalButtonStyle: "yes" (default) or "confirm"
* ModalButtonCancelText
* ModalButtonOkText

2. Rename `delete_modal_actions.tmpl` template to
`modal_actions_confirm.tmpl` because it is not only used for action
modals deletion now.

3. Refactored css related to modals into `web_src/css/modules/modal.css`
and improved the styles.

4. Also use the template for PR deletion modal and remove issue
dependency modal.

5. Some modals should also use the template, but not sure how to open
them, so mark these modal actions by `{{/* TODO: Convert to
base/modal_actions_confirm */}}`

After (Also tested on arc green):

Hovering on the left buttons

<img width="711" alt="Screen Shot 2023-04-23 at 15 17 12"
src="https://user-images.githubusercontent.com/17645053/233825650-76307e65-9255-44bb-80e8-7062f58ead1b.png">

<img width="786" alt="Screen Shot 2023-04-23 at 15 17 21"
src="https://user-images.githubusercontent.com/17645053/233825652-4dc6f7d1-a180-49fb-a468-d60950eaee0d.png">

Test for functionalities:

https://user-images.githubusercontent.com/17645053/233826857-76376fda-022c-42d0-b0f3-339c17ca4e59.mov

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-04-23 17:24:19 +08:00
Felipe Leopoldo Sologuren Gutiérrez
15c035775a
Add main landmark to templates and adjust titles (#22670)
* Add main aria landmark to templates
 * Adjust some titles to improve understanding of location in navigation

Contributed by @Forgejo
2023-02-01 22:56:10 +00:00
Gusted
d55a0b7238
Refactor i18n to locale (#20153)
* Refactor `i18n` to `locale`

- Currently we're using the `i18n` variable naming for the `locale`
struct. This contains locale's specific information and cannot be used
for general i18n purpose, therefore refactoring it to `locale` makes
more sense.
- Ref: https://github.com/go-gitea/gitea/pull/20096#discussion_r906699200

* Update routers/install/install.go
2022-06-27 15:58:46 -05:00
Gusted
3725fa28cc
Improve UI on mobile (#19546)
Start making the mobile experience not painful and be actually usable. This contains a few smaller changes to enhance this experience.

- Submit buttons on the review forms aren't columns anymore and are now allowed to be displayed on one row.
- The label/milestone & New Issue buttons were given each own row even tough, there's enough place to do it one the same row. This commit fixes that.
- The issues+Pull tab on repo's has a third item besides the label/milestone & New Issue buttons, the search bar. On desktop there's enough place to do this on one row, for mobile it isn't, currently it was using for each item a new row. This commits fixes that by only giving the searchbar a new row and have the other two buttons on the same row.
- The notification table will now be show a scrollbar instead of overflow.
- The repo buttons(Watch, Star, Fork) on mobile were showing quite big and the SVG wasn't even displayed on the same line, if the count of those numbers were too high it would even overflow. This commit removes the SVG, as there isn't any place to show them on the same row and allows them to have a new row if the counts of those buttons are high.
- The admin page can show you a lot of interesting information, on mobile the System Status + Configuration weren't properly displayed as the margin's were too high. This commit fixes that by reducing the margin to a number that makes sense on mobile.
- Fixes to not overflow the tables but instead force them to be scrollable.
- When viewing a issue or pull request, the comments aren't full-width but instead 80% and aligned to right, on mobile this is a annoyance as there isn't much width to begin with. This commits fixes that by forcing full-width and removing the avatars on the left side and instead including them inline in the comment header.
2022-05-01 18:11:21 +02:00
zeripath
90e0a402c1
Show last cron messages on monitor page (#19223)
As discussed on #19221 we should store the results of the last task message on the
crontask and show them on the monitor page.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-29 02:31:07 +01:00
zeripath
4e57bd1d30
Add number in queue status to monitor page (#18712)
Add number in queue status to the monitor page so that administrators can
assess how much work is left to be done in the queues.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-02-12 13:31:26 +08:00
zeripath
01087e9eef
Make Requests Processes and create process hierarchy. Associate OpenRepository with context. (#17125)
This PR registers requests with the process manager and manages hierarchy within the processes.

Git repos are then associated with a context, (usually the request's context) - with sub commands using this context as their base context.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-30 20:06:32 +00:00
silverwind
1a03fa7a4f
Update JS dependencies (#15033)
* Update JS dependencies

- Update all JS dependencies
- For octicons, rename trashcan to trash
- For svgo, migrate to v2 api, output seems to have slightly changed but icons look the same
- For stylelint, update config, fix custom property duplicates
- For monaco, drop legacy Edge support
- For eslint, enable new rules, fix new issues
- For less-loader, remove deprecated import syntax

* update svgo usage in generate-images and rebuild logo.svg with it
2021-03-22 05:04:19 +01:00
silverwind
87a0396719
Fix admin monitoring margin (#14134) 2020-12-23 21:38:29 +01:00
silverwind
c85bb62635
Replace more icons with SVG, repo search tweaks (#13860)
* Replace more icons with SVG

- Replace remaining icons on admin page with SVG
- Fix vertical menu background on arc-green
- Minor improvments to frontpage repo search
- More icon replacements here and there

* fix integration

* whitespace tweak

* add comment

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-12-09 14:03:19 -05:00
silverwind
b1cf7f4df1
Add class to page content to unify top margin (#13766)
* Add class to page content to unify top margin

Previously pages would individually set this margin but some didn't so
content would stick to the header without any space. Resolve this by
adding a new class that is added on all pages. The only place where we
remove this margin again is on the pages with menu or wrapper in the
header.

* fix admin notices

* fix team pages

* fix loading segment on gitgraph for arc-green

* fix last missing case

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-11-30 23:00:14 -05:00
silverwind
ffddf3f8a6
Make SVG size argument optional (#12814)
Now defaults to 16 on both frontend and backend.

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-09-11 16:19:00 -04:00
zeripath
217647f331
Multiple small admin dashboard fixes (#12153)
* Remove spurious spacing between Maintenance Operations and its table on dashboard
* Prevent (EXTRA string) comments in Task headers
* Redirect tasks started from monitor page back to monitor
* Fix #12107 - redirects from process cancel should use AppSubUrl
* When wrapping queues set the name correctly

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-07-05 22:38:03 +03:00
Matthew R. McDougal
689e7e8831
Fix form action template substitutions on admin pages (#11519)
* Fix form action template substitution on admin dashboard

* Fix form action template substitution on admin monitoring page

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-05-20 14:35:16 -04:00
zeripath
9a2e47b23a
Refactor Cron and merge dashboard tasks (#10745)
* Refactor Cron and merge dashboard tasks

* Merge Cron and Dashboard tasks
* Make every cron task report a system notice on completion
* Refactor the creation of these tasks
* Ensure that execution counts of tasks is correct
* Allow cron tasks to be started from the cron page

* golangci-lint fixes

* Enforce that only one task with the same name can be registered

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix name check

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per @guillep2k

* as per @lafriks

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add git.CommandContext variants

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-05-16 19:31:38 -04:00
zeripath
62eb1b0f25 Graceful Queues: Issue Indexing and Tasks (#9363)
* Queue: Add generic graceful queues with settings

* Queue & Setting: Add worker pool implementation

* Queue: Add worker settings

* Queue: Make resizing worker pools

* Queue: Add name variable to queues

* Queue: Add monitoring

* Queue: Improve logging

* Issues: Gracefulise the issues indexer

Remove the old now unused specific queues

* Task: Move to generic queue and gracefulise

* Issues: Standardise the issues indexer queue settings

* Fix test

* Queue: Allow Redis to connect to unix

* Prevent deadlock during early shutdown of issue indexer

* Add MaxWorker settings to queues

* Merge branch 'master' into graceful-queues

* Update modules/indexer/issues/indexer.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Update modules/indexer/issues/indexer.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Update modules/queue/queue_channel.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Update modules/queue/queue_disk.go

* Update modules/queue/queue_disk_channel.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Rename queue.Description to queue.ManagedQueue as per @guillep2k

* Cancel pool workers when removed

* Remove dependency on queue from setting

* Update modules/queue/queue_redis.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* As per @guillep2k add mutex locks on shutdown/terminate

* move unlocking out of setInternal

* Add warning if number of workers < 0

* Small changes as per @guillep2k

* No redis host specified not found

* Clean up documentation for queues

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

* Update modules/indexer/issues/indexer_test.go

* Ensure that persistable channel queue is added to manager

* Rename QUEUE_NAME REDIS_QUEUE_NAME

* Revert "Rename QUEUE_NAME REDIS_QUEUE_NAME"

This reverts commit 1f83b4fc9b.

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-01-07 12:23:09 +01:00
zeripath
60c5339042 Graceful: Cancel Process on monitor pages & HammerTime (#9213)
* Graceful: Create callbacks to with contexts

* Graceful: Say when Gitea is completely finished

* Graceful: Git and Process within HammerTime

Force all git commands to terminate at HammerTime
Force all process commands to terminate at HammerTime
Move almost all git processes to run as git Commands

* Graceful: Always Hammer after Shutdown

* ProcessManager: Add cancel functionality

* Fix tests

* Make sure that process.Manager.Kill() cancels

* Make threadsafe access to Processes and remove own unused Kill

* Remove cmd from the process manager as it is no longer used

* the default context is the correct context

* get rid of double till
2019-11-30 16:40:22 +02:00
David Schneiderbauer
af57d6ab8a undo #3142 because processes don't use the util.Timestamp (#4203) 2018-06-10 09:48:45 +08:00
Lunny Xiao
f2e20c81b6 Refactor struct's time to remove unnecessary memory usage (#3142)
* refactor struct's time to remove unnecessary memory usage

* use AsTimePtr simple code

* fix tests

* fix time compare

* fix template on gpg

* use AddDuration instead of Add
2017-12-11 06:37:04 +02:00
Patrick G
efbb895ebe Cleaner ui for admin, repo settings, and user settings page (#1269) (#1270) 2017-03-15 23:39:38 +01:00
Jonas
9ad3a07989 Fix for #1037: Corrected process ID placeholder to PID from Pid (#1048)
Signed-off-by: Jonas Östanbäck <jonas.ostanback@gmail.com>
2017-02-25 11:12:46 +08:00
Adam Strzelecki
da2585c11e Indent all templates with tabs
This commit improves templates readability, since all of them use consistent
indent with all template command blocks indented too.

1. Indents both HTML containers such as <div>, <p> and Go HTML template blocks
   such as {{if}} {{with}}

2. Cleans all trailing white-space

3. Adds trailing last line-break to each file
2015-12-08 00:57:46 +01:00
Unknwon
81b8427438 new UI on admin/monitor 2015-08-19 00:34:11 +08:00
Unknwon
963354c5d7 Add raw, history file button, and other mirror fixes 2014-10-11 18:02:48 -04:00
Vyacheslav Bakhmutov
5103f16f78 add omitted lang 2014-10-11 13:43:21 +07:00
Unknwon
9bc63bda7d Finish all new admin pages. 2014-08-30 20:49:51 +08:00