Our drone by necessity runs on git repositories not owned by the drone process. Unfortunately this means that git operations and thence CI builds will fail without the `safe.directory` option being set.
See: https://drone.gitea.io/go-gitea/gitea/54632/2/8
* use go1.18 to build gitea& update min go version to 1.17
* bump in a few more places
* add a few simple tests for isipprivate
* update go.mod
* update URL to https://go.dev/dl/
* golangci-lint
* attempt golangci-lint workaround
* change version
* bump fumpt version
* skip strings.title test
* go mod tidy
* update tests as some aren't private??
* update tests
This ensures the tools only run in the versions we've tested and it also
does not polute PATH with those tools so they are truly isolated. This
syntax of `go run` requires go 1.17, so the minimum version is set
accordingly.
Fixes: https://github.com/go-gitea/gitea/issues/18867
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
The CI currently downloads all go modules in each pipeline step because
go modules reside outside the project directory. Fix this by introducing
a volume for the `/go` directory [1] so modules are only downloaded once per
pipeline using a new `deps-backend` make target.
For completeness, I also included new `deps` and `deps-frontend` targets
and the frontend one is also triggered explicitly on CI where needed.
[1] https://docs.drone.io/pipeline/kubernetes/examples/language/golang/#dependencies
* Fix pushing to 1-x-dev docker tag
It appears that #18551 and #18573 have a mistake in that raymond does not have
an {{else}} on {{#equal}}. This PR notes that Sprig has a hasPrefix function
and so we use this with another if.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix pushing to 1-x-dev docker tag (part 2)
Although we now have the manifest working, we need to create the images.
Here we adjust the .drone.yml to force building of the images
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix pushing to 1-x-dev docker tag
OK now we have the images building we should make sure that the main ones stays
dev and the release/v* ones become *-dev-*
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Apply suggestions from code review
* Make docker gitea/gitea:v1.16-dev etc refer to the latest build on that branch
One of the problems with our current docker tagging is that although we
have strict version tags, latest and dev we do not have a way for docker
users to track the current release branch. This PR simply suggests that
we use the 1.x-dev tag for these and we build and push these. This will
give users who want or need unreleased bug fixes the option of tracking
the pre-release version instead of simply jumping to dev.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This PR should resolve the permission problems during CI, if the uid doesn't match, a more clear message is shown.
* CI fails with unknown permission problems #17710
The new drone step dependencies:
(root)prepare-test-env -> (gitea)build -> (gitea)test
- this fixes the CI release upload issues, as the docker image for this is freshly built (unlike the mostly unmaintained "official" drone plugins), thus containing current CA certs needed for letsencrypt since 2021-09-31.
- woodpecker is a drone-ci fork maintained partially by @6543. it's API compatible with current drone plugins afaik
Now that node 16.6.1 is out we can (if desired) switch back to node 16.
This PR proposes changing drone to run node:16
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Upgrade to golang-jwt 3.2.2
Upgrade to the latest version of golang-jwt
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Forcibly replace the 3.2.1 version of golang-jwt/jwt and increase minimum Go version
Using go.mod we can forcibly replace the 3.2.1 version used by goth to 3.2.2.
Further given golang-jwt/jwts stated policy of only supporting supported go versions
we should just raise our minimal version of go to 1.16 for 1.16 as by time of release
1.15 will be out of support.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* update minimal go required
Signed-off-by: Andrew Thornton <art27@cantab.net>
* update config.yaml
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Disable frontend testing
Jest does not appear to work on the latest node 16.6.0 and fails with an inscrutable
message.
I have been unable to work out what the problem is. This PR simply disables the
test-frontend part in the makefile.
Another alternative would be to drop node to node 14 - which is the LTS for node.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* actually just tell on 16.5 instead
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Use node 16.5 instead of 16
Signed-off-by: Andrew Thornton <art27@cantab.net>
* bump postgres and mysql DB versions
* posgres test against v10
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
- Update all JS dependencies to latest version
- Use Node 16 on CI
- Add new lint rules
- Add some gitignore entries for debug files
Co-authored-by: Lauris BH <lauris@nix.lv>
- Don't package node_modules in tarballs, they are not cross-platform
anymore and npm cache should not be messed with directly. Instead,
require an internet connection to rebuild the UI, which is not necessary
in the general use case because prebuilt UI files are shipped in the
public directory.
- Simplify the fomantic build and make the target phony. We don't need
anything more for something that is rarely ran.
- Use regular tar again to build tarballs and add variable for excludes
- Disable annoying npm update notifications
Fixes: https://github.com/go-gitea/gitea/pull/14578
Fixes: https://github.com/go-gitea/gitea/pull/15256
Fixes: https://github.com/go-gitea/gitea/pull/15262
Co-authored-by: 6543 <6543@obermui.de>
- Add basic frontend unit testing infrastructure using jest in ESM mode
- Rename 'make test' to 'make test-backend'
- Introduce 'make test-frontend' and 'make test' that runs both
- Bump Node.js requirement to v12. v10 will be EOL in less than a month.
- Convert all build-related JS files to ESM.
I opted to run frontend tests run as part of the compliance pipeline because
they complete fast and are not platform-specific like the golang tests.
* Vendor node mods as cache; fix esbuild/fomantic offline build
* Fix --exclude; use bsdtar for consistent globbing
* Fall back to GNU tar; forward-compatible for APT 2.0
* Avoid having extd. attrs with bsdtar
* Dependency and misc. optimizations
* Remove extra code after esbuild-loader update
Co-authored-by: Mike L <cl.jeremy@qq.com>
* use certmagic for more extensible/robust ACME cert handling
* accept TOS based on config option
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Add compliance check for windows to ensure cross platform build
* no race condition
* woops. meant to change windows, not arm64
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* fix build
* take flash error message back and fix more windows lint error
* performance optimization
* own step to check lint for windows
Co-authored-by: 6543 <6543@obermui.de>
* Move last commit cache back into modules/git
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Remove go-git from the interface for last commit cache
Signed-off-by: Andrew Thornton <art27@cantab.net>
* move cacheref to last_commit_cache
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Remove go-git from routers/private/hook
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move FindLFSFiles to pipeline
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make no-go-git variants
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Submodule RefID
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix issue with GetCommitsInfo
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix GetLastCommitForPaths
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Improve efficiency
Signed-off-by: Andrew Thornton <art27@cantab.net>
* More efficiency
Signed-off-by: Andrew Thornton <art27@cantab.net>
* even faster
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Reduce duplication
* As per @lunny
Signed-off-by: Andrew Thornton <art27@cantab.net>
* attempt to fix drone
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix test-tags
Signed-off-by: Andrew Thornton <art27@cantab.net>
* default to use no-go-git variants and add gogit build tag
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @6543
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* docker: rootless image
* improve docs + remove check for write perm on custom
* add more info on ssh passtrough
* Add comment for internal ssh server in container config
* Support elastic search for code search
* Finished elastic search implementation and add some tests
* Enable test on drone and added docs
* Add new fields to elastic search
* Fix bug
* remove unused changes
* Use indexer alias to keep the gitea indexer version
* Improve codes
* Some code improvements
* The real indexer name changed to xxx.v1
Co-authored-by: zeripath <art27@cantab.net>
* Use Node 14 on CI
Node 14 is sufficiently stable now, use it on CI.
* also run build on node 14
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: zeripath <art27@cantab.net>
* Add a storage layer for attachments
* Fix some bug
* fix test
* Fix copyright head and lint
* Fix bug
* Add setting for minio and flags for migrate-storage
* Add documents
* fix lint
* Add test for minio store type on attachments
* fix test
* fix test
* Apply suggestions from code review
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Add warning when storage migrated successfully
* Fix drone
* fix test
* rebase
* Fix test
* display the error on console
* Move minio test to amd64 since minio docker don't support arm64
* refactor the codes
* add trace
* Fix test
* remove log on xorm
* Fi download bug
* Add a storage layer for attachments
* Add setting for minio and flags for migrate-storage
* fix lint
* Add test for minio store type on attachments
* Apply suggestions from code review
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Fix drone
* fix test
* Fix test
* display the error on console
* Move minio test to amd64 since minio docker don't support arm64
* refactor the codes
* add trace
* Fix test
* Add URL function to serve attachments directly from S3/Minio
* Add ability to enable/disable redirection in attachment configuration
* Fix typo
* Add a storage layer for attachments
* Add setting for minio and flags for migrate-storage
* fix lint
* Add test for minio store type on attachments
* Apply suggestions from code review
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Fix drone
* fix test
* Fix test
* display the error on console
* Move minio test to amd64 since minio docker don't support arm64
* don't change unrelated files
* Fix lint
* Fix build
* update go.mod and go.sum
* Use github.com/minio/minio-go/v6
* Remove unused function
* Upgrade minio to v7 and some other improvements
* fix lint
* Fix go mod
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Tyler <tystuyfzand@gmail.com>
* Add 'checks' Make and CI tasks
Introduce new "checks" targets that perform tasks that we've been piling
onto the linting tasks. This will make the linter tasks faster and
hopefully encourage some users to use them locally.
* add checks to --help
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* go1.15
* update makefile xgo version
* fix vet issue
* update docs to version of go in use
* add TODO for asyncpreemptoff
Co-authored-by: Lauris BH <lauris@nix.lv>
* First stab at a Gitlab migrations interface.
* Modify JS to show migration for Gitlab
* Properly strip out #gitlab tag from repo name
* Working Gitlab migrations!
Still need to figure out how to hide tokens/etc from showing up in opts.CloneAddr
* Try #2 at trying to hide credentials.
CloneAddr was being used as OriginalURL.
Now passing OriginalURL through from the form and saving it.
* Add go-gitlab dependency
* Vendor go-gitlab
* Use gitlab.BasicAuthClient
Correct CloneURL.
This should be functioning!
Previous commits fixed "Migrated from"
from including the migration credentials.
* Replaced repoPath with repoID globally.
RepoID is grabbed in NewGitlabDownloader
* Logging touchup
* Properly set private repo status.
Properly set milestone deadline time.
Consistently use Gitlab username for 'Name'.
* Add go-gitlab vendor cache
* Fix PR migrations:
- Count of issues is kept to set a non-conflicting PR.ID
- Bool is used to tell whether to fetch Issue or PR comments
* Ensure merged PRs are closed and set with the proper time
* Remove copyright and some commented code
* Rip out '#gitlab' based self-hosted Gitlab support
* Hide given credentials for migrated repos.
CloneAddr was being saved as OriginalURL.
Now passing OriginalURL through from the form and
saving it in it's place
* Use asset.URL directly, no point in parsing.
Opened PRs should fall through to false.
* Fix importing Milestones.
Allow importing using Personal Tokens or anonymous access.
* Fix Gitlab Milestone migration if DueDate isn't set
* Empty Milestone due dates properly return nil, not zero time
* Add GITLAB_READ_TOKEN to drone unit-test step
* Add working gitlab_test.go.
A Personal Access Token, given in env variable GITLAB_READ_TOKEN
is required to run the test.
* Fix linting issues
* Add modified JS files
* Remove pre-build JS files
* Only merged PRs are marged as merged/closed
* Test topics
* Skip test if gitlab is inaccessible
* Grab personal token from username, not password.
Matches Github migration implementation
* Add SetContext() to GitlabDownloader.
* Checking Updated field in Issues.
* Actually fetch Issue Updated time from Gitlab
* Add Gitlab migration GetReviews() stub
* Fix Patch and Clone URLs
* check Updated too
* fix mod
* make vendor with go1.14
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Add option to use /tmp for test repositories
* Fix exit status
* Add feedback about using tmp repos
Co-authored-by: Guillermo Prandi <guillep2k@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Turn off go modules for xgo and gxz
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Add test release for PR
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Try with go modules off
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Revert Makefile, force a release test
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Turn on GOPROXY for release
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* CI
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* CI
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Final commit
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Fix cross compile
* Add test for cross compile
* Fix drone
* Fix drone
* Also prevent CC environment not to generate
Co-authored-by: zeripath <art27@cantab.net>
- Added 'lint', 'lint-frontend', 'lint-backend' targets
- Added 'lint-frontend', 'lint-backend' ci steps and restructure the
'compliance' pipeline to have a clear separation between frontend and
backend and use parallelism where possible. Also, the main build
pipelines now depend on 'compliance' so they will skip if it fails.
- Added dependencies on ci steps so they skip when 'compliance' fails
- Moved JS linters to devDependencies
- Removed deprecated 'js' and 'css' targets
- added new 'make webpack' target
- deprecated 'make js' and 'make css'
- extend webpack config to load the less files
- updated docs
I had to rename the source file of `arc-green.less` to avoid generating
a useless JS entrypoint via webpack-fix-style-only-entries which would
not work with different source/destination filenames. I hear that there
should be cleaner solutions possible once we upgrade to Webpack 5.
Co-authored-by: zeripath <art27@cantab.net>
* combine make calls on drone
combine most make calls on drone into one command. this should give a
speedup of about 5-15 seconds per make invocation because make only
needs to resolve file globs once.
* combine timeout calls as well
* ci: configure remaining github limmit
* prepend with github since package is common to all migrations
* add RefreshRate
* Update github.go
* add missing space
* go fmt
* Read env variable GITHUB_READ_TOKEN for token
* Update .drone.yml
* ci: try re-ordering for optimizing time
* ci: try re-ordering for optimizing time
* ci: try re-ordering for optimizing time
* ci: try re-ordering for optimizing time
* ci: try re-ordering for optimizing time
* ci: try re-ordering for optimizing time
* ci: try offloading mysql8 to arm64
* Revert "ci: try offloading mysql8 to arm64"
This reverts commit c60de5db1c.
* ci: try offloading pgsql to arm64
* ci: activate ldap on arm64
* ci: test mysql8 in place pgsql arm64
* chore: clean un-needed move
* typo
* ci: revert runnning mysql on arm64
* ci: run compliance on arm
* chore: limit change
* chore: readd maybe need for release fetch-tags
* ci: remove docker-linux-amd64-dry-run
* ci: remove docker-linux-amd64-dry-run
* Revert "ci: remove docker-linux-amd64-dry-run"
This reverts commit 0715f65b11.
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
