Commit graph

14 commits

Author SHA1 Message Date
Lunny Xiao
28f4029e40
Display deprecated warning in admin panel pages as well as in the log file (#26094) (#26154)
backport #26094
Temporily resolve #25915
Related #25994

This PR includes #26007 's changes but have a UI to prompt administrator
about the deprecated settings as well as the log or console warning.
Then users will have enough time to notice the problem and don't have
surprise like before.

<img width="1293" alt="图片"
src="https://github.com/go-gitea/gitea/assets/81045/c33355f0-1ea7-4fb3-ad43-cd23cd15391d">

(cherry picked from commit c598741f01)
2023-07-30 07:42:53 +02:00
Giteabot
f890856cf4
Update path related documents (#25417) (#25982)
Backport #25417 by @wxiaoguang

Update WorkPath/WORK_PATH related documents, remove out-dated
information.

Remove "StaticRootPath" on the admin config display page, because few
end user really need it, it only causes misconfiguration.

![image](https://github.com/go-gitea/gitea/assets/2114189/8095afa4-da76-436b-9e89-2a92c229c01d)

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit ee47face12)
2023-07-24 07:58:56 +02:00
silverwind
24e64fe372
Replace interface{} with any (#25686) (#25687)
Same perl replacement as https://github.com/go-gitea/gitea/pull/25686
but for 1.20 to ease future backporting.
2023-07-04 23:41:32 -04:00
wxiaoguang
061b68e995
Refactor path & config system (#25330) (#25416)
Backport #25330

# The problem

There were many "path tricks":

* By default, Gitea uses its program directory as its work path
* Gitea tries to use the "work path" to guess its "custom path" and
"custom conf (app.ini)"
* Users might want to use other directories as work path
* The non-default work path should be passed to Gitea by GITEA_WORK_DIR
or "--work-path"
* But some Gitea processes are started without these values
    * The "serv" process started by OpenSSH server
    * The CLI sub-commands started by site admin
* The paths are guessed by SetCustomPathAndConf again and again
* The default values of "work path / custom path / custom conf" can be
changed when compiling

# The solution

* Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use
test code to cover its behaviors.
* When Gitea's web server runs, write the WORK_PATH to "app.ini", this
value must be the most correct one, because if this value is not right,
users would find that the web UI doesn't work and then they should be
able to fix it.
* Then all other sub-commands can use the WORK_PATH in app.ini to
initialize their paths.
* By the way, when Gitea starts for git protocol, it shouldn't output
any log, otherwise the git protocol gets broken and client blocks
forever.

The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path
> env var GITEA_WORK_DIR > builtin default

The "app.ini" searching order is: cmd arg --config > cmd arg "work path
/ custom path" > env var "work path / custom path" > builtin default


## ⚠️ BREAKING

If your instance's "work path / custom path / custom conf" doesn't meet
the requirements (eg: work path must be absolute), Gitea will report a
fatal error and exit. You need to set these values according to the
error log.
2023-06-22 16:27:18 +00:00
wxiaoguang
4647660776
Rewrite logger system (#24726)
## ⚠️ Breaking

The `log.<mode>.<logger>` style config has been dropped. If you used it,
please check the new config manual & app.example.ini to make your
instance output logs as expected.

Although many legacy options still work, it's encouraged to upgrade to
the new options.

The SMTP logger is deleted because SMTP is not suitable to collect logs.

If you have manually configured Gitea log options, please confirm the
logger system works as expected after upgrading.

## Description

Close #12082 and maybe more log-related issues, resolve some related
FIXMEs in old code (which seems unfixable before)

Just like rewriting queue #24505 : make code maintainable, clear legacy
bugs, and add the ability to support more writers (eg: JSON, structured
log)

There is a new document (with examples): `logging-config.en-us.md`

This PR is safer than the queue rewriting, because it's just for
logging, it won't break other logic.

## The old problems

The logging system is quite old and difficult to maintain:
* Unclear concepts: Logger, NamedLogger, MultiChannelledLogger,
SubLogger, EventLogger, WriterLogger etc
* Some code is diffuclt to konw whether it is right:
`log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs
`log.DelLogger("console")`
* The old system heavily depends on ini config system, it's difficult to
create new logger for different purpose, and it's very fragile.
* The "color" trick is difficult to use and read, many colors are
unnecessary, and in the future structured log could help
* It's difficult to add other log formats, eg: JSON format
* The log outputer doesn't have full control of its goroutine, it's
difficult to make outputer have advanced behaviors
* The logs could be lost in some cases: eg: no Fatal error when using
CLI.
* Config options are passed by JSON, which is quite fragile.
* INI package makes the KEY in `[log]` section visible in `[log.sub1]`
and `[log.sub1.subA]`, this behavior is quite fragile and would cause
more unclear problems, and there is no strong requirement to support
`log.<mode>.<logger>` syntax.


## The new design

See `logger.go` for documents.


## Screenshot

<details>


![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff)


![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9)


![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee)

</details>

## TODO

* [x] add some new tests
* [x] fix some tests
* [x] test some sub-commands (manually ....)

---------

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-21 22:35:11 +00:00
Hester Gong
63a401ac40
Move secrets and runners settings to actions settings (#24200)
This PR moves the secrets and runners settings to actions settings on
all settings(repo,org,user,admin) levels.

After this PR, if
[ENABLED](5e7543fcf4/custom/conf/app.example.ini (L2604))
inside `app.ini` under `[actions]` is set to `false`, the "Actions" tab
(including runners management and secrets management) will not be shown.

After, the settings under actions settings for each level:

1. Admin Level
"Runners Management"
<img width="1437" alt="Screen Shot 2023-04-26 at 14 34 20"
src="https://user-images.githubusercontent.com/17645053/234489731-15822d21-38e1-4560-8bbe-69f122376abc.png">

2. User Level
"Secrets Management"
<img width="1427" alt="Screen Shot 2023-04-26 at 14 34 30"
src="https://user-images.githubusercontent.com/17645053/234489795-68c9c0cb-24f8-4f09-95c6-458ab914c313.png">

3. Repo and Organization Levels
"Runners Management" and "Secrets Management" 
   Org:
<img width="1437" alt="Screen Shot 2023-04-26 at 14 35 07"
src="https://user-images.githubusercontent.com/17645053/234489996-f3af5ebb-d354-46ca-9087-a0b586845281.png">

<img width="1433" alt="Screen Shot 2023-04-26 at 14 35 14"
src="https://user-images.githubusercontent.com/17645053/234490004-3abf8fed-81fd-4ce2-837a-935dade1793d.png">

    Repo:
<img width="1419" alt="Screen Shot 2023-04-26 at 14 34 50"
src="https://user-images.githubusercontent.com/17645053/234489904-80c11038-4b58-462c-9d0b-8b7cf70bc2b3.png">
    
<img width="1430" alt="Screen Shot 2023-04-26 at 14 34 57"
src="https://user-images.githubusercontent.com/17645053/234489918-4e8d1fe2-9bcd-4d8a-96c1-238a8088d92e.png">

It also finished these tasks :

- [x] rename routers function "runners" to "actions", and refactor
related file names
- [x] check and modify part of the runners related functions to match
their name
- [x] Fix backend check caused by fmt check

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-04-27 20:08:47 -04:00
wxiaoguang
ff2f479a4b
Clean template/helper.go (#23922)
The first step of #23328

This PR cleans:

1. Dead function like `SubStr`
2. Unnecessary function like `UseHTTPS`, it should be provided by
`window.origin`
3. Duplicate function like `IsShowFullName`, there was already a
`DeafultShowFullName`

I have searched these removed functions globally, no use in code.
2023-04-07 03:31:41 -04:00
Lunny Xiao
91fa0eb9d7
Avoid warning for system setting when start up (#23054)
Partially fix #23050

After #22294 merged, it always has a warning log like `cannot get
context cache` when starting up. This should not affect any real life
but it's annoying. This PR will fix the problem. That means when
starting up, getting the system settings will not try from the cache but
will read from the database directly.

---------

Co-authored-by: Lauris BH <lauris@nix.lv>
2023-02-24 18:23:13 +08:00
Lunny Xiao
c53ad052d8
Refactor the setting to make unit test easier (#22405)
Some bugs caused by less unit tests in fundamental packages. This PR
refactor `setting` package so that create a unit test will be easier
than before.

- All `LoadFromXXX` files has been splited as two functions, one is
`InitProviderFromXXX` and `LoadCommonSettings`. The first functions will
only include the code to create or new a ini file. The second function
will load common settings.
- It also renames all functions in setting from `newXXXService` to
`loadXXXSetting` or `loadXXXFrom` to make the function name less
confusing.
- Move `XORMLog` to `SQLLog` because it's a better name for that.

Maybe we should finally move these `loadXXXSetting` into the `XXXInit`
function? Any idea?

---------

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: delvh <dev.lh@web.de>
2023-02-20 00:12:01 +08:00
Lunny Xiao
bd820aa9c5
Add context cache as a request level cache (#22294)
To avoid duplicated load of the same data in an HTTP request, we can set
a context cache to do that. i.e. Some pages may load a user from a
database with the same id in different areas on the same page. But the
code is hidden in two different deep logic. How should we share the
user? As a result of this PR, now if both entry functions accept
`context.Context` as the first parameter and we just need to refactor
`GetUserByID` to reuse the user from the context cache. Then it will not
be loaded twice on an HTTP request.

But of course, sometimes we would like to reload an object from the
database, that's why `RemoveContextData` is also exposed.

The core context cache is here. It defines a new context
```go
type cacheContext struct {
	ctx  context.Context
	data map[any]map[any]any
        lock sync.RWMutex
}

var cacheContextKey = struct{}{}

func WithCacheContext(ctx context.Context) context.Context {
	return context.WithValue(ctx, cacheContextKey, &cacheContext{
		ctx:  ctx,
		data: make(map[any]map[any]any),
	})
}
```

Then you can use the below 4 methods to read/write/del the data within
the same context.

```go
func GetContextData(ctx context.Context, tp, key any) any
func SetContextData(ctx context.Context, tp, key, value any)
func RemoveContextData(ctx context.Context, tp, key any)
func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error)
```

Then let's take a look at how `system.GetString` implement it.

```go
func GetSetting(ctx context.Context, key string) (string, error) {
	return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) {
		return cache.GetString(genSettingCacheKey(key), func() (string, error) {
			res, err := GetSettingNoCache(ctx, key)
			if err != nil {
				return "", err
			}
			return res.SettingValue, nil
		})
	})
}
```

First, it will check if context data include the setting object with the
key. If not, it will query from the global cache which may be memory or
a Redis cache. If not, it will get the object from the database. In the
end, if the object gets from the global cache or database, it will be
set into the context cache.

An object stored in the context cache will only be destroyed after the
context disappeared.
2023-02-15 21:37:34 +08:00
Jason Song
21c91b7dff
Set disable_gravatar/enable_federated_avatar when offline mode is true (#22479)
When offline mode is true, we should set `disable_gravatar` to `true`
and `enable_federated_avatar` to `false` in system settings.
2023-01-17 17:00:19 +02:00
flynnnnnnnnnn
e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
Lunny Xiao
385462d36c
Fix dashboard ignored system setting cache (#21621)
This is a performance regression from #18058

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2022-11-10 14:43:53 +08:00
Lunny Xiao
f860a6d2e4
Add system setting table with cache and also add cache supports for user setting (#18058) 2022-10-17 07:29:26 +08:00