Commit graph

10 commits

Author SHA1 Message Date
Earl Warren
209610075a
[TESTS] add log.Level to test.NewLogChecker
So the caller can check log events at the desired level instead of
being limited to the default level log.INFO

(cherry picked from commit 2fbf5f9555)
(cherry picked from commit e2137a3147)
2024-02-05 16:54:44 +01:00
Gusted
d1cb590c78
[GITEA] Log SQL queries when the database return error
- When the database returns an error about the SQL query, the error is
logged but not the SQL query and arguments, which is just as valuable as
the vague deeply hidden documented error that the database returns.
It's possible to log the SQL query by logging **all** SQL queries. For
bigger instances such as Codeberg, this is not a viable option.
- Adds a new hook, enabled by default, to log SQL queries with their
arguments and the error returned by the database when the database
returns an error.
- This likely needs some fine tuning in the future to decide when to
enable this, as the error is already logged and if people have the
`[database].LOG_SQL` option enabled, the SQL would be logged twice. But
given that it's an rare occurence for SQL queries to error, it's fine to
leave that as-is.
- Ref: https://codeberg.org/forgejo/forgejo/issues/1998

(cherry picked from commit 866229bc32)
(cherry picked from commit 96dd3e87cf)
(cherry picked from commit e165510317)
(cherry picked from commit 1638e2b3f5)
2024-02-05 16:09:43 +01:00
Gusted
664192767c
[GITEA] Add slow SQL query warning
- Databases are one of the most important parts of Forgejo, every
interaction with Forgejo uses the database in one way or another.
Therefore, it is important to maintain the database and recognize when
Forgejo is not doing well with the database. Forgejo already has the
option to log *every* SQL query along with its execution time, but
monitoring becomes impractical for larger instances and takes up
unnecessary storage in the logs.
- Add a QoL enhancement that allows instance administrators to specify a
threshold value beyond which query execution time is logged as a warning
in the xorm logger. The default value is a conservative five seconds to
avoid this becoming a source of spam in the logs.
- The use case for this patch is that with an instance the size of Codeberg, monitoring SQL logs is not very fruitful and most of them are uninteresting. Recently, in the context of persistent deadlock issues (https://codeberg.org/forgejo/forgejo/issues/220), I have noticed that certain queries hold locks on tables like comment and issue for several seconds. This patch helps to identify which queries these are and when they happen.
- Added unit test.

(cherry picked from commit 24bbe7886f)
(cherry picked from commit 6e29145b3c)
(cherry picked from commit 63731e3071)
(cherry picked from commit 3ce1a09736)
(cherry picked from commit a64426907d)
(cherry picked from commit 4b19215691)
(cherry picked from commit e635674435)
(cherry picked from commit 9cf501f1af)
(cherry picked from commit 0d6b934eba)
(cherry picked from commit 4b6c273879)
(cherry picked from commit 89b1315338)
(cherry picked from commit edd8e66ce9)

[GITEA] Add slow SQL query warning (squash) document the setting

(cherry picked from commit ce38599c51)
(cherry picked from commit 794aa67c68)
(cherry picked from commit a4c2c6b004)
(cherry picked from commit 97912752bc)
(cherry picked from commit 00b5327c97)
(cherry picked from commit 1069c860e7)
(cherry picked from commit 84241f42c8)
(cherry picked from commit e4bda0e845)
(cherry picked from commit 7357fb91bf)
(cherry picked from commit a8dd7f6da2)
(cherry picked from commit e636e9f4be)
(cherry picked from commit bf04ae8603)
(cherry picked from commit 93b19e3568)
(cherry picked from commit 83f91363ad)
(cherry picked from commit e34a05bc73)
(cherry picked from commit 68569aeee9)
2024-02-05 16:05:50 +01:00
Earl Warren
081a155cc1
[SEMVER] store SemVer in ForgejoSemVer after a database upgrade
(cherry picked from commit b7fe7cf401)
(cherry picked from commit cf339eed4f)
(cherry picked from commit 4f3a16168b)
(cherry picked from commit 6f5bbc53fc)
(cherry picked from commit aca42b422e)
(cherry picked from commit 5a7f7580e5)
(cherry picked from commit 06c383c807)
(cherry picked from commit fe831dcb53)
(cherry picked from commit cd12cd0dbc)
(cherry picked from commit cc79163703)
(cherry picked from commit 0102a5715e)
(cherry picked from commit 403f7520b3)
(cherry picked from commit a3b61510a2)
(cherry picked from commit f83f0f9feb)
(cherry picked from commit fd1c3a6d09)
(cherry picked from commit f7cdc3d6f1)
(cherry picked from commit 060121b644)
(cherry picked from commit 62c847ff02)
(cherry picked from commit 4d051b51c2)
(cherry picked from commit 86e6981a93)
(cherry picked from commit c1fc9e441b)
(cherry picked from commit 8bb2f0871a)
(cherry picked from commit 0cd9fe5251)
(cherry picked from commit b0b44778b4)
(cherry picked from commit 7c2f4f749f)
2024-02-05 14:44:33 +01:00
delvh
e10ba5e5e0
Rename Sync2 -> Sync (#26479)
The xorm `Sync2` has already been deprecated in favor of `Sync`,
so let's do the same inside the Gitea codebase.

Command used to replace everything:
```sh
for i in $(ag Sync2 --files-with-matches); do vim $i -c ':%sno/Sync2/Sync/g' -c ':wq'; done
```
2023-08-13 21:17:21 +02:00
Jason Song
71ca3067bc
Check primary keys for all tables and drop ForeignReference (#21721)
Some dbs require that all tables have primary keys, see
- #16802
- #21086

We can add a test to keep it from being broken again.

Edit:

~Added missing primary key for `ForeignReference`~ Dropped the
`ForeignReference` table to satisfy the check, so it closes #21086.

More context can be found in comments.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
2022-12-23 19:35:43 +08:00
flynnnnnnnnnn
e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
KN4CK3R
044c754ea5
Add context.Context to more methods (#21546)
This PR adds a context parameter to a bunch of methods. Some helper
`xxxCtx()` methods got replaced with the normal name now.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-19 16:12:33 +08:00
Eng Zer Jun
8b0aaa5f86
test: use T.TempDir to create temporary test directory (#21043)
A testing cleanup. 

This pull request replaces `os.MkdirTemp` with `t.TempDir`. We can use the `T.TempDir` function from the `testing` package to create temporary directory. The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete. 

This saves us at least 2 lines (error check, and cleanup) on every instance, or in some cases adds cleanup that we forgot.

Reference: https://pkg.go.dev/testing#T.TempDir

```go
func TestFoo(t *testing.T) {
	// before
	tmpDir, err := os.MkdirTemp("", "")
	require.NoError(t, err)
	defer os.RemoveAll(tmpDir)

	// now
	tmpDir := t.TempDir()
}
```

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-09-04 16:14:53 +01:00
Lunny Xiao
d91d4db344
Move tests as seperate sub packages to reduce duplicated file names (#19951) 2022-06-15 09:02:00 +02:00