c3325c2cbf
Instrument logs ( #3566 )
...
* Add spans to show worker_name and job_id
Always show worker_name and job_id when relevant, by creating spans that
live as long as the worker and the job.
* Remove redundant worker names from fn run_worker
Remove the now redundant worker_name tracing, as the information now
lives at the span level
* Use span when spawning new thread
* Instrument more pertinent functions
Move spans to more pertinent functions. Remove more redundant info, and
change worker_name to just worker
* Change tracing subscriber to log path:line_no
Change from logging target (module name usually) to file path and line
number
2024-04-18 01:11:14 +02:00
da3ded6196
fix: measure memory usage on postgres scripts
2024-04-18 01:05:16 +02:00
cb6efb08a2
feat: show more for logs on s3 directly possible from browser log viewer
2024-04-17 21:42:46 +02:00
c23fdc1585
fix: add mysql decimal support ( #3557 )
2024-04-15 14:26:17 +02:00
a1d2c59fce
feat: flow concurrency limits support custom concurrency key
2024-04-09 22:03:39 +02:00
6ad1b37844
fix: improve performance of for-loops with parralelism
2024-04-09 19:47:50 +02:00
05998e91a1
fix: improve performance of for-loops with parralelism
2024-04-09 19:41:56 +02:00
65b280c84b
prevent parallelism from timing out lock monitor
2024-04-09 18:50:19 +02:00
424a5e344e
fix: improve deadlocks for parallel branches with parallelism constraints
2024-04-09 11:52:40 +02:00
34fd745b2f
more error messages
2024-04-09 10:27:14 +02:00
ce18be6431
fix: improve deadlocks for parallel branches with parallelism constraints
2024-04-09 01:38:55 +02:00
10d739537d
fix: parallel flow with parallelism constraint could deadlock
2024-04-06 13:25:34 +02:00
ab89792187
nit global cache python improvements
2024-04-04 22:07:14 +02:00
1402c72212
fix: improve global cache deps
2024-04-04 21:33:44 +02:00
a67f093184
fix: improve chars splitting for logs with utf-8 chars
2024-04-04 13:38:19 +02:00
d5c1a5707b
fix: fix char excess size module for logs truncation
2024-04-04 03:25:11 +02:00
de66141362
add checksum log to s3 piptar pull
2024-04-02 20:55:31 +02:00
288b9c8ce4
feat: while loop as new flow primitive
2024-04-02 17:38:11 +02:00
e66d4b6708
continue on error option for steps
2024-04-02 10:32:47 +02:00
a913e23c35
fix: improve git sync for script renames
2024-03-26 18:21:58 +01:00
6b842b3c53
feat: add set_flow_user_states and get_flow_user_states api and sdk support
2024-03-26 17:31:17 +01:00
32de1ddfef
improve vacuum queue
2024-03-24 17:40:21 +01:00
ddfc7f14ec
improve vacuum queue
2024-03-24 17:28:15 +01:00
d63f5ee26e
fix: add support for azure storage on distributed cache/logs
2024-03-23 18:42:38 +01:00
3279996fd2
feat(frontend): fetch logs just-in-time only when necessary
2024-03-23 15:57:23 +01:00
6b719fecea
feat: add workspace environment vars (custom contextual vars) ( #3455 )
...
* feat: custom contextual variables
* chore: sqlx prepare
* fix: main merge
2024-03-23 11:40:55 +01:00
2339060de1
nit newline
2024-03-23 11:33:10 +01:00
7714221f3e
feat: large log disk and distributed storage compaction
2024-03-23 10:55:29 +01:00
c2d18a6d6f
feat keep sql columns ordering ( #3444 )
...
* feat: keep pg columns ordering
* feat: pass column order through flow status
* feat: sql ordering for all sql langs
* fix: sqlx build on mac + rename to _metadata
* Update JobPreview.svelte
* Update +page.svelte
* Update +page.svelte
* Update LogPanel.svelte
* Update +page.svelte
* Update LogPanel.svelte
* Update JobPreview.svelte
* fix: build
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2024-03-22 10:34:32 +01:00
f8d3ba12af
generalize cache object store
2024-03-21 08:50:10 +01:00
a012ea0768
fix compile on non ee
2024-03-20 16:55:06 +01:00
169bfd761d
fix compile on non ee
2024-03-20 16:41:23 +01:00
d616735174
fix compile on non ee
2024-03-20 15:34:25 +01:00
fcf5cadd69
remove tmpcache
2024-03-20 15:32:55 +01:00
8718d038f4
fix compile on non ee
2024-03-20 15:18:54 +01:00
7dafa3d62a
fix compile on non ee
2024-03-20 15:09:28 +01:00
a0a42067fb
feat: s3 cache config added to instance settings + parallelized tar pulling
2024-03-20 14:19:34 +01:00
10d2cd41de
fix: ai prevent python async + max db schema length ( #3440 )
2024-03-19 08:17:51 +01:00
4788c393e9
avoid caching binary on cloud
2024-03-17 19:13:51 +01:00
ab18058405
fix: allow multiple PIP_SECRETS
2024-03-17 15:15:36 +01:00
8056ce49af
removing pulling all piptars at start
2024-03-17 11:21:43 +01:00
6854728569
fix: disapproval does not trigger flow error handler anymore
2024-03-17 11:08:02 +01:00
fd57b40064
log nits
2024-03-16 19:01:58 +01:00
9745eec4d4
bash nits
2024-03-16 18:52:17 +01:00
6f2f9b7ec2
nits logging
2024-03-16 18:30:51 +01:00
958df4f62c
feat: refactor logging to improve performance by order of magnitude for longer jobs
2024-03-16 17:55:34 +01:00
543d2a1577
fix: s3 cache is an instance settings ( #3421 )
...
* s3
* s3
2024-03-15 15:09:05 +01:00
e5aec9c445
fix: improve pip_secret interpolation
2024-03-15 11:17:41 +01:00
bda7ac3669
fix: avoid unecessary re-schedule for retyied flows
2024-03-15 09:53:58 +01:00
8144813d61
fix jobs metrics
2024-03-14 19:53:38 +01:00
wendrul
Ruben Fiszel
HugoCasa
Ruben Fiszel