Compare commits

...
Author SHA1 Message Date
CityFun 79703e3c71 fix: resolve missing arrow in compose file comparison during app upgrade (#12663) 2026-05-06 07:38:15 +00:00
ssongliu aee4e4e282 fix: adjust sidebar collapse popover (#12662) 2026-05-06 07:36:15 +00:00
CityFun 744c7559f3 feat: add package-lock.json (#12661) 2026-05-06 14:47:56 +08:00
ssongliu d94c3ec9cc fix: adjust frontend panel sizing (#12660) 2026-05-06 06:22:15 +00:00
ssongliu f2625eab16 fix: refactor local agent proxy handling (#12659) 2026-05-06 06:20:14 +00:00
CityFun c6d498a9be style: remove table column sorting (#12658) 2026-05-06 03:26:14 +00:00
Sanjay Santhanam b06bc0a455 fix(cmd): use exec.LookPath in Which() so detection works without external 'which' (#12651)
cmd.Which() previously shelled out to `which <name>` to determine whether
a binary was on PATH. On distributions that do not ship a `which` package
by default — Arch Linux is the canonical example, but the same applies to
several minimal container images — `which` itself is missing, so every
call to Which() returned false and 1Panel reported core dependencies
(notably Docker) as 'not installed' even when they were running normally.

Switch the primary path to Go's exec.LookPath, which uses the process
PATH directly and has no external dependency. The original shell-out is
preserved as a fallback so any environment where the agent's PATH
differs from the user's interactive shell PATH (the original reason the
shell-out existed) keeps working unchanged.

Both copies are updated (agent/utils/cmd/cmd.go and core/utils/cmd/cmd.go)
and a small unit test is added to each package to guard the regression.

Fixes #12605

Signed-off-by: Sanjay Santhanam <51058514+Sanjays2402@users.noreply.github.com>
2026-05-06 10:47:02 +08:00
Sanjay Santhanam 68411bddd9 fix(ssl): support IPv6 hosts in panel SSL self-signed certificate flow (#12652)
Enabling Panel SSL with the self-sign provider rejected IPv6 hosts with
"domain format invalid". Two coupled bugs caused this:

1. The frontend extracted the host from window.location.href with
   href.split('//')[1].split(':')[0]. For an IPv6 URL like
   https://[::1]:1234 that yields '[' \u2014 not a valid host \u2014 because
   the second split splits on the first colon inside the bracketed
   address. Use window.location.hostname, which natively returns the
   bracket-stripped IPv6 host.

2. The backend ObtainSSL flow used net.ParseIP(domain) directly. Even if
   the frontend sent the bracketed form ('[::1]'), net.ParseIP rejects
   brackets, so the value flowed into IsValidDomain() and failed the
   regex.

Add common.ParseIPLoose() that accepts both bare and bracketed IPv6 in
addition to bare IPv4. Use it at both call sites in ObtainSSL (renew
path and create path). A unit test guards the regression.

Files:
  - agent/utils/common/common.go               (new ParseIPLoose helper)
  - agent/utils/common/parse_ip_test.go        (12 cases, all green)
  - agent/app/service/website_ca.go            (call sites switched)
  - frontend/src/views/setting/safe/ssl/index.vue
                                                (host extraction fix)

Fixes #12646

Signed-off-by: Sanjay Santhanam <51058514+Sanjays2402@users.noreply.github.com>
2026-05-06 10:36:23 +08:00
Ran a2ac2e0b22 fix(agent/file): correct return type of NewIFileService (#12648)
`NewIFileService` returns the bare struct type `FileService`, but its
body returns a pointer (`&FileService{}`), and the function name suggests
it should return the interface (`IFileService`). The current signature
breaks `go build ./...` on the `agent` module:

    app/service/file.go:95:9: cannot use &FileService{} (value of type
        *FileService) as FileService value in return statement
    app/service/website_proxy.go:276:36: cannot call pointer method
        GetFileList on FileService

Both errors resolve when the constructor returns the interface, since
`*FileService` implements every method on `IFileService` (verified with
`go vet ./...`).

After this change, `go build ./...` and `go vet ./...` are clean on the
`agent` module.
2026-05-06 10:33:04 +08:00
ssongliu a5a5d9f1ac fix: adjust container log download timeout (#12632) 2026-04-29 09:41:53 +00:00
ssongliu 82876eac8d fix: close task log dialog state (#12631)
Refs #12595
2026-04-29 02:43:42 +00:00
ssongliu 19c9c0f4e5 fix: adjust agent terminal height (#12630)
Refs #12611
2026-04-29 02:41:42 +00:00
ssongliu 70f044dc50 fix: adjust home memo empty state (#12629)
Refs #12609
2026-04-29 02:39:41 +00:00
Waynexxxx 1c8418026c fix: clean up docker process in DownloadContainerLogs on timeout (#12607) 2026-04-28 15:47:03 +08:00
CityFun 17f8471c14 fix: Fix the issue where *.domain.com fails to redirect. (#12625) 2026-04-28 07:41:40 +00:00
CityFun 4deb2ea484 feat: add deepseek-v4-pro and deepseek-v4-flash model (#12622)
https://github.com/1Panel-dev/1Panel/issues/12614
2026-04-28 07:39:40 +00:00
蘭 d05317dbf5 feat: enhance VS Code connection instructions and add SSH setup script details (#12613)
#9695
2026-04-28 07:37:42 +00:00
蘭 1db6ccd72b fix: Fix the right-click and filtering function of Monaco editor (#12612)
#12593
2026-04-28 07:35:41 +00:00
ssongliu bc00760404 fix: fix home monitor chart loading (#12591) 2026-04-28 07:33:40 +00:00
蘭 e57dc1240b feat: add stop functionality for decompress tasks and improve UI for task management (#12582) 2026-04-28 07:31:40 +00:00
ssongliu 4c396d68ca fix: simplify login page i18n labels (#12584) 2026-04-23 18:22:51 +08:00
CityFun ab0fe4286a fix: fixed issue with model account invalid in hermes (#12590) 2026-04-23 18:01:54 +08:00
蘭 5ab758d626 fix: Fix the pagination issue in the historical version list (#12583) 2026-04-23 11:20:12 +08:00
王贺 3a0a6537f0 ci: skip add-labels for dependabot PRs
* ci: skip add-labels for dependabot PRs
2026-04-22 17:31:53 +08:00
蘭 4eda92f25c refactor: simplify pagination logic and update theme change listener (#12564) 2026-04-22 16:53:24 +08:00
王贺 94f8c5d2f1 chore: add dependabot config (#12565) 2026-04-22 16:53:12 +08:00
王贺 92598effc1 fix: keep app tags on one line (#12563) 2026-04-22 15:26:38 +08:00
CityFun 748cecaee8 feat: add fu-step (#12562) 2026-04-22 15:26:26 +08:00
王贺 b783034d0a refactor: refine translations (#12561) 2026-04-22 14:13:06 +08:00
CityFun 94a4e39793 feat: remove some config for hermes channel (#12560) 2026-04-22 13:43:24 +08:00
CityFun 1864dee591 style: change file history table style (#12559) 2026-04-22 03:03:01 +00:00
ssongliu 8c4ea85d02 feat: support fxplay resource URLs (#12558) 2026-04-22 03:01:00 +00:00
CityFun a714d5a84e feat: update hermes qq wecom channel config (#12555) 2026-04-21 19:49:32 +08:00
Wu MX 4f97a7890f feat: Enhance container data handling and status synchronization (#12554) 2026-04-21 18:45:50 +08:00
蘭 f15f91e5fa feat: add support for stopping compression tasks and enhance compression functionality (#12553) 2026-04-21 18:40:45 +08:00
CityFun 02bd500917 style: change table style (#12552) 2026-04-21 16:35:34 +08:00
CityFun 810c6fd232 feat: change hermes channel logic (#12551) 2026-04-21 16:35:24 +08:00
ssongliu d50a8983bc fix: adjust home dashboard styles (#12550) 2026-04-21 16:35:15 +08:00
ssongliu 89647304bb fix: adjust home dashboard styles (#12549) 2026-04-21 16:35:04 +08:00
蘭 9ebad13bf8 fix: Fix the error of switching between the current file and the historical version of the opened file (#12548) 2026-04-21 16:34:54 +08:00
王贺 cc0ac8cc43 fix mongodb root password sync (#12547)
#### What this PR does / why we need it?

#### Summary of your change

#### Please indicate you've done the following:

- [ ] Made sure tests are passing and test coverage is added if needed.
- [ ] Made sure commit message follow the rule of [Conventional Commits specification](https://www.conventionalcommits.org/).
- [ ] Considered the docs impact and opened a new docs issue or PR with docs changes if needed.
2026-04-21 04:00:57 +00:00
王贺 9f5a1d842a fix mongodb password action (#12546)
#### What this PR does / why we need it?

#### Summary of your change

#### Please indicate you've done the following:

- [ ] Made sure tests are passing and test coverage is added if needed.
- [ ] Made sure commit message follow the rule of [Conventional Commits specification](https://www.conventionalcommits.org/).
- [ ] Considered the docs impact and opened a new docs issue or PR with docs changes if needed.
2026-04-21 03:58:58 +00:00
王贺 beb33957be fix mongodb upload alert visibility (#12545)
#### What this PR does / why we need it?

#### Summary of your change

#### Please indicate you've done the following:

- [ ] Made sure tests are passing and test coverage is added if needed.
- [ ] Made sure commit message follow the rule of [Conventional Commits specification](https://www.conventionalcommits.org/).
- [ ] Considered the docs impact and opened a new docs issue or PR with docs changes if needed.
2026-04-21 03:56:57 +00:00
蘭 ffcbbb705b fix: restore codicons and folding support in Monaco editor (#12544) 2026-04-21 03:54:58 +00:00
王贺 058bc71630 fix mongodb remote edit action (#12543)
#### What this PR does / why we need it?

#### Summary of your change

#### Please indicate you've done the following:

- [ ] Made sure tests are passing and test coverage is added if needed.
- [ ] Made sure commit message follow the rule of [Conventional Commits specification](https://www.conventionalcommits.org/).
- [ ] Considered the docs impact and opened a new docs issue or PR with docs changes if needed.
2026-04-21 03:52:58 +00:00
王贺 0a38979bd9 fix mongodb uninstall cleanup (#12542)
#### What this PR does / why we need it?

#### Summary of your change

#### Please indicate you've done the following:

- [ ] Made sure tests are passing and test coverage is added if needed.
- [ ] Made sure commit message follow the rule of [Conventional Commits specification](https://www.conventionalcommits.org/).
- [ ] Considered the docs impact and opened a new docs issue or PR with docs changes if needed.
2026-04-21 03:50:57 +00:00
CityFun 5fa4a15914 style: Optimize the table styling. (#12540) 2026-04-20 22:21:45 +08:00
CityFun 81c551a11b feat: Optimize the change detection for application Compose files. (#12539) 2026-04-20 17:47:52 +08:00
CityFun 5681449353 fix: Optimize the website name badge style. (#12538) 2026-04-20 17:47:42 +08:00
CityFun 88849e6355 fix: Fix the issue where the directory can still be modified while editing the runtime environment. (#12537) 2026-04-20 17:47:31 +08:00
蘭 d6bbcdc32f fix: improve pagination handling and directory naming in recycle bin (#12535) 2026-04-20 06:44:56 +00:00
KOMATA abd1581c1d fix: enhance file and trash services with metadata validation (#12529)
* feat: Enhance file and recycle bin services with metadata handling and validation

* refactor: Rename recycle bin metadata handling to trash info and update related functions

* refactor: Remove orphan trash info pruning logic and related map usage in recycle bin service
2026-04-20 14:11:15 +08:00
CityFun cd30ec7632 feat: support session delete for hermes-agent (#12533) 2026-04-20 06:08:56 +00:00
蘭 646826769a feat: Enhance file sharing functions with optional headers for improved localization support (#12530) 2026-04-20 06:06:56 +00:00
卢亦之 074c0abb3b fix: correct listening status display on the firewall page (#12516)
The bug only appeared when the same process had both TCP and UDP sockets:
every listening port on that process was attributed to whichever protocol
was encountered first. For example, if a process listened on TCP 8000 and
UDP 8001, the UI could wrongly show TCP 8001 as in use while UDP 8001
looked unused.

Use (PID, conn.Type) as the cache key so each protocol has its own
ListeningProcess entry.
2026-04-20 11:27:18 +08:00
ssongliu 3a00e893bc fix: fix psession atomic alignment (#12528) 2026-04-20 03:26:55 +00:00
蘭 2f37677650 fix: Refactor file history management to include current content and improve error handling (#12527) 2026-04-20 02:58:56 +00:00
CityFun 12ae89923b feat: support skill management for hermes-agent (#12514) 2026-04-17 16:36:17 +08:00
蘭 031a203a74 fix: Enhance file history settings with default values and validation (#12512) 2026-04-17 16:05:57 +08:00
蘭 646b99974a feat: File editing supports Dockerfile language (#12511) 2026-04-17 16:05:46 +08:00
蘭 254ed954e9 feat: Add file history management features (#12510) 2026-04-16 19:11:05 +08:00
CityFun a0050dca09 feat: update golang Dependency (#12509) 2026-04-16 17:36:52 +08:00
王贺 cd80b4e6c3 fix(frontend): clean up build warnings (#12508) 2026-04-16 16:53:24 +08:00
CityFun df5d3d782c feat: Optimize the openclaw channel bot add flow (#12507) 2026-04-16 08:52:48 +00:00
CityFun 3c5b3c3467 feat: Optimize the Hermes WeChat login flow (#12506) 2026-04-16 16:42:21 +08:00
Copilotandwan92hen 2bdba42d87 docs: add translation contribution guide (#12505)
* docs: add translation contribution guide with reference PR and checklist

Agent-Logs-Url: https://github.com/1Panel-dev/1Panel/sessions/e6a41d10-557e-4ca1-a78b-18888a9f17be

Co-authored-by: wan92hen <27671436+wan92hen@users.noreply.github.com>

* docs: clarify frontend vs backend locale file naming conventions

Agent-Logs-Url: https://github.com/1Panel-dev/1Panel/sessions/e6a41d10-557e-4ca1-a78b-18888a9f17be

Co-authored-by: wan92hen <27671436+wan92hen@users.noreply.github.com>

* docs: update reference PR to #10352 (merged Spanish locale)

Agent-Logs-Url: https://github.com/1Panel-dev/1Panel/sessions/2b14f230-892a-412e-baed-07e67614d1e9

Co-authored-by: wan92hen <27671436+wan92hen@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: wan92hen <27671436+wan92hen@users.noreply.github.com>
2026-04-16 15:00:11 +08:00
CityFun 0b146f7ff3 feat: add unbind website for agent (#12500) 2026-04-15 18:32:46 +08:00
BugKing c28a4f6d69 Remove link to development guideline from CONTRIBUTING.md (#12502)
Removed a link to the development guideline to streamline the contributing process.
2026-04-15 18:32:13 +08:00
CityFun f93a34acc4 fix(ai): Fixed issue with delete hermes-agent failed (#12499) 2026-04-15 08:24:46 +00:00
CityFun 4e78841bf6 feat(ai): support Hermes Agent chat sessions in 1Panel (#12497)
* feat(ai): support Hermes Agent chat sessions in 1Panel

* feat(ai): support Hermes Agent chat sessions in 1Panel

* feat(ai): change docker command
2026-04-15 15:44:20 +08:00
王贺 7f5bb58c72 fix: mongodb restore fixes (#12496)
* fix mongodb restore flow

* chore cleanup i18n and terminal style
2026-04-15 15:26:30 +08:00
王贺 6a28359ac2 fix mongodb route fallback (#12495) 2026-04-15 13:51:20 +08:00
王贺 59655695b5 fix(frontend): correct anti-leech component import path (#12494) 2026-04-15 11:29:27 +08:00
CityFun 786c5c1d7c feat: support hermes-agent (#12493) 2026-04-15 03:10:45 +00:00
KOMATAandssongliu 08388124b3 feat: Migrate to MinIO SDK and enhance S3 client with TLS and context (#12207)
* chore: update dependencies and migrate S3 client to MinIO SDK

- Removed AWS SDK dependency and replaced S3 client implementation with MinIO SDK.
- Updated various dependencies in go.mod and go.sum, including version upgrades for `klauspost/compress`, `minio/minio-go`, and `rs/xid`.
- Adjusted S3 client methods to utilize MinIO's API for bucket listing, object existence checks, uploads, downloads, and deletions.

* refactor: enhance S3 client with context management and TLS support

- Introduced context management with timeouts for S3 client operations to improve reliability.
- Added TLS configuration to support secure connections based on the endpoint scheme.
- Updated S3 client methods to utilize the new context and transport settings for enhanced performance and security.

---------

Co-authored-by: ssongliu <sloooop1x@gmail.com>
2026-04-15 10:00:08 +08:00
王贺 34ccc68b3a fix: clear firewall rule description (#12491)
#### What this PR does / why we need it?

Refs https://github.com/1Panel-dev/1Panel/issues/12488

#### Summary of your change

#### Please indicate you've done the following:

- [ ] Made sure tests are passing and test coverage is added if needed.
- [ ] Made sure commit message follow the rule of [Conventional Commits specification](https://www.conventionalcommits.org/).
- [ ] Considered the docs impact and opened a new docs issue or PR with docs changes if needed.
2026-04-14 10:44:46 +00:00
王贺 97ef386f17 fix: correct tamper english i18n (#12490)
#### What this PR does / why we need it?

Refs https://github.com/1Panel-dev/1Panel/issues/12482
#### Summary of your change

#### Please indicate you've done the following:

- [ ] Made sure tests are passing and test coverage is added if needed.
- [ ] Made sure commit message follow the rule of [Conventional Commits specification](https://www.conventionalcommits.org/).
- [ ] Considered the docs impact and opened a new docs issue or PR with docs changes if needed.
2026-04-14 10:42:45 +00:00
王贺 394339d7dc feat: support mongodb management (#12486)
#### What this PR does / why we need it?

Refs https://github.com/1Panel-dev/1Panel/issues/7759

#### Summary of your change

#### Please indicate you've done the following:

- [ ] Made sure tests are passing and test coverage is added if needed.
- [ ] Made sure commit message follow the rule of [Conventional Commits specification](https://www.conventionalcommits.org/).
- [ ] Considered the docs impact and opened a new docs issue or PR with docs changes if needed.
2026-04-14 10:40:44 +00:00
蘭 8362578355 fix: Resolve the issue of QR code sharing files not being able to download (#12485) 2026-04-14 07:12:44 +00:00
CityFun 6dc218cf80 perf: Optimize the frontend architecture. (#12479) 2026-04-13 18:36:28 +08:00
CityFun 89888cef73 feat: Optimize frontend structure (#12475) 2026-04-13 05:48:42 +00:00
ssongliu 4d06f444ac refactor: restore symlink-based xpack integration (#12473) 2026-04-13 10:34:59 +08:00
CityFun 6ea07421ac feat: change feishu channel logic (#12465) 2026-04-10 13:36:25 +08:00
蘭 326d52e311 fix: Fix some issues with sharing files (#12466) 2026-04-10 13:36:10 +08:00
CityFun 6e3f751dba feat: add model name required check for agent (#12463) 2026-04-10 03:30:40 +00:00
ssongliu 6982fd22d4 fix: extend agent role command timeout (#12464) 2026-04-10 03:28:39 +00:00
ssongliu 04fd456b63 fix: silence csrf forbidden toast (#12461) 2026-04-09 22:23:47 +08:00
ssongliu bea4bfeee8 feat: update ai agent config texts 2026-04-09 21:33:16 +08:00
蘭 fa6157b610 feat: enhance AI search functionality with response language support (#12460) 2026-04-09 19:00:26 +08:00
CityFun 30af26779c style: change openclaw channel style (#12459) 2026-04-09 10:48:38 +00:00
CityFun 16d5c00b9a fix: fix issue with openclaw discord config error (#12456) 2026-04-09 17:39:01 +08:00
蘭 3f14f8f30e feat: File management supports link share file (#12453) 2026-04-09 17:38:38 +08:00
ssongliu 218f92a960 fix: fallback to password login when mfa needs captcha (#12438) 2026-04-09 10:28:39 +08:00
ssongliu 75be8d3a23 fix: extend ai agent api timeouts (#12443) 2026-04-09 10:27:28 +08:00
ssongliu e2d8125b3d chore: shorten tamper translations (#12442) 2026-04-09 10:26:41 +08:00
CityFun 7ec3c517c2 feat: Modify the configuration of the OpenClaw DingTalk channel. (#12440) 2026-04-09 10:26:22 +08:00
CityFun e186e30d8c feat: remove deployment type for agent website binding (#12435) 2026-04-09 10:26:00 +08:00
CityFun 67a8ba2421 feat: Add app status sync logic for agent (#12436) 2026-04-08 08:36:40 +00:00
dependabot[bot] 24fc15beaf chore(deps): bump github.com/go-jose/go-jose/v4 in /agent (#12432)
Bumps [github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose) from 4.1.3 to 4.1.4.
- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Commits](https://github.com/go-jose/go-jose/compare/v4.1.3...v4.1.4)

---
updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v4
  dependency-version: 4.1.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-07 18:12:42 +08:00
王贺 c1fe70b53e chore: align build workflow runtime versions (#12434) 2026-04-07 18:11:20 +08:00
CityFun 30bf4a8d46 feat: add agent website check for delete (#12433) 2026-04-07 09:56:39 +00:00
CityFun 9e501ce02b feat: Modify the certificate auto-renewal logic. (#12430)
Refs https://github.com/1Panel-dev/1Panel/issues/12389
2026-04-07 09:54:38 +00:00
王贺 d74692a68b fix: optimize runtime script dropdown (#12431) 2026-04-07 17:45:09 +08:00
王贺 57fd80fb3d fix: optimize openresty other page layout (#12429) 2026-04-07 17:44:28 +08:00
KOMATA 39d2c34682 fix: ensure proper resource management in file operations (#12358)
* fix: Ensure proper resource management in file operations by closing files after creation and compression

* fix: Improve error handling for log file operations in MediaFile function
2026-04-07 17:35:31 +08:00
CityFun 3e4a48c892 feat: Modify the OpenClaw plugin installation parameters. (#12427) 2026-04-07 07:33:32 +00:00
CityFun 2c8f8f32fd feat: Add default external port access to application settings. (#12426) 2026-04-07 07:31:32 +00:00
maninhill 3cf6d9ac51 Update user count in README 2026-04-07 15:26:42 +08:00
BugKing 64d8fe8bd0 docs: Update README.md 2026-04-07 15:26:13 +08:00
蘭 8129260508 feat: File management supports AI search (#12415) 2026-04-03 22:11:01 +08:00
ssongliu a40786a7fa refactor: clean up ollama pull log parsing (#12414) 2026-04-03 22:10:47 +08:00
CityFun 368193a777 feat: Add website binding support for the agent (#12413) 2026-04-03 09:15:54 +00:00
ssongliu 61ef725892 feat: improve terminal ai runtime updates (#12409) 2026-04-03 07:58:13 +00:00
CityFun 0c49c41085 feat: change ai menu (#12407) 2026-04-03 07:56:12 +00:00
ssongliu 2a498de96e fix: tighten clean tree delete semantics (#12406)
Refs #12399
2026-04-03 07:54:12 +00:00
ssongliu d055cbb053 feat: improve terminal ai notice interaction 2026-04-03 14:28:09 +08:00
ssongliu 4b3f8a9fd0 style: refine app launcher card layout (#12404) 2026-04-03 11:23:27 +08:00
CityFun 4c26945910 feat: update default model config (#12403) 2026-04-02 10:30:07 +00:00
ssongliu 55b6104b56 fix: reset dashboard delta baseline after resume (#12402)
Refs #12067
2026-04-02 10:28:08 +00:00
CityFun 12dc8aae81 feat: Optimize Monaco editor loading (#12401) 2026-04-02 10:26:08 +00:00
ssongliu 37e2b93693 refactor: replace el-dialog with DialogPro (#12400) 2026-04-02 10:24:09 +00:00
ssongliu 62daf1e019 fix: improve ssh config file editing and include handling (#12398)
Refs #11899
2026-04-02 10:22:09 +00:00
CityFun f5e75d4bfa feat: The skill marketplace supports ClawHub China acceleration. (#12396) 2026-04-02 06:02:06 +00:00
ssongliu 32fed45c1e fix: skip proxy check when disabled (#12388)
Refs #12379
2026-04-01 11:22:08 +00:00
ssongliu 8b7597abef chore: upgrade grpc buildkit and x/image (#12387) 2026-04-01 11:20:08 +00:00
CityFun 01fb9a461b feat: OpenClaw supports configuring backup models. (#12386) 2026-04-01 11:18:09 +00:00
ssongliu bc022a4d21 refactor: simplify agent role binding inputs (#12385) 2026-04-01 11:16:09 +00:00
蘭 973cab2837 fix: Update ESLint plugin and solve the issue of code filter overflow in the development environment (#12383) 2026-04-01 09:14:09 +00:00
ssongliu 5ce155eee6 feat: add agent role binding management (#12382) 2026-04-01 09:12:07 +00:00
CityFun d83b49cc70 feat: Add more configuration options to channels. (#12380) 2026-04-01 08:24:08 +00:00
ssongliu ec981f8153 fix: harden login IP checks and entrance cookie (#12378) 2026-04-01 08:22:09 +00:00
ssongliu b2853277e7 fix: improve auth lock handling and login feedback (#12377) 2026-03-31 11:00:06 +00:00
ssongliu f573a0fa0e fix: improve request validation and session lifecycle (#12375) 2026-03-31 10:58:05 +00:00
CityFun f787734feb feat: Support uninstallation and upgrades of OpenClaw plugins. (#12373) 2026-03-31 10:08:05 +00:00
CityFun 52778f520a feat: QQ and Feishu use official plugins.QQ, Feishu, DingTalk, Telegram, and Discord support multiple accounts. (#12369)
feat: QQ and Feishu use official plugins.QQ, Feishu, DingTalk, Telegram, and Discord support multiple accounts.
2026-03-31 05:48:04 +00:00
ssongliu b5aba18248 fix: harden core memory session handling (#12366) 2026-03-30 14:30:06 +00:00
ssongliu bc9882da50 fix: guard mfa login with ip tracker (#12365) 2026-03-30 22:28:32 +08:00
ssongliu 0ba7e98e8d feat: add agent config role management (#12364) 2026-03-30 08:59:06 +00:00
王贺 e6dc44d135 chore: refine i18n (#12350)
* chore: refine i18n

* fix overview i18n

* feat: remove joinLabel funciton
2026-03-30 16:54:19 +08:00
蘭 a9f30e9f5f feat: Enhance Bark alert configuration options and visibility (#12362) 2026-03-30 06:55:05 +00:00
CityFun f3bf9d0905 feat: support remark for openclaw (#12361)
Refs https://github.com/1Panel-dev/1Panel/issues/12191
2026-03-30 06:53:05 +00:00
Alan 05ea384a00 feat: Add Bark as a new alert notification channel. (#12338) 2026-03-30 10:58:52 +08:00
maninhill 62c50aafe1 Add video introduction link to README
Added a link to a video introduction of 1Panel.
2026-03-27 09:59:17 +08:00
CityFun 6e052b8ef2 fix: Fixed issue with openclaw skill page return empty (#12347) 2026-03-26 17:57:07 +08:00
ssongliu 1d40bdc11b fix: resolve menu item wrapping issue (#12346) 2026-03-26 09:16:24 +00:00
CityFun 3b62197f29 fix: Fixed issue with openclaw skill page return empty (#12345) 2026-03-26 09:10:24 +00:00
CityFun 2d78ca5a86 fix: Fixed issue with agent acount model name is empty (#12343) 2026-03-26 05:44:27 +00:00
KOMATA a9b65b7ce5 feat: add support for trusted SSL certificates in website proxy configuration (#12342) 2026-03-26 13:42:11 +08:00
wanghe-fit2cloud 291b40a45d chore: fix ru locale formatting 2026-03-26 11:08:36 +08:00
CityFun a959da7d44 feat: Optimize the agent's decision-making logic (#12341) 2026-03-26 10:57:48 +08:00
ssongliu 26c4e8323f fix: update quick command batch separator (#12340) 2026-03-26 02:46:10 +00:00
王贺 0534badce7 fix: unify vLLM copy casing (#12339) 2026-03-26 10:32:36 +08:00
CityFun 594d818db0 style: change openclaw style (#12336) 2026-03-25 10:58:17 +00:00
ssongliu 04dceb93a7 fix: unify terminal scroll sensitivity defaults (#12334)
Refs #11911
2026-03-25 09:34:18 +00:00
CityFun 7b554c0d61 feat: Add skill hub for openclaw (#12333) 2026-03-25 09:32:18 +00:00
ssongliu bb642cd8a8 feat: improve terminal quick command workflow (#12331) 2026-03-25 08:56:18 +00:00
CityFun 40249b527d feat: add version control for openclaw config menu (#12329) 2026-03-25 08:54:18 +00:00
蘭 74ff7ddc19 feat: Upload multiple files and add overall and current file upload progress messages (#12327)
https://github.com/1Panel-dev/1Panel/issues/10593
2026-03-25 07:30:19 +00:00
ssongliu 096ec8edc7 chore: refine terminal ai prefix handling (#12328) 2026-03-25 07:28:22 +00:00
ssongliu 0d19069059 feat: improve terminal ai command interception (#12326) 2026-03-25 06:04:18 +00:00
CityFun 0da833302b feat: change openclaw upgrade logic (#12325)
#### What this PR does / why we need it?

#### Summary of your change

#### Please indicate you've done the following:

- [ ] Made sure tests are passing and test coverage is added if needed.
- [ ] Made sure commit message follow the rule of [Conventional Commits specification](https://www.conventionalcommits.org/).
- [ ] Considered the docs impact and opened a new docs issue or PR with docs changes if needed.
2026-03-25 06:02:17 +00:00
ssongliu 4a69cf0ff5 fix: migrate ai agents menu title 2026-03-24 19:54:38 +08:00
CityFun 39704ef4e5 feat: Add openclaw overview button (#12319) 2026-03-24 10:42:05 +00:00
王贺 e52cfeeb78 fix: sync fit2cloud table locale on first load (#12316) 2026-03-24 18:37:17 +08:00
蘭 476429005c ref: Change tooltip placement from top to left for better visibility (#12317)
#11922
2026-03-24 10:26:05 +00:00
ssongliu 6fb41dc711 feat: refine terminal ai assistant settings (#12315) 2026-03-24 09:10:07 +00:00
CityFun 2cb2058cfa feat: support edit config file (#12313) 2026-03-24 09:08:07 +00:00
蘭 8533dd1d33 fix: Fix the issue where the gz file cannot be previewed after decompression (#12314) 2026-03-24 09:06:06 +00:00
CityFun 0cfbcd25fc feat: Optimize plugin installation logic (#12312) 2026-03-24 08:04:05 +00:00
CityFun b91ca8bdb3 feat: add openclaw skill management menu (#12311) 2026-03-24 05:48:06 +00:00
ssongliu 72764863bc fix: unify terminal ai validation and loading state (#12310) 2026-03-24 05:46:06 +00:00
蘭 a7b9fd0181 feat: add validation and error messages for wget URL input (#12309) 2026-03-24 05:44:07 +00:00
ssongliu 2b3164fe82 chore: format terminal ai i18n entries (#12303) 2026-03-23 17:26:50 +08:00
ssongliu 2c6307049a fix: handle upgrade clean leaf selection (#12302) 2026-03-23 17:15:50 +08:00
CityFun acc1b9270f feat: add npm registry config for openclaw (#12300) 2026-03-23 09:12:03 +00:00
王贺 60e4143525 Format aiSummary string for better readability (#12301) 2026-03-23 16:48:09 +08:00
KOMATA 99bcd6d059 feat: add SSL verification option to website proxy configuration (#12258)
* feat: add SSL verification option to website proxy configuration

* localization: update proxy SSL verification messages in multiple languages
2026-03-23 16:32:27 +08:00
ssongliu c4259dd004 feat: warn when compose file is missing (#12299) 2026-03-23 08:22:03 +00:00
ssongliu 1c5a4bc978 feat: improve container file browser path jump (#12293)
Refs #12282
2026-03-23 08:20:04 +00:00
CityFun 42f6e9a9a6 feat: add wenxin channel for openclaw (#12298) 2026-03-23 07:54:03 +00:00
蘭 569363530a fix: Fix the issue of preserving archive modification time during the decompression process of gz files (#12297) 2026-03-23 07:52:04 +00:00
蘭 ec9eba0b9a feat: enhance tab management and file handling in editor (#12294)
#12098
2026-03-23 07:50:03 +00:00
ssongliu 5376d54089 feat: refine terminal ai settings and safeguards (#12289) 2026-03-23 07:48:04 +00:00
KOMATA c10e5747b0 fix: trim whitespace from command in ExecComposer method (#12290) 2026-03-23 15:47:31 +08:00
jakub961241 5a80931a49 fix: auto theme does not switch when OS changes day/night mode (#6813) (#12278)
When theme is set to "auto" (follow system), changing the OS from
light to dark (or vice versa) without reloading the page had no
effect. The theme was only evaluated once on page load.

Added a matchMedia 'change' event listener that calls switchTheme()
whenever the OS color scheme changes, but only when theme is "auto".
Listener is properly cleaned up on component unmount.

Fixes #6813 (point 1)
2026-03-23 14:57:15 +08:00
jakub961241 f541379daa fix: open container directory at WorkingDir instead of root (#12265)
When clicking "Directory" on a container, the file browser always
opened at `/` (root). Now it inspects the container first and opens
at the container's configured WorkingDir (e.g., `/app`, `/var/www`).

Falls back to `/` if WorkingDir is not set or inspect fails.

Fixes #12255
2026-03-23 14:52:57 +08:00
王贺 0018335d64 fix: unify sidebar menu item heights (#12253) 2026-03-20 18:32:17 +08:00
CityFun df90286253 feat: optimize the validation logic for model accounts (#12251) 2026-03-20 09:55:54 +00:00
ssongliu 8dfa7945d7 feat: add xpack node multi overview entry (#12248)
Refs #11735
2026-03-20 09:53:54 +00:00
王贺 1f4292d9cf fix: add missing ssl column controls (#12252) 2026-03-20 17:41:50 +08:00
蘭 e635a7beba feat: add stop functionality for wget file downloads (#12245)
#9791
2026-03-20 09:05:54 +00:00
CityFun 59c657e403 feat: agent model account support xiaomi (#12242)
https://github.com/1Panel-dev/1Panel/issues/12220
2026-03-20 07:27:55 +00:00
ssongliu ff8f79ed52 fix: align compose partial status actions (#12239)
Refs #12232
2026-03-20 06:51:54 +00:00
CityFun 57a7edf481 feat: improve Minimax validation (#12240) 2026-03-20 06:39:56 +00:00
CityFun bbd33c66f1 fix: fix the issue where the website fails to start after being stopped (#12241) 2026-03-20 14:38:18 +08:00
ssongliu b05ed1a0ca fix: refine sidebar node dropdown style (#12238)
Refs #12231
2026-03-20 03:21:54 +00:00
蘭 b645c4edb4 feat: enhance file operations to preserve file attributes during copy (#12237)
#11542
2026-03-20 03:19:54 +00:00
CityFun 9f61aac8be feat: improve the OpenClaw terminal (#12230) 2026-03-19 18:26:37 +08:00
CityFun 11d78644ee feat: add dingTalk channel for openclaw (#12229) 2026-03-19 09:55:44 +00:00
蘭 b96428e1d6 feat: Extract the file and retain the original file modification time (#12228)
#11542
2026-03-19 09:53:47 +00:00
蘭 6432c35b1a feat: File Selection Add Favorite Pop Up Window for Quick Access to Favorite Files (#12227)
#11326
2026-03-19 09:53:14 +00:00
CityFun fa843f0940 feat: improve agent module logic (#12226)
* feat: openclaw supports configuring multiple models

* feat: change openclaw bind type

* feat: change openclaw bind type

* feat: improve agent module logic
2026-03-19 17:00:15 +08:00
ssongliu 5c7ffc561b chore: upgrade agent docker dependencies (#12224) 2026-03-19 07:27:20 +00:00
ssongliu c2f0c8c6ac feat: add ai terminal settings (#12215) 2026-03-19 07:25:19 +00:00
ssongliu 5bf6d17451 fix: handle mfa and entrance auth errors (#12222) 2026-03-19 15:11:37 +08:00
王贺 7f799004c3 fix: fix openresty logs (#12219) 2026-03-19 14:31:54 +08:00
CityFun c68251cada feat: openclaw supports configuring multiple models (#12213) 2026-03-18 10:36:11 +00:00
王贺 23e7d03d8d fix: missing form fields in es/pt (#12211) 2026-03-18 18:32:28 +08:00
王贺 5085db7a2f chore: upgrade qiniu sdk to fix kodo hash mismatch (#12214) 2026-03-18 18:31:51 +08:00
CityFun a9cf7a7d1d feat: Optimize the logic of the OpenClaw Caddyfile (#12209)
* feat: Optimize the logic of the OpenClaw Caddyfile

* feat: Optimize the logic of the OpenClaw Caddyfile
2026-03-18 15:38:01 +08:00
CityFun cf18995182 fix: Fix the issue preventing the successful creation of the OpenClaw deployment website. (#12206)
* fix: Fix the issue preventing the successful creation of the OpenClaw  deployment website.

* fix: change update app_install port logic
2026-03-18 15:37:48 +08:00
CityFun 24d7dc0786 fix: Fixed the issue where the OpenClaw access address could not be set to a domain name. (#12201) 2026-03-17 09:55:21 +00:00
CityFun 142c9b9a71 fix: Fixed issue with build backend failed (#12199) 2026-03-17 14:50:45 +08:00
ssongliu f3c7603f6e fix: respect region-based release doc source (#12198) 2026-03-17 14:33:11 +08:00
ssongliuandssongliu db26da1700 chore: format english proxy helper text (#12197)
Co-authored-by: ssongliu <songlius11@163.com>
2026-03-17 14:06:20 +08:00
ssongliuandssongliu d5ff59f4e7 fix: remove unused compose backup status param (#12196)
Co-authored-by: ssongliu <songlius11@163.com>
2026-03-17 13:56:34 +08:00
王贺 85b88db99e docs: refine Docker proxy desc (#12195) 2026-03-17 11:15:27 +08:00
ssongliuandssongliu f7053934b5 fix: improve compose backup and container restore handling (#12192)
Co-authored-by: ssongliu <songlius11@163.com>
2026-03-16 22:39:34 +08:00
wangdage12andfanbook-wangdage bf1339bb40 fix: show loading state when confirming deletion of multiple folders and reduce unnecessary basedir requests
Co-authored-by: fanbook-wangdage <124357765+fanbook-wangdage@users.noreply.github.com>
2026-03-16 19:07:19 +08:00
CityFun 43c5e62c23 feat: change allowedOrigin param for openclaw upgrade (#12188) 2026-03-16 10:53:50 +00:00
ssongliuandssongliu 28634ce631 fix(middleware): preserve numeric format in operation logs (#12173) (#12185)
Co-authored-by: ssongliu <songlius11@163.com>
2026-03-16 17:33:07 +08:00
ssongliuandssongliu 72263dfdda fix(container): allow backup for stopped containers (#12184)
Co-authored-by: ssongliu <songlius11@163.com>
2026-03-16 16:39:16 +08:00
ssongliuandssongliu 479d36e25f fix(log): localize upgrade and rebuild operation labels (#12183)
Co-authored-by: ssongliu <songlius11@163.com>
2026-03-16 16:25:34 +08:00
ssongliu 2f3ad3537e feat: add docs source config and runtime env linkage (#12182) 2026-03-16 08:13:51 +00:00
CityFun 2dfe5096f0 feat: Optimize reverse proxy handling for applications with HTTPS ports. (#12181) 2026-03-16 08:07:51 +00:00
CityFun 4375722aef feat: add checkOnStart config for openclaw (#12178) 2026-03-16 06:45:49 +00:00
CityFun bde4d06842 fix: Fixed issue with build source code failed (#12177)
Refs https://github.com/1Panel-dev/1Panel/issues/12176
2026-03-16 05:51:50 +00:00
CityFun 17a486b2c4 feat: Modify the OpenClaw upgrade logic. (#12175) 2026-03-16 04:11:50 +00:00
CityFun 8499c09571 feat: Add OpenClaw security adaptation (#12168) 2026-03-13 18:41:44 +08:00
CityFun 7cb70f785a feat: change openclaw port jump logic (#12164) 2026-03-12 14:27:34 +00:00
CityFun f3781320de feat: add icon for agent (#12163) 2026-03-12 18:52:04 +08:00
CityFun 869ee75020 feat: Add allowedOrigins config for openclaw (#12162) 2026-03-12 09:37:33 +00:00
ssongliu 1d5a26373d chore: optimize lint and cleanup warnings (#12161) 2026-03-12 09:35:33 +00:00
ssongliu 176fd0f8ad feat: support file operations inside containers (#12160)
Refs #523
2026-03-12 09:33:33 +00:00
ssongliu 341a55ae63 feat: add Agent quick jump with priority show strategy (#12159)
Refs #12158
2026-03-12 09:31:33 +00:00
CityFun fb87cf545f feat: Modify the Gemini account verification logic (#12155)
Refs https://github.com/1Panel-dev/1Panel/issues/12149
2026-03-12 09:07:33 +00:00
ssongliu 6db2a9352e feat: refactor image update flow and align i18n messages (#12154)
Refs #10786
2026-03-12 09:05:34 +00:00
ssongliu 21826bcc1a feat: complete backup and recovery for containers and compose (#12153)
Refs #9656
2026-03-12 09:03:34 +00:00
ssongliu eedbc2e95c feat: load registry auth from docker cli config when pulling image (#12152)
Refs #10460
2026-03-12 09:01:33 +00:00
CityFun 46763df7bb fix: Fixed issue with ai menu change failed (#12147) 2026-03-11 10:55:27 +00:00
CityFun 175162753e feat: openclaw channel support wecom (#12146) 2026-03-11 10:53:27 +00:00
CityFun d114933fe9 feat: agent account add openRouter (#12143) 2026-03-11 04:00:55 +00:00
蘭 432412e596 ref: Adjust the column widths of the file system (#12141) 2026-03-11 10:38:15 +08:00
ssongliu 1b577e3d41 fix(core): resolve operation data by agent for non-local nodes (#12136) 2026-03-10 17:23:06 +08:00
CityFun eda9641d35 feat: change some prompt (#12135) 2026-03-10 16:11:31 +08:00
CityFun 6808e4892c fix: Fixed ai menu router error (#12134) 2026-03-10 14:55:00 +08:00
CityFun 0d71cd4db2 feat: change custom model name logic (#12132) 2026-03-10 03:25:58 +00:00
CityFun b6b3304a71 style: change some prompt (#12131) 2026-03-10 11:09:20 +08:00
CityFun 59a0a573ed feat: 增加 vLLM 管理 (#12129) 2026-03-09 20:01:39 +08:00
CityFun e82d686940 feat: add vLLm management (#12128) 2026-03-09 19:55:20 +08:00
ssongliu b48bb88aa1 fix(postgresql): check active sessions before dropping db (#12127) 2026-03-09 16:04:58 +08:00
ssongliu 5ce9ec5b64 feat(container): refine compose and setting i18n (#12125)
Refs #12109
2026-03-09 08:03:52 +00:00
ssongliu 740f3549ad fix: optimize operation log resolve flow and i18n replacements (#12123) 2026-03-09 16:02:12 +08:00
zhengkunwang223 aa8a9e8b1e feat: add QQbot channel for ai Agent 2026-03-09 15:11:04 +08:00
ssongliu 14218e3495 fix(compose): load directory when compose path changes (#12122)
Refs #12106
2026-03-09 06:57:50 +00:00
蘭 c526eb2962 fix: Resolve the issue of not receiving emails when the sender's name is in Chinese (#12121)
#12006
2026-03-09 06:55:50 +00:00
CityFun 1ff8292be4 feat: add ai agent create limit (#12124) 2026-03-09 13:54:06 +08:00
A_Words d4394616af feat: support toggling file list columns (#12099)
* feat: support toggling file list columns

* feat: use fu table column select for file list

* chore: remove unused file column i18n keys

* feat: simplify file column visibility logic and remove unused configurations
2026-03-09 09:51:53 +08:00
王贺 462ca6b373 feat: rename AI model menu to local model (#12108) 2026-03-09 09:51:24 +08:00
ssongliu 657c3ad37a chore: localize newly added i18n keys (#12104) 2026-03-06 14:05:40 +00:00
王贺 90d11a3b74 remove ssl helper (#12102) 2026-03-06 15:31:28 +08:00
CityFun 883f3db725 feat: add openclaw default tool config (#12096) 2026-03-06 13:40:03 +08:00
ssongliu 75da72c265 fix: avoid pulling build-only compose images (#12095)
Refs  #12092
2026-03-06 02:29:39 +00:00
ssongliu 78eef15f5b feat(ai): hide coding plan providers in intl mode (#12090) 2026-03-05 10:47:38 +00:00
ssongliu 735cf3cc71 chore(agent): downgrade docker/cli to v28.5.1 (#12091) 2026-03-05 18:38:42 +08:00
Rowan Chen 99c5d9a8b2 feat: show SSL certificate issuer and expiry date in certificate selectors (#12071) 2026-03-05 15:44:12 +08:00
Rowan Chen 3a0d2039d0 fix: set sort_order for newly installed apps to avoid random positioning (#12069) 2026-03-05 15:43:18 +08:00
dependabot[bot] 98e2d6390a chore(deps): bump github.com/docker/cli in /agent (#12079)
Bumps [github.com/docker/cli](https://github.com/docker/cli) from 28.5.1+incompatible to 29.2.0+incompatible.
- [Commits](https://github.com/docker/cli/compare/v28.5.1...v29.2.0)

---
updated-dependencies:
- dependency-name: github.com/docker/cli
  dependency-version: 29.2.0+incompatible
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-05 12:02:34 +08:00
ssongliu 975e7f9f58 revert: drop nginx safe string validation changes (#12076) 2026-03-05 11:55:23 +08:00
王贺 77c90a26f8 chore: change screenshot (#12075)
Updated UI display screenshot in README.
2026-03-05 11:07:33 +08:00
ssongliu 17cb12df8b fix: avoid sidebar menu flicker when loading and sorting (#12066) 2026-03-05 02:37:04 +00:00
王贺 c986b04667 Update installation script URL in README (#12065) 2026-03-04 18:11:06 +08:00
CityFun 754e016f20 feat: Optimize the Ollama configuration workflow (#12063) 2026-03-04 08:39:03 +00:00
ssongliu f67ce75492 fix: update hook initialization handling (#12062) 2026-03-04 16:38:59 +08:00
ssongliu e236ca5607 fix(swagger): correct api annotation types and methods (#12060) 2026-03-04 07:13:02 +00:00
CityFun 10e74c4fe4 feat: add agent analytics req (#12059) 2026-03-04 07:01:03 +00:00
MoeShin 473c0a5ff2 chore: optimize website proxy api and log (#12052) 2026-03-04 14:35:05 +08:00
ssongliu 7fbe6e4091 fix(auth): update 1panel token hash validation (#12056) 2026-03-04 14:30:17 +08:00
ssongliu 65dbb53a07 fix(terminal): remove command completion protocol (#12055) 2026-03-04 06:25:05 +00:00
KOMATA c21910c203 feat: enhance container compose view with additional status indicators and environment configuration options (#12042)
- Added visual indicators for container status and creation date.
- Introduced a new radio button for environment configuration alongside existing options.
- Improved layout and styling for better user experience.
- Adjusted table column widths and added a class for styling.
2026-03-03 18:31:47 +08:00
CityFun 1728511bad feat: update kimi-coding logic (#12049) 2026-03-03 09:32:03 +00:00
ssongliu e63f6672f8 fix: persist terminal font settings (#12047) 2026-03-03 09:30:03 +00:00
ssongliu 7fa9754ce7 refine passkey prerequisite checks and i18n guidance 2026-03-03 17:08:06 +08:00
王贺 475e4ca629 ci: configure Go dependency cache (#12046)
Add cache dependency path for Go setup
2026-03-03 16:04:26 +08:00
ssongliu 32dee74edf fix: update intl domain links to 1panel.pro (#12045) 2026-03-03 07:20:02 +00:00
CityFun 6715559925 feat: support z.ai agent account (#12044) 2026-03-03 15:18:24 +08:00
KOMATA 2f380c9f2f feat: enhance app synchronization with concurrent processing and detailed result handling (#12041)
- Introduced concurrent processing for app synchronization tasks to improve performance.
- Added structures for managing app work items and results.
- Implemented detailed logging of synchronization progress and HTTP request outcomes.
- Enhanced error handling for icon downloads and app details updates.
2026-03-03 13:46:22 +08:00
CityFun 0aebb0d2db [Feature] Add App Store status check on the Agent page (#12039) 2026-03-03 12:02:42 +08:00
CityFun b7cf9aab2b feat(ai-agents): support ollama apiType selection (#12037) 2026-03-03 11:57:17 +08:00
Rowan Chen ea2f1875bc fix: disable favorite toggle during sort mode and fix drag rebound issue (#12026)
* Optimize sorting-related issues, disable the "favorite stars" function during sorting, and optimize sorting behavior caused by DOM conflicts.

* fix: prevent app icon from being squished in detail drawer
2026-03-03 11:06:53 +08:00
KOMATA 83524940ed refactor: remove unused pprof import from main.go (#12031) 2026-03-03 11:05:26 +08:00
MoeShin 8c33eb8409 feat: Operate website proxy config (#12005)
* feat: Operate website proxy config

* chore: Update api comment

* chore: update swagger

* chore: update x-log

* feat: split api
2026-03-03 11:05:00 +08:00
ssongliu 644655af86 feat: add extra_hosts support for container config (#12035)
Related: https://github.com/1Panel-dev/1Panel/issues/11861
2026-03-03 03:02:02 +00:00
CityFun 8082f4d2ca fix(ai-agents): mark skipped-verification accounts as unverified and normalize legacy status (#12034) 2026-03-03 11:00:53 +08:00
ssongliu 3c896bc5f8 fix: improve MFA interval compatibility i18n hints (#12033)
Related: https://github.com/1Panel-dev/1Panel/issues/11915
2026-03-03 03:00:02 +00:00
ssongliu 5c7ce6d8d4 fix: add PostgreSQL recreation and DB host env sync in app restore (#12029) 2026-03-03 02:58:01 +00:00
CityFun 222702f66b feat(ai-agents): Add ark coding plan for agent account (#12027) 2026-03-02 22:04:57 +08:00
ssongliu b3b3feb1b9 feat: improve container log viewer and download options (#12025)
#11898
2026-03-02 14:04:02 +00:00
ssongliu 133140040b fix: store loadingText as key and localize on render (#12022) 2026-03-02 10:32:01 +00:00
ssongliu d70f903432 fix: hide upage for intl and reload after edition switch (#12021) 2026-03-02 10:30:00 +00:00
CityFun f77f305316 feat(ai-agents): add time zone config for openclaw (#12020) 2026-03-02 10:28:01 +00:00
ssongliu 13b5908442 fix: handle sqlite wal/shm mismatch during restore (#12024) 2026-03-02 18:27:38 +08:00
CityFun d1335db986 fix(ai-agents): fixed issue with minimax-m2.5 is not work for bailian-coding-plan (#12023) 2026-03-02 18:27:25 +08:00
KOMATA 1e8836ad95 feat: add passkey prerequisites messages and update passkey settings UI (#12018)
* feat: add passkey prerequisites messages and update passkey settings UI

* refactor: enhance passkey settings UI by restructuring alert component and adding prerequisite checks
2026-03-02 16:05:55 +08:00
CityFun d155217ce3 feat(ai-agents): Add copaw type for agent (#12017) 2026-03-02 06:53:58 +00:00
ssongliu 9ddb996a44 fix: fix menu sort mismatch between load and migration (#12016) 2026-03-02 06:25:58 +00:00
ssongliu 226eccf5f1 fix: fix terminal extra line after settings change (#12015) 2026-03-02 06:24:00 +00:00
CityFun 4b2d6f0ef2 feat(ai-agents): add bailian-coding-plan account support (#12014) 2026-03-02 14:22:09 +08:00
蘭 f1c9ad45a7 feat: Improve file decompression processing (#12012) 2026-03-02 05:53:58 +00:00
KOMATA 93489c85fa feat: Enhance compose item interaction and styling (#12008)
- Updated the compose item to include a new class for active state, improving visual feedback.
- Refactored action buttons into a dedicated container for better organization and hover effects.
- Added CSS transitions for smoother interactions on hover and active states.
2026-03-02 13:43:34 +08:00
CityFun 8cead06d7b feat(ai-agents): add anthropic-messages support for custom account apiType (#12013) 2026-03-02 13:43:06 +08:00
ssongliu 625cd9f32f chore: update installation package path (#12009) 2026-03-02 04:05:59 +00:00
dependabot[bot] 9aab3b42af chore(deps): bump go.opentelemetry.io/otel/sdk in /agent (#12010)
Bumps [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go) from 1.38.0 to 1.40.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.38.0...v1.40.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/sdk
  dependency-version: 1.40.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 12:04:30 +08:00
A_Words 6684cd57d7 fix: open compressed files in decompress drawer and improve archive detection (#11997) 2026-03-02 11:29:19 +08:00
A_Words 00d95d6e19 fix(mkdown-editor): avoid pre-sanitizing markdown source (#11998) (#12004) 2026-03-02 10:54:25 +08:00
ssongliu 36f03e5daf chore: refine partial i18n copy (#11992) 2026-02-28 15:01:54 +00:00
ssongliu 7c395a6bf8 fix: resolve GPU menu selected state style issue 2026-02-28 18:58:18 +08:00
zhengkunwang223 e9a7cac5fd fix(frontend): change table style 2026-02-28 18:58:06 +08:00
zhengkunwang223 9f00b431d4 fix(frontend): stabilize installed app card name truncation 2026-02-28 18:35:34 +08:00
maninhill 1705d44378 Update README.md 2026-02-28 17:54:03 +08:00
CityFun 537e2c730c feat: preserve Feishu account map on config updates (#11990) 2026-02-28 09:19:52 +00:00
ssongliu b4dc6b1ab0 chore: optimize runtime region description (#11989) 2026-02-28 09:17:53 +00:00
CityFun 1f22285cf4 chore: refine other language i18n wording (#11988) 2026-02-28 16:41:28 +08:00
A_Words 1d546236e4 fix: restore subsite nginx config on start to avoid stale stop page (#11978) 2026-02-28 15:25:18 +08:00
Kadxy 982e0bbd3e feat: add copy connection URL button to database connection info drawers (#11981)
Add a copy connection URL button to the MySQL, PostgreSQL, and Redis connection info drawers, allowing users to one-click copy the full connection URI for both container and remote connections.
2026-02-28 15:13:58 +08:00
CityFun 5a67a62733 chore: refine en and zh-Hant i18n wording (#11987) 2026-02-28 07:11:52 +00:00
ssongliu 961ebd30a1 fix: resolve overseas script repository sync failure (#11985) 2026-02-28 10:38:35 +08:00
ssongliu 9819546694 chore: optimize internationalization content (#11980) 2026-02-27 14:57:50 +00:00
ssongliu 5c8d391a73 fix: resolve partial application restore failure (#11979)
Refs #11716
2026-02-27 14:43:50 +00:00
CityFun 0e84b9a2a7 feat: fix error after deleting website deployed via deployment for "openclaw" (#11975) 2026-02-27 17:54:19 +08:00
王贺 1f58da501e ci: update build workflow (#11976) 2026-02-27 17:53:52 +08:00
ssongliu 0041184a3e fix: resolve version synchronization issue (#11971) 2026-02-27 07:17:51 +00:00
ssongliu 112468949d feat: Update sqlite connection settings (#11969) 2026-02-27 15:16:59 +08:00
KOMATA 7e03e94460 chore: remove stylelint configuration and ignore files, update lint-staged for stylelint removal (#11968) 2026-02-27 15:08:37 +08:00
ssongliu fd9b299b14 chore: update Edition key (#11962) 2026-02-26 10:05:46 +00:00
ssongliu 37932d7fb2 chore: Complete system security upgrade (#11961) 2026-02-26 10:01:47 +00:00
CityFun f0f2087f54 feat: add browser config for Agent (#11958) 2026-02-26 09:59:48 +00:00
CityFun f3de09e5d7 feat: Add discord channel for Agent (#11950) 2026-02-25 13:59:46 +00:00
maninhill 1a8c1613a4 Update README to eliminate redundant text
Removed duplicate description of 1Panel.
2026-02-25 16:56:40 +08:00
KOMATA 5209275f28 feat: Implement caching for settings retrieval and update logic (#11886)
* feat: Implement caching for settings retrieval and update logic

- Introduced a caching mechanism for settings using go-cache to improve performance.
- Updated the Create, Update, and UpdateOrCreate methods to cache values after database operations.
- Modified the Get and GetValueByKey methods to utilize the cache for faster access.
- Adjusted middleware to use the new GetValueByKey method for retrieving settings, enhancing code consistency.

* feat: Enhance setting cache management with dynamic TTL

- Introduced a function to determine cache TTL based on setting keys, allowing critical settings to have shorter cache durations.
- Updated cache logic in Create, Update, Get, and GetValueByKey methods to utilize the new TTL management, improving performance and consistency in settings retrieval.

* refactor: Simplify setting cache TTL management

- Removed the dynamic TTL function and standardized the cache TTL to a fixed duration for all settings.
- Updated cache logic in Create, Update, Get, and GetValueByKey methods to use the new fixed TTL, enhancing code clarity and maintainability.
- Added a new function to restart the core service after resetting settings, improving the reset command's functionality.

* refactor: Remove core restart functionality from reset commands

- Eliminated the restartCoreAfterReset function to simplify the reset command logic.
- Updated reset commands to return nil after setting changes, enhancing clarity and reducing unnecessary complexity.
2026-02-25 16:40:20 +08:00
ssongliu 924e59d948 fix: Fix login log recording abnormality in some scenarios (#11948) 2026-02-25 08:39:46 +00:00
CityFun 87a39cab3a feat: Add telegram channel for Agent (#11947) 2026-02-25 16:38:34 +08:00
CityFun f9a70757a1 feat: add rememberApiKey for AgentAccount (#11945) 2026-02-25 05:53:45 +00:00
ssongliu 741acee393 fix: Modify command invocation method (#11944) 2026-02-25 05:51:44 +00:00
dependabot[bot] c964a02088 chore(deps): bump filippo.io/edwards25519 from 1.1.0 to 1.1.1 in /agent (#11946)
Bumps [filippo.io/edwards25519](https://github.com/FiloSottile/edwards25519) from 1.1.0 to 1.1.1.
- [Commits](https://github.com/FiloSottile/edwards25519/compare/v1.1.0...v1.1.1)

---
updated-dependencies:
- dependency-name: filippo.io/edwards25519
  dependency-version: 1.1.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-25 11:58:33 +08:00
ssongliu ba71613ae6 fix: Fix docker compose parsing failure issue (#11942)
Refs #11934
2026-02-25 03:51:44 +00:00
ssongliu 5493b41381 fix: Modify monitoring interval prompt information (#11940)
Refs #11918
2026-02-25 02:57:47 +00:00
ssongliu c3f45d4787 feat: v2 supports international version (#11939) 2026-02-25 10:56:06 +08:00
rowanchen-com a8a92da483 Added the ability to sort installed applications. (#11936) 2026-02-25 10:35:29 +08:00
CityFun a77e61a861 feat: Agent Model Account support custom config (#11933)
* feat: Agent Model Account support custom config

* feat: Add agent gatewayControlUi config
2026-02-24 18:26:02 +08:00
kgbow e589728987 refactor(website-ssl): improve API flexibility for WebsiteSSLSearch validation (#11926) 2026-02-24 17:30:37 +08:00
maninhill cc3cdef5f3 Revise 1Panel description and add feature highlights
Updated the description of 1Panel to clarify its purpose and features.
2026-02-24 17:28:14 +08:00
A_Words 7101af0594 fix: Allow wildcard domains in domainOrIP validator for SSL form (#11923) 2026-02-24 17:22:30 +08:00
KOMATA df6cda7143 chore: change esbuild to native version (#11904)
* chore: change esbuild to native version

* chore: add @parcel/watcher and esbuild to trustedDependencies
2026-02-24 17:21:20 +08:00
KOMATA 62beb534d2 feat: enhance passkey function 2026-02-24 17:19:13 +08:00
adriandadri 9a70bf1c2d feat: Add support for Technitium DNS (#11896)
Co-authored-by: adri <adri@digitalunited.net>
2026-02-24 17:13:59 +08:00
KOMATA 6f4559c7cd style: Update Prettier configuration and adjust SCSS formatting (#11885)
- Added iconfont directory to .prettierignore
- Removed jsxBracketSameLine option from .prettierrc.js
- Fixed indentation in common.scss and reset.scss files
2026-02-24 17:03:20 +08:00
KOMATA 8b2bcdc814 feat: Enhance domain handling in website creation (#11884)
* feat: Enhance domain handling in website creation

* fix: Improve domain trimming and validation in website alias assignment
2026-02-24 16:58:30 +08:00
maninhill 0f6ddf26fd Refine README.md content for conciseness
Removed redundant phrases for clarity.
2026-02-24 14:11:31 +08:00
王贺 ebe472a9db feat(i18n): refine translations (#11931) 2026-02-24 13:42:10 +08:00
ssongliu c1a87baad7 feat: Terminal supports foreground and background colors (#11929) 2026-02-24 13:39:02 +08:00
Scars c7f185a184 feat: Terminal supports custom fonts (#11889) 2026-02-13 18:15:12 +08:00
蘭 4ca2ba5482 feat: File editing supports last opened file (#11893)
#11572
2026-02-13 10:01:14 +00:00
蘭 727b74895d ref: Adjust the column widths of the file system (#11887)
#11699
2026-02-13 09:59:13 +00:00
773 changed files with 88027 additions and 16098 deletions
+14
View File
@@ -0,0 +1,14 @@
version: 2
updates:
- package-ecosystem: "gomod"
directories:
- "/agent"
- "/core"
schedule:
interval: "daily"
- package-ecosystem: "npm"
directory: "/frontend"
schedule:
interval: "daily"
+1 -1
View File
@@ -6,7 +6,7 @@ permissions:
jobs:
generic_handler:
name: Add Labels to PR
if: github.repository == '1Panel-dev/1Panel'
if: github.repository == '1Panel-dev/1Panel' && github.event.pull_request.user.login != 'dependabot[bot]'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
@@ -1,33 +1,43 @@
name: Create Release And Upload Cloudflare R2
name: Build And Publish (OSS + R2)
on:
push:
tags:
- 'v*'
jobs:
create-release:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v4
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: '22.19.0'
node-version: '24.11.1'
- name: Build Web
run: |
cd frontend && npm install && npm run build:pro
env:
NODE_OPTIONS: --max-old-space-size=8192
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.24'
go-version: '1.25'
cache-dependency-path: |
core/go.sum
agent/go.sum
- name: Build Release
uses: goreleaser/goreleaser-action@v6
with:
distribution: goreleaser
version: '~> v2'
args: release --skip=publish --clean
- name: Upload Assets
uses: softprops/action-gh-release@v1
if: startsWith(github.ref, 'refs/tags/')
@@ -36,10 +46,24 @@ jobs:
files: |
dist/*.tar.gz
dist/checksums.txt
- name: Setup OSSUTIL
uses: yizhoumo/setup-ossutil@v2
with:
endpoint: ${{ secrets.OSS_ENDPOINT }}
access-key-id: ${{ secrets.OSS_ACCESS_KEY_ID }}
access-key-secret: ${{ secrets.OSS_ACCESS_KEY_SECRET }}
ossutil-version: '1.7.18'
- name: Upload Assets to OSS
run: |
ossutil cp -r dist/ oss://resource-fit2cloud-com/1panel/package/v2/stable/${{ github.ref_name }}/release/ --include "*.tar.gz" --include "checksums.txt" --only-current-dir --force
- name: Setup Rclone
uses: AnimMouse/setup-rclone@v1
with:
rclone_config: ${{ secrets.RCLONE_CONFIG }}
- name: Upload to Cloudflare R2
run: |
rclone copy dist/ cloudflare_r2:package/v2/stable/${{ github.ref_name }}/release/ --include "*.tar.gz" --include "checksums.txt" --progress
@@ -1,47 +0,0 @@
name: Create Release And Upload assets
on:
push:
tags:
- 'v*'
jobs:
create-release:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v4
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: '22.19.0'
- name: Build Web
run: |
cd frontend && npm install && npm run build:pro
env:
NODE_OPTIONS: --max-old-space-size=8192
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.24'
- name: Build Release
uses: goreleaser/goreleaser-action@v6
with:
distribution: goreleaser
version: '~> v2'
args: release --skip=publish --clean
- name: Upload Assets
uses: softprops/action-gh-release@v1
if: startsWith(github.ref, 'refs/tags/')
with:
draft: true
files: |
dist/*.tar.gz
dist/checksums.txt
- name: Setup OSSUTIL
uses: yizhoumo/setup-ossutil@v2
with:
endpoint: ${{ secrets.OSS_ENDPOINT }}
access-key-id: ${{ secrets.OSS_ACCESS_KEY_ID }}
access-key-secret: ${{ secrets.OSS_ACCESS_KEY_SECRET }}
ossutil-version: '1.7.18'
- name: Upload Assets to OSS
run: ossutil cp -r dist/ oss://resource-fit2cloud-com/1panel/package/v2/stable/${{ github.ref_name }}/release/ --include "*.tar.gz" --include "checksums.txt" --only-current-dir --force
+5 -13
View File
@@ -9,7 +9,6 @@ build/1panel-core
# Mac
.DS_Store
*/.DS_Store
# VS Code
.vscode
@@ -41,20 +40,11 @@ core/cmd/server/web/index.html
frontend/auto-imports.d.ts
frontend/components.d.ts
frontend/src/xpack
frontend/src/xpack-ee
agent/xpack
agent/router/entry_xpack.go
agent/server/init_xpack.go
agent/utils/xpack/xpack.go
agent/utils/xpack/xpack_xpack.go
agent/xpack-ee
core/xpack
core/router/entry_xpack.go
core/server/init_xpack.go
core/utils/xpack/xpack.go
core/utils/xpack/xpack_xpack.go
core/xpack-ee
core/router/entry_xpackee.go
core/server/init_xpackee.go
core/utils/xpack/xpack_xpackee.go
.history/
dist/
@@ -73,4 +63,6 @@ agent/.golangci.yml
openspec
CLAUDE.md
AGENTS.md
opencode.json
opencode.json
superpowers
.worktrees/
+4 -2
View File
@@ -10,10 +10,12 @@ PR are always welcome, even if they only contain small fixes like typos or a few
Please submit a PR broken down into small changes bit by bit. A PR consisting of a lot of features and code changes may be hard to review. It is recommended to submit PRs in an incremental fashion.
This [development guideline](https://docs.1panel.pro/dev_manual/dev_manual/) contains information about repository structure, how to set up development environment, how to run it, and more.
Note: If you split your pull request to small changes, please make sure any of the changes goes to master will not break anything. Otherwise, it can not be merged until this feature complete.
## Add a new translation
If you'd like to help translate 1Panel into a new language, please read the [Translation Contribution Guide](docs/TRANSLATION.md). It lists every file you need to create or modify, along with a reference PR you can use as a concrete example.
## Report issues
It is a great way to contribute by reporting an issue. Well-written and complete bug reports are always welcome! Please open an issue and follow the template to fill in required information.
+65 -36
View File
@@ -1,16 +1,23 @@
<p align="center"><a href="https://1panel.pro"><img src="https://resource.1panel.pro/img/1panel-logo.png" alt="1Panel" width="300" /></a></p>
<h3 align="center">The open-source VPS control panel with native AI agent support</h3>
<p align="center">
Trusted by <strong>2,000,000+</strong> self-hosters worldwide
</p>
<p align="center">
<a href="https://trendshift.io/repositories/2462" target="_blank"><img src="https://trendshift.io/api/badge/repositories/2462" alt="1Panel-dev%2F1Panel | Trendshift" style="width: 240px; height: auto;" /></a>
</p>
<p align="center">
<a href="https://www.gnu.org/licenses/gpl-3.0.html"><img src="https://shields.io/github/license/1Panel-dev/1Panel?color=%231890FF" alt="License: GPL v3"></a>
<a href="https://app.codacy.com/gh/1Panel-dev/1Panel?utm_source=github.com&utm_medium=referral&utm_content=1Panel-dev/1Panel&utm_campaign=Badge_Grade_Dashboard"><img src="https://app.codacy.com/project/badge/Grade/da67574fd82b473992781d1386b937ef" alt="Codacy"></a>
<a href="https://discord.gg/bUpUqWqdRr" target="_blank">
<img src="https://img.shields.io/discord/1318846410149335080?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb"
alt="chat on Discord"></a>
<a href="https://app.codacy.com/gh/1Panel-dev/1Panel"><img src="https://app.codacy.com/project/badge/Grade/da67574fd82b473992781d1386b937ef" alt="Codacy"></a>
<a href="https://discord.gg/bUpUqWqdRr"><img src="https://img.shields.io/discord/1318846410149335080?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb" alt="Discord"></a>
<a href="https://github.com/1Panel-dev/1Panel/releases"><img src="https://img.shields.io/github/v/release/1Panel-dev/1Panel" alt="GitHub release"></a>
<a href="https://github.com/1Panel-dev/1Panel"><img src="https://img.shields.io/github/stars/1Panel-dev/1Panel?color=%231890FF&style=flat-square" alt="Stars"></a><br>
<a href="https://github.com/1Panel-dev/1Panel"><img src="https://img.shields.io/github/stars/1Panel-dev/1Panel?color=%231890FF&style=flat-square" alt="Stars"></a>
</p>
<p align="center">
<a href="/README.md"><img alt="English" src="https://img.shields.io/badge/English-d9d9d9"></a>
<a href="/docs/README.zh-Hans.md"><img alt="中文(简体)" src="https://img.shields.io/badge/中文(简体)-d9d9d9"></a>
@@ -18,69 +25,91 @@
<a href="/docs/README.pt-br.md"><img alt="Português (Brasil)" src="https://img.shields.io/badge/Português (Brasil)-d9d9d9"></a>
<a href="/docs/README.ar.md"><img alt="العربية" src="https://img.shields.io/badge/العربية-d9d9d9"></a>
<a href="/docs/README.de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-d9d9d9"></a>
<a href="/docs/README.es.md"><img alt="Español" src="https://img.shields.io/badge/Español-d9d9d9"></a><br>
<a href="/docs/README.es.md"><img alt="Español" src="https://img.shields.io/badge/Español-d9d9d9"></a>
<a href="/docs/README.fr.md"><img alt="français" src="https://img.shields.io/badge/français-d9d9d9"></a>
<a href="/docs/README.ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-d9d9d9"></a>
<a href="/docs/README.id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-d9d9d9"></a>
<a href="/docs/README.zh-Hant.md"><img alt="中文(繁體)" src="https://img.shields.io/badge/中文(繁體)-d9d9d9"></a>
<a href="/docs/README.tr.md"><img alt="Türkçe" src="https://img.shields.io/badge/Türkçe-d9d9d9"></a>
<a href="/docs/README.ru.md"><img alt="Русский" src="https://img.shields.io/badge/%D0%A0%D1%83%D1%81%D1%81%D0%BA%D0%B8%D0%B9-d9d9d9"></a>
<a href="/docs/README.ru.md"><img alt="Русский" src="https://img.shields.io/badge/Русский-d9d9d9"></a>
<a href="/docs/README.ms.md"><img alt="Bahasa Melayu" src="https://img.shields.io/badge/Bahasa Melayu-d9d9d9"></a>
</p>
------------------------------
## What is 1Panel?
---
1Panel is a modern, open-source Linux server management panel that streamlines administration through an intuitive, clean web interface.
## What is 1Panel?
## Key Features
1Panel is a modern, open-source VPS control panel — and the only one with **native AI agent support**. Run Ollama models, deploy OpenClaw agents, and manage your entire server stack from one clean web interface. No CLI memorization required.
🔹 **AI Integration**
Seamlessly deploy OpenClaw agents, host local LLMs via Ollama, and monitor GPU utilization—all within a unified workspace engineered for efficient AI development and inference.
👉 Watch the [2-minute introduction](https://www.youtube.com/watch?v=Jl_wqp-XA08)
🔹 **One-Click Website Deployment**
Launch production-ready websites instantly with automatic domain binding, SSL certificate provisioning, and configuration—zero manual setup required.
## Why 1Panel?
🔹 **Curated App Marketplace**
Discover, install, and update trusted open-source applications (WordPress, MySQL, Redis, and more) with a single click.
| | 1Panel | cPanel / Plesk | aaPanel | Webmin |
|--|--------|----------------|---------|--------|
| Free & open source | ✅ | ❌ | Partial | ✅ |
| Native AI agent runtime | ✅ | ❌ | ❌ | ❌ |
| One-click app marketplace | ✅ 165+ apps | ❌ | ✅ | ❌ |
| Modern UI (post-2020) | ✅ | ❌ | Partial | ❌ |
| Docker / container management | ✅ | ❌ | ❌ | ❌ |
| Active development | ✅ | ✅ | ✅ | Slow |
🔹 **Centralized Server Management**
Monitor system resources and manage files, databases, containers, and services effortlessly through a unified, intuitive dashboard.
## Key Features
🔹 **Enterprise-Grade Security**
Strengthen defenses with container isolation, firewall controls, audit logging, and secure deployment practices—ensuring compliance and peace of mind.
🔹 **Intelligent Backup & Restore**
Schedule automated backups or perform instant restores to local storage or leading cloud platforms (AWS S3, Cloudflare R2, and more), with flexible retention and destination options.
- **AI Agent Runtime**: Deploy Ollama LLMs, spin up OpenClaw personal agents, and monitor GPU utilization — all from the dashboard. No separate AI stack to manage.
- **One-Click Website Deployment**: Launch production-ready websites with automatic domain binding, SSL provisioning, and Nginx config — zero manual setup.
- **App Marketplace**: 165+ trusted open-source apps (Nextcloud, Bitwarden, Umami, NocoBase, and more) installed and updated with a single click.
- **Docker & Container Management**: Create, start, stop, and inspect containers, images, networks, and volumes through a visual UI — no CLI juggling.
- **Security Out of the Box**: Firewall rules, fail2ban, container isolation, WAF, and audit logs — configured and running from day one.
- **Backup & Restore**: Schedule automated backups to AWS S3, Cloudflare R2, or local storage. Restore any snapshot in one click.
## Quick Start
Execute the script below and follow the prompts to install 1Panel:
> **Requirements:** Linux VPS (Debian / Ubuntu / CentOS / Rocky), 1 GB RAM, internet access.
> Takes ~60 seconds.
```bash
bash -c "$(curl -sSL https://resource.fit2cloud.com/1panel/package/v2/quick_start.sh)"
bash -c "$(curl -sSL https://resource.1panel.pro/v2/quick_start.sh)"
```
After installation, open `http://<your-server-ip>:<port>/<security-path>` in your browser.
Run `1pctl user-info` via SSH if you need to retrieve your access credentials.
## Screenshot
![UI Display](https://resource.1panel.pro/img/1panel.png)
![1Panel UI](https://resource.1panel.pro/img/overview_en_v2.png)
## Pro Edition
1Panel OSS is free forever. Pro adds features built for teams and production workloads:
| Feature | OSS | Pro |
|---------|:---:|:---:|
| One-click app installs | ✅ | ✅ |
| AI agents (OpenClaw) | 1 agent | Unlimited |
| WAF & advanced security | Basic | ✅ |
| Website tamper protection | ❌ | ✅ |
| Website uptime monitoring | ❌ | ✅ |
| Multi-node management | ❌ | ✅ |
| Custom logo & theme | ❌ | ✅ |
| Priority support | ❌ | ✅ |
**From $80/year.** [Compare plans & start 30-day free trial →](https://1panel.pro/pricing)
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=1Panel-dev/1Panel&type=Date)](https://star-history.com/#1Panel-dev/1Panel&Date)
## Pro Edition
## Community & Support
Compared to the OSS Edition, 1Panel Pro Edition provides users with a wealth of enhanced features and technical support services. Enhanced features include WAF enhancement, website tamper protection, website monitoring, GPU monitoring, custom logo and theme color, etc. [Click to view the detailed introduction of the Pro Edition](https://1panel.pro/pricing).
- **Discord** — [Join the community](https://discord.gg/bUpUqWqdRr) for help, feature requests, and show-and-tell
- **Docs** — [1panel.pro/docs](https://1panel.pro/docs)
- **Issues** — [GitHub Issues](https://github.com/1Panel-dev/1Panel/issues) for bug reports
## Security Information
## Security
If you discover any security issues, please refer to [SECURITY.md](/SECURITY.md).
Found a vulnerability? Please read [SECURITY.md](/SECURITY.md) before disclosing.
## License
Licensed under The GNU General Public License version 3 (GPLv3) (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
<https://www.gnu.org/licenses/gpl-3.0.html>
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Licensed under the [GNU General Public License v3.0](https://www.gnu.org/licenses/gpl-3.0.html).
File diff suppressed because it is too large Load Diff
+13 -1
View File
@@ -109,7 +109,7 @@ func (b *BaseApi) LoadPort(c *gin.Context) {
// @Success 200 {object} response.DatabaseConn
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /apps/installed/conninfo [POST]
// @Router /apps/installed/conninfo [post]
func (b *BaseApi) LoadConnInfo(c *gin.Context) {
var req dto.OperationWithNameAndType
if err := helper.CheckBindAndValidate(&req, c); err != nil {
@@ -349,3 +349,15 @@ func (b *BaseApi) GetAppInstallInfo(c *gin.Context) {
}
helper.SuccessWithData(c, info)
}
func (b *BaseApi) UpdateAppInstallSort(c *gin.Context) {
var req request.AppInstallSort
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := appInstallService.UpdateSort(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
+45
View File
@@ -415,6 +415,11 @@ func (b *BaseApi) Backup(c *gin.Context) {
helper.InternalServer(c, err)
return
}
case "mongodb":
if err := backupService.MongodbBackup(req); err != nil {
helper.InternalServer(c, err)
return
}
case constant.AppPostgresql, constant.AppPostgresqlCluster:
if err := backupService.PostgresqlBackup(req); err != nil {
helper.InternalServer(c, err)
@@ -430,6 +435,16 @@ func (b *BaseApi) Backup(c *gin.Context) {
helper.InternalServer(c, err)
return
}
case "container":
if err := backupService.ContainerBackup(req); err != nil {
helper.InternalServer(c, err)
return
}
case "compose":
if err := backupService.ComposeBackup(req); err != nil {
helper.InternalServer(c, err)
return
}
}
helper.Success(c)
}
@@ -465,6 +480,11 @@ func (b *BaseApi) Recover(c *gin.Context) {
helper.InternalServer(c, err)
return
}
case "mongodb":
if err := backupService.MongodbRecover(req); err != nil {
helper.InternalServer(c, err)
return
}
case constant.AppPostgresql, constant.AppPostgresqlCluster:
if err := backupService.PostgresqlRecover(req); err != nil {
helper.InternalServer(c, err)
@@ -485,6 +505,16 @@ func (b *BaseApi) Recover(c *gin.Context) {
helper.InternalServer(c, err)
return
}
case "container":
if err := backupService.ContainerRecover(req); err != nil {
helper.InternalServer(c, err)
return
}
case "compose":
if err := backupService.ComposeRecover(req); err != nil {
helper.InternalServer(c, err)
return
}
}
helper.Success(c)
}
@@ -510,6 +540,11 @@ func (b *BaseApi) RecoverByUpload(c *gin.Context) {
helper.InternalServer(c, err)
return
}
case "mongodb":
if err := backupService.MongodbRecoverByUpload(req); err != nil {
helper.InternalServer(c, err)
return
}
case constant.AppPostgresql, constant.AppPostgresqlCluster:
if err := backupService.PostgresqlRecoverByUpload(req); err != nil {
helper.InternalServer(c, err)
@@ -525,6 +560,16 @@ func (b *BaseApi) RecoverByUpload(c *gin.Context) {
helper.InternalServer(c, err)
return
}
case "container":
if err := backupService.ContainerRecover(req); err != nil {
helper.InternalServer(c, err)
return
}
case "compose":
if err := backupService.ComposeRecover(req); err != nil {
helper.InternalServer(c, err)
return
}
}
helper.Success(c)
}
+159 -21
View File
@@ -1,6 +1,9 @@
package v2
import (
"net/http"
"net/url"
"path"
"strconv"
"github.com/1Panel-dev/1Panel/agent/app/api/v2/helper"
@@ -53,6 +56,162 @@ func (b *BaseApi) LoadContainerUsers(c *gin.Context) {
helper.SuccessWithData(c, containerService.LoadUsers(req))
}
// @Tags Container
// @Summary List container files
// @Accept json
// @Param request body dto.ContainerFileReq true "request"
// @Success 200 {array} dto.ContainerFileInfo
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /containers/files/search [post]
func (b *BaseApi) ListContainerFiles(c *gin.Context) {
var req dto.ContainerFileReq
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
files, err := containerService.ListContainerFiles(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, files)
}
// @Tags Container
// @Summary Upload container file
// @Accept multipart/form-data
// @Param containerID formData string true "containerID"
// @Param path formData string true "path"
// @Param file formData file true "file"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /containers/files/upload [post]
// @x-panel-log {"bodyKeys":["containerID","path"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"容器 [containerID] 上传文件到 [path]","formatEN":"Upload file to [path] in container [containerID]"}
func (b *BaseApi) UploadContainerFile(c *gin.Context) {
form, err := c.MultipartForm()
if err != nil {
helper.BadRequest(c, err)
return
}
containerIDs := form.Value["containerID"]
paths := form.Value["path"]
uploadFiles := form.File["file"]
if len(containerIDs) == 0 || len(paths) == 0 || len(uploadFiles) == 0 {
helper.BadRequest(c, errors.New("invalid container file upload params"))
return
}
req := dto.ContainerFileReq{
ContainerID: containerIDs[0],
Path: paths[0],
}
for _, uploadFile := range uploadFiles {
file, err := uploadFile.Open()
if err != nil {
helper.InternalServer(c, err)
return
}
err = containerService.UploadContainerFile(req, path.Base(uploadFile.Filename), uploadFile.Size, file)
_ = file.Close()
if err != nil {
helper.InternalServer(c, err)
return
}
}
helper.Success(c)
}
// @Tags Container
// @Summary Get container file content
// @Accept json
// @Param request body dto.ContainerFileReq true "request"
// @Success 200 {object} dto.ContainerFileContent
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /containers/files/content [post]
func (b *BaseApi) GetContainerFileContent(c *gin.Context) {
var req dto.ContainerFileReq
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
content, err := containerService.GetContainerFileContent(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, content)
}
// @Tags Container
// @Summary Get container file size
// @Accept json
// @Param request body dto.ContainerFileReq true "request"
// @Success 200 {int} size
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /containers/files/size [post]
func (b *BaseApi) GetContainerFileSize(c *gin.Context) {
var req dto.ContainerFileReq
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
size, err := containerService.GetContainerFileSize(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, size)
}
// @Tags Container
// @Summary Delete container file
// @Accept json
// @Param request body dto.ContainerFileBatchDeleteReq true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /containers/files/del [post]
// @x-panel-log {"bodyKeys":["containerID","paths"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"删除容器 [containerID] 文件 [paths]","formatEN":"Delete files [paths] in container [containerID]"}
func (b *BaseApi) DeleteContainerFile(c *gin.Context) {
var req dto.ContainerFileBatchDeleteReq
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := containerService.DeleteContainerFile(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags Container
// @Summary Download container file
// @Accept json
// @Param request body dto.ContainerFileReq true "request"
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /containers/files/download [post]
// @x-panel-log {"bodyKeys":["containerID","path"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"下载容器 [containerID] 文件 [path]","formatEN":"Download file [path] from container [containerID]"}
func (b *BaseApi) DownloadContainerFile(c *gin.Context) {
var req dto.ContainerFileReq
if err := c.ShouldBindJSON(&req); err != nil {
helper.BadRequest(c, err)
return
}
if req.ContainerID == "" || req.Path == "" {
helper.BadRequest(c, errors.New("invalid container file download params"))
return
}
reader, fileName, contentType, err := containerService.DownloadContainerFile(req)
if err != nil {
helper.InternalServer(c, err)
return
}
defer reader.Close()
c.Header("Content-Disposition", "attachment; filename*=utf-8''"+url.PathEscape(fileName))
c.DataFromReader(http.StatusOK, -1, contentType, reader, nil)
}
// @Tags Container
// @Summary List containers
// @Accept json
@@ -306,27 +465,6 @@ func (b *BaseApi) ContainerCreate(c *gin.Context) {
helper.Success(c)
}
// @Tags Container
// @Summary Create container by command
// @Accept json
// @Param request body dto.ContainerCreateByCommand true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /containers/command [post]
func (b *BaseApi) ContainerCreateByCommand(c *gin.Context) {
var req dto.ContainerCreateByCommand
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := containerService.ContainerCreateByCommand(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags Container
// @Summary Upgrade container
// @Accept json
+294
View File
@@ -0,0 +1,294 @@
package v2
import (
"context"
"encoding/base64"
"github.com/1Panel-dev/1Panel/agent/app/api/v2/helper"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/gin-gonic/gin"
)
// @Tags Database Mongodb
// @Summary Create mongodb database
// @Accept json
// @Param request body dto.MongodbDBCreate true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /databases/mongodb [post]
// @x-panel-log {"bodyKeys":["name"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"创建 mongodb 数据库 [name]","formatEN":"create mongodb database [name]"}
func (b *BaseApi) CreateMongodb(c *gin.Context) {
var req dto.MongodbDBCreate
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if len(req.Password) != 0 {
password, err := base64.StdEncoding.DecodeString(req.Password)
if err != nil {
helper.BadRequest(c, err)
return
}
req.Password = string(password)
}
if _, err := mongodbService.Create(context.Background(), req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags Database Mongodb
// @Summary Page mongodb databases
// @Accept json
// @Param request body dto.MongodbDBSearch true "request"
// @Success 200 {object} dto.PageResult
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /databases/mongodb/search [post]
func (b *BaseApi) SearchMongodb(c *gin.Context) {
var req dto.MongodbDBSearch
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
total, list, err := mongodbService.SearchWithPage(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, dto.PageResult{
Items: list,
Total: total,
})
}
// @Tags Database Mongodb
// @Summary Update mongodb database description
// @Accept json
// @Param request body dto.UpdateDescription true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /databases/mongodb/description [post]
// @x-panel-log {"bodyKeys":["id","description"],"paramKeys":[],"BeforeFunctions":[{"input_column":"id","input_value":"id","isList":false,"db":"database_mongodbs","output_column":"name","output_value":"name"}],"formatZH":"mongodb 数据库 [name] 描述信息修改 [description]","formatEN":"The description of the mongodb database [name] is modified => [description]"}
func (b *BaseApi) UpdateMongodbDescription(c *gin.Context) {
var req dto.UpdateDescription
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := mongodbService.UpdateDescription(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags Database Mongodb
// @Summary Load mongodb database from remote
// @Accept json
// @Param request body dto.MongodbLoadDB true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /databases/mongodb/load [post]
func (b *BaseApi) LoadMongodbFromRemote(c *gin.Context) {
var req dto.MongodbLoadDB
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := mongodbService.LoadFromRemote(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags Database Mongodb
// @Summary Bind mongodb database user info
// @Accept json
// @Param request body dto.MongodbBind true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /databases/mongodb/bind [post]
// @x-panel-log {"bodyKeys":["database", "name", "username"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"绑定 mongodb 数据库 [database] [name] 用户 [username]","formatEN":"bind mongodb database [database] [name] user [username]"}
func (b *BaseApi) BindMongodbUser(c *gin.Context) {
var req dto.MongodbBind
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if len(req.Password) != 0 {
password, err := base64.StdEncoding.DecodeString(req.Password)
if err != nil {
helper.BadRequest(c, err)
return
}
req.Password = string(password)
}
if err := mongodbService.BindUser(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags Database Mongodb
// @Summary Change mongodb database password
// @Accept json
// @Param request body dto.MongodbPassword true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /databases/mongodb/password [post]
// @x-panel-log {"bodyKeys":["database", "name"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"更新 mongodb 数据库 [database] [name] 密码","formatEN":"update mongodb database [database] [name] password"}
func (b *BaseApi) ChangeMongodbPassword(c *gin.Context) {
var req dto.MongodbPassword
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if len(req.Password) != 0 {
password, err := base64.StdEncoding.DecodeString(req.Password)
if err != nil {
helper.BadRequest(c, err)
return
}
req.Password = string(password)
}
if err := mongodbService.ChangePassword(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags Database Mongodb
// @Summary Change mongodb root password
// @Accept json
// @Param request body dto.ChangeDBInfo true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /databases/mongodb/root/password [post]
// @x-panel-log {"bodyKeys":["database"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"更新 mongodb 数据库 [database] root 密码","formatEN":"update mongodb database [database] root password"}
func (b *BaseApi) ChangeMongodbRootPassword(c *gin.Context) {
var req dto.ChangeDBInfo
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if len(req.Value) != 0 {
value, err := base64.StdEncoding.DecodeString(req.Value)
if err != nil {
helper.BadRequest(c, err)
return
}
req.Value = string(value)
}
if err := mongodbService.ChangeRootPassword(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags Database Mongodb
// @Summary Load mongodb privileges
// @Accept json
// @Param request body dto.MongodbPrivilegesLoad true "request"
// @Success 200 {string} string
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /databases/mongodb/privileges [post]
func (b *BaseApi) LoadMongodbPrivileges(c *gin.Context) {
var req dto.MongodbPrivilegesLoad
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
permission, err := mongodbService.LoadPrivileges(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, permission)
}
// @Tags Database Mongodb
// @Summary Change mongodb privileges
// @Accept json
// @Param request body dto.MongodbPrivileges true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /databases/mongodb/privileges/change [post]
// @x-panel-log {"bodyKeys":["database", "username"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"更新 mongodb 数据库 [database] 用户 [username] 权限","formatEN":"update mongodb database [database] user [username] privileges"}
func (b *BaseApi) ChangeMongodbPrivileges(c *gin.Context) {
var req dto.MongodbPrivileges
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := mongodbService.ChangePrivileges(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags Database Mongodb
// @Summary Check before delete mongodb database
// @Accept json
// @Param request body dto.MongodbDBDeleteCheck true "request"
// @Success 200 {array} string
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /databases/mongodb/del/check [post]
func (b *BaseApi) DeleteCheckMongodb(c *gin.Context) {
var req dto.MongodbDBDeleteCheck
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
apps, err := mongodbService.DeleteCheck(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, apps)
}
// @Tags Database Mongodb
// @Summary Delete mongodb database
// @Accept json
// @Param request body dto.MongodbDBDelete true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /databases/mongodb/del [post]
// @x-panel-log {"bodyKeys":["id"],"paramKeys":[],"BeforeFunctions":[{"input_column":"id","input_value":"id","isList":false,"db":"database_mongodbs","output_column":"name","output_value":"name"}],"formatZH":"删除 mongodb 数据库 [name]","formatEN":"delete mongodb database [name]"}
func (b *BaseApi) DeleteMongodb(c *gin.Context) {
var req dto.MongodbDBDelete
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
tx, ctx := helper.GetTxAndContext()
if err := mongodbService.Delete(ctx, req); err != nil {
helper.InternalServer(c, err)
tx.Rollback()
return
}
tx.Commit()
helper.Success(c)
}
+2 -2
View File
@@ -12,7 +12,7 @@ import (
// @Tags Container Docker
// @Summary Load docker status
// @Produce json
// @Success 200 {string} dto.DockerStatus
// @Success 200 {object} dto.DockerStatus
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /containers/docker/status [get]
@@ -24,7 +24,7 @@ func (b *BaseApi) LoadDockerStatus(c *gin.Context) {
// @Tags Container Docker
// @Summary Load docker daemon.json
// @Produce json
// @Success 200 {object} string
// @Success 200 {string} string
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /containers/daemonjson/file [get]
+10 -6
View File
@@ -31,17 +31,20 @@ var (
dbCommonService = service.NewIDBCommonService()
mysqlService = service.NewIMysqlService()
postgresqlService = service.NewIPostgresqlService()
mongodbService = service.NewIMongodbService()
databaseService = service.NewIDatabaseService()
redisService = service.NewIRedisService()
cronjobService = service.NewICronjobService()
fileService = service.NewIFileService()
sshService = service.NewISSHService()
firewallService = service.NewIFirewallService()
iptablesService = service.NewIIptablesService()
monitorService = service.NewIMonitorService()
systemService = service.NewISystemService()
fileService = service.NewIFileService()
fileHistoryService = service.NewIFileHistoryService()
fileShareService = service.NewIFileShareService()
sshService = service.NewISSHService()
firewallService = service.NewIFirewallService()
iptablesService = service.NewIIptablesService()
monitorService = service.NewIMonitorService()
systemService = service.NewISystemService()
deviceService = service.NewIDeviceService()
fail2banService = service.NewIFail2BanService()
@@ -70,6 +73,7 @@ var (
recycleBinService = service.NewIRecycleBinService()
favoriteService = service.NewIFavoriteService()
hostService = service.NewIHostService()
websiteCAService = service.NewIWebsiteCAService()
taskService = service.NewITaskService()
+330
View File
@@ -25,6 +25,7 @@ import (
websocket2 "github.com/1Panel-dev/1Panel/agent/utils/websocket"
"github.com/gin-gonic/gin"
"github.com/gorilla/websocket"
qrcode "github.com/skip2/go-qrcode"
)
// @Tags File
@@ -48,6 +49,31 @@ func (b *BaseApi) ListFiles(c *gin.Context) {
helper.SuccessWithData(c, fileList)
}
// @Tags File
// @Summary File search: content grep + optional AI summary
// @Description When file-management AI is enabled, returns mode=ai with summary and hits. When disabled, returns mode=grep with hits only. Scans file contents only. Supports match options, extension/size/time filters, and scan limits.
// @Accept json
// @Param request body request.FileAISearch true "request"
// @Success 200 {object} response.FileAISearchResult
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/ai-search [post]
func (b *BaseApi) FileAISearch(c *gin.Context) {
var req request.FileAISearch
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if strings.TrimSpace(req.ResponseLanguage) == "" {
req.ResponseLanguage = strings.TrimSpace(c.GetHeader("Accept-Language"))
}
res, err := fileService.AISearch(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, res)
}
// @Tags File
// @Summary Page file
// @Accept json
@@ -224,6 +250,26 @@ func (b *BaseApi) CompressFile(c *gin.Context) {
helper.Success(c)
}
// @Tags File
// @Summary Stop compress task
// @Accept json
// @Param request body request.FileCompressStopReq true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/compress/stop [post]
func (b *BaseApi) StopCompressFile(c *gin.Context) {
var req request.FileCompressStopReq
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := fileService.StopCompress(req.TaskID); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags File
// @Summary Decompress file
// @Accept json
@@ -246,6 +292,26 @@ func (b *BaseApi) DeCompressFile(c *gin.Context) {
helper.Success(c)
}
// @Tags File
// @Summary Stop decompress task
// @Accept json
// @Param request body request.FileDeCompressStopReq true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/decompress/stop [post]
func (b *BaseApi) StopDeCompressFile(c *gin.Context) {
var req request.FileDeCompressStopReq
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := fileService.StopDeCompress(req.TaskID); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags File
// @Summary Load file content
// @Accept json
@@ -515,6 +581,29 @@ func (b *BaseApi) WgetFile(c *gin.Context) {
})
}
// @Tags File
// @Summary Stop wget file download
// @Accept json
// @Param request body request.FileProcessReq true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/wget/stop [post]
// @x-panel-log {"bodyKeys":["key"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"停止下载任务 [key]","formatEN":"Stop wget task [key]"}
func (b *BaseApi) StopWget(c *gin.Context) {
var req request.FileProcessReq
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if strings.TrimSpace(req.Key) == "" {
helper.BadRequest(c, errors.New("key is required"))
return
}
files.CancelDownload(req.Key)
helper.Success(c)
}
// @Tags File
// @Summary Move file
// @Accept json
@@ -1015,3 +1104,244 @@ func (b *BaseApi) SetFileRemark(c *gin.Context) {
}
helper.Success(c)
}
// @Tags File
// @Summary List file shares
// @Accept json
// @Param request body dto.PageInfo true "request"
// @Success 200 {object} dto.PageResult
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/share/search [post]
func (b *BaseApi) SearchFileShare(c *gin.Context) {
var req dto.PageInfo
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
total, list, err := fileShareService.Page(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, dto.PageResult{
Total: total,
Items: list,
})
}
// @Tags File
// @Summary Get file share detail by path
// @Accept json
// @Param request body dto.FilePath true "request"
// @Success 200 {object} response.FileShareInfo
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/share/detail [post]
func (b *BaseApi) GetFileShareDetail(c *gin.Context) {
var req dto.FilePath
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
info, err := fileShareService.GetByPath(req.Path)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, info)
}
// @Tags File
// @Summary Get file share detail by code (no login)
// @Param code query string true "share code"
// @Success 200 {object} response.FileSharePublicInfo
// @Router /files/share/info [get]
func (b *BaseApi) GetPublicFileShareInfo(c *gin.Context) {
code := strings.TrimSpace(c.Query("code"))
if code == "" {
helper.BadRequest(c, errors.New("code is required"))
return
}
info, err := fileShareService.GetPublicByCode(code)
if err != nil {
if be, ok := err.(buserr.BusinessError); ok {
helper.ErrorWithDetail(c, http.StatusBadRequest, be.Msg, be)
return
}
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, info)
}
func buildSharePublicURL(c *gin.Context, code, operateNode string) string {
scheme := strings.TrimSpace(c.GetHeader("X-Forwarded-Proto"))
if scheme == "" {
if c.Request.TLS != nil {
scheme = "https"
} else {
scheme = "http"
}
}
host := strings.TrimSpace(c.GetHeader("X-Forwarded-Host"))
if host == "" {
host = c.Request.Host
}
shareURL := url.URL{
Scheme: scheme,
Host: host,
Path: "/s/" + url.PathEscape(code),
}
query := shareURL.Query()
if strings.TrimSpace(operateNode) != "" {
query.Set("operateNode", operateNode)
}
shareURL.RawQuery = query.Encode()
return shareURL.String()
}
// @Tags File
// @Summary Get file share QR code image
// @Produce png
// @Param code query string true "share code"
// @Param operateNode query string false "operate node"
// @Success 200 {file} file
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/share/qrcode [get]
func (b *BaseApi) GetFileShareQRCode(c *gin.Context) {
code := strings.TrimSpace(c.Query("code"))
if code == "" {
helper.BadRequest(c, errors.New("code is required"))
return
}
if _, err := fileShareService.GetByCode(code); err != nil {
if be, ok := err.(buserr.BusinessError); ok {
helper.ErrorWithDetail(c, http.StatusBadRequest, be.Msg, be)
return
}
helper.InternalServer(c, err)
return
}
png, err := qrcode.Encode(buildSharePublicURL(c, code, c.Query("operateNode")), qrcode.Medium, 256)
if err != nil {
helper.InternalServer(c, err)
return
}
c.Header("Cache-Control", "private, max-age=300")
c.Data(http.StatusOK, "image/png", png)
}
// @Tags File
// @Summary Create temporary file share link
// @Accept json
// @Param request body request.FileShareCreate true "request"
// @Success 200 {object} response.FileShareInfo
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/share/create [post]
// @x-panel-log {"bodyKeys":["path","expireMinutes"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"创建文件分享 [path]","formatEN":"Create file share [path]"}
func (b *BaseApi) CreateFileShare(c *gin.Context) {
var req request.FileShareCreate
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
res, err := fileShareService.Create(req)
if err != nil {
if be, ok := err.(buserr.BusinessError); ok {
helper.ErrorWithDetail(c, http.StatusInternalServerError, be.Msg, be.Err)
return
}
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, res)
}
// @Tags File
// @Summary Delete file share by path
// @Accept json
// @Param request body dto.FilePath true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/share/del [post]
// @x-panel-log {"bodyKeys":["path"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"关闭文件分享 [path]","formatEN":"Close file share [path]"}
func (b *BaseApi) DeleteFileShare(c *gin.Context) {
var req dto.FilePath
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := fileShareService.DeleteByPath(req.Path); err != nil {
if be, ok := err.(buserr.BusinessError); ok {
helper.ErrorWithDetail(c, http.StatusInternalServerError, be.Msg, be.Err)
return
}
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags File
// @Summary Check file share code (no login)
// @Param code query string true "share code"
// @Param password query string false "optional password"
// @Success 200 {object} dto.Response
// @Router /files/share/check [get]
func (b *BaseApi) CheckFileShare(c *gin.Context) {
code := strings.TrimSpace(c.Query("code"))
password := c.Query("password")
if code == "" {
helper.BadRequest(c, errors.New("code is required"))
return
}
if err := fileShareService.Check(code, password); err != nil {
if be, ok := err.(buserr.BusinessError); ok {
helper.ErrorWithDetail(c, http.StatusBadRequest, be.Msg, be)
return
}
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags File
// @Summary Download file by share code (no login)
// @Produce octet-stream
// @Param code query string true "share code"
// @Param password query string false "optional password"
// @Success 200 {file} file
// @Router /files/share/download [get]
func (b *BaseApi) DownloadFileShare(c *gin.Context) {
code := strings.TrimSpace(c.Query("code"))
password := c.Query("password")
if code == "" {
helper.BadRequest(c, errors.New("code is required"))
return
}
filePath, displayName, err := fileShareService.PrepareDownload(code, password)
if err != nil {
if be, ok := err.(buserr.BusinessError); ok {
helper.ErrorWithDetail(c, http.StatusBadRequest, be.Msg, be)
return
}
helper.InternalServer(c, err)
return
}
file, err := os.Open(filePath)
if err != nil {
helper.InternalServer(c, err)
return
}
defer file.Close()
info, err := file.Stat()
if err != nil {
helper.InternalServer(c, err)
return
}
c.Header("Content-Length", strconv.FormatInt(info.Size(), 10))
c.Header("Content-Disposition", "attachment; filename*=utf-8''"+url.PathEscape(displayName))
http.ServeContent(c.Writer, c.Request, displayName, info.ModTime(), file)
}
+91
View File
@@ -0,0 +1,91 @@
package v2
import (
"github.com/1Panel-dev/1Panel/agent/app/api/v2/helper"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/dto/request"
"github.com/gin-gonic/gin"
)
// @Tags File
// @Summary Load file history list
// @Accept json
// @Param request body request.FileHistorySearchReq true "request"
// @Success 200 {object} response.PageResult
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/history/search [post]
func (b *BaseApi) SearchFileHistory(c *gin.Context) {
var req request.FileHistorySearchReq
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
total, items, err := fileHistoryService.Search(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, dto.PageResult{Items: items, Total: total})
}
// @Tags File
// @Summary Load file history content
// @Accept json
// @Param request body request.FileHistoryContentReq true "request"
// @Success 200 {object} response.FileHistoryInfo
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/history/content [post]
func (b *BaseApi) GetFileHistoryContent(c *gin.Context) {
var req request.FileHistoryContentReq
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
item, err := fileHistoryService.GetContent(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, item)
}
// @Tags File
// @Summary Delete file history record
// @Accept json
// @Param request body request.FileHistoryDeleteReq true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/history/del [post]
func (b *BaseApi) DeleteFileHistory(c *gin.Context) {
var req request.FileHistoryDeleteReq
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := fileHistoryService.Delete(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags File
// @Summary Restore file history record
// @Accept json
// @Param request body request.FileHistoryRestoreReq true "request"
// @Success 200 {object} response.FileInfo
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /files/history/restore [post]
func (b *BaseApi) RestoreFileHistory(c *gin.Context) {
var req request.FileHistoryRestoreReq
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
info, err := fileHistoryService.Restore(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, info)
}
+167
View File
@@ -0,0 +1,167 @@
package v2
import (
"github.com/1Panel-dev/1Panel/agent/app/api/v2/helper"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/utils/encrypt"
"github.com/gin-gonic/gin"
)
func (b *BaseApi) CreateHost(c *gin.Context) {
var req dto.HostOperate
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
host, err := hostService.Create(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, host)
}
func (b *BaseApi) TestByInfo(c *gin.Context) {
var req dto.HostConnTest
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
helper.SuccessWithData(c, hostService.TestByInfo(req))
}
func (b *BaseApi) TestByID(c *gin.Context) {
var req dto.OperateByID
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
helper.SuccessWithData(c, hostService.TestLocalConn(req.ID))
}
func (b *BaseApi) HostTree(c *gin.Context) {
var req dto.SearchForTree
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
data, err := hostService.SearchForTree(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, data)
}
func (b *BaseApi) SearchHost(c *gin.Context) {
var req dto.SearchPageWithGroup
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
total, list, err := hostService.SearchWithPage(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, dto.PageResult{Items: list, Total: total})
}
func (b *BaseApi) DeleteHost(c *gin.Context) {
var req dto.OperateByIDs
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := hostService.Delete(req.IDs); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
func (b *BaseApi) UpdateHost(c *gin.Context) {
var req dto.HostOperate
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
var err error
if len(req.Password) != 0 && req.AuthMode == "password" {
req.Password, err = hostService.EncryptHost(req.Password)
if err != nil {
helper.BadRequest(c, err)
return
}
req.PrivateKey = ""
req.PassPhrase = ""
}
if len(req.PrivateKey) != 0 && req.AuthMode == "key" {
req.PrivateKey, err = hostService.EncryptHost(req.PrivateKey)
if err != nil {
helper.BadRequest(c, err)
return
}
if len(req.PassPhrase) != 0 {
req.PassPhrase, err = encrypt.StringEncrypt(req.PassPhrase)
if err != nil {
helper.BadRequest(c, err)
return
}
}
req.Password = ""
}
upMap := map[string]interface{}{
"name": req.Name,
"group_id": req.GroupID,
"addr": req.Addr,
"port": req.Port,
"user": req.User,
"auth_mode": req.AuthMode,
"remember_password": req.RememberPassword,
"description": req.Description,
}
if req.AuthMode == "password" {
upMap["password"] = req.Password
upMap["private_key"] = ""
upMap["pass_phrase"] = ""
} else {
upMap["password"] = ""
upMap["private_key"] = req.PrivateKey
upMap["pass_phrase"] = req.PassPhrase
}
hostItem, err := hostService.Update(req.ID, upMap)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, hostItem)
}
func (b *BaseApi) UpdateHostGroup(c *gin.Context) {
var req dto.ChangeGroup
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if _, err := hostService.Update(req.ID, map[string]interface{}{"group_id": req.GroupID}); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
func (b *BaseApi) GetHostByID(c *gin.Context) {
var req dto.OperateByID
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
info, err := hostService.GetHostByID(req.ID)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, info)
}
+79
View File
@@ -5,6 +5,7 @@ import (
"github.com/1Panel-dev/1Panel/agent/app/api/v2/helper"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/dto/request"
"github.com/1Panel-dev/1Panel/agent/app/model"
"github.com/1Panel-dev/1Panel/agent/global"
"github.com/1Panel-dev/1Panel/agent/utils/ssh"
@@ -27,6 +28,15 @@ func (b *BaseApi) GetSettingInfo(c *gin.Context) {
helper.SuccessWithData(c, setting)
}
func (b *BaseApi) GetTerminalAISettingInfo(c *gin.Context) {
setting, err := settingService.GetTerminalAIInfo()
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, setting)
}
// @Tags System Setting
// @Summary Load system available status
// @Success 200
@@ -59,6 +69,75 @@ func (b *BaseApi) UpdateSetting(c *gin.Context) {
helper.Success(c)
}
func (b *BaseApi) UpdateTerminalAISetting(c *gin.Context) {
var req dto.TerminalAIInfo
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := settingService.UpdateTerminalAI(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
func (b *BaseApi) GetFileManageAISettingInfo(c *gin.Context) {
setting, err := settingService.GetFileManageAIInfo()
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, setting)
}
func (b *BaseApi) UpdateFileManageAISetting(c *gin.Context) {
var req dto.FileManageAIInfo
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := settingService.UpdateFileManageAI(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags System Setting
// @Summary Load file history setting info
// @Success 200 {object} response.FileHistorySettingInfo
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /settings/file-history/search [post]
func (b *BaseApi) GetFileHistorySettingInfo(c *gin.Context) {
setting, err := settingService.GetFileHistorySettingInfo()
if err != nil {
helper.InternalServer(c, err)
return
}
helper.SuccessWithData(c, setting)
}
// @Tags System Setting
// @Summary Update file history setting
// @Accept json
// @Param request body request.FileHistorySettingUpdate true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /settings/file-history/update [post]
func (b *BaseApi) UpdateFileHistorySetting(c *gin.Context) {
var req request.FileHistorySettingUpdate
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
if err := settingService.UpdateFileHistorySetting(req); err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags System Setting
// @Summary Load local backup dir
// @Success 200 {string} path
+2 -2
View File
@@ -249,14 +249,14 @@ func (b *BaseApi) LoadSSHFile(c *gin.Context) {
// @Tags SSH
// @Summary Update host SSH setting by file
// @Accept json
// @Param request body dto.SSHConf true "request"
// @Param request body dto.SSHConfUpdate true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /hosts/ssh/file/update [post]
// @x-panel-log {"bodyKeys":["key"],"paramKeys":[],"BeforeFunctions":[],"formatZH":"修改 SSH 配置文件 [key]","formatEN":"update SSH conf [key]"}
func (b *BaseApi) UpdateSSHByFile(c *gin.Context) {
var req dto.SettingUpdate
var req dto.SSHConfUpdate
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
+1 -1
View File
@@ -32,7 +32,7 @@ func (b *BaseApi) PageTasks(c *gin.Context) {
// @Tags TaskLog
// @Summary Get the number of executing tasks
// @Success 200 {object} int64
// @Success 200 {integer} int64
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /logs/tasks/executing/count [get]
+40 -4
View File
@@ -9,8 +9,10 @@ import (
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/service"
"github.com/1Panel-dev/1Panel/agent/global"
"github.com/1Panel-dev/1Panel/agent/utils/cmd"
"github.com/1Panel-dev/1Panel/agent/utils/ssh"
"github.com/1Panel-dev/1Panel/agent/utils/terminal"
"github.com/gin-gonic/gin"
"github.com/gorilla/websocket"
@@ -40,9 +42,33 @@ func (b *BaseApi) WsSSH(c *gin.Context) {
return
}
client, err := loadLocalConn()
if wshandleError(wsConn, errors.WithMessage(err, "failed to set up the connection. Please check the host information")) {
return
hostID, _ := strconv.Atoi(c.DefaultQuery("id", "0"))
var client *ssh.SSHClient
if hostID > 0 {
host, err := service.GetHostInfo(uint(hostID))
if wshandleError(wsConn, errors.WithMessage(err, "load host info by id failed")) {
return
}
connInfo := ssh.ConnInfo{
Addr: host.Addr,
Port: int(host.Port),
User: host.User,
AuthMode: host.AuthMode,
Password: host.Password,
PrivateKey: []byte(host.PrivateKey),
}
if len(host.PassPhrase) != 0 {
connInfo.PassPhrase = []byte(host.PassPhrase)
}
client, err = ssh.NewClient(connInfo)
if wshandleError(wsConn, errors.WithMessage(err, "failed to set up the connection. Please check the host information")) {
return
}
} else {
client, err = loadLocalConn()
if wshandleError(wsConn, errors.WithMessage(err, "failed to set up the connection. Please check the host information")) {
return
}
}
defer client.Close()
command := c.DefaultQuery("command", "")
@@ -187,19 +213,29 @@ func loadContainerInitCmd(c *gin.Context) ([]string, error) {
func loadDatabaseInitCmd(c *gin.Context) ([]string, error) {
database := c.Query("database")
databaseType := c.Query("databaseType")
if len(database) == 0 || len(databaseType) == 0 {
if len(databaseType) == 0 {
return nil, fmt.Errorf("error param of database: %s or database type: %s", database, databaseType)
}
databaseConn, err := appInstallService.LoadConnInfo(dto.OperationWithNameAndType{Type: databaseType, Name: database})
if err != nil {
return nil, fmt.Errorf("no such database in db, err: %v", err)
}
if len(databaseConn.ContainerName) == 0 {
return nil, fmt.Errorf("no such database container for database: %s or database type: %s", database, databaseType)
}
commands := []string{"exec", "-it", databaseConn.ContainerName}
switch databaseType {
case "mysql", "mysql-cluster":
commands = append(commands, []string{"mysql", "-uroot", "-p" + databaseConn.Password}...)
case "mariadb":
commands = append(commands, []string{"mariadb", "-uroot", "-p" + databaseConn.Password}...)
case "mongodb":
commands = append(commands, []string{
"mongosh",
"--username", databaseConn.Username,
"--password", databaseConn.Password,
"--authenticationDatabase", "admin",
}...)
case "postgresql", "postgresql-cluster":
commands = []string{"exec", "-e", fmt.Sprintf("PGPASSWORD=%s", databaseConn.Password), "-it", databaseConn.ContainerName, "psql", "-t", "-U", databaseConn.Username}
}
+46 -2
View File
@@ -506,7 +506,7 @@ func (b *BaseApi) GetProxyConfig(c *gin.Context) {
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /websites/proxies/update [post]
// @x-panel-log {"bodyKeys":["id"],"paramKeys":[],"BeforeFunctions":[{"input_column":"id","input_value":"id","isList":false,"db":"websites","output_column":"primary_domain","output_value":"domain"}],"formatZH":"修改网站 [domain] 反向代理配置 ","formatEN":"Update domain [domain] proxy config"}
// @x-panel-log {"bodyKeys":["id","name","operate"],"paramKeys":[],"BeforeFunctions":[{"input_column":"id","input_value":"id","isList":false,"db":"websites","output_column":"primary_domain","output_value":"domain"}],"formatZH":"修改 [operate] 网站 [domain] 反向代理配置 [name] ","formatEN":"Update [operate] domain [domain] proxy config [name]"}
func (b *BaseApi) UpdateProxyConfig(c *gin.Context) {
var req request.WebsiteProxyConfig
if err := helper.CheckBindAndValidate(&req, c); err != nil {
@@ -520,6 +520,50 @@ func (b *BaseApi) UpdateProxyConfig(c *gin.Context) {
helper.Success(c)
}
// @Tags Website
// @Summary Delete proxy config
// @Accept json
// @Param request body request.WebsiteProxyDel true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /websites/proxies/delete [post]
// @x-panel-log {"bodyKeys":["id","name"],"paramKeys":[],"BeforeFunctions":[{"input_column":"id","input_value":"id","isList":false,"db":"websites","output_column":"primary_domain","output_value":"domain"}],"formatZH":"删除网站 [domain] 反向代理配置 [name] ","formatEN":"Delete domain [domain] proxy config [name]"}
func (b *BaseApi) DeleteProxyConfig(c *gin.Context) {
var req request.WebsiteProxyDel
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
err := websiteService.DeleteProxy(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags Website
// @Summary Update proxy config status
// @Accept json
// @Param request body request.WebsiteProxyStatusUpdate true "request"
// @Success 200
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /websites/proxies/status [post]
// @x-panel-log {"bodyKeys":["id","name","status"],"paramKeys":[],"BeforeFunctions":[{"input_column":"id","input_value":"id","isList":false,"db":"websites","output_column":"primary_domain","output_value":"domain"}],"formatZH":"修改网站 [domain] 反向代理配置 [name] 状态 [status] ","formatEN":"Update domain [domain] proxy config [name] status [status]"}
func (b *BaseApi) UpdateProxyConfigStatus(c *gin.Context) {
var req request.WebsiteProxyStatusUpdate
if err := helper.CheckBindAndValidate(&req, c); err != nil {
return
}
err := websiteService.UpdateProxyStatus(req)
if err != nil {
helper.InternalServer(c, err)
return
}
helper.Success(c)
}
// @Tags Website
// @Summary Update proxy file
// @Accept json
@@ -528,7 +572,7 @@ func (b *BaseApi) UpdateProxyConfig(c *gin.Context) {
// @Security ApiKeyAuth
// @Security Timestamp
// @Router /websites/proxies/file [post]
// @x-panel-log {"bodyKeys":["websiteID"],"paramKeys":[],"BeforeFunctions":[{"input_column":"id","input_value":"websiteID","isList":false,"db":"websites","output_column":"primary_domain","output_value":"domain"}],"formatZH":"更新反向代理文件 [domain]","formatEN":"Nginx conf proxy file update [domain]"}
// @x-panel-log {"bodyKeys":["websiteID","name"],"paramKeys":[],"BeforeFunctions":[{"input_column":"id","input_value":"websiteID","isList":false,"db":"websites","output_column":"primary_domain","output_value":"domain"}],"formatZH":"修改网站 [domain] 反向代理配置文件 [name] ","formatEN":"Update domain [domain] proxy config file [name]"}
func (b *BaseApi) UpdateProxyConfigFile(c *gin.Context) {
var req request.NginxProxyUpdate
if err := helper.CheckBindAndValidate(&req, c); err != nil {
+577 -76
View File
@@ -3,51 +3,60 @@ package dto
import "time"
type AgentCreateReq struct {
Name string `json:"name" validate:"required"`
AppVersion string `json:"appVersion" validate:"required"`
WebUIPort int `json:"webUIPort" validate:"required"`
BridgePort int `json:"bridgePort" validate:"required"`
Provider string `json:"provider" validate:"required"`
Model string `json:"model" validate:"required"`
AccountID uint `json:"accountId"`
APIKey string `json:"apiKey"`
BaseURL string `json:"baseURL"`
Token string `json:"token"`
TaskID string `json:"taskID"`
Advanced bool `json:"advanced"`
ContainerName string `json:"containerName"`
AllowPort bool `json:"allowPort"`
SpecifyIP string `json:"specifyIP"`
RestartPolicy string `json:"restartPolicy"`
CpuQuota float64 `json:"cpuQuota"`
MemoryLimit float64 `json:"memoryLimit"`
MemoryUnit string `json:"memoryUnit"`
PullImage bool `json:"pullImage"`
EditCompose bool `json:"editCompose"`
DockerCompose string `json:"dockerCompose"`
Name string `json:"name" validate:"required"`
Remark string `json:"remark"`
AppVersion string `json:"appVersion" validate:"required"`
WebUIPort int `json:"webUIPort" validate:"required,min=1,max=65535"`
BridgePort int `json:"bridgePort"`
AllowedOrigins []string `json:"allowedOrigins"`
AgentType string `json:"agentType" validate:"required,oneof=openclaw copaw hermes-agent"`
Model string `json:"model"`
AccountID uint `json:"accountId"`
Token string `json:"token"`
TaskID string `json:"taskID"`
Advanced bool `json:"advanced"`
ContainerName string `json:"containerName"`
AllowPort bool `json:"allowPort"`
SpecifyIP string `json:"specifyIP"`
RestartPolicy string `json:"restartPolicy"`
CpuQuota float64 `json:"cpuQuota"`
MemoryLimit float64 `json:"memoryLimit"`
MemoryUnit string `json:"memoryUnit"`
PullImage bool `json:"pullImage"`
EditCompose bool `json:"editCompose"`
DockerCompose string `json:"dockerCompose"`
}
type AgentItem struct {
ID uint `json:"id"`
Name string `json:"name"`
Provider string `json:"provider"`
ProviderName string `json:"providerName"`
Model string `json:"model"`
BaseURL string `json:"baseUrl"`
APIKey string `json:"apiKey"`
Token string `json:"token"`
Status string `json:"status"`
Message string `json:"message"`
AppInstallID uint `json:"appInstallId"`
AccountID uint `json:"accountId"`
AppVersion string `json:"appVersion"`
Container string `json:"containerName"`
WebUIPort int `json:"webUIPort"`
BridgePort int `json:"bridgePort"`
Path string `json:"path"`
ConfigPath string `json:"configPath"`
Upgradable bool `json:"upgradable"`
CreatedAt time.Time `json:"createdAt"`
ID uint `json:"id"`
Name string `json:"name"`
Remark string `json:"remark"`
AgentType string `json:"agentType"`
Provider string `json:"provider"`
ProviderName string `json:"providerName"`
Model string `json:"model"`
APIType string `json:"apiType"`
MaxTokens int `json:"maxTokens"`
ContextWindow int `json:"contextWindow"`
BaseURL string `json:"baseUrl"`
APIKey string `json:"apiKey"`
Token string `json:"token"`
Status string `json:"status"`
Message string `json:"message"`
AppInstallID uint `json:"appInstallId"`
WebsiteID uint `json:"websiteId"`
WebsitePrimaryDomain string `json:"websitePrimaryDomain"`
WebsiteType string `json:"websiteType"`
WebsiteProtocol string `json:"websiteProtocol"`
AccountID uint `json:"accountId"`
AppVersion string `json:"appVersion"`
Container string `json:"containerName"`
WebUIPort int `json:"webUIPort"`
BridgePort int `json:"bridgePort"`
Path string `json:"path"`
ConfigPath string `json:"configPath"`
Upgradable bool `json:"upgradable"`
CreatedAt time.Time `json:"createdAt"`
}
type AgentDeleteReq struct {
@@ -60,27 +69,193 @@ type AgentTokenResetReq struct {
ID uint `json:"id" validate:"required"`
}
type AgentRemarkUpdateReq struct {
ID uint `json:"id" validate:"required"`
Remark string `json:"remark"`
}
type AgentWebsiteBindReq struct {
AgentID uint `json:"agentId" validate:"required"`
WebsiteID uint `json:"websiteId" validate:"required"`
}
type AgentModelConfigUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
AccountID uint `json:"accountId" validate:"required"`
Model string `json:"model" validate:"required"`
Fallbacks []string `json:"fallbacks"`
}
type AgentModelConfig struct {
AccountID uint `json:"accountId"`
Model string `json:"model"`
Fallbacks []string `json:"fallbacks"`
}
type AgentHermesChatSessionItem struct {
ID string `json:"id"`
Title string `json:"title"`
Model string `json:"model"`
MessageCount int64 `json:"messageCount"`
StartedAt string `json:"startedAt"`
LastActive string `json:"lastActive"`
}
type AgentOverviewReq struct {
AgentID uint `json:"agentId" validate:"required"`
}
type AgentIDReq struct {
AgentID uint `json:"agentId" validate:"required"`
}
type AgentHermesChatSessionRenameReq struct {
AgentID uint `json:"agentId" validate:"required"`
ID string `json:"id" validate:"required"`
Title string `json:"title" validate:"required"`
}
type AgentHermesChatSessionDeleteReq struct {
AgentID uint `json:"agentId" validate:"required"`
ID string `json:"id" validate:"required"`
}
type AgentOverview struct {
Snapshot AgentOverviewSnapshot `json:"snapshot"`
}
type AgentRoleBinding struct {
Channel string `json:"channel" validate:"required"`
AccountID string `json:"accountId"`
}
type AgentRoleCreateReq struct {
AgentID uint `json:"agentId" validate:"required"`
Name string `json:"name" validate:"required"`
Model string `json:"model"`
Bindings []AgentRoleBinding `json:"bindings"`
}
type AgentRoleCreateResp struct {
Output string `json:"output"`
}
type AgentRoleDeleteReq struct {
AgentID uint `json:"agentId" validate:"required"`
ID string `json:"id" validate:"required"`
}
type AgentRoleBindReq struct {
AgentID uint `json:"agentId" validate:"required"`
AccountID uint `json:"accountId" validate:"required"`
Model string `json:"model" validate:"required"`
ID string `json:"id" validate:"required"`
Channel string `json:"channel" validate:"required"`
AccountID string `json:"accountId"`
}
type AgentConfiguredAgentsReq struct {
AgentID uint `json:"agentId" validate:"required"`
}
type AgentRoleChannelsReq struct {
AgentID uint `json:"agentId" validate:"required"`
}
type AgentRoleChannelItem struct {
Name string `json:"name"`
Bound bool `json:"bound"`
AccountIDs []string `json:"accountIds"`
}
type AgentRoleMarkdownFilesReq struct {
AgentID uint `json:"agentId" validate:"required"`
Workspace string `json:"workspace" validate:"required"`
}
type AgentConfiguredAgentItem struct {
ID string `json:"id"`
Name string `json:"name"`
Workspace string `json:"workspace"`
Model string `json:"model"`
AgentDir string `json:"agentDir"`
Bindings []AgentRoleBinding `json:"bindings"`
}
type AgentRoleMarkdownFileItem struct {
Name string `json:"name"`
Content string `json:"content"`
}
type AgentRoleMarkdownFileUpdateItem struct {
Name string `json:"name" validate:"required,oneof=AGENTS.md SOUL.md USER.md IDENTITY.md TOOLS.md HEARTBEAT.md BOOT.md BOOTSTRAP.md"`
Content string `json:"content"`
}
type AgentRoleMarkdownFilesUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
Workspace string `json:"workspace" validate:"required"`
Restart bool `json:"restart"`
Files []AgentRoleMarkdownFileUpdateItem `json:"files" validate:"required"`
}
type AgentOverviewSnapshot struct {
ContainerStatus string `json:"containerStatus"`
AppVersion string `json:"appVersion"`
DefaultModel string `json:"defaultModel"`
ChannelCount int `json:"channelCount"`
SkillCount int `json:"skillCount"`
JobCount int `json:"jobCount"`
SessionCount int `json:"sessionCount"`
}
type AgentAccountModel struct {
RecordID uint `json:"recordId"`
ID string `json:"id"`
Name string `json:"name"`
ContextWindow int `json:"contextWindow"`
MaxTokens int `json:"maxTokens"`
Reasoning bool `json:"reasoning"`
Input []string `json:"input"`
}
type AgentAccountModelReq struct {
AccountID uint `json:"accountId" validate:"required"`
}
type AgentAccountModelCreateReq struct {
AccountID uint `json:"accountId" validate:"required"`
Model AgentAccountModel `json:"model" validate:"required"`
}
type AgentAccountModelUpdateReq struct {
AccountID uint `json:"accountId" validate:"required"`
Model AgentAccountModel `json:"model" validate:"required"`
}
type AgentAccountModelDeleteReq struct {
AccountID uint `json:"accountId" validate:"required"`
RecordID uint `json:"recordId" validate:"required"`
}
type AgentAccountCreateReq struct {
Provider string `json:"provider" validate:"required"`
Name string `json:"name" validate:"required"`
APIKey string `json:"apiKey" validate:"required"`
BaseURL string `json:"baseURL"`
Remark string `json:"remark"`
Provider string `json:"provider" validate:"required"`
Name string `json:"name" validate:"required"`
APIKey string `json:"apiKey" validate:"required"`
RememberAPIKey bool `json:"rememberApiKey"`
BaseURL string `json:"baseURL"`
Models []AgentAccountModel `json:"models"`
APIType string `json:"apiType" validate:"required"`
Remark string `json:"remark"`
}
type AgentAccountUpdateReq struct {
ID uint `json:"id" validate:"required"`
Name string `json:"name" validate:"required"`
APIKey string `json:"apiKey" validate:"required"`
BaseURL string `json:"baseURL"`
Remark string `json:"remark"`
SyncAgents bool `json:"syncAgents"`
ID uint `json:"id" validate:"required"`
Name string `json:"name" validate:"required"`
APIKey string `json:"apiKey" validate:"required"`
RememberAPIKey bool `json:"rememberApiKey"`
BaseURL string `json:"baseURL"`
APIType string `json:"apiType" validate:"required"`
Remark string `json:"remark"`
SyncAgents bool `json:"syncAgents"`
}
type AgentAccountVerifyReq struct {
@@ -100,20 +275,27 @@ type AgentAccountSearch struct {
}
type AgentAccountInfo struct {
ID uint `json:"id"`
Provider string `json:"provider"`
ProviderName string `json:"providerName"`
Name string `json:"name"`
APIKey string `json:"apiKey"`
BaseURL string `json:"baseUrl"`
Verified bool `json:"verified"`
Remark string `json:"remark"`
CreatedAt time.Time `json:"createdAt"`
ID uint `json:"id"`
Provider string `json:"provider"`
ProviderName string `json:"providerName"`
Name string `json:"name"`
APIKey string `json:"apiKey"`
RememberAPIKey bool `json:"rememberApiKey"`
BaseURL string `json:"baseUrl"`
Models []AgentAccountModel `json:"models"`
APIType string `json:"apiType"`
Verified bool `json:"verified"`
Remark string `json:"remark"`
CreatedAt time.Time `json:"createdAt"`
}
type ProviderModelInfo struct {
ID string `json:"id"`
Name string `json:"name"`
ID string `json:"id"`
Name string `json:"name"`
ContextWindow int `json:"contextWindow"`
MaxTokens int `json:"maxTokens"`
Reasoning bool `json:"reasoning"`
Input []string `json:"input"`
}
type ProviderInfo struct {
@@ -129,12 +311,17 @@ type AgentFeishuConfigReq struct {
}
type AgentFeishuConfigUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
BotName string `json:"botName" validate:"required"`
AppID string `json:"appId" validate:"required"`
AppSecret string `json:"appSecret" validate:"required"`
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy" validate:"required"`
AgentID uint `json:"agentId" validate:"required"`
Enabled bool `json:"enabled"`
ThreadSession bool `json:"threadSession"`
ReplyMode string `json:"replyMode" validate:"required"`
Streaming bool `json:"streaming"`
RequireMention string `json:"requireMention" validate:"required,oneof=true false open"`
GroupPolicy string `json:"groupPolicy" validate:"required,oneof=open allowlist disabled"`
GroupAllowFrom []string `json:"groupAllowFrom"`
Domain string `json:"domain"`
ConnectionMode string `json:"connectionMode"`
Bots []AgentFeishuBot `json:"bots" validate:"required,min=1"`
}
type AgentFeishuPairingApproveReq struct {
@@ -143,9 +330,323 @@ type AgentFeishuPairingApproveReq struct {
}
type AgentFeishuConfig struct {
Enabled bool `json:"enabled"`
ThreadSession bool `json:"threadSession"`
ReplyMode string `json:"replyMode"`
Streaming bool `json:"streaming"`
RequireMention string `json:"requireMention"`
GroupPolicy string `json:"groupPolicy"`
GroupAllowFrom []string `json:"groupAllowFrom"`
Domain string `json:"domain"`
ConnectionMode string `json:"connectionMode"`
Bots []AgentFeishuBot `json:"bots"`
Installed bool `json:"installed"`
}
type AgentTelegramConfigReq struct {
AgentID uint `json:"agentId" validate:"required"`
}
type AgentTelegramConfigUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy" validate:"required,oneof=pairing open allowlist disabled"`
AllowFrom []string `json:"allowFrom"`
RequireMention bool `json:"requireMention"`
GroupPolicy string `json:"groupPolicy" validate:"required,oneof=open allowlist disabled"`
GroupAllowFrom []string `json:"groupAllowFrom"`
Proxy string `json:"proxy"`
Streaming string `json:"streaming" validate:"required,oneof=off partial block progress"`
DefaultAccount string `json:"defaultAccount" validate:"required"`
Bots []AgentTelegramBot `json:"bots" validate:"required,min=1"`
}
type AgentTelegramConfig struct {
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy"`
AllowFrom []string `json:"allowFrom"`
RequireMention bool `json:"requireMention"`
GroupPolicy string `json:"groupPolicy"`
GroupAllowFrom []string `json:"groupAllowFrom"`
Proxy string `json:"proxy"`
Streaming string `json:"streaming"`
DefaultAccount string `json:"defaultAccount"`
Bots []AgentTelegramBot `json:"bots"`
}
type AgentChannelPairingApproveReq struct {
AgentID uint `json:"agentId" validate:"required"`
Type string `json:"type" validate:"required,oneof=feishu telegram discord wecom qqbot dingtalk"`
PairingCode string `json:"pairingCode" validate:"required"`
AccountID string `json:"accountId"`
}
type AgentChannelDeleteReq struct {
AgentID uint `json:"agentId" validate:"required"`
Type string `json:"type" validate:"required,oneof=feishu telegram discord wecom qqbot dingtalk weixin"`
}
type AgentWecomConfigUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy" validate:"required,oneof=pairing open allowlist disabled"`
AllowFrom []string `json:"allowFrom"`
GroupPolicy string `json:"groupPolicy" validate:"required,oneof=open allowlist disabled"`
GroupAllowFrom []string `json:"groupAllowFrom"`
BotID string `json:"botId" validate:"required"`
Secret string `json:"secret" validate:"required"`
}
type AgentWecomConfig struct {
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy"`
AllowFrom []string `json:"allowFrom"`
GroupPolicy string `json:"groupPolicy"`
GroupAllowFrom []string `json:"groupAllowFrom"`
BotID string `json:"botId"`
Secret string `json:"secret"`
Installed bool `json:"installed"`
}
type AgentDingTalkConfigUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy" validate:"required,oneof=pairing allowlist open disabled"`
AllowFrom []string `json:"allowFrom"`
GroupPolicy string `json:"groupPolicy" validate:"required,oneof=open allowlist disabled"`
GroupAllowFrom []string `json:"groupAllowFrom"`
SeparateSessionByConversation bool `json:"separateSessionByConversation"`
GroupSessionScope string `json:"groupSessionScope" validate:"required,oneof=group group_sender"`
SharedMemoryAcrossConversations bool `json:"sharedMemoryAcrossConversations"`
AsyncMode bool `json:"asyncMode"`
AckText string `json:"ackText"`
Bots []AgentDingTalkBot `json:"bots" validate:"required,min=1"`
}
type AgentDingTalkConfig struct {
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy"`
AllowFrom []string `json:"allowFrom"`
GroupPolicy string `json:"groupPolicy"`
GroupAllowFrom []string `json:"groupAllowFrom"`
SeparateSessionByConversation bool `json:"separateSessionByConversation"`
GroupSessionScope string `json:"groupSessionScope"`
SharedMemoryAcrossConversations bool `json:"sharedMemoryAcrossConversations"`
AsyncMode bool `json:"asyncMode"`
AckText string `json:"ackText"`
Bots []AgentDingTalkBot `json:"bots"`
Installed bool `json:"installed"`
}
type AgentWeixinLoginReq struct {
AgentID uint `json:"agentId" validate:"required"`
TaskID string `json:"taskID" validate:"required"`
}
type AgentWeixinConfig struct {
Enabled bool `json:"enabled"`
}
type AgentQQBotConfigUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy"`
AllowFrom []string `json:"allowFrom"`
GroupPolicy string `json:"groupPolicy"`
GroupAllowFrom []string `json:"groupAllowFrom"`
Bots []AgentQQBotBot `json:"bots" validate:"required,min=1"`
}
type AgentQQBotConfig struct {
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy"`
AllowFrom []string `json:"allowFrom"`
GroupPolicy string `json:"groupPolicy"`
GroupAllowFrom []string `json:"groupAllowFrom"`
Bots []AgentQQBotBot `json:"bots"`
Installed bool `json:"installed"`
}
type AgentPluginInstallReq struct {
AgentID uint `json:"agentId" validate:"required"`
Type string `json:"type" validate:"required,oneof=feishu qqbot wecom dingtalk weixin"`
TaskID string `json:"taskID" validate:"required"`
}
type AgentPluginUpgradeReq struct {
AgentID uint `json:"agentId" validate:"required"`
Type string `json:"type" validate:"required,oneof=feishu qqbot wecom dingtalk weixin"`
TaskID string `json:"taskID" validate:"required"`
}
type AgentPluginUninstallReq struct {
AgentID uint `json:"agentId" validate:"required"`
Type string `json:"type" validate:"required,oneof=feishu qqbot wecom dingtalk weixin"`
TaskID string `json:"taskID" validate:"required"`
}
type AgentPluginCheckReq struct {
AgentID uint `json:"agentId" validate:"required"`
Type string `json:"type" validate:"required,oneof=feishu qqbot wecom dingtalk weixin"`
CheckLatest bool `json:"checkLatest"`
}
type AgentPluginStatus struct {
Installed bool `json:"installed"`
CurrentVersion string `json:"currentVersion"`
LatestVersion string `json:"latestVersion"`
Upgradable bool `json:"upgradable"`
}
type AgentDiscordConfigUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy" validate:"required"`
AllowFrom []string `json:"allowFrom"`
RequireMention bool `json:"requireMention"`
GroupPolicy string `json:"groupPolicy" validate:"required,oneof=open allowlist disabled"`
Proxy string `json:"proxy"`
DefaultAccount string `json:"defaultAccount" validate:"required"`
Bots []AgentDiscordBot `json:"bots" validate:"required,min=1"`
}
type AgentDiscordConfig struct {
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy"`
AllowFrom []string `json:"allowFrom"`
RequireMention bool `json:"requireMention"`
GroupPolicy string `json:"groupPolicy"`
Proxy string `json:"proxy"`
DefaultAccount string `json:"defaultAccount"`
Bots []AgentDiscordBot `json:"bots"`
}
type AgentChannelBotBase struct {
AccountID string `json:"accountId"`
Name string `json:"name"`
Enabled bool `json:"enabled"`
DmPolicy string `json:"dmPolicy"`
BotName string `json:"botName"`
AppID string `json:"appId"`
AppSecret string `json:"appSecret"`
IsDefault bool `json:"isDefault"`
}
func (b AgentChannelBotBase) IsEnabled() bool {
return b.Enabled
}
type AgentFeishuBot struct {
AgentChannelBotBase
AppID string `json:"appId"`
AppSecret string `json:"appSecret"`
DmPolicy string `json:"dmPolicy"`
AllowFrom []string `json:"allowFrom"`
}
type AgentTelegramBot struct {
AgentChannelBotBase
BotToken string `json:"botToken"`
DmPolicy string `json:"dmPolicy"`
GroupPolicy string `json:"groupPolicy"`
Streaming string `json:"streaming"`
}
type AgentDiscordBot struct {
AgentChannelBotBase
Token string `json:"token"`
}
type AgentQQBotBot struct {
AgentChannelBotBase
AppID string `json:"appId"`
ClientSecret string `json:"clientSecret"`
AllowFrom []string `json:"allowFrom"`
SystemPrompt string `json:"systemPrompt"`
}
type AgentDingTalkBot struct {
AgentChannelBotBase
ClientID string `json:"clientId"`
ClientSecret string `json:"clientSecret"`
}
type AgentSecurityConfigUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
AllowedOrigins []string `json:"allowedOrigins"`
}
type AgentSecurityConfig struct {
AllowedOrigins []string `json:"allowedOrigins"`
}
type AgentOtherConfigUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
UserTimezone string `json:"userTimezone" validate:"required"`
BrowserEnabled bool `json:"browserEnabled"`
NPMRegistry string `json:"npmRegistry" validate:"required"`
}
type AgentOtherConfig struct {
UserTimezone string `json:"userTimezone"`
BrowserEnabled bool `json:"browserEnabled"`
NPMRegistry string `json:"npmRegistry"`
}
type AgentConfigFileReq struct {
AgentID uint `json:"agentId" validate:"required"`
}
type AgentConfigFileUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
Content string `json:"content" validate:"required"`
}
type AgentConfigFile struct {
Content string `json:"content"`
}
type AgentSkillSearchReq struct {
AgentID uint `json:"agentId" validate:"required"`
Source string `json:"source" validate:"required,oneof=clawhub-global clawhub-cn skillhub official skills-sh"`
Keyword string `json:"keyword" validate:"required"`
}
type AgentSkillItem struct {
Name string `json:"name"`
Description string `json:"description"`
Category string `json:"category"`
Tags []string `json:"tags"`
Source string `json:"source"`
Trust string `json:"trust"`
Identifier string `json:"identifier"`
Bundled bool `json:"bundled"`
Disabled bool `json:"disabled"`
Uninstallable bool `json:"uninstallable"`
}
type AgentSkillSearchItem struct {
Slug string `json:"slug"`
Identifier string `json:"identifier"`
Name string `json:"name"`
Description string `json:"description"`
Summary string `json:"summary"`
Version string `json:"version"`
Source string `json:"source"`
Trust string `json:"trust"`
Score string `json:"score"`
}
type AgentSkillUpdateReq struct {
AgentID uint `json:"agentId" validate:"required"`
Name string `json:"name" validate:"required"`
Enabled bool `json:"enabled"`
}
type AgentSkillInstallReq struct {
AgentID uint `json:"agentId" validate:"required"`
Source string `json:"source" validate:"required,oneof=clawhub-global clawhub-cn skillhub official skills-sh"`
Slug string `json:"slug" validate:"required"`
TaskID string `json:"taskID" validate:"required"`
}
type AgentSkillUninstallReq struct {
AgentID uint `json:"agentId" validate:"required"`
Name string `json:"name" validate:"required"`
}
+12 -10
View File
@@ -65,10 +65,11 @@ type UploadForRecover struct {
}
type CommonBackup struct {
Type string `json:"type" validate:"required,oneof=app mysql mariadb redis website postgresql mysql-cluster postgresql-cluster redis-cluster"`
Type string `json:"type" validate:"required,oneof=app mysql mariadb redis website postgresql mongodb mysql-cluster postgresql-cluster redis-cluster container compose"`
Name string `json:"name"`
DetailName string `json:"detailName"`
Secret string `json:"secret"`
StopBefore bool `json:"stopBefore"`
TaskID string `json:"taskID"`
FileName string `json:"fileName"`
Args []string `json:"args"`
@@ -76,15 +77,16 @@ type CommonBackup struct {
Description string `json:"description"`
}
type CommonRecover struct {
DownloadAccountID uint `json:"downloadAccountID" validate:"required"`
Type string `json:"type" validate:"required,oneof=app mysql mariadb redis website postgresql mysql-cluster postgresql-cluster redis-cluster"`
Name string `json:"name"`
DetailName string `json:"detailName"`
File string `json:"file"`
Secret string `json:"secret"`
TaskID string `json:"taskID"`
BackupRecordID uint `json:"backupRecordID"`
Timeout int `json:"timeout"`
DownloadAccountID uint `json:"downloadAccountID" validate:"required"`
Type string `json:"type" validate:"required,oneof=app mysql mariadb redis website postgresql mongodb mysql-cluster postgresql-cluster redis-cluster container compose"`
Name string `json:"name"`
DetailName string `json:"detailName"`
File string `json:"file"`
Secret string `json:"secret"`
DropAllCollections bool `json:"dropAllCollections"`
TaskID string `json:"taskID"`
BackupRecordID uint `json:"backupRecordID"`
Timeout int `json:"timeout"`
}
type RecordSearch struct {
+6
View File
@@ -11,6 +11,12 @@ type SearchPageWithType struct {
Type string `json:"type"`
}
type SearchPageWithGroup struct {
PageInfo
GroupID uint `json:"groupID"`
Info string `json:"info"`
}
type PageInfo struct {
Page int `json:"page" validate:"required,number"`
PageSize int `json:"pageSize" validate:"required,number"`
+45 -10
View File
@@ -48,6 +48,34 @@ type ContainerOptions struct {
State string `json:"state"`
}
type ContainerFileReq struct {
ContainerID string `json:"containerID" validate:"required"`
Path string `json:"path" validate:"required"`
}
type ContainerFileBatchDeleteReq struct {
ContainerID string `json:"containerID" validate:"required"`
Paths []string `json:"paths" validate:"required,min=1,dive,required"`
}
type ContainerFileInfo struct {
Name string `json:"name"`
Path string `json:"path"`
IsDir bool `json:"isDir"`
IsLink bool `json:"isLink"`
LinkTo string `json:"linkTo"`
Size int64 `json:"size"`
Mode string `json:"mode"`
ModTime string `json:"modTime"`
}
type ContainerFileContent struct {
Content string `json:"content"`
Size int64 `json:"size"`
Truncated bool `json:"truncated"`
IsBinary bool `json:"isBinary"`
}
type ContainerStatus struct {
Created int `json:"created"`
Running int `json:"running"`
@@ -95,10 +123,16 @@ type ContainerOperate struct {
Privileged bool `json:"privileged"`
AutoRemove bool `json:"autoRemove"`
Volumes []VolumeHelper `json:"volumes"`
ExtraHosts []ExtraHost `json:"extraHosts"`
Labels []string `json:"labels"`
Env []string `json:"env"`
RestartPolicy string `json:"restartPolicy"`
}
type ExtraHost struct {
Hostname string `json:"hostname"`
IP string `json:"ip"`
}
type ContainerNetwork struct {
Network string `json:"network"`
Ipv4 string `json:"ipv4"`
@@ -253,16 +287,17 @@ type BatchDelete struct {
}
type ComposeInfo struct {
Name string `json:"name"`
CreatedAt string `json:"createdAt"`
CreatedBy string `json:"createdBy"`
ContainerCount int `json:"containerCount"`
RunningCount int `json:"runningCount"`
ConfigFile string `json:"configFile"`
Workdir string `json:"workdir"`
Path string `json:"path"`
Containers []ComposeContainer `json:"containers"`
Env string `json:"env"`
Name string `json:"name"`
CreatedAt string `json:"createdAt"`
CreatedBy string `json:"createdBy"`
ContainerCount int `json:"containerCount"`
RunningCount int `json:"runningCount"`
ConfigFile string `json:"configFile"`
Workdir string `json:"workdir"`
ComposeFileExists bool `json:"composeFileExists"`
Path string `json:"path"`
Containers []ComposeContainer `json:"containers"`
Env string `json:"env"`
}
type ComposeContainer struct {
ContainerID string `json:"containerID"`
+1
View File
@@ -4,6 +4,7 @@ import "time"
type DashboardBase struct {
WebsiteNumber int `json:"websiteNumber"`
AgentNumber int `json:"agentNumber"`
DatabaseNumber int `json:"databaseNumber"`
CronjobNumber int `json:"cronjobNumber"`
AppInstalledNumber int `json:"appInstalledNumber"`
+78 -1
View File
@@ -11,7 +11,7 @@ type DBConfUpdateByFile struct {
type ChangeDBInfo struct {
ID uint `json:"id"`
From string `json:"from" validate:"required,oneof=local remote"`
Type string `json:"type" validate:"required,oneof=mysql mariadb postgresql redis mysql-cluster postgresql-cluster redis-cluster"`
Type string `json:"type" validate:"required,oneof=mysql mariadb postgresql redis mongodb mysql-cluster postgresql-cluster redis-cluster"`
Database string `json:"database" validate:"required"`
Value string `json:"value" validate:"required"`
}
@@ -137,6 +137,83 @@ type MysqlStatus struct {
Position string `json:"Position"`
}
// mongodb
type MongodbDBSearch struct {
PageInfo
Info string `json:"info"`
Database string `json:"database" validate:"required"`
OrderBy string `json:"orderBy" validate:"required,oneof=name createdAt"`
Order string `json:"order" validate:"required,oneof=null ascending descending"`
}
type MongodbDBInfo struct {
ID uint `json:"id"`
CreatedAt time.Time `json:"createdAt"`
Name string `json:"name"`
From string `json:"from"`
MongodbName string `json:"mongodbName"`
Username string `json:"username"`
Password string `json:"password"`
IsDelete bool `json:"isDelete"`
Description string `json:"description"`
}
type MongodbDBCreate struct {
Name string `json:"name" validate:"required"`
From string `json:"from" validate:"required,oneof=local remote"`
Database string `json:"database" validate:"required"`
Username string `json:"username" validate:"required"`
Password string `json:"password" validate:"required"`
Permission string `json:"permission" validate:"required,oneof=dbOwner read readWrite userAdmin"`
Description string `json:"description"`
}
type MongodbLoadDB struct {
From string `json:"from" validate:"required,oneof=local remote"`
Type string `json:"type" validate:"required,oneof=mongodb"`
Database string `json:"database" validate:"required"`
}
type MongodbDBDeleteCheck struct {
ID uint `json:"id" validate:"required"`
Type string `json:"type" validate:"required,oneof=mongodb"`
Database string `json:"database" validate:"required"`
}
type MongodbDBDelete struct {
ID uint `json:"id" validate:"required"`
Type string `json:"type" validate:"required,oneof=mongodb"`
Database string `json:"database" validate:"required"`
ForceDelete bool `json:"forceDelete"`
DeleteBackup bool `json:"deleteBackup"`
}
type MongodbBind struct {
Database string `json:"database" validate:"required"`
Name string `json:"name" validate:"required"`
Username string `json:"username" validate:"required"`
Password string `json:"password" validate:"required"`
}
type MongodbPassword struct {
Database string `json:"database" validate:"required"`
Name string `json:"name" validate:"required"`
Password string `json:"password" validate:"required"`
}
type MongodbPrivileges struct {
Database string `json:"database" validate:"required"`
Name string `json:"name" validate:"required"`
Username string `json:"username" validate:"required"`
Permission string `json:"permission" validate:"required,oneof=dbOwner read readWrite userAdmin"`
}
type MongodbPrivilegesLoad struct {
Database string `json:"database" validate:"required"`
Name string `json:"name" validate:"required"`
Username string `json:"username" validate:"required"`
}
type MysqlVariables struct {
BinlogCacheSize string `json:"binlog_cache_size"`
InnodbBufferPoolSize string `json:"innodb_buffer_pool_size"`
@@ -1,8 +1,6 @@
package dto
import (
"time"
)
import "time"
type HostOperate struct {
ID uint `json:"id"`
@@ -34,11 +32,6 @@ type SearchForTree struct {
Info string `json:"info"`
}
type ChangeHostGroup struct {
ID uint `json:"id" validate:"required"`
GroupID uint `json:"groupID" validate:"required"`
}
type HostInfo struct {
ID uint `json:"id"`
CreatedAt time.Time `json:"createdAt"`
+1
View File
@@ -7,6 +7,7 @@ import (
type SearchTaskLogReq struct {
Status string `json:"status"`
Type string `json:"type"`
TaskID string `json:"taskID"`
PageInfo
}
+9
View File
@@ -138,3 +138,12 @@ type AppUpdateVersion struct {
AppInstallID uint `json:"appInstallID" validate:"required"`
UpdateVersion string `json:"updateVersion"`
}
type AppInstallSortItem struct {
InstallID uint `json:"installID"`
SortOrder int `json:"sortOrder"`
}
type AppInstallSort struct {
Items []AppInstallSortItem `json:"items" validate:"required"`
}
+39
View File
@@ -9,6 +9,29 @@ type FileOption struct {
files.FileOption
}
type FileAISearch struct {
Path string `json:"path" validate:"required"`
Query string `json:"query" validate:"required"`
ResponseLanguage string `json:"responseLanguage,omitempty"`
ContainSub *bool `json:"containSub,omitempty"`
MaxItems int `json:"maxItems" validate:"omitempty,min=1,max=2000"`
MatchCase bool `json:"matchCase"`
WholeWord bool `json:"wholeWord"`
UseRegex bool `json:"useRegex"`
Extensions []string `json:"extensions,omitempty"`
MinSize int64 `json:"minSize"`
MaxSize int64 `json:"maxSize"`
ModifiedAfter string `json:"modifiedAfter,omitempty"`
ModifiedBefore string `json:"modifiedBefore,omitempty"`
MaxScanFiles int `json:"maxScanFiles"`
MaxFileBytes int64 `json:"maxFileBytes"`
MaxHitsPerFile int `json:"maxHitsPerFile"`
MaxTotalHits int `json:"maxTotalHits"`
ContentHitsPromptMaxBytes int `json:"contentHitsPromptMaxBytes"`
LlmMaxOutputTokens int `json:"llmMaxOutputTokens"`
}
type FileContentReq struct {
Path string `json:"path" validate:"required"`
IsDetail bool `json:"isDetail"`
@@ -55,6 +78,11 @@ type FileCompress struct {
Name string `json:"name" validate:"required"`
Replace bool `json:"replace"`
Secret string `json:"secret"`
TaskID string `json:"taskID"`
}
type FileCompressStopReq struct {
TaskID string `json:"taskID" validate:"required"`
}
type FileDeCompress struct {
@@ -62,6 +90,11 @@ type FileDeCompress struct {
Type string `json:"type" validate:"required"`
Path string `json:"path" validate:"required"`
Secret string `json:"secret"`
TaskID string `json:"taskID"`
}
type FileDeCompressStopReq struct {
TaskID string `json:"taskID" validate:"required"`
}
type FileEdit struct {
@@ -172,3 +205,9 @@ type FileRemarkUpdate struct {
Path string `json:"path" validate:"required"`
Remark string `json:"remark"`
}
type FileShareCreate struct {
Path string `json:"path" validate:"required"`
ExpireMinutes int `json:"expireMinutes" validate:"min=0,max=10080"`
Password *string `json:"password"`
}
+52
View File
@@ -0,0 +1,52 @@
package request
import (
"encoding/json"
"github.com/1Panel-dev/1Panel/agent/app/dto"
)
type FileHistorySearchReq struct {
dto.PageInfo
Path string `json:"path"`
Scope string `json:"scope" validate:"required,oneof=current all"`
Operation string `json:"operation"`
}
type FileHistoryContentReq struct {
ID uint `json:"id" validate:"required"`
}
type FileHistoryIDs []uint
func (ids *FileHistoryIDs) UnmarshalJSON(data []byte) error {
type alias []uint
var direct alias
if err := json.Unmarshal(data, &direct); err == nil {
*ids = FileHistoryIDs(direct)
return nil
}
var wrapped struct {
IDs []uint `json:"ids"`
}
if err := json.Unmarshal(data, &wrapped); err != nil {
return err
}
*ids = FileHistoryIDs(wrapped.IDs)
return nil
}
type FileHistoryDeleteReq struct {
IDs FileHistoryIDs `json:"ids" validate:"required"`
}
type FileHistoryRestoreReq struct {
ID uint `json:"id" validate:"required"`
}
type FileHistorySettingUpdate struct {
Enable string `json:"enable" validate:"required,oneof=Enable Disable"`
MaxPerPath int `json:"maxPerPath" validate:"min=0,max=1000"`
DiskQuotaMB int `json:"diskQuotaMB" validate:"min=0,max=1048576"`
}
-1
View File
@@ -1 +0,0 @@
package request
+12
View File
@@ -269,9 +269,21 @@ type WebsiteProxyConfig struct {
Replaces map[string]string `json:"replaces"`
SNI bool `json:"sni"`
ProxySSLName string `json:"proxySSLName"`
SSLVerify bool `json:"sslVerify"`
CorsConfig
}
type WebsiteProxyDelete struct {
ID uint `json:"id" validate:"required"`
Name string `json:"name" validate:"required"`
}
type WebsiteProxyStatusUpdate struct {
ID uint `json:"id" validate:"required"`
Name string `json:"name" validate:"required"`
Status string `json:"status" validate:"required"`
}
type CorsConfig struct {
Cors bool `json:"cors"`
AllowOrigins string `json:"allowOrigins"`
+2 -3
View File
@@ -6,8 +6,8 @@ type WebsiteSSLSearch struct {
dto.PageInfo
AcmeAccountID string `json:"acmeAccountID"`
Domain string `json:"domain"`
OrderBy string `json:"orderBy" validate:"required,oneof=expire_date"`
Order string `json:"order" validate:"required,oneof=null ascending descending"`
OrderBy string `json:"orderBy" validate:"omitempty,oneof=created_at expire_date"`
Order string `json:"order" validate:"omitempty,oneof=null ascending descending"`
}
type WebsiteSSLListReq struct {
@@ -51,7 +51,6 @@ type WebsiteSSLApply struct {
ID uint `json:"ID" validate:"required"`
SkipDNSCheck bool `json:"skipDNSCheck"`
Nameservers []string `json:"nameservers"`
DisableLog bool `json:"disableLog"`
}
type WebsiteSSLObtain struct {
+1
View File
@@ -126,6 +126,7 @@ type AppInstallDTO struct {
WebUI string `json:"webUI"`
CreatedAt time.Time `json:"createdAt"`
Favorite bool `json:"favorite"`
SortOrder int `json:"sortOrder"`
App AppDetail `json:"app"`
Container string `json:"container"`
IsEdit bool `json:"isEdit"`
+38
View File
@@ -85,3 +85,41 @@ type FileConvertLog struct {
type FileRemarksRes struct {
Remarks map[string]string `json:"remarks"`
}
type FileShareInfo struct {
Code string `json:"code"`
Path string `json:"path"`
FileName string `json:"fileName"`
ExpiresAt int64 `json:"expiresAt"`
Permanent bool `json:"permanent"`
HasPassword bool `json:"hasPassword"`
Password string `json:"password,omitempty"`
}
type FileSharePublicInfo struct {
FileName string `json:"fileName"`
ExpiresAt int64 `json:"expiresAt"`
Permanent bool `json:"permanent"`
HasPassword bool `json:"hasPassword"`
}
type FileAIContentHit struct {
Path string `json:"path"`
Line int `json:"line"`
Text string `json:"text"`
}
type FileAISearchResult struct {
Mode string `json:"mode"`
Summary string `json:"summary"`
Hits []FileAIContentHit `json:"hits"`
ContentScannedFiles int `json:"contentScannedFiles"`
ContentHitsTruncated bool `json:"contentHitsTruncated"`
Truncated bool `json:"truncated"`
PreFiltered bool `json:"preFiltered"`
ItemCount int `json:"itemCount"`
PromptTokens int `json:"promptTokens"`
CompletionTokens int `json:"completionTokens"`
TotalTokens int `json:"totalTokens"`
Duration string `json:"duration"`
}
+31
View File
@@ -0,0 +1,31 @@
package response
import "time"
type FileHistoryInfo struct {
ID uint `json:"id"`
FileID string `json:"fileId"`
Path string `json:"path"`
CurrentPath string `json:"currentPath"`
PreviousID uint `json:"previousId"`
SourcePath string `json:"sourcePath"`
TargetPath string `json:"targetPath"`
FileName string `json:"fileName"`
Extension string `json:"extension"`
FileMode string `json:"fileMode"`
Operation string `json:"operation"`
Deleted bool `json:"deleted"`
ContentSize int64 `json:"contentSize"`
ContentSHA string `json:"contentSHA"`
StoragePath string `json:"storagePath,omitempty"`
Content string `json:"content,omitempty"`
CurrentContent string `json:"currentContent,omitempty"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
type FileHistorySettingInfo struct {
Enable string `json:"enable"`
MaxPerPath int `json:"maxPerPath"`
DiskQuotaMB int `json:"diskQuotaMB"`
}
+14
View File
@@ -55,6 +55,7 @@ type CleanTree struct {
Size uint64 `json:"size"`
IsCheck bool `json:"isCheck"`
IsRecommend bool `json:"isRecommend"`
CanDelete bool `json:"canDelete"`
}
type Clean struct {
@@ -87,6 +88,19 @@ type SystemProxy struct {
Password string `json:"password"`
}
type TerminalAIInfo struct {
AIStatus string `json:"aiStatus"`
AIAccountID string `json:"aiAccountId"`
AIPrefix string `json:"aiPrefix" validate:"required,oneof=@ai #ai /ai"`
AIRiskCommands string `json:"aiRiskCommands"`
AIRiskCommandsDefault string `json:"aiRiskCommandsDefault"`
}
type FileManageAIInfo struct {
AIStatus string `json:"aiStatus"`
AIAccountID string `json:"aiAccountId"`
}
type CommonDescription struct {
ID string `json:"id" validate:"required"`
Type string `json:"type" validate:"required"`
+6 -1
View File
@@ -4,7 +4,6 @@ import "time"
type SSHUpdate struct {
Key string `json:"key" validate:"required"`
OldValue string `json:"oldValue"`
NewValue string `json:"newValue"`
}
@@ -50,6 +49,12 @@ type GenerateLoad struct {
type SSHConf struct {
File string `json:"file"`
}
type SSHConfUpdate struct {
Key string `json:"key" validate:"required"`
Path string `json:"path"`
Value string `json:"value"`
}
type SearchSSHLog struct {
PageInfo
Info string `json:"info"`
+17 -11
View File
@@ -2,15 +2,21 @@ package model
type Agent struct {
BaseModel
Name string `json:"name" gorm:"not null;unique"`
Provider string `json:"provider"`
Model string `json:"model"`
BaseURL string `json:"baseUrl"`
APIKey string `json:"apiKey"`
Token string `json:"token"`
Status string `json:"status"`
Message string `json:"message"`
AppInstallID uint `json:"appInstallId"`
AccountID uint `json:"accountId"`
ConfigPath string `json:"configPath"`
Name string `json:"name" gorm:"not null;unique"`
Remark string `json:"remark"`
AgentType string `json:"agentType" gorm:"default:openclaw"`
Provider string `json:"provider"`
Model string `json:"model"`
APIType string `json:"apiType"`
MaxTokens int `json:"maxTokens"`
ContextWindow int `json:"contextWindow"`
BaseURL string `json:"baseUrl"`
APIKey string `json:"apiKey"`
Token string `json:"token"`
Status string `json:"status"`
Message string `json:"message"`
AppInstallID uint `json:"appInstallId"`
WebsiteID uint `json:"websiteId"`
AccountID uint `json:"accountId"`
ConfigPath string `json:"configPath"`
}
+8 -6
View File
@@ -2,12 +2,14 @@ package model
type AgentAccount struct {
BaseModel
Provider string `json:"provider"`
Name string `json:"name"`
APIKey string `json:"apiKey"`
BaseURL string `json:"baseUrl"`
Verified bool `json:"verified"`
Remark string `json:"remark"`
Provider string `json:"provider"`
Name string `json:"name"`
APIKey string `json:"apiKey"`
BaseURL string `json:"baseUrl"`
APIType string `json:"apiType"`
RememberAPIKey bool `json:"rememberApiKey"`
Verified bool `json:"verified"`
Remark string `json:"remark"`
}
func (AgentAccount) TableName() string {
+17
View File
@@ -0,0 +1,17 @@
package model
type AgentAccountModel struct {
BaseModel
AccountID uint `json:"accountId" gorm:"index"`
Model string `json:"model" gorm:"index"`
Name string `json:"name"`
ContextWindow int `json:"contextWindow"`
MaxTokens int `json:"maxTokens"`
Reasoning bool `json:"reasoning"`
Input string `json:"input" gorm:"type:text"`
SortOrder int `json:"sortOrder" gorm:"index"`
}
func (AgentAccountModel) TableName() string {
return "agent_account_models"
}
+1
View File
@@ -26,6 +26,7 @@ type AppInstall struct {
HttpsPort int `json:"httpsPort"`
WebUI string `json:"webUI"`
Favorite bool `json:"favorite"`
SortOrder int `json:"sortOrder" gorm:"default:0"`
App App `json:"app" gorm:"-:migration"`
}
+12
View File
@@ -0,0 +1,12 @@
package model
type DatabaseMongodb struct {
BaseModel
Name string `json:"name" gorm:"not null"`
From string `json:"from" gorm:"not null;default:local"`
MongodbName string `json:"mongodbName" gorm:"not null"`
Username string `json:"username" gorm:"not null"`
Password string `json:"password" gorm:"not null"`
IsDelete bool `json:"isDelete"`
Description string `json:"description"`
}
+19
View File
@@ -0,0 +1,19 @@
package model
type FileHistory struct {
BaseModel
FileID string `json:"fileId" gorm:"not null;index:idx_file_history_file_id_created_at,priority:1;index:idx_file_history_file_id_deleted_created_at,priority:1"`
Path string `json:"path" gorm:"not null;index:idx_file_history_path_created_at,priority:1;index:idx_file_history_path_hash_created_at,priority:1;index:idx_file_history_related_path,priority:1"`
PathHash string `json:"pathHash" gorm:"not null;index:idx_file_history_path_created_at,priority:2;index:idx_file_history_path_hash_created_at,priority:2"`
PreviousID uint `json:"previousId" gorm:"index"`
SourcePath string `json:"sourcePath" gorm:"index:idx_file_history_related_path,priority:2"`
TargetPath string `json:"targetPath" gorm:"index:idx_file_history_related_path,priority:3"`
FileName string `json:"fileName" gorm:"not null"`
Extension string `json:"extension"`
FileMode string `json:"fileMode"`
Operation string `json:"operation" gorm:"not null;index:idx_file_history_operation_created_at,priority:1"`
Deleted bool `json:"deleted" gorm:"not null;default:false;index:idx_file_history_deleted_created_at,priority:1;index:idx_file_history_file_id_deleted_created_at,priority:2"`
ContentSize int64 `json:"contentSize" gorm:"not null"`
ContentSHA string `json:"contentSHA" gorm:"not null;index"`
StoragePath string `json:"storagePath" gorm:"not null;uniqueIndex"`
}
+14
View File
@@ -0,0 +1,14 @@
package model
type FileShare struct {
BaseModel
Path string `gorm:"not null;uniqueIndex" json:"path"`
Token string `gorm:"not null;uniqueIndex" json:"token"`
FileName string `gorm:"not null" json:"fileName"`
ExpiresUnix int64 `json:"expiresUnix"`
PasswordEnc string `json:"-"`
PasswordSalt string `json:"passwordSalt"`
PasswordHash string `json:"passwordHash"`
MaxDownloads int `json:"maxDownloads"`
DownloadCount int `json:"downloadCount"`
}
+325 -88
View File
@@ -1,10 +1,23 @@
package provider
import "strings"
import (
"strings"
)
type Model struct {
ID string
Name string
ID string
Name string
ContextWindow int
MaxTokens int
Reasoning bool
Input []string
}
type RuntimeDefault struct {
APIType string
ContextWindow int
MaxTokens int
Input []string
}
type Meta struct {
@@ -13,138 +26,298 @@ type Meta struct {
Sort uint
DefaultBaseURL string
EnvKey string
Default RuntimeDefault
Models []Model
Enabled bool
}
var catalog = map[string]Meta{
"custom": {
Key: "custom",
DisplayName: "Custom",
Sort: 10,
DefaultBaseURL: "",
EnvKey: "CUSTOM_API_KEY",
Default: RuntimeDefault{
APIType: "openai-completions",
ContextWindow: 128000,
MaxTokens: 8192,
Input: []string{"text"},
},
Models: []Model{},
},
"ollama": {
Key: "ollama",
DisplayName: "Ollama",
Sort: 1,
Enabled: true,
Sort: 15,
Default: RuntimeDefault{
APIType: "openai-responses",
ContextWindow: 160000,
MaxTokens: 8192,
Input: []string{"text"},
},
},
"vllm": {
Key: "vllm",
DisplayName: "vLLM",
Sort: 20,
DefaultBaseURL: "",
EnvKey: "VLLM_API_KEY",
Default: RuntimeDefault{
APIType: "openai-completions",
ContextWindow: 128000,
MaxTokens: 8192,
Input: []string{"text"},
},
Models: []Model{},
},
"deepseek": {
Key: "deepseek",
DisplayName: "DeepSeek",
Sort: 2,
Sort: 25,
DefaultBaseURL: "https://api.deepseek.com/v1",
EnvKey: "DEEPSEEK_API_KEY",
Enabled: true,
Default: RuntimeDefault{
APIType: "openai-completions",
ContextWindow: 131072,
MaxTokens: 8192,
Input: []string{"text"},
},
Models: []Model{
{ID: "deepseek/deepseek-chat", Name: "DeepSeek Chat"},
{ID: "deepseek/deepseek-reasoner", Name: "DeepSeek Reasoner"},
{ID: "deepseek/deepseek-r1:1.5b", Name: "DeepSeek R1 1.5B"},
{ID: "deepseek/deepseek-v4-flash", Name: "deepseek-v4-flash"},
{ID: "deepseek/deepseek-v4-pro", Name: "deepseek-v4-pro"},
},
},
"openai": {
Key: "openai",
DisplayName: "OpenAI",
Sort: 3,
DefaultBaseURL: "https://api.openai.com/v1",
EnvKey: "OPENAI_API_KEY",
Enabled: true,
"bailian-coding-plan": {
Key: "bailian-coding-plan",
DisplayName: "阿里云百炼 Coding Plan",
Sort: 30,
DefaultBaseURL: "https://coding.dashscope.aliyuncs.com/v1",
EnvKey: "QWEN_API_KEY",
Default: RuntimeDefault{
APIType: "openai-completions",
ContextWindow: 256000,
MaxTokens: 8192,
Input: []string{"text"},
},
Models: []Model{
{ID: "openai/codex-mini-latest", Name: "Codex Mini"},
{ID: "openai/gpt-4.1", Name: "GPT-4.1"},
{ID: "openai/gpt-4o", Name: "GPT-4o"},
{ID: "openai/gpt-4o-mini", Name: "GPT-4o Mini"},
{ID: "openai/gpt-5", Name: "GPT-5"},
{ID: "openai/gpt-5-mini", Name: "GPT-5 Mini"},
{ID: "bailian-coding-plan/qwen3.5-plus", Name: "Qwen3.5-Plus", Reasoning: true},
{ID: "bailian-coding-plan/qwen3-max", Name: "Qwen3-Max", Reasoning: true},
{ID: "bailian-coding-plan/qwen3-coder-next", Name: "Qwen3-Coder-Next", Reasoning: true},
{ID: "bailian-coding-plan/qwen3-coder-plus", Name: "Qwen3-Coder-Plus", Reasoning: true},
{ID: "bailian-coding-plan/minimax-m2.5", Name: "MiniMax M2.5", Reasoning: true},
{ID: "bailian-coding-plan/glm-5", Name: "GLM-5", Reasoning: true},
{ID: "bailian-coding-plan/kimi-k2.5", Name: "Kimi-k2.5", Reasoning: true},
{ID: "bailian-coding-plan/glm-4.7", Name: "GLM-4.7", Reasoning: true},
},
},
"anthropic": {
Key: "anthropic",
DisplayName: "Anthropic",
Sort: 4,
DefaultBaseURL: "https://api.anthropic.com",
EnvKey: "ANTHROPIC_API_KEY",
Enabled: true,
"ark-coding-plan": {
Key: "ark-coding-plan",
DisplayName: "方舟 Coding Plan",
Sort: 35,
DefaultBaseURL: "https://ark.cn-beijing.volces.com/api/coding/v3",
EnvKey: "ARK_API_KEY",
Default: RuntimeDefault{
APIType: "openai-completions",
ContextWindow: 256000,
MaxTokens: 4096,
Input: []string{"text"},
},
Models: []Model{
{ID: "anthropic/claude-3-haiku-20240307", Name: "Claude 3 Haiku"},
{ID: "anthropic/claude-3-5-haiku-latest", Name: "Claude 3.5 Haiku"},
{ID: "anthropic/claude-3-5-sonnet-20241022", Name: "Claude 3.5 Sonnet"},
{ID: "anthropic/claude-3-7-sonnet-20250219", Name: "Claude 3.7 Sonnet"},
{ID: "anthropic/claude-opus-4-1", Name: "Claude Opus 4.1"},
{ID: "ark-coding-plan/ark-code-latest", Name: "Ark Coding Plan", ContextWindow: 256000, MaxTokens: 4096},
{ID: "ark-coding-plan/doubao-seed-code", Name: "Doubao Seed Code", ContextWindow: 256000, MaxTokens: 4096},
{ID: "ark-coding-plan/glm-4.7", Name: "GLM 4.7 Coding", ContextWindow: 200000, MaxTokens: 4096},
{ID: "ark-coding-plan/kimi-k2-thinking", Name: "Kimi K2 Thinking", ContextWindow: 256000, MaxTokens: 4096},
{ID: "ark-coding-plan/kimi-k2.5", Name: "Kimi K2.5 Coding", ContextWindow: 256000, MaxTokens: 4096},
{ID: "ark-coding-plan/doubao-seed-code-preview-251028", Name: "Doubao Seed Code Preview", ContextWindow: 256000, MaxTokens: 4096},
},
},
"gemini": {
Key: "gemini",
DisplayName: "Gemini",
Sort: 5,
DefaultBaseURL: "https://generativelanguage.googleapis.com",
EnvKey: "GEMINI_API_KEY",
Enabled: true,
"zai": {
Key: "zai",
DisplayName: "Z.ai",
Sort: 40,
DefaultBaseURL: "https://open.bigmodel.cn/api/paas/v4",
EnvKey: "ZAI_API_KEY",
Default: RuntimeDefault{
APIType: "openai-completions",
ContextWindow: 204800,
MaxTokens: 131072,
Input: []string{"text"},
},
Models: []Model{
{ID: "google/gemini-1.5-flash", Name: "Gemini 1.5 Flash"},
{ID: "google/gemini-1.5-pro", Name: "Gemini 1.5 Pro"},
{ID: "google/gemini-2.0-flash", Name: "Gemini 2.0 Flash"},
{ID: "google/gemini-2.5-flash", Name: "Gemini 2.5 Flash"},
{ID: "google/gemini-2.5-pro", Name: "Gemini 2.5 Pro"},
{ID: "google/gemini-3-flash-preview", Name: "Gemini 3 Flash Preview"},
{ID: "zai/glm-5", Name: "GLM-5", Reasoning: true},
{ID: "zai/glm-4.7", Name: "GLM-4.7", Reasoning: true},
{ID: "zai/glm-4.7-flash", Name: "GLM-4.7-Flash", Reasoning: true},
{ID: "zai/glm-4.7-flashx", Name: "GLM-4.7-FlashX", Reasoning: true},
},
},
"minimax": {
Key: "minimax",
DisplayName: "MiniMax (CN)",
Sort: 6,
Sort: 45,
DefaultBaseURL: "https://api.minimaxi.com/anthropic",
EnvKey: "MINIMAX_API_KEY",
Enabled: true,
Default: RuntimeDefault{
APIType: "anthropic-messages",
ContextWindow: 200000,
MaxTokens: 8192,
Input: []string{"text"},
},
Models: []Model{
{ID: "minimax/MiniMax-M2.1", Name: "MiniMax M2.1"},
{ID: "minimax/MiniMax-M2.1-lightning", Name: "MiniMax M2.1 Lightning"},
{ID: "minimax/MiniMax-M2.7", Name: "MiniMax M2.7"},
{ID: "minimax/MiniMax-M2.7-highspeed", Name: "MiniMax M2.7 highspeed"},
{ID: "minimax/MiniMax-M2.5", Name: "MiniMax M2.5", Reasoning: true},
{ID: "minimax/MiniMax-M2.5-highspeed", Name: "MiniMax M2.5 highspeed"},
},
},
"moonshot": {
Key: "moonshot",
DisplayName: "Moonshot (Global)",
Sort: 7,
DefaultBaseURL: "https://api.moonshot.ai/v1",
EnvKey: "MOONSHOT_API_KEY",
Enabled: true,
"xiaomi": {
Key: "xiaomi",
DisplayName: "Xiaomi",
Sort: 46,
DefaultBaseURL: "https://api.xiaomimimo.com/v1",
EnvKey: "XIAOMI_API_KEY",
Default: RuntimeDefault{
APIType: "openai-completions",
ContextWindow: 262144,
MaxTokens: 8192,
Input: []string{"text"},
},
Models: []Model{
{ID: "moonshot/kimi-k2.5", Name: "Kimi K2.5"},
{ID: "moonshot/kimi-k2-0905-preview", Name: "Kimi K2 0905 Preview"},
{ID: "moonshot/kimi-k2-thinking", Name: "Kimi K2 Thinking"},
{ID: "xiaomi/mimo-v2-flash", Name: "Xiaomi MiMo V2 Flash", ContextWindow: 262144, MaxTokens: 8192, Input: []string{"text"}},
{ID: "xiaomi/mimo-v2-pro", Name: "Xiaomi MiMo V2 Pro", ContextWindow: 1048576, MaxTokens: 32000, Reasoning: true, Input: []string{"text"}},
{ID: "xiaomi/mimo-v2-omni", Name: "Xiaomi MiMo V2 Omni", ContextWindow: 262144, MaxTokens: 32000, Reasoning: true, Input: []string{"text", "image"}},
},
},
"kimi": {
Key: "kimi",
DisplayName: "Kimi (CN)",
Sort: 8,
Sort: 50,
DefaultBaseURL: "https://api.moonshot.cn/v1",
EnvKey: "KIMI_API_KEY",
Enabled: true,
Default: RuntimeDefault{
APIType: "openai-completions",
ContextWindow: 256000,
MaxTokens: 8192,
Input: []string{"text", "image"},
},
Models: []Model{
{ID: "kimi/kimi-k2.5", Name: "Kimi K2.5"},
{ID: "kimi/kimi-k2.5", Name: "Kimi K2.5", Reasoning: true},
{ID: "kimi/kimi-k2-0905-preview", Name: "Kimi K2 0905 Preview"},
{ID: "kimi/kimi-k2-thinking", Name: "Kimi K2 Thinking"},
{ID: "kimi/kimi-k2-thinking", Name: "Kimi K2 Thinking", Reasoning: true},
},
},
"kimi-coding": {
Key: "kimi-coding",
DisplayName: "Kimi Coding",
Sort: 9,
DefaultBaseURL: "https://api.moonshot.cn/anthropic/v1",
Sort: 51,
DefaultBaseURL: "https://api.kimi.com/coding/",
EnvKey: "KIMI_API_KEY",
Enabled: true,
Default: RuntimeDefault{
APIType: "anthropic-messages",
ContextWindow: 262144,
MaxTokens: 32768,
Input: []string{"text", "image"},
},
Models: []Model{
{ID: "kimi-coding/k2p5", Name: "Kimi K2.5"},
{ID: "kimi-coding/kimi-code", Name: "Kimi Code", ContextWindow: 262144, MaxTokens: 32768, Reasoning: true, Input: []string{"text", "image"}},
{ID: "kimi-coding/k2p5", Name: "Kimi K2.5", ContextWindow: 262144, MaxTokens: 32768, Reasoning: true, Input: []string{"text", "image"}},
},
},
"qwen": {
Key: "qwen",
DisplayName: "Qwen",
Sort: 10,
DefaultBaseURL: "https://dashscope.aliyuncs.com/compatible-mode/v1",
EnvKey: "QWEN_API_KEY",
Enabled: false,
"openai": {
Key: "openai",
DisplayName: "OpenAI",
Sort: 55,
DefaultBaseURL: "https://api.openai.com/v1",
EnvKey: "OPENAI_API_KEY",
Default: RuntimeDefault{
APIType: "openai-responses",
ContextWindow: 272000,
MaxTokens: 128000,
Input: []string{"text", "image"},
},
Models: []Model{
{ID: "openai/gpt-5.4", Name: "gpt-5.4", ContextWindow: 272000, MaxTokens: 128000, Reasoning: true, Input: []string{"text", "image"}},
{ID: "openai/gpt-5.4-pro", Name: "gpt-5.4-pro", ContextWindow: 1050000, MaxTokens: 128000, Reasoning: true, Input: []string{"text", "image"}},
{ID: "openai/gpt-5.4-mini", Name: "gpt-5.4-mini", ContextWindow: 400000, MaxTokens: 128000, Reasoning: true, Input: []string{"text", "image"}},
{ID: "openai/gpt-5.4-nano", Name: "gpt-5.4-nano", ContextWindow: 400000, MaxTokens: 128000, Reasoning: true, Input: []string{"text", "image"}},
},
},
"openrouter": {
Key: "openrouter",
DisplayName: "OpenRouter",
Sort: 56,
DefaultBaseURL: "https://openrouter.ai/api/v1",
EnvKey: "OPENROUTER_API_KEY",
Default: RuntimeDefault{
APIType: "openai-completions",
ContextWindow: 128000,
MaxTokens: 8192,
Input: []string{"text"},
},
Models: []Model{
{ID: "openrouter/free", Name: "openrouter/free"},
{ID: "openrouter/auto", Name: "openrouter/auto"},
},
},
"anthropic": {
Key: "anthropic",
DisplayName: "Anthropic",
Sort: 60,
DefaultBaseURL: "https://api.anthropic.com",
EnvKey: "ANTHROPIC_API_KEY",
Default: RuntimeDefault{
APIType: "anthropic-messages",
ContextWindow: 256000,
MaxTokens: 8192,
Input: []string{"text", "image"},
},
Models: []Model{
{ID: "anthropic/claude-sonnet-4-6", Name: "Claude Sonnet 4.6", Reasoning: true},
{ID: "anthropic/claude-opus-4-6", Name: "Claude Opus 4.6", Reasoning: true},
{ID: "anthropic/claude-opus-4-5", Name: "Claude Opus 4.5"},
{ID: "anthropic/claude-sonnet-4-5", Name: "Claude Sonnet 4.5"},
{ID: "anthropic/claude-haiku-4-5", Name: "Claude Haiku 4.5"},
},
},
"gemini": {
Key: "gemini",
DisplayName: "Gemini",
Sort: 65,
DefaultBaseURL: "https://generativelanguage.googleapis.com",
EnvKey: "GEMINI_API_KEY",
Default: RuntimeDefault{
APIType: "openai-completions",
ContextWindow: 256000,
MaxTokens: 8192,
Input: []string{"text", "image"},
},
Models: []Model{
{ID: "google/gemini-3-flash-preview", Name: "Gemini 3 Flash Preview", Reasoning: true},
{ID: "google/gemini-flash-latest", Name: "Gemini Flash Latest"},
{ID: "google/gemini-3-pro-preview", Name: "Gemini 3 Pro Preview", Reasoning: true},
},
},
"moonshot": {
Key: "moonshot",
DisplayName: "Moonshot (Global)",
Sort: 70,
DefaultBaseURL: "https://api.moonshot.ai/v1",
EnvKey: "MOONSHOT_API_KEY",
Default: RuntimeDefault{
APIType: "openai-completions",
ContextWindow: 256000,
MaxTokens: 8192,
Input: []string{"text"},
},
Models: []Model{
{ID: "moonshot/kimi-k2.5", Name: "Kimi K2.5", Reasoning: true},
{ID: "moonshot/kimi-k2-0905-preview", Name: "Kimi K2 0905 Preview"},
{ID: "moonshot/kimi-k2-thinking", Name: "Kimi K2 Thinking", Reasoning: true},
},
},
}
func Get(key string) (Meta, bool) {
meta, ok := catalog[strings.ToLower(strings.TrimSpace(key))]
meta, ok := catalog[key]
if !ok {
return Meta{}, false
}
@@ -159,13 +332,8 @@ func All() map[string]Meta {
return result
}
func IsEnabled(key string) bool {
meta, ok := catalog[strings.ToLower(strings.TrimSpace(key))]
return ok && meta.Enabled
}
func DefaultBaseURL(key string) (string, bool) {
meta, ok := catalog[strings.ToLower(strings.TrimSpace(key))]
meta, ok := catalog[key]
if !ok || strings.TrimSpace(meta.DefaultBaseURL) == "" {
return "", false
}
@@ -173,7 +341,7 @@ func DefaultBaseURL(key string) (string, bool) {
}
func EnvKey(key string) string {
meta, ok := catalog[strings.ToLower(strings.TrimSpace(key))]
meta, ok := catalog[key]
if !ok {
return ""
}
@@ -181,7 +349,7 @@ func EnvKey(key string) string {
}
func DisplayName(key string) string {
meta, ok := catalog[strings.ToLower(strings.TrimSpace(key))]
meta, ok := catalog[key]
if !ok {
return key
}
@@ -191,11 +359,80 @@ func DisplayName(key string) string {
return meta.DisplayName
}
func FindModel(key, modelID string) (Model, bool) {
meta, ok := Get(key)
if !ok {
return Model{}, false
}
for _, item := range meta.Models {
if item.ID == modelID {
return item, true
}
}
return Model{}, false
}
func cloneMeta(meta Meta) Meta {
clone := meta
if len(meta.Default.Input) > 0 {
clone.Default.Input = make([]string, len(meta.Default.Input))
copy(clone.Default.Input, meta.Default.Input)
}
if len(meta.Models) > 0 {
clone.Models = make([]Model, len(meta.Models))
copy(clone.Models, meta.Models)
for i, item := range meta.Models {
clone.Models[i] = normalizeModel(meta, item)
}
}
return clone
}
func normalizeModel(meta Meta, model Model) Model {
clone := model
clone.ID = strings.TrimSpace(clone.ID)
clone.Name = strings.TrimSpace(clone.Name)
if clone.Name == "" {
clone.Name = clone.ID
}
if clone.MaxTokens <= 0 {
clone.MaxTokens = meta.Default.MaxTokens
}
if clone.ContextWindow <= 0 {
clone.ContextWindow = meta.Default.ContextWindow
}
if len(clone.Input) == 0 && len(meta.Default.Input) > 0 {
clone.Input = make([]string, len(meta.Default.Input))
copy(clone.Input, meta.Default.Input)
}
return clone
}
func ResolveRuntimeParams(provider, apiType string, maxTokens, contextWindow int) (string, int, int) {
defaultAPIType := "openai-completions"
defaultMaxTokens := 8192
defaultContextWindow := 256000
if meta, ok := Get(provider); ok {
if meta.Default.APIType != "" {
defaultAPIType = meta.Default.APIType
}
if meta.Default.MaxTokens > 0 {
defaultMaxTokens = meta.Default.MaxTokens
}
if meta.Default.ContextWindow > 0 {
defaultContextWindow = meta.Default.ContextWindow
}
}
resolvedAPI := apiType
if strings.TrimSpace(apiType) == "" {
resolvedAPI = defaultAPIType
}
resolvedMaxTokens := defaultMaxTokens
resolvedContextWindow := defaultContextWindow
if maxTokens > 0 {
resolvedMaxTokens = maxTokens
}
if contextWindow > 0 {
resolvedContextWindow = contextWindow
}
return resolvedAPI, resolvedMaxTokens, resolvedContextWindow
}
+86
View File
@@ -0,0 +1,86 @@
package provider
import (
"fmt"
"strings"
)
type OpenClawProviderPatch struct {
PrimaryModel string
ProviderKey string
ModelID string
APIKey string
BaseURL string
APIType string
AuthHeader bool
}
func BuildOpenClawProviderPatch(provider, modelName, apiType, baseURL, apiKey string) (*OpenClawProviderPatch, error) {
if modelName == "" {
return nil, fmt.Errorf("model is required")
}
resolvedAPIType, _, _ := ResolveRuntimeParams(provider, apiType, 0, 0)
modelID := resolveOpenClawModelID(provider, modelName)
switch provider {
case "deepseek":
return newOpenClawProviderPatch(modelName, "deepseek", modelID, apiKey, baseURL, "openai-completions", false), nil
case "gemini":
return newOpenClawProviderPatch("google/"+modelID, "google", modelID, apiKey, baseURL, resolvedAPIType, false), nil
case "moonshot", "kimi":
return buildMoonshotProviderPatch(provider, modelName, modelID, baseURL, apiKey), nil
case "bailian-coding-plan":
return newOpenClawProviderPatch("bailian-coding-plan/"+modelID, "bailian-coding-plan", modelID, apiKey, baseURL, "openai-completions", false), nil
case "ark-coding-plan":
return newOpenClawProviderPatch("ark-coding-plan/"+modelID, "ark-coding-plan", modelID, apiKey, baseURL, "openai-completions", false), nil
case "minimax":
return newOpenClawProviderPatch("minimax/"+modelID, "minimax", modelID, apiKey, baseURL, "anthropic-messages", true), nil
case "xiaomi":
return newOpenClawProviderPatch("xiaomi/"+modelID, "xiaomi", modelID, apiKey, baseURL, "openai-completions", false), nil
case "custom", "vllm":
return newOpenClawProviderPatch(provider+"/"+modelID, provider, modelID, apiKey, baseURL, resolvedAPIType, false), nil
case "ollama":
return newOpenClawProviderPatch(modelName, "ollama", modelID, "ollama", baseURL, resolvedAPIType, false), nil
case "kimi-coding":
return newOpenClawProviderPatch(modelName, "kimi-coding", modelID, apiKey, baseURL, "anthropic-messages", false), nil
case "zai":
return newOpenClawProviderPatch("zai/"+modelID, "zai", modelID, apiKey, baseURL, "openai-completions", false), nil
default:
return newOpenClawProviderPatch(modelName, provider, modelID, apiKey, baseURL, resolvedAPIType, false), nil
}
}
func buildMoonshotProviderPatch(provider, modelName, modelID, baseURL, apiKey string) *OpenClawProviderPatch {
providerKey := provider
primaryModel := modelName
if provider == "kimi" {
providerKey = "moonshot"
primaryModel = "moonshot/" + modelID
}
return newOpenClawProviderPatch(primaryModel, providerKey, modelID, apiKey, baseURL, "openai-completions", false)
}
func newOpenClawProviderPatch(primaryModel, providerKey, modelID, apiKey, baseURL, apiType string, authHeader bool) *OpenClawProviderPatch {
return &OpenClawProviderPatch{
PrimaryModel: primaryModel,
ProviderKey: providerKey,
ModelID: modelID,
APIKey: apiKey,
BaseURL: baseURL,
APIType: apiType,
AuthHeader: authHeader,
}
}
func resolveOpenClawModelID(provider, modelName string) string {
if provider == "custom" || provider == "vllm" {
target := strings.TrimLeft(modelName, "/")
if parts := strings.SplitN(target, "/", 2); len(parts) == 2 && parts[0] == "custom" {
return strings.TrimLeft(parts[1], "/")
}
return target
}
if parts := strings.SplitN(modelName, "/", 2); len(parts) == 2 {
return parts[1]
}
return modelName
}
+181
View File
@@ -0,0 +1,181 @@
package provider
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/buserr"
)
type VerifyRequest struct {
Method string
URL string
Headers map[string]string
Body []byte
}
const (
defaultVerifyTimeout = 30 * time.Second
)
func SkipVerification(key string) bool {
switch key {
case "custom", "vllm", "ollama", "kimi-coding":
return true
default:
return false
}
}
func VerifyAccount(provider, baseURL, apiKey string) error {
req := BuildVerifyRequest(provider, baseURL, apiKey)
var body *bytes.Buffer
if len(req.Body) > 0 {
body = bytes.NewBuffer(req.Body)
} else {
body = bytes.NewBuffer(nil)
}
httpReq, err := http.NewRequest(req.Method, req.URL, body)
if err != nil {
return err
}
for key, value := range req.Headers {
httpReq.Header.Set(key, value)
}
resp, err := (&http.Client{Timeout: verifyTimeout()}).Do(httpReq)
if err != nil {
return buserr.WithErr("ErrAgentAccountUnavailable", err)
}
defer resp.Body.Close()
if resp.StatusCode >= 400 {
return buserr.WithErr("ErrAgentAccountUnavailable", fmt.Errorf("verify failed: %s", resp.Status))
}
return nil
}
func verifyTimeout() time.Duration {
return defaultVerifyTimeout
}
func BuildVerifyRequest(provider, baseURL, apiKey string) VerifyRequest {
base := strings.TrimRight(strings.TrimSpace(baseURL), "/")
headers := map[string]string{}
request := VerifyRequest{Method: http.MethodGet, Headers: headers}
switch provider {
case "anthropic", "kimi-coding":
headers["x-api-key"] = apiKey
headers["anthropic-version"] = "2023-06-01"
if strings.Contains(base, "/v1") {
request.URL = base + "/models"
} else {
request.URL = base + "/v1/models"
}
case "gemini":
request.Method = http.MethodPost
if strings.Contains(base, "/v1beta") {
request.URL = base + "/models/gemini-3-flash-preview:generateContent"
} else {
request.URL = base + "/v1beta/models/gemini-3-flash-preview:generateContent"
}
headers["x-goog-api-key"] = apiKey
headers["Content-Type"] = "application/json"
request.Body = mustJSON(map[string]interface{}{
"contents": []map[string]interface{}{{
"parts": []map[string]string{{
"text": "Explain how AI works in a few words",
}},
}},
})
case "zai":
headers["Authorization"] = fmt.Sprintf("Bearer %s", apiKey)
request.URL = base + "/models"
case "bailian-coding-plan":
request.Method = http.MethodPost
if !strings.Contains(base, "/v1") {
base = base + "/v1"
}
request.URL = base + "/chat/completions"
headers["Authorization"] = fmt.Sprintf("Bearer %s", apiKey)
headers["Content-Type"] = "application/json"
request.Body = mustJSON(map[string]interface{}{
"model": "qwen3.5-plus",
"messages": []map[string]string{{"role": "user", "content": "test"}},
"max_tokens": 1,
})
case "ark-coding-plan":
request.Method = http.MethodPost
if !strings.Contains(base, "/api/coding/v3") {
base = "https://ark.cn-beijing.volces.com/api/coding/v3"
}
request.URL = base + "/chat/completions"
headers["Authorization"] = fmt.Sprintf("Bearer %s", apiKey)
headers["Content-Type"] = "application/json"
request.Body = mustJSON(map[string]interface{}{
"model": "ark-code-latest",
"messages": []map[string]string{{"role": "user", "content": "test"}},
"max_tokens": 1,
})
case "minimax":
request.Method = http.MethodPost
headers["x-api-key"] = apiKey
headers["anthropic-version"] = "2023-06-01"
headers["Content-Type"] = "application/json"
if strings.Contains(base, "/v1") {
request.URL = base + "/messages"
} else {
request.URL = base + "/v1/messages"
}
request.Body = mustJSON(map[string]interface{}{
"model": "MiniMax-M2.5",
"max_tokens": 1,
"messages": []map[string]interface{}{{
"role": "user",
"content": []map[string]string{{
"type": "text",
"text": "test",
}},
}},
})
case "xiaomi":
request.Method = http.MethodPost
headers["Authorization"] = fmt.Sprintf("Bearer %s", apiKey)
headers["Content-Type"] = "application/json"
if !strings.Contains(base, "/v1") {
base = base + "/v1"
}
request.URL = base + "/chat/completions"
request.Body = mustJSON(map[string]interface{}{
"model": "mimo-v2-flash",
"max_tokens": 1,
"messages": []map[string]string{{"role": "user", "content": "test"}},
})
case "openrouter":
headers["Authorization"] = fmt.Sprintf("Bearer %s", apiKey)
if strings.Contains(base, "/v1") {
request.URL = base + "/key"
} else {
request.URL = base + "/v1/key"
}
default:
headers["Authorization"] = fmt.Sprintf("Bearer %s", apiKey)
if strings.Contains(base, "/v1") {
request.URL = base + "/models"
} else {
request.URL = base + "/v1/models"
}
}
return request
}
func mustJSON(value interface{}) []byte {
payload, err := json.Marshal(value)
if err != nil {
return []byte("{}")
}
return payload
}
+8
View File
@@ -16,6 +16,7 @@ type IAgentRepo interface {
DeleteByID(id uint) error
DeleteByAppInstallID(appInstallID uint) error
DeleteByAppInstallIDWithCtx(ctx context.Context, appInstallID uint) error
ClearWebsiteIDByWebsiteIDWithCtx(ctx context.Context, websiteID uint) error
List(opts ...DBOption) ([]model.Agent, error)
}
@@ -66,6 +67,13 @@ func (a AgentRepo) DeleteByAppInstallIDWithCtx(ctx context.Context, appInstallID
return getTx(ctx).Where("app_install_id = ?", appInstallID).Delete(&model.Agent{}).Error
}
func (a AgentRepo) ClearWebsiteIDByWebsiteIDWithCtx(ctx context.Context, websiteID uint) error {
if websiteID == 0 {
return nil
}
return getTx(ctx).Model(&model.Agent{}).Where("website_id = ?", websiteID).Update("website_id", 0).Error
}
func (a AgentRepo) List(opts ...DBOption) ([]model.Agent, error) {
var agents []model.Agent
if err := getDb(opts...).Find(&agents).Error; err != nil {
+50
View File
@@ -0,0 +1,50 @@
package repo
import "github.com/1Panel-dev/1Panel/agent/app/model"
type AgentAccountModelRepo struct{}
type IAgentAccountModelRepo interface {
List(opts ...DBOption) ([]model.AgentAccountModel, error)
GetFirst(opts ...DBOption) (*model.AgentAccountModel, error)
Create(item *model.AgentAccountModel) error
Save(item *model.AgentAccountModel) error
DeleteByID(id uint) error
Delete(opts ...DBOption) error
}
func NewIAgentAccountModelRepo() IAgentAccountModelRepo {
return &AgentAccountModelRepo{}
}
func (a AgentAccountModelRepo) List(opts ...DBOption) ([]model.AgentAccountModel, error) {
var list []model.AgentAccountModel
if err := getDb(opts...).Find(&list).Error; err != nil {
return nil, err
}
return list, nil
}
func (a AgentAccountModelRepo) GetFirst(opts ...DBOption) (*model.AgentAccountModel, error) {
var item model.AgentAccountModel
if err := getDb(opts...).First(&item).Error; err != nil {
return nil, err
}
return &item, nil
}
func (a AgentAccountModelRepo) Create(item *model.AgentAccountModel) error {
return getDb().Create(item).Error
}
func (a AgentAccountModelRepo) Save(item *model.AgentAccountModel) error {
return getDb().Save(item).Error
}
func (a AgentAccountModelRepo) DeleteByID(id uint) error {
return getDb().Delete(&model.AgentAccountModel{}, id).Error
}
func (a AgentAccountModelRepo) Delete(opts ...DBOption) error {
return getDb(opts...).Delete(&model.AgentAccountModel{}).Error
}
+2 -1
View File
@@ -80,9 +80,10 @@ func (u *LauncherRepo) ListQuickJump(withAll bool) []model.QuickJump {
}
if !withAll && len(quicks) == 0 {
return []model.QuickJump{
{Name: "Agent", Title: "aiTools.agents.agent", Recommend: 1, IsShow: true, Router: "/ai/agents/agent"},
{Name: "Website", Title: "menu.website", Recommend: 10, IsShow: true, Router: "/websites"},
{Name: "Database", Title: "menu.database", Recommend: 30, IsShow: true, Router: "/databases"},
{Name: "Cronjob", Title: "menu.cronjob", Recommend: 50, IsShow: true, Router: "/cronjobs"},
{Name: "Cronjob", Title: "menu.cronjob", Recommend: 50, IsShow: false, Router: "/cronjobs"},
{Name: "AppInstalled", Title: "home.appInstalled", Recommend: 70, IsShow: true, Router: "/apps/installed"},
}
}
+30
View File
@@ -19,6 +19,12 @@ func WithByID(id uint) DBOption {
}
}
func WithByGroupID(id uint) DBOption {
return func(g *gorm.DB) *gorm.DB {
return g.Where("group_id = ?", id)
}
}
func WithByNOTID(id uint) DBOption {
return func(g *gorm.DB) *gorm.DB {
return g.Where("id != ?", id)
@@ -43,6 +49,12 @@ func WithByName(name string) DBOption {
}
}
func WithByAddr(addr string) DBOption {
return func(g *gorm.DB) *gorm.DB {
return g.Where("addr = ?", addr)
}
}
func WithByKey(key string) DBOption {
return func(g *gorm.DB) *gorm.DB {
return g.Where("key = ?", key)
@@ -100,6 +112,24 @@ func WithByAccountID(accountID uint) DBOption {
}
}
func WithByWebsiteID(websiteID uint) DBOption {
return func(g *gorm.DB) *gorm.DB {
if websiteID == 0 {
return g
}
return g.Where("website_id = ?", websiteID)
}
}
func WithByAppInstallID(appInstallID uint) DBOption {
return func(g *gorm.DB) *gorm.DB {
if appInstallID == 0 {
return g
}
return g.Where("app_install_id = ?", appInstallID)
}
}
func WithByType(tp string) DBOption {
return func(g *gorm.DB) *gorm.DB {
return g.Where("`type` = ?", tp)
+113
View File
@@ -0,0 +1,113 @@
package repo
import (
"context"
"fmt"
"github.com/1Panel-dev/1Panel/agent/app/model"
"github.com/1Panel-dev/1Panel/agent/global"
"github.com/1Panel-dev/1Panel/agent/utils/encrypt"
"gorm.io/gorm"
)
type MongodbRepo struct{}
type IMongodbRepo interface {
Get(opts ...DBOption) (model.DatabaseMongodb, error)
WithByMongodbName(mongodbName string) DBOption
List(opts ...DBOption) ([]model.DatabaseMongodb, error)
Page(limit, offset int, opts ...DBOption) (int64, []model.DatabaseMongodb, error)
Create(ctx context.Context, mongodb *model.DatabaseMongodb) error
Delete(ctx context.Context, opts ...DBOption) error
Update(id uint, vars map[string]interface{}) error
DeleteLocal(ctx context.Context) error
}
func NewIMongodbRepo() IMongodbRepo {
return &MongodbRepo{}
}
func (u *MongodbRepo) Get(opts ...DBOption) (model.DatabaseMongodb, error) {
var mongodb model.DatabaseMongodb
db := global.DB
for _, opt := range opts {
db = opt(db)
}
if err := db.First(&mongodb).Error; err != nil {
return mongodb, err
}
pass, err := encrypt.StringDecrypt(mongodb.Password)
if err != nil {
global.LOG.Errorf("decrypt mongodb db %s password failed, err: %v", mongodb.Name, err)
}
mongodb.Password = pass
return mongodb, err
}
func (u *MongodbRepo) List(opts ...DBOption) ([]model.DatabaseMongodb, error) {
var mongodbs []model.DatabaseMongodb
db := global.DB.Model(&model.DatabaseMongodb{})
for _, opt := range opts {
db = opt(db)
}
if err := db.Find(&mongodbs).Error; err != nil {
return mongodbs, err
}
for i := 0; i < len(mongodbs); i++ {
pass, err := encrypt.StringDecrypt(mongodbs[i].Password)
if err != nil {
global.LOG.Errorf("decrypt mongodb db %s password failed, err: %v", mongodbs[i].Name, err)
}
mongodbs[i].Password = pass
}
return mongodbs, nil
}
func (u *MongodbRepo) Page(page, size int, opts ...DBOption) (int64, []model.DatabaseMongodb, error) {
var mongodbs []model.DatabaseMongodb
db := global.DB.Model(&model.DatabaseMongodb{})
for _, opt := range opts {
db = opt(db)
}
count := int64(0)
db = db.Count(&count)
if err := db.Limit(size).Offset(size * (page - 1)).Find(&mongodbs).Error; err != nil {
return count, mongodbs, err
}
for i := 0; i < len(mongodbs); i++ {
pass, err := encrypt.StringDecrypt(mongodbs[i].Password)
if err != nil {
global.LOG.Errorf("decrypt mongodb db %s password failed, err: %v", mongodbs[i].Name, err)
}
mongodbs[i].Password = pass
}
return count, mongodbs, nil
}
func (u *MongodbRepo) Create(ctx context.Context, mongodb *model.DatabaseMongodb) error {
pass, err := encrypt.StringEncrypt(mongodb.Password)
if err != nil {
return fmt.Errorf("encrypt mongodb db %s password failed, err: %v", mongodb.Name, err)
}
mongodb.Password = pass
return getTx(ctx).Create(mongodb).Error
}
func (u *MongodbRepo) Delete(ctx context.Context, opts ...DBOption) error {
return getTx(ctx, opts...).Delete(&model.DatabaseMongodb{}).Error
}
func (u *MongodbRepo) DeleteLocal(ctx context.Context) error {
return getTx(ctx).Where("`from` = ?", "local").Delete(&model.DatabaseMongodb{}).Error
}
func (u *MongodbRepo) Update(id uint, vars map[string]interface{}) error {
return global.DB.Model(&model.DatabaseMongodb{}).Where("id = ?", id).Updates(vars).Error
}
func (u *MongodbRepo) WithByMongodbName(mongodbName string) DBOption {
return func(g *gorm.DB) *gorm.DB {
return g.Where("mongodb_name = ?", mongodbName)
}
}
+121
View File
@@ -0,0 +1,121 @@
package repo
import (
"github.com/1Panel-dev/1Panel/agent/app/model"
"github.com/1Panel-dev/1Panel/agent/global"
"gorm.io/gorm"
)
type FileHistoryRepo struct{}
type IFileHistoryRepo interface {
Get(opts ...DBOption) (model.FileHistory, error)
Page(limit, offset int, opts ...DBOption) (int64, []model.FileHistory, error)
Create(history *model.FileHistory) error
Delete(opts ...DBOption) error
DeleteByIDs(ids []uint) error
WithByID(id uint) DBOption
WithByFileID(fileID string) DBOption
WithByPath(path string) DBOption
WithByPathHash(pathHash string) DBOption
WithByPathLike(path string) DBOption
WithByOperation(operation string) DBOption
WithNotOperation(operation string) DBOption
WithByRelatedPath(path string) DBOption
}
func NewIFileHistoryRepo() IFileHistoryRepo {
return &FileHistoryRepo{}
}
func (r *FileHistoryRepo) Get(opts ...DBOption) (model.FileHistory, error) {
var item model.FileHistory
db := global.DB.Model(&model.FileHistory{})
for _, opt := range opts {
db = opt(db)
}
return item, db.First(&item).Error
}
func (r *FileHistoryRepo) Page(limit, offset int, opts ...DBOption) (int64, []model.FileHistory, error) {
var total int64
var items []model.FileHistory
db := global.DB.Model(&model.FileHistory{})
for _, opt := range opts {
db = opt(db)
}
if err := db.Count(&total).Error; err != nil {
return 0, nil, err
}
if err := db.Order("created_at desc").Limit(limit).Offset(offset).Find(&items).Error; err != nil {
return 0, nil, err
}
return total, items, nil
}
func (r *FileHistoryRepo) Create(history *model.FileHistory) error {
return global.DB.Create(history).Error
}
func (r *FileHistoryRepo) Delete(opts ...DBOption) error {
db := global.DB.Model(&model.FileHistory{})
for _, opt := range opts {
db = opt(db)
}
return db.Delete(&model.FileHistory{}).Error
}
func (r *FileHistoryRepo) DeleteByIDs(ids []uint) error {
if len(ids) == 0 {
return nil
}
return global.DB.Where("id in ?", ids).Delete(&model.FileHistory{}).Error
}
func (r *FileHistoryRepo) WithByID(id uint) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("id = ?", id)
}
}
func (r *FileHistoryRepo) WithByFileID(fileID string) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("file_id = ?", fileID)
}
}
func (r *FileHistoryRepo) WithByPath(path string) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("path = ?", path)
}
}
func (r *FileHistoryRepo) WithByPathHash(pathHash string) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("path_hash = ?", pathHash)
}
}
func (r *FileHistoryRepo) WithByPathLike(path string) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("path like ?", "%"+path+"%")
}
}
func (r *FileHistoryRepo) WithByOperation(operation string) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("operation = ?", operation)
}
}
func (r *FileHistoryRepo) WithNotOperation(operation string) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("operation <> ?", operation)
}
}
func (r *FileHistoryRepo) WithByRelatedPath(path string) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("path = ? OR source_path = ? OR target_path = ?", path, path, path)
}
}
+77
View File
@@ -0,0 +1,77 @@
package repo
import (
"github.com/1Panel-dev/1Panel/agent/app/model"
"github.com/1Panel-dev/1Panel/agent/global"
"gorm.io/gorm"
)
type FileShareRepo struct{}
type IFileShareRepo interface {
Page(page, size int, opts ...DBOption) (int64, []model.FileShare, error)
Create(fileShare *model.FileShare) error
Save(fileShare *model.FileShare) error
Delete(opts ...DBOption) error
GetFirst(opts ...DBOption) (model.FileShare, error)
All() ([]model.FileShare, error)
WithByPath(path string) DBOption
WithByCode(code string) DBOption
}
func NewIFileShareRepo() IFileShareRepo {
return &FileShareRepo{}
}
func (r *FileShareRepo) WithByPath(path string) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("path = ?", path)
}
}
func (r *FileShareRepo) WithByCode(code string) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("token = ?", code)
}
}
func (r *FileShareRepo) Page(page, size int, opts ...DBOption) (int64, []model.FileShare, error) {
var (
items []model.FileShare
count int64
)
db := getDb(opts...).Model(&model.FileShare{})
db = db.Count(&count)
err := db.Order("path asc").Limit(size).Offset(size * (page - 1)).Find(&items).Error
return count, items, err
}
func (r *FileShareRepo) Create(fileShare *model.FileShare) error {
return global.DB.Create(fileShare).Error
}
func (r *FileShareRepo) Save(fileShare *model.FileShare) error {
return global.DB.Save(fileShare).Error
}
func (r *FileShareRepo) GetFirst(opts ...DBOption) (model.FileShare, error) {
var item model.FileShare
db := getDb(opts...).Model(&model.FileShare{})
if err := db.First(&item).Error; err != nil {
return item, err
}
return item, nil
}
func (r *FileShareRepo) Delete(opts ...DBOption) error {
db := getDb(opts...).Model(&model.FileShare{})
return db.Delete(&model.FileShare{}).Error
}
func (r *FileShareRepo) All() ([]model.FileShare, error) {
var items []model.FileShare
if err := getDb().Order("path asc").Find(&items).Error; err != nil {
return nil, err
}
return items, nil
}
+89
View File
@@ -10,6 +10,18 @@ import (
type HostRepo struct{}
type IHostRepo interface {
Get(opts ...DBOption) (model.Host, error)
GetList(opts ...DBOption) ([]model.Host, error)
Page(limit, offset int, opts ...DBOption) (int64, []model.Host, error)
Create(host *model.Host) error
Update(id uint, vars map[string]interface{}) error
UpdateGroup(group, newGroup uint) error
Delete(opts ...DBOption) error
WithByInfo(info string) DBOption
WithByPort(port uint) DBOption
WithByUser(user string) DBOption
GetFirewallRecord(opts ...DBOption) (model.Firewall, error)
ListFirewallRecord(opts ...DBOption) ([]model.Firewall, error)
SaveFirewallRecord(firewall *model.Firewall) error
@@ -30,6 +42,80 @@ func NewIHostRepo() IHostRepo {
return &HostRepo{}
}
func (h *HostRepo) Get(opts ...DBOption) (model.Host, error) {
var host model.Host
db := global.DB
for _, opt := range opts {
db = opt(db)
}
err := db.First(&host).Error
return host, err
}
func (h *HostRepo) GetList(opts ...DBOption) ([]model.Host, error) {
var hosts []model.Host
db := global.DB.Model(&model.Host{})
for _, opt := range opts {
db = opt(db)
}
err := db.Find(&hosts).Error
return hosts, err
}
func (h *HostRepo) Page(page, size int, opts ...DBOption) (int64, []model.Host, error) {
var hosts []model.Host
db := global.DB.Model(&model.Host{})
for _, opt := range opts {
db = opt(db)
}
count := int64(0)
db = db.Count(&count)
err := db.Limit(size).Offset(size * (page - 1)).Find(&hosts).Error
return count, hosts, err
}
func (h *HostRepo) WithByInfo(info string) DBOption {
return func(g *gorm.DB) *gorm.DB {
if len(info) == 0 {
return g
}
infoStr := "%" + info + "%"
return g.Where("name LIKE ? OR addr LIKE ?", infoStr, infoStr)
}
}
func (h *HostRepo) WithByPort(port uint) DBOption {
return func(g *gorm.DB) *gorm.DB {
return g.Where("port = ?", port)
}
}
func (h *HostRepo) WithByUser(user string) DBOption {
return func(g *gorm.DB) *gorm.DB {
return g.Where("user = ?", user)
}
}
func (h *HostRepo) Create(host *model.Host) error {
return global.DB.Create(host).Error
}
func (h *HostRepo) Update(id uint, vars map[string]interface{}) error {
return global.DB.Model(&model.Host{}).Where("id = ?", id).Updates(vars).Error
}
func (h *HostRepo) UpdateGroup(group, newGroup uint) error {
return global.DB.Model(&model.Host{}).Where("group_id = ?", group).Updates(map[string]interface{}{"group_id": newGroup}).Error
}
func (h *HostRepo) Delete(opts ...DBOption) error {
db := global.DB
for _, opt := range opts {
db = opt(db)
}
return db.Delete(&model.Host{}).Error
}
func (h *HostRepo) GetFirewallRecord(opts ...DBOption) (model.Firewall, error) {
var firewall model.Firewall
db := global.DB
@@ -79,6 +165,9 @@ func (h *HostRepo) SaveFirewallRecord(firewall *model.Firewall) error {
firewall.Strategy,
).First(&data).Error
}
if data.ID != 0 {
firewall.ID = data.ID
}
return global.DB.Save(firewall).Error
}
+10
View File
@@ -10,6 +10,7 @@ import (
type IWebsiteRepo interface {
WithAppInstallId(appInstallId uint) DBOption
WithAppInstallIds(appInstallIds []uint) DBOption
WithDomain(domain string) DBOption
WithAlias(alias string) DBOption
WithWebsiteSSLID(sslId uint) DBOption
@@ -48,6 +49,15 @@ func (w *WebsiteRepo) WithAppInstallId(appInstallID uint) DBOption {
}
}
func (w *WebsiteRepo) WithAppInstallIds(appInstallIDs []uint) DBOption {
return func(db *gorm.DB) *gorm.DB {
if len(appInstallIDs) == 0 {
return db
}
return db.Where("app_install_id in (?)", appInstallIDs)
}
}
func (w *WebsiteRepo) WithRuntimeID(runtimeID uint) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("runtime_id = ?", runtimeID)
+10
View File
@@ -13,6 +13,7 @@ type WebsiteDomainRepo struct {
type IWebsiteDomainRepo interface {
WithWebsiteId(websiteId uint) DBOption
WithWebsiteIds(websiteIds []uint) DBOption
WithPort(port int) DBOption
WithDomain(domain string) DBOption
WithDomainLike(domain string) DBOption
@@ -36,6 +37,15 @@ func (w WebsiteDomainRepo) WithWebsiteId(websiteId uint) DBOption {
}
}
func (w WebsiteDomainRepo) WithWebsiteIds(websiteIds []uint) DBOption {
return func(db *gorm.DB) *gorm.DB {
if len(websiteIds) == 0 {
return db
}
return db.Where("website_id in (?)", websiteIds)
}
}
func (w WebsiteDomainRepo) WithPort(port int) DBOption {
return func(db *gorm.DB) *gorm.DB {
return db.Where("port = ?", port)
File diff suppressed because it is too large Load Diff
+533
View File
@@ -0,0 +1,533 @@
package service
import (
"os"
"path"
"sort"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/dto/request"
"github.com/1Panel-dev/1Panel/agent/buserr"
"github.com/1Panel-dev/1Panel/agent/constant"
"github.com/1Panel-dev/1Panel/agent/global"
"github.com/1Panel-dev/1Panel/agent/utils/cmd"
)
var agentMarkdownFileNames = []string{"AGENTS.md", "SOUL.md", "USER.md", "IDENTITY.md", "TOOLS.md", "HEARTBEAT.md", "BOOT.md", "BOOTSTRAP.md"}
func (a AgentService) CreateRole(req dto.AgentRoleCreateReq) (*dto.AgentRoleCreateResp, error) {
_, install, err := a.loadAgentAndInstall(req.AgentID)
if err != nil {
return nil, err
}
args := []string{"exec", install.ContainerName, "openclaw", "agents", "add", req.Name}
workspace := "/home/node/.openclaw/workspace-agent_" + req.Name
agentDir := "/home/node/.openclaw/agents/" + req.Name
args = append(args, "--workspace", workspace)
if model := strings.TrimSpace(req.Model); model != "" {
args = append(args, "--model", model)
}
for _, binding := range req.Bindings {
channel := binding.Channel
if channel == "" {
continue
}
if accountID := binding.AccountID; accountID != "" {
channel = channel + ":" + accountID
}
args = append(args, "--bind", channel)
}
args = append(args, "--agent-dir", agentDir)
args = append(args, "--non-interactive", "--json")
mgr := cmd.NewCommandMgr(cmd.WithTimeout(5 * time.Minute))
output, err := mgr.RunWithStdout("docker", args...)
if err != nil {
return nil, err
}
return &dto.AgentRoleCreateResp{Output: strings.TrimSpace(output)}, nil
}
func (a AgentService) GetConfiguredAgents(req dto.AgentConfiguredAgentsReq) ([]dto.AgentConfiguredAgentItem, error) {
agent, _, conf, err := a.loadAgentConfig(req.AgentID)
if err != nil {
return nil, err
}
agents, ok := conf["agents"].(map[string]interface{})
if !ok {
return []dto.AgentConfiguredAgentItem{}, nil
}
rawList, ok := agents["list"].([]interface{})
if !ok {
return []dto.AgentConfiguredAgentItem{}, nil
}
result := make([]dto.AgentConfiguredAgentItem, 0, len(rawList))
baseDir := path.Join(global.Dir.AppInstallDir, agent.AgentType, agent.Name, "data")
for _, item := range rawList {
record, ok := item.(map[string]interface{})
if !ok {
continue
}
configured := extractConfiguredAgentItem(baseDir, record)
if strings.EqualFold(configured.ID, "main") {
continue
}
result = append(result, configured)
}
applyConfiguredAgentBindings(result, conf["bindings"])
return result, nil
}
func (a AgentService) GetRoleChannels(req dto.AgentRoleChannelsReq) ([]dto.AgentRoleChannelItem, error) {
_, _, conf, err := a.loadAgentConfig(req.AgentID)
if err != nil {
return nil, err
}
channels, ok := conf["channels"].(map[string]interface{})
if !ok || len(channels) == 0 {
return []dto.AgentRoleChannelItem{}, nil
}
boundBindings := loadBoundChannelBindings(conf["bindings"])
result := make([]dto.AgentRoleChannelItem, 0, len(channels))
for key := range channels {
if key == "" {
continue
}
accountIDs := extractRoleChannelAccountIDs(conf, key)
availableAccountIDs := filterAvailableChannelAccountIDs(boundBindings, key, accountIDs)
result = append(result, dto.AgentRoleChannelItem{
Name: key,
Bound: isRoleChannelFullyBound(boundBindings, key, accountIDs, availableAccountIDs),
AccountIDs: availableAccountIDs,
})
}
sort.Slice(result, func(i, j int) bool {
return result[i].Name < result[j].Name
})
return result, nil
}
func (a AgentService) DeleteRole(req dto.AgentRoleDeleteReq) error {
agent, install, conf, err := a.loadAgentConfig(req.AgentID)
if err != nil {
return err
}
baseDir := path.Join(global.Dir.AppInstallDir, agent.AgentType, agent.Name, "data")
if req.ID == "" {
return buserr.New("ErrRecordNotFound")
}
target, ok := findConfiguredAgentByID(baseDir, conf, req.ID)
if !ok {
return buserr.New("ErrRecordNotFound")
}
args := []string{"exec", install.ContainerName, "openclaw", "agents", "delete", req.ID, "--force"}
mgr := cmd.NewCommandMgr(cmd.WithTimeout(5 * time.Minute))
if _, err = mgr.RunWithStdout("docker", args...); err != nil {
return err
}
if target.Workspace != "" {
if err := os.RemoveAll(target.Workspace); err != nil {
return err
}
}
if target.AgentDir != "" {
if err := os.RemoveAll(target.AgentDir); err != nil {
return err
}
}
return nil
}
func (a AgentService) BindRole(req dto.AgentRoleBindReq) error {
return a.operateRoleBinding(req, "bind")
}
func (a AgentService) UnbindRole(req dto.AgentRoleBindReq) error {
return a.operateRoleBinding(req, "unbind")
}
func (a AgentService) operateRoleBinding(req dto.AgentRoleBindReq, action string) error {
agent, install, conf, err := a.loadAgentConfig(req.AgentID)
if err != nil {
return err
}
baseDir := path.Join(global.Dir.AppInstallDir, agent.AgentType, agent.Name, "data")
if req.ID == "" {
return buserr.New("ErrRecordNotFound")
}
if _, ok := findConfiguredAgentByID(baseDir, conf, req.ID); !ok {
return buserr.New("ErrRecordNotFound")
}
binding := formatRoleBinding(req.Channel, req.AccountID)
if binding == "" {
return buserr.New("ErrInvalidParams")
}
args := []string{
"exec",
install.ContainerName,
"openclaw",
"agents",
action,
"--agent",
req.ID,
"--bind",
binding,
}
args = append(args, "--json")
mgr := cmd.NewCommandMgr(cmd.WithTimeout(5 * time.Minute))
_, err = mgr.RunWithStdout("docker", args...)
return err
}
func (a AgentService) GetRoleMarkdownFiles(req dto.AgentRoleMarkdownFilesReq) ([]dto.AgentRoleMarkdownFileItem, error) {
agent, err := loadOpenclawAgentByID(req.AgentID)
if err != nil {
return nil, err
}
baseDir := path.Join(global.Dir.AppInstallDir, agent.AgentType, agent.Name, "data")
workspaceDir, err := resolveMarkdownWorkspaceDir(baseDir, req.Workspace)
if err != nil {
return nil, err
}
items := make([]dto.AgentRoleMarkdownFileItem, 0, len(agentMarkdownFileNames))
for _, name := range agentMarkdownFileNames {
item := dto.AgentRoleMarkdownFileItem{
Name: name,
}
content, readErr := os.ReadFile(path.Join(workspaceDir, name))
if readErr == nil {
item.Content = string(content)
} else if !os.IsNotExist(readErr) {
return nil, readErr
}
items = append(items, item)
}
return items, nil
}
func (a AgentService) UpdateRoleMarkdownFiles(req dto.AgentRoleMarkdownFilesUpdateReq) error {
agent, install, err := a.loadOpenclawAgentAndInstall(req.AgentID)
if err != nil {
return err
}
baseDir := path.Join(global.Dir.AppInstallDir, agent.AgentType, agent.Name, "data")
dirPath, err := resolveMarkdownWorkspaceDir(baseDir, req.Workspace)
if err != nil {
return err
}
if err = os.MkdirAll(dirPath, 0755); err != nil {
return err
}
for _, item := range req.Files {
file := path.Join(dirPath, item.Name)
if err = os.WriteFile(file, []byte(item.Content), 0644); err != nil {
return err
}
}
if req.Restart {
return NewIAppInstalledService().Operate(request.AppInstalledOperate{
InstallId: install.ID,
Operate: constant.Restart,
})
}
return nil
}
func extractConfiguredAgentItem(installDir string, record map[string]interface{}) dto.AgentConfiguredAgentItem {
item := dto.AgentConfiguredAgentItem{Bindings: []dto.AgentRoleBinding{}}
if id, ok := record["id"].(string); ok {
item.ID = id
} else if id, ok := record["agentId"].(string); ok {
item.ID = id
}
if name, ok := record["name"].(string); ok {
item.Name = name
}
if workspace, ok := record["workspace"].(string); ok {
item.Workspace = resolveRoleDir(installDir, workspace)
}
if model, ok := record["model"].(string); ok {
item.Model = model
}
if agentDir, ok := record["agentDir"].(string); ok {
item.AgentDir = agentDir
} else if agentDir, ok := record["agent_dir"].(string); ok {
item.AgentDir = agentDir
}
item.AgentDir = resolveRoleDir(installDir, item.AgentDir)
return item
}
func findConfiguredAgentByID(baseDir string, conf map[string]interface{}, id string) (dto.AgentConfiguredAgentItem, bool) {
agents, ok := conf["agents"].(map[string]interface{})
if !ok {
return dto.AgentConfiguredAgentItem{}, false
}
rawList, ok := agents["list"].([]interface{})
if !ok {
return dto.AgentConfiguredAgentItem{}, false
}
for _, item := range rawList {
record, ok := item.(map[string]interface{})
if !ok {
continue
}
configured := extractConfiguredAgentItem(baseDir, record)
if strings.EqualFold(configured.ID, id) {
return configured, true
}
}
return dto.AgentConfiguredAgentItem{}, false
}
func formatRoleBinding(channel, accountID string) string {
if channel == "" {
return ""
}
if accountID == "" {
return channel
}
return channel + ":" + accountID
}
func applyConfiguredAgentBindings(agents []dto.AgentConfiguredAgentItem, value interface{}) {
bindings, ok := value.([]interface{})
if !ok || len(agents) == 0 {
return
}
indexByID := make(map[string]int, len(agents))
indexByName := make(map[string]int, len(agents))
for i, agent := range agents {
if agent.ID != "" {
indexByID[agent.ID] = i
}
if agent.Name != "" {
indexByName[agent.Name] = i
}
}
for _, binding := range bindings {
record, ok := binding.(map[string]interface{})
if !ok {
continue
}
if bindingType, _ := record["type"].(string); !strings.EqualFold(bindingType, "route") {
continue
}
targetID, _ := record["agentId"].(string)
if targetID == "" {
continue
}
match, ok := record["match"].(map[string]interface{})
if !ok {
continue
}
channel, _ := match["channel"].(string)
if channel == "" {
continue
}
index, ok := indexByID[targetID]
if !ok {
index, ok = indexByName[targetID]
}
if !ok {
continue
}
accountID, _ := match["accountId"].(string)
if accountID == "" {
accountID, _ = record["accountId"].(string)
}
agents[index].Bindings = append(agents[index].Bindings, dto.AgentRoleBinding{
Channel: channel,
AccountID: accountID,
})
}
}
func loadBoundChannelBindings(value interface{}) map[string]map[string]struct{} {
result := make(map[string]map[string]struct{})
bindings, ok := value.([]interface{})
if !ok {
return result
}
for _, binding := range bindings {
record, ok := binding.(map[string]interface{})
if !ok {
continue
}
if bindingType, _ := record["type"].(string); !strings.EqualFold(bindingType, "route") {
continue
}
match, ok := record["match"].(map[string]interface{})
if !ok {
continue
}
channel, _ := match["channel"].(string)
if channel == "" {
continue
}
accountID, _ := match["accountId"].(string)
if accountID == "" {
accountID, _ = record["accountId"].(string)
}
if _, ok := result[channel]; !ok {
result[channel] = make(map[string]struct{})
}
result[channel][accountID] = struct{}{}
}
return result
}
func extractRoleChannelAccountIDs(conf map[string]interface{}, channel string) []string {
switch channel {
case "feishu":
config := extractFeishuConfig(conf)
accountIDs := make([]string, 0, len(config.Bots))
for _, item := range config.Bots {
if accountID := item.AccountID; accountID != "" {
accountIDs = append(accountIDs, accountID)
}
}
sort.Strings(accountIDs)
return accountIDs
case "telegram":
config := extractTelegramConfig(conf)
accountIDs := make([]string, 0, len(config.Bots))
for _, item := range config.Bots {
if accountID := item.AccountID; accountID != "" {
accountIDs = append(accountIDs, accountID)
}
}
sort.Strings(accountIDs)
return accountIDs
case "discord":
config := extractDiscordConfig(conf)
accountIDs := make([]string, 0, len(config.Bots))
for _, item := range config.Bots {
if accountID := item.AccountID; accountID != "" {
accountIDs = append(accountIDs, accountID)
}
}
sort.Strings(accountIDs)
return accountIDs
case "qqbot":
config := extractQQBotConfig(conf)
accountIDs := make([]string, 0, len(config.Bots))
for _, item := range config.Bots {
if accountID := item.AccountID; accountID != "" {
accountIDs = append(accountIDs, accountID)
}
}
sort.Strings(accountIDs)
return accountIDs
case "dingtalk-connector":
config := extractDingTalkConfig(conf)
accountIDs := make([]string, 0, len(config.Bots))
for _, item := range config.Bots {
if accountID := item.AccountID; accountID != "" {
accountIDs = append(accountIDs, accountID)
}
}
sort.Strings(accountIDs)
return accountIDs
case "wecom":
return []string{}
default:
return extractRawChannelAccountIDs(getChannelConfig(conf, channel))
}
}
func filterAvailableChannelAccountIDs(bindings map[string]map[string]struct{}, channel string, accountIDs []string) []string {
channelBindings, ok := bindings[channel]
if !ok || len(channelBindings) == 0 {
return append([]string(nil), accountIDs...)
}
if _, ok := channelBindings[""]; ok {
return []string{}
}
result := make([]string, 0, len(accountIDs))
for _, accountID := range accountIDs {
if _, ok := channelBindings[accountID]; ok {
continue
}
result = append(result, accountID)
}
return result
}
func isRoleChannelFullyBound(bindings map[string]map[string]struct{}, channel string, allAccountIDs, availableAccountIDs []string) bool {
channelBindings, ok := bindings[channel]
if !ok || len(channelBindings) == 0 {
return false
}
if _, ok := channelBindings[""]; ok {
return true
}
if len(allAccountIDs) == 0 {
return true
}
return len(availableAccountIDs) == 0
}
func extractRawChannelAccountIDs(channel map[string]interface{}) []string {
if len(channel) == 0 {
return []string{}
}
accounts, ok := channel["accounts"].(map[string]interface{})
if !ok || len(accounts) == 0 {
return []string{}
}
result := make([]string, 0, len(accounts))
for key := range accounts {
if key == "" {
continue
}
result = append(result, key)
}
sort.Strings(result)
return result
}
func resolveRoleDir(installDir, workspace string) string {
if workspace == "" {
return ""
}
if strings.HasPrefix(workspace, "/home/node/.openclaw/workspace/") {
return strings.ReplaceAll(workspace, "/home/node/.openclaw", installDir)
}
return strings.ReplaceAll(workspace, "/home/node/.openclaw", path.Join(installDir, "conf"))
}
func resolveMarkdownWorkspaceDir(installDir, workspace string) (string, error) {
if workspace == "" {
return "", buserr.New("ErrRecordNotFound")
}
confDir := path.Join(installDir, "conf")
allowedOpenclawPrefixes := []string{
"/home/node/.openclaw/workspace/",
"/home/node/.openclaw/workspace-agent_",
}
for _, prefix := range allowedOpenclawPrefixes {
if strings.HasPrefix(workspace, prefix) {
return resolveRoleDir(installDir, workspace), nil
}
}
cleanWorkspace := path.Clean(workspace)
cleanConfDir := path.Clean(confDir)
if cleanWorkspace == cleanConfDir || strings.HasPrefix(cleanWorkspace, cleanConfDir+"/") {
return cleanWorkspace, nil
}
return "", buserr.New("ErrRecordNotFound")
}
File diff suppressed because it is too large Load Diff
+731
View File
@@ -0,0 +1,731 @@
package service
import (
"errors"
"fmt"
"path"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/model"
providercatalog "github.com/1Panel-dev/1Panel/agent/app/provider"
"github.com/1Panel-dev/1Panel/agent/buserr"
"github.com/1Panel-dev/1Panel/agent/constant"
"github.com/1Panel-dev/1Panel/agent/utils/common"
agentenv "github.com/1Panel-dev/1Panel/agent/utils/env"
"github.com/1Panel-dev/1Panel/agent/utils/files"
"github.com/joho/godotenv"
"gopkg.in/yaml.v3"
)
const hermesWorkspaceDir = "/opt/data/workspace"
const hermesExecutablePath = "/opt/hermes/.venv/bin/hermes"
type hermesConfig struct {
Model hermesModelConfig `yaml:"model"`
Terminal hermesTerminalConfig `yaml:"terminal"`
Timezone string `yaml:"timezone,omitempty"`
}
type hermesModelConfig struct {
Default string `yaml:"default"`
Provider string `yaml:"provider"`
BaseURL string `yaml:"base_url,omitempty"`
APIKey string `yaml:"api_key,omitempty"`
}
type hermesTerminalConfig struct {
Backend string `yaml:"backend"`
Cwd string `yaml:"cwd"`
}
type hermesEnvEntry struct {
Key string
Value string
}
func buildHermesDockerExecCommandArgs(containerName, command string, commandArgs ...string) []string {
args := []string{"exec", "-u", "hermes", containerName, command}
return append(args, commandArgs...)
}
func buildHermesDockerExecArgs(containerName string, hermesArgs ...string) []string {
return buildHermesDockerExecCommandArgs(containerName, "hermes", hermesArgs...)
}
func writeHermesConfig(confDir string, account *model.AgentAccount, modelName string, timezone string) error {
if strings.TrimSpace(confDir) == "" {
return fmt.Errorf("config dir is required")
}
if account == nil {
return fmt.Errorf("account is required")
}
if strings.TrimSpace(modelName) == "" {
return fmt.Errorf("model is required")
}
fileOp := files.NewFileOp()
if !fileOp.Stat(confDir) {
if err := fileOp.CreateDir(confDir, constant.DirPerm); err != nil {
return err
}
}
provider := resolveHermesProvider(account.Provider)
configPath := path.Join(confDir, "config.yaml")
cfg, err := readHermesConfigMap(configPath)
if err != nil {
return err
}
model := map[string]interface{}{
"default": resolveHermesModel(account.Provider, provider, modelName),
"provider": provider,
"base_url": account.BaseURL,
}
if provider == "custom" && account.APIKey != "" {
model["api_key"] = "${CUSTOM_API_KEY}"
}
cfg["model"] = model
cfg["terminal"] = map[string]interface{}{
"backend": "local",
"cwd": hermesWorkspaceDir,
}
if timezone != "" {
cfg["timezone"] = timezone
} else {
delete(cfg, "timezone")
}
if err := writeHermesConfigMap(configPath, cfg); err != nil {
return err
}
return writeHermesModelEnv(path.Join(confDir, ".env"), account)
}
func prepareHermesInstallFiles(appInstall *model.AppInstall, account *model.AgentAccount, modelName string) error {
if appInstall == nil {
return fmt.Errorf("app install is required")
}
dataDir := path.Join(appInstall.GetPath(), "data")
if err := writeHermesConfig(dataDir, account, modelName, normalizeHermesTimezone(common.LoadTimeZoneByCmd())); err != nil {
return err
}
return files.NewFileOp().ChownR(dataDir, "1000", "1000", true)
}
func readHermesConfig(configPath string) (*hermesConfig, error) {
content, err := files.NewFileOp().GetContent(configPath)
if err != nil {
return nil, err
}
var cfg hermesConfig
if err := yaml.Unmarshal(content, &cfg); err != nil {
return nil, err
}
return &cfg, nil
}
func readHermesConfigMap(configPath string) (map[string]interface{}, error) {
fileOp := files.NewFileOp()
if !fileOp.Stat(configPath) {
return map[string]interface{}{}, nil
}
content, err := fileOp.GetContent(configPath)
if err != nil {
return nil, err
}
cfg := map[string]interface{}{}
if strings.TrimSpace(string(content)) == "" {
return cfg, nil
}
if err := yaml.Unmarshal(content, &cfg); err != nil {
return nil, err
}
return cfg, nil
}
func writeHermesConfigMap(configPath string, cfg map[string]interface{}) error {
payload, err := yaml.Marshal(cfg)
if err != nil {
return err
}
return files.NewFileOp().SaveFile(configPath, string(payload), 0600)
}
func readHermesTelegramChannelConfig(confDir string) (*dto.AgentTelegramConfig, error) {
envMap, err := readHermesEnvMap(path.Join(confDir, ".env"))
if err != nil {
return nil, err
}
cfg, err := readHermesConfigMap(path.Join(confDir, "config.yaml"))
if err != nil {
return nil, err
}
allowFrom := splitHermesEnvList(envMap["TELEGRAM_ALLOWED_USERS"])
requireMention := extractBoolValue(childMap(cfg, "telegram")["require_mention"], false)
dmPolicy := ""
if extractHermesEnvBool(envMap, "TELEGRAM_ALLOW_ALL_USERS", false) {
dmPolicy = "open"
} else if len(allowFrom) > 0 {
dmPolicy = "allowlist"
} else if envMap["TELEGRAM_BOT_TOKEN"] != "" {
dmPolicy = "pairing"
}
groupPolicy := ""
if requireMention {
groupPolicy = "allowlist"
} else if envMap["TELEGRAM_BOT_TOKEN"] != "" {
groupPolicy = "open"
}
token := envMap["TELEGRAM_BOT_TOKEN"]
result := &dto.AgentTelegramConfig{
Enabled: token != "",
DmPolicy: dmPolicy,
AllowFrom: allowFrom,
RequireMention: requireMention,
GroupPolicy: groupPolicy,
GroupAllowFrom: []string{},
Streaming: "",
DefaultAccount: "",
}
if token != "" {
result.Bots = []dto.AgentTelegramBot{
{
AgentChannelBotBase: dto.AgentChannelBotBase{
AccountID: "default",
Name: "Default",
Enabled: true,
IsDefault: true,
},
BotToken: token,
DmPolicy: dmPolicy,
GroupPolicy: groupPolicy,
Streaming: "",
},
}
}
return result, nil
}
func writeHermesTelegramChannelConfig(confDir string, config dto.AgentTelegramConfig) error {
envPath := path.Join(confDir, ".env")
envMap, err := readHermesEnvMap(envPath)
if err != nil {
return err
}
token := firstHermesTelegramBotToken(config.Bots, config.DefaultAccount)
if token != "" {
envMap["TELEGRAM_BOT_TOKEN"] = token
} else {
delete(envMap, "TELEGRAM_BOT_TOKEN")
}
delete(envMap, "TELEGRAM_ALLOWED_USERS")
delete(envMap, "TELEGRAM_ALLOW_ALL_USERS")
switch config.DmPolicy {
case "open":
envMap["TELEGRAM_ALLOW_ALL_USERS"] = "true"
case "allowlist":
if allow := joinHermesEnvList(config.AllowFrom); allow != "" {
envMap["TELEGRAM_ALLOWED_USERS"] = allow
}
}
if err := writeHermesEnvMap(envPath, envMap, []string{
"TELEGRAM_BOT_TOKEN",
"TELEGRAM_ALLOWED_USERS",
"TELEGRAM_ALLOW_ALL_USERS",
}); err != nil {
return err
}
configPath := path.Join(confDir, "config.yaml")
cfg, err := readHermesConfigMap(configPath)
if err != nil {
return err
}
telegram := ensureChildMap(cfg, "telegram")
telegram["require_mention"] = config.RequireMention
return writeHermesConfigMap(configPath, cfg)
}
func readHermesDiscordChannelConfig(confDir string) (*dto.AgentDiscordConfig, error) {
envMap, err := readHermesEnvMap(path.Join(confDir, ".env"))
if err != nil {
return nil, err
}
cfg, err := readHermesConfigMap(path.Join(confDir, "config.yaml"))
if err != nil {
return nil, err
}
allowFrom := splitHermesEnvList(envMap["DISCORD_ALLOWED_USERS"])
requireMention := extractBoolValue(childMap(cfg, "discord")["require_mention"], false)
dmPolicy := ""
if extractHermesEnvBool(envMap, "DISCORD_ALLOW_ALL_USERS", false) {
dmPolicy = "open"
} else if len(allowFrom) > 0 {
dmPolicy = "allowlist"
} else if envMap["DISCORD_BOT_TOKEN"] != "" {
dmPolicy = "pairing"
}
groupPolicy := ""
if requireMention {
groupPolicy = "allowlist"
} else if envMap["DISCORD_BOT_TOKEN"] != "" {
groupPolicy = "open"
}
token := envMap["DISCORD_BOT_TOKEN"]
result := &dto.AgentDiscordConfig{
Enabled: token != "",
DmPolicy: dmPolicy,
AllowFrom: allowFrom,
RequireMention: requireMention,
GroupPolicy: groupPolicy,
DefaultAccount: "",
}
if token != "" {
result.Bots = []dto.AgentDiscordBot{
{
AgentChannelBotBase: dto.AgentChannelBotBase{
AccountID: "default",
Name: "Default",
Enabled: true,
IsDefault: true,
},
Token: token,
},
}
}
return result, nil
}
func writeHermesDiscordChannelConfig(confDir string, config dto.AgentDiscordConfig) error {
envPath := path.Join(confDir, ".env")
envMap, err := readHermesEnvMap(envPath)
if err != nil {
return err
}
token := firstHermesDiscordBotToken(config.Bots, config.DefaultAccount)
if token != "" {
envMap["DISCORD_BOT_TOKEN"] = token
} else {
delete(envMap, "DISCORD_BOT_TOKEN")
}
delete(envMap, "DISCORD_ALLOWED_USERS")
delete(envMap, "DISCORD_ALLOW_ALL_USERS")
switch config.DmPolicy {
case "open":
envMap["DISCORD_ALLOW_ALL_USERS"] = "true"
case "allowlist":
if allow := joinHermesEnvList(config.AllowFrom); allow != "" {
envMap["DISCORD_ALLOWED_USERS"] = allow
}
}
if err := writeHermesEnvMap(envPath, envMap, []string{
"DISCORD_BOT_TOKEN",
"DISCORD_ALLOWED_USERS",
"DISCORD_ALLOW_ALL_USERS",
}); err != nil {
return err
}
configPath := path.Join(confDir, "config.yaml")
cfg, err := readHermesConfigMap(configPath)
if err != nil {
return err
}
discord := ensureChildMap(cfg, "discord")
discord["require_mention"] = config.RequireMention
return writeHermesConfigMap(configPath, cfg)
}
func deleteHermesEnvKeys(confDir string, keys ...string) error {
envPath := path.Join(confDir, ".env")
envMap, err := readHermesEnvMap(envPath)
if err != nil {
return err
}
for _, key := range keys {
delete(envMap, key)
}
return writeHermesEnvMap(envPath, envMap, keys)
}
func deleteHermesConfigSections(confDir string, topLevelKeys []string, platformKeys []string) error {
configPath := path.Join(confDir, "config.yaml")
cfg, err := readHermesConfigMap(configPath)
if err != nil {
return err
}
for _, key := range topLevelKeys {
delete(cfg, key)
}
if len(platformKeys) > 0 {
if platforms, ok := cfg["platforms"].(map[string]interface{}); ok {
for _, key := range platformKeys {
delete(platforms, key)
}
if len(platforms) == 0 {
delete(cfg, "platforms")
}
}
}
return writeHermesConfigMap(configPath, cfg)
}
func deleteHermesTelegramChannelConfig(confDir string) error {
if err := deleteHermesEnvKeys(confDir,
"TELEGRAM_BOT_TOKEN",
"TELEGRAM_ALLOWED_USERS",
"TELEGRAM_ALLOW_ALL_USERS",
"TELEGRAM_HOME_CHANNEL",
); err != nil {
return err
}
return deleteHermesConfigSections(confDir, []string{"telegram"}, []string{"telegram"})
}
func deleteHermesDiscordChannelConfig(confDir string) error {
if err := deleteHermesEnvKeys(confDir,
"DISCORD_BOT_TOKEN",
"DISCORD_ALLOWED_USERS",
"DISCORD_ALLOW_ALL_USERS",
"DISCORD_HOME_CHANNEL",
); err != nil {
return err
}
return deleteHermesConfigSections(confDir, []string{"discord"}, []string{"discord"})
}
func normalizeHermesTimezone(timezone string) string {
timezone = strings.TrimSpace(timezone)
if timezone == "" {
return ""
}
if _, err := time.LoadLocation(timezone); err != nil {
return ""
}
return timezone
}
func resolveHermesProvider(provider string) string {
switch provider {
case "":
return "custom"
case "openrouter", "anthropic", "gemini", "zai", "kimi-coding", "xiaomi":
return provider
case "minimax":
return "minimax-cn"
default:
return "custom"
}
}
func resolveHermesModel(sourceProvider, targetProvider, modelName string) string {
target := strings.TrimSpace(modelName)
if target == "" {
return ""
}
if targetProvider != "custom" {
return target
}
if sourceProvider == "custom" || sourceProvider == "vllm" {
return normalizeCustomModel(target)
}
if strings.Contains(target, "/") {
parts := strings.SplitN(target, "/", 2)
model := strings.TrimSpace(parts[1])
if model != "" {
return model
}
}
return target
}
func resolveHermesConfiguredModelID(account *model.AgentAccount, accountModels []dto.AgentAccountModel, configuredModel string) (string, error) {
if account == nil {
return "", buserr.New("ErrAgentModelNotInAccount")
}
configuredModel = strings.TrimSpace(configuredModel)
if configuredModel == "" {
return "", buserr.New("ErrAgentModelNotInAccount")
}
provider := resolveHermesProvider(account.Provider)
for _, item := range accountModels {
if resolveHermesModel(account.Provider, provider, item.ID) == configuredModel {
return item.ID, nil
}
}
return "", buserr.New("ErrAgentModelNotInAccount")
}
func resolveHermesEnvEntries(account *model.AgentAccount) []hermesEnvEntry {
if account == nil {
return nil
}
if resolveHermesProvider(account.Provider) == "custom" {
if account.APIKey == "" {
return nil
}
return []hermesEnvEntry{
{Key: "CUSTOM_API_KEY", Value: account.APIKey},
}
}
apiKey := account.APIKey
baseURL := account.BaseURL
entries := make([]hermesEnvEntry, 0, 4)
appendEntry := func(key, value string) {
if key == "" || value == "" {
return
}
entries = append(entries, hermesEnvEntry{Key: key, Value: value})
}
switch account.Provider {
case "openrouter":
appendEntry("OPENROUTER_API_KEY", apiKey)
appendEntry("OPENROUTER_BASE_URL", baseURL)
case "anthropic":
appendEntry("ANTHROPIC_API_KEY", apiKey)
case "gemini":
appendEntry("GOOGLE_API_KEY", apiKey)
appendEntry("GEMINI_API_KEY", apiKey)
appendEntry("GEMINI_BASE_URL", baseURL)
case "zai":
appendEntry("GLM_API_KEY", apiKey)
appendEntry("ZAI_API_KEY", apiKey)
appendEntry("Z_AI_API_KEY", apiKey)
appendEntry("GLM_BASE_URL", baseURL)
case "kimi-coding", "moonshot":
appendEntry("KIMI_API_KEY", apiKey)
appendEntry("KIMI_BASE_URL", baseURL)
case "kimi":
appendEntry("KIMI_CN_API_KEY", apiKey)
appendEntry("KIMI_BASE_URL", baseURL)
case "minimax":
appendEntry("MINIMAX_CN_API_KEY", apiKey)
appendEntry("MINIMAX_CN_BASE_URL", baseURL)
case "xiaomi":
appendEntry("XIAOMI_API_KEY", apiKey)
appendEntry("XIAOMI_BASE_URL", baseURL)
case "deepseek":
appendEntry("DEEPSEEK_API_KEY", apiKey)
appendEntry("DEEPSEEK_BASE_URL", baseURL)
case "bailian-coding-plan":
appendEntry("DASHSCOPE_API_KEY", apiKey)
appendEntry("DASHSCOPE_BASE_URL", baseURL)
case "openai":
appendEntry("OPENAI_API_KEY", apiKey)
appendEntry("OPENAI_BASE_URL", baseURL)
case "custom", "vllm":
if account.APIType == "openai-completions" || account.APIType == "openai-responses" {
appendEntry("OPENAI_API_KEY", apiKey)
appendEntry("OPENAI_BASE_URL", baseURL)
}
default:
if envKey := providercatalog.EnvKey(account.Provider); envKey != "" {
appendEntry(envKey, apiKey)
}
}
return entries
}
func writeHermesModelEnv(envPath string, account *model.AgentAccount) error {
envMap, err := readHermesEnvMap(envPath)
if err != nil {
return err
}
for _, key := range hermesManagedModelEnvKeys() {
delete(envMap, key)
}
entries := resolveHermesEnvEntries(account)
order := make([]string, 0, len(entries))
for _, entry := range entries {
if entry.Key == "" || entry.Value == "" {
continue
}
envMap[entry.Key] = entry.Value
order = append(order, entry.Key)
}
return writeHermesEnvMap(envPath, envMap, order)
}
func hermesManagedModelEnvKeys() []string {
keys := []string{
"OPENROUTER_API_KEY",
"OPENROUTER_BASE_URL",
"ANTHROPIC_API_KEY",
"GOOGLE_API_KEY",
"GEMINI_API_KEY",
"GEMINI_BASE_URL",
"GLM_API_KEY",
"ZAI_API_KEY",
"Z_AI_API_KEY",
"GLM_BASE_URL",
"KIMI_API_KEY",
"KIMI_CN_API_KEY",
"KIMI_BASE_URL",
"MINIMAX_CN_API_KEY",
"MINIMAX_CN_BASE_URL",
"XIAOMI_API_KEY",
"XIAOMI_BASE_URL",
"DEEPSEEK_API_KEY",
"DEEPSEEK_BASE_URL",
"DASHSCOPE_API_KEY",
"DASHSCOPE_BASE_URL",
"OPENAI_API_KEY",
"OPENAI_BASE_URL",
}
seen := make(map[string]struct{}, len(keys))
result := make([]string, 0, len(keys))
appendKey := func(key string) {
if key == "" {
return
}
if _, ok := seen[key]; ok {
return
}
seen[key] = struct{}{}
result = append(result, key)
}
for _, key := range keys {
appendKey(key)
}
for _, meta := range providercatalog.All() {
appendKey(meta.EnvKey)
}
return result
}
func writeHermesEnv(envPath string, entries []hermesEnvEntry) error {
envMap, err := readHermesEnvMap(envPath)
if err != nil {
return err
}
for _, entry := range entries {
if entry.Key == "" || entry.Value == "" {
continue
}
envMap[entry.Key] = entry.Value
}
order := make([]string, 0, len(entries))
for _, entry := range entries {
if entry.Key == "" {
continue
}
order = append(order, entry.Key)
}
return writeHermesEnvMap(envPath, envMap, order)
}
func readHermesEnvMap(envPath string) (map[string]string, error) {
fileOp := files.NewFileOp()
if !fileOp.Stat(envPath) {
return map[string]string{}, nil
}
envMap, err := godotenv.Read(envPath)
if err != nil {
return nil, err
}
return envMap, nil
}
func writeHermesEnvMap(envPath string, envMap map[string]string, order []string) error {
if len(envMap) == 0 {
return files.NewFileOp().SaveFile(envPath, "", 0600)
}
return agentenv.WriteWithOrder(envMap, envPath, order)
}
func splitHermesEnvList(value string) []string {
if value == "" {
return []string{}
}
parts := strings.Split(value, ",")
result := make([]string, 0, len(parts))
seen := make(map[string]struct{}, len(parts))
for _, part := range parts {
item := strings.TrimSpace(part)
if item == "" {
continue
}
if _, ok := seen[item]; ok {
continue
}
seen[item] = struct{}{}
result = append(result, item)
}
return result
}
func joinHermesEnvList(values []string) string {
if len(values) == 0 {
return ""
}
result := make([]string, 0, len(values))
seen := make(map[string]struct{}, len(values))
for _, value := range values {
item := strings.TrimSpace(value)
if item == "" {
continue
}
if _, ok := seen[item]; ok {
continue
}
seen[item] = struct{}{}
result = append(result, item)
}
return strings.Join(result, ",")
}
func firstHermesTelegramBotToken(bots []dto.AgentTelegramBot, defaultAccount string) string {
for _, bot := range bots {
if bot.AccountID == defaultAccount || bot.IsDefault {
return bot.BotToken
}
}
if len(bots) == 0 {
return ""
}
return bots[0].BotToken
}
func firstHermesDiscordBotToken(bots []dto.AgentDiscordBot, defaultAccount string) string {
for _, bot := range bots {
if bot.AccountID == defaultAccount || bot.IsDefault {
return bot.Token
}
}
if len(bots) == 0 {
return ""
}
return bots[0].Token
}
func extractHermesEnvBool(envMap map[string]string, key string, defaultValue bool) bool {
value, ok := envMap[key]
if !ok || value == "" {
return defaultValue
}
return strings.EqualFold(value, "true")
}
func validateHermesPairingApproveResult(output string, err error) error {
if strings.Contains(output, "Approved!") {
return nil
}
if strings.Contains(output, "not found or expired for platform") {
return buserr.New("ErrHermesPairingCodeUnavailable")
}
if err == nil {
if strings.TrimSpace(output) == "" {
return fmt.Errorf("unexpected hermes pairing approve result")
}
return errors.New(strings.TrimSpace(output))
}
if strings.Contains(err.Error(), "not found or expired for platform") {
return buserr.New("ErrHermesPairingCodeUnavailable")
}
return err
}
+591
View File
@@ -0,0 +1,591 @@
package service
import (
"path"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/buserr"
)
const hermesWeixinLoginScript = `import asyncio
from gateway.platforms.weixin import qr_login
from hermes_cli.config import save_env_value
creds = asyncio.run(qr_login('/opt/data'))
print('RESULT =', creds)
if not creds:
raise SystemExit(1)
save_env_value('WEIXIN_ACCOUNT_ID', creds.get('account_id', ''))
save_env_value('WEIXIN_TOKEN', creds.get('token', ''))
if creds.get('base_url'):
save_env_value('WEIXIN_BASE_URL', creds['base_url'])
save_env_value('WEIXIN_CDN_BASE_URL', 'https://novac2c.cdn.weixin.qq.com/c2c')
save_env_value('WEIXIN_DM_POLICY', 'open')
save_env_value('WEIXIN_ALLOW_ALL_USERS', 'true')
save_env_value('WEIXIN_ALLOWED_USERS', '')
save_env_value('WEIXIN_GROUP_POLICY', 'disabled')
save_env_value('WEIXIN_GROUP_ALLOWED_USERS', '')
save_env_value('WEIXIN_HOME_CHANNEL', creds.get('user_id', ''))
print('WEIXIN ENV WRITTEN')
print('Restart the Hermes-Agent container to apply the new Weixin settings.')
`
func readHermesQQBotChannelConfig(confDir string) (*dto.AgentQQBotConfig, error) {
envMap, err := readHermesEnvMap(path.Join(confDir, ".env"))
if err != nil {
return nil, err
}
cfg, err := readHermesConfigMap(path.Join(confDir, "config.yaml"))
if err != nil {
return nil, err
}
platform := childMap(childMap(cfg, "platforms"), "qq")
extra := childMap(platform, "extra")
appID := envMap["QQ_APP_ID"]
clientSecret := envMap["QQ_CLIENT_SECRET"]
dmPolicy := extractStringValue(extra["dm_policy"])
allowFrom := splitHermesEnvList(envMap["QQ_ALLOWED_USERS"])
groupPolicy := extractStringValue(extra["group_policy"])
groupAllowFrom := splitHermesEnvList(envMap["QQ_GROUP_ALLOWED_USERS"])
result := &dto.AgentQQBotConfig{
Enabled: extractBoolValue(platform["enabled"], false) && appID != "" && clientSecret != "",
DmPolicy: dmPolicy,
AllowFrom: allowFrom,
GroupPolicy: groupPolicy,
GroupAllowFrom: groupAllowFrom,
}
if appID != "" || clientSecret != "" {
result.Bots = []dto.AgentQQBotBot{
{
AgentChannelBotBase: dto.AgentChannelBotBase{
AccountID: "default",
Name: "Default",
Enabled: true,
IsDefault: true,
},
AppID: appID,
ClientSecret: clientSecret,
},
}
}
return result, nil
}
func writeHermesQQBotChannelConfig(confDir string, config dto.AgentQQBotConfig) error {
envPath := path.Join(confDir, ".env")
envMap, err := readHermesEnvMap(envPath)
if err != nil {
return err
}
defaultBot := getDefaultQQBot(config.Bots)
envMap["QQ_APP_ID"] = defaultBot.AppID
envMap["QQ_CLIENT_SECRET"] = defaultBot.ClientSecret
delete(envMap, "QQ_ALLOWED_USERS")
delete(envMap, "QQ_GROUP_ALLOWED_USERS")
delete(envMap, "QQ_ALLOW_ALL_USERS")
delete(envMap, "QQ_MARKDOWN_SUPPORT")
if config.DmPolicy == "open" {
envMap["QQ_ALLOW_ALL_USERS"] = "true"
} else if config.DmPolicy == "allowlist" {
if allow := joinHermesEnvList(config.AllowFrom); allow != "" {
envMap["QQ_ALLOWED_USERS"] = allow
}
}
if config.GroupPolicy == "allowlist" {
if allow := joinHermesEnvList(config.GroupAllowFrom); allow != "" {
envMap["QQ_GROUP_ALLOWED_USERS"] = allow
}
}
if err := writeHermesEnvMap(envPath, envMap, []string{
"QQ_APP_ID",
"QQ_CLIENT_SECRET",
"QQ_ALLOW_ALL_USERS",
"QQ_ALLOWED_USERS",
"QQ_GROUP_ALLOWED_USERS",
"QQ_HOME_CHANNEL",
"QQ_HOME_CHANNEL_NAME",
"QQ_STT_API_KEY",
"QQ_STT_BASE_URL",
"QQ_STT_MODEL",
}); err != nil {
return err
}
configPath := path.Join(confDir, "config.yaml")
cfg, err := readHermesConfigMap(configPath)
if err != nil {
return err
}
platform := ensureChildMap(ensureChildMap(cfg, "platforms"), "qq")
platform["enabled"] = config.Enabled && defaultBot.AppID != "" && defaultBot.ClientSecret != ""
extra := ensureChildMap(platform, "extra")
extra["markdown_support"] = true
extra["dm_policy"] = config.DmPolicy
extra["group_policy"] = config.GroupPolicy
delete(extra, "app_id")
delete(extra, "client_secret")
if config.DmPolicy == "allowlist" {
extra["allow_from"] = append([]string(nil), config.AllowFrom...)
} else {
delete(extra, "allow_from")
}
if config.GroupPolicy == "allowlist" {
extra["group_allow_from"] = append([]string(nil), config.GroupAllowFrom...)
} else {
delete(extra, "group_allow_from")
}
return writeHermesConfigMap(configPath, cfg)
}
func deleteHermesQQBotChannelConfig(confDir string) error {
if err := deleteHermesEnvKeys(confDir,
"QQ_APP_ID",
"QQ_CLIENT_SECRET",
"QQ_ALLOW_ALL_USERS",
"QQ_ALLOWED_USERS",
"QQ_GROUP_ALLOWED_USERS",
"QQ_HOME_CHANNEL",
"QQ_HOME_CHANNEL_NAME",
"QQ_STT_API_KEY",
"QQ_STT_BASE_URL",
"QQ_STT_MODEL",
); err != nil {
return err
}
return deleteHermesConfigSections(confDir, nil, []string{"qq"})
}
func readHermesWecomChannelConfig(confDir string) (*dto.AgentWecomConfig, error) {
envMap, err := readHermesEnvMap(path.Join(confDir, ".env"))
if err != nil {
return nil, err
}
cfg, err := readHermesConfigMap(path.Join(confDir, "config.yaml"))
if err != nil {
return nil, err
}
platform := childMap(childMap(cfg, "platforms"), "wecom")
allowFrom := splitHermesEnvList(envMap["WECOM_ALLOWED_USERS"])
groupAllowFrom := splitHermesEnvList(envMap["WECOM_GROUP_ALLOWED_USERS"])
dmPolicy := envMap["WECOM_DM_POLICY"]
groupPolicy := envMap["WECOM_GROUP_POLICY"]
botID := envMap["WECOM_BOT_ID"]
secret := envMap["WECOM_SECRET"]
return &dto.AgentWecomConfig{
Enabled: extractBoolValue(platform["enabled"], false) && botID != "" && secret != "",
DmPolicy: dmPolicy,
AllowFrom: allowFrom,
GroupPolicy: groupPolicy,
GroupAllowFrom: groupAllowFrom,
BotID: botID,
Secret: secret,
}, nil
}
func writeHermesWecomChannelConfig(confDir string, config dto.AgentWecomConfig) error {
envPath := path.Join(confDir, ".env")
envMap, err := readHermesEnvMap(envPath)
if err != nil {
return err
}
if config.BotID != "" {
envMap["WECOM_BOT_ID"] = config.BotID
} else {
delete(envMap, "WECOM_BOT_ID")
}
if config.Secret != "" {
envMap["WECOM_SECRET"] = config.Secret
} else {
delete(envMap, "WECOM_SECRET")
}
delete(envMap, "WECOM_ALLOWED_USERS")
delete(envMap, "WECOM_GROUP_ALLOWED_USERS")
delete(envMap, "WECOM_ALLOW_ALL_USERS")
delete(envMap, "WECOM_DM_POLICY")
delete(envMap, "WECOM_GROUP_POLICY")
if config.DmPolicy != "" {
envMap["WECOM_DM_POLICY"] = config.DmPolicy
}
if config.GroupPolicy != "" {
envMap["WECOM_GROUP_POLICY"] = config.GroupPolicy
}
if config.DmPolicy == "allowlist" {
if allow := joinHermesEnvList(config.AllowFrom); allow != "" {
envMap["WECOM_ALLOWED_USERS"] = allow
}
}
if config.GroupPolicy == "allowlist" {
if allow := joinHermesEnvList(config.GroupAllowFrom); allow != "" {
envMap["WECOM_GROUP_ALLOWED_USERS"] = allow
}
}
if err := writeHermesEnvMap(envPath, envMap, []string{
"WECOM_BOT_ID",
"WECOM_SECRET",
"WECOM_DM_POLICY",
"WECOM_ALLOWED_USERS",
"WECOM_GROUP_POLICY",
"WECOM_GROUP_ALLOWED_USERS",
}); err != nil {
return err
}
configPath := path.Join(confDir, "config.yaml")
cfg, err := readHermesConfigMap(configPath)
if err != nil {
return err
}
platform := ensureChildMap(ensureChildMap(cfg, "platforms"), "wecom")
platform["enabled"] = config.Enabled && config.BotID != "" && config.Secret != ""
extra := ensureChildMap(platform, "extra")
extra["dm_policy"] = config.DmPolicy
extra["group_policy"] = config.GroupPolicy
delete(extra, "bot_id")
delete(extra, "secret")
if config.DmPolicy == "allowlist" {
extra["allow_from"] = append([]string(nil), config.AllowFrom...)
} else {
delete(extra, "allow_from")
}
if config.GroupPolicy == "allowlist" {
extra["group_allow_from"] = append([]string(nil), config.GroupAllowFrom...)
} else {
delete(extra, "group_allow_from")
}
return writeHermesConfigMap(configPath, cfg)
}
func deleteHermesWecomChannelConfig(confDir string) error {
if err := deleteHermesEnvKeys(confDir,
"WECOM_BOT_ID",
"WECOM_SECRET",
"WECOM_ALLOW_ALL_USERS",
"WECOM_DM_POLICY",
"WECOM_ALLOWED_USERS",
"WECOM_GROUP_POLICY",
"WECOM_GROUP_ALLOWED_USERS",
"WECOM_HOME_CHANNEL",
); err != nil {
return err
}
return deleteHermesConfigSections(confDir, nil, []string{"wecom"})
}
func readHermesDingTalkChannelConfig(confDir string) (*dto.AgentDingTalkConfig, error) {
envMap, err := readHermesEnvMap(path.Join(confDir, ".env"))
if err != nil {
return nil, err
}
cfg, err := readHermesConfigMap(path.Join(confDir, "config.yaml"))
if err != nil {
return nil, err
}
platform := childMap(childMap(cfg, "platforms"), "dingtalk")
extra := childMap(platform, "extra")
clientID := envMap["DINGTALK_CLIENT_ID"]
clientSecret := envMap["DINGTALK_CLIENT_SECRET"]
allowFrom := splitHermesEnvList(envMap["DINGTALK_ALLOWED_USERS"])
dmPolicy := ""
if extractHermesEnvBool(envMap, "DINGTALK_ALLOW_ALL_USERS", false) {
dmPolicy = "open"
} else if len(allowFrom) > 0 {
dmPolicy = "allowlist"
} else if extractStringValue(extra["unauthorized_dm_behavior"]) == "ignore" {
dmPolicy = "disabled"
} else if clientID != "" || clientSecret != "" {
dmPolicy = "pairing"
}
result := &dto.AgentDingTalkConfig{
Enabled: extractBoolValue(platform["enabled"], clientID != "" && clientSecret != ""),
DmPolicy: dmPolicy,
AllowFrom: allowFrom,
GroupPolicy: "",
GroupAllowFrom: []string{},
}
if clientID != "" || clientSecret != "" {
result.Bots = []dto.AgentDingTalkBot{
{
AgentChannelBotBase: dto.AgentChannelBotBase{
AccountID: "default",
Name: "Default",
Enabled: true,
IsDefault: true,
},
ClientID: clientID,
ClientSecret: clientSecret,
},
}
}
return result, nil
}
func writeHermesDingTalkChannelConfig(confDir string, config dto.AgentDingTalkConfig) error {
envPath := path.Join(confDir, ".env")
envMap, err := readHermesEnvMap(envPath)
if err != nil {
return err
}
clientID, clientSecret := firstHermesDingTalkBotCredentials(config.Bots)
if config.Enabled && clientID != "" {
envMap["DINGTALK_CLIENT_ID"] = clientID
} else {
delete(envMap, "DINGTALK_CLIENT_ID")
}
if config.Enabled && clientSecret != "" {
envMap["DINGTALK_CLIENT_SECRET"] = clientSecret
} else {
delete(envMap, "DINGTALK_CLIENT_SECRET")
}
delete(envMap, "DINGTALK_ALLOWED_USERS")
delete(envMap, "DINGTALK_ALLOW_ALL_USERS")
if config.DmPolicy == "allowlist" {
if allow := joinHermesEnvList(config.AllowFrom); allow != "" {
envMap["DINGTALK_ALLOWED_USERS"] = allow
}
} else if config.DmPolicy == "open" {
envMap["DINGTALK_ALLOW_ALL_USERS"] = "true"
}
if err := writeHermesEnvMap(envPath, envMap, []string{
"DINGTALK_CLIENT_ID",
"DINGTALK_CLIENT_SECRET",
"DINGTALK_ALLOW_ALL_USERS",
"DINGTALK_ALLOWED_USERS",
}); err != nil {
return err
}
configPath := path.Join(confDir, "config.yaml")
cfg, err := readHermesConfigMap(configPath)
if err != nil {
return err
}
platform := ensureChildMap(ensureChildMap(cfg, "platforms"), "dingtalk")
platform["enabled"] = config.Enabled && clientID != "" && clientSecret != ""
extra := ensureChildMap(platform, "extra")
switch config.DmPolicy {
case "pairing":
extra["unauthorized_dm_behavior"] = "pair"
case "disabled":
extra["unauthorized_dm_behavior"] = "ignore"
default:
delete(extra, "unauthorized_dm_behavior")
}
return writeHermesConfigMap(configPath, cfg)
}
func deleteHermesDingTalkChannelConfig(confDir string) error {
if err := deleteHermesEnvKeys(confDir,
"DINGTALK_CLIENT_ID",
"DINGTALK_CLIENT_SECRET",
"DINGTALK_ALLOW_ALL_USERS",
"DINGTALK_ALLOWED_USERS",
"DINGTALK_HOME_CHANNEL",
); err != nil {
return err
}
return deleteHermesConfigSections(confDir, nil, []string{"dingtalk"})
}
func readHermesFeishuChannelConfig(confDir string) (*dto.AgentFeishuConfig, error) {
envMap, err := readHermesEnvMap(path.Join(confDir, ".env"))
if err != nil {
return nil, err
}
cfg, err := readHermesConfigMap(path.Join(confDir, "config.yaml"))
if err != nil {
return nil, err
}
platform := childMap(childMap(cfg, "platforms"), "feishu")
appID := envMap["FEISHU_APP_ID"]
appSecret := envMap["FEISHU_APP_SECRET"]
allowFrom := splitHermesEnvList(envMap["FEISHU_ALLOWED_USERS"])
dmPolicy := ""
if extractHermesEnvBool(envMap, "FEISHU_ALLOW_ALL_USERS", false) {
dmPolicy = "open"
} else if len(allowFrom) > 0 {
dmPolicy = "allowlist"
} else if appID != "" || appSecret != "" {
dmPolicy = "pairing"
}
groupPolicy := envMap["FEISHU_GROUP_POLICY"]
result := &dto.AgentFeishuConfig{
Enabled: extractBoolValue(platform["enabled"], appID != "" && appSecret != ""),
ThreadSession: false,
ReplyMode: "",
Streaming: false,
RequireMention: "",
GroupPolicy: groupPolicy,
GroupAllowFrom: []string{},
Domain: envMap["FEISHU_DOMAIN"],
ConnectionMode: envMap["FEISHU_CONNECTION_MODE"],
}
if appID != "" || appSecret != "" {
result.Bots = []dto.AgentFeishuBot{
{
AgentChannelBotBase: dto.AgentChannelBotBase{
AccountID: "default",
Name: "Default",
Enabled: true,
IsDefault: true,
},
AppID: appID,
AppSecret: appSecret,
DmPolicy: dmPolicy,
AllowFrom: allowFrom,
},
}
}
return result, nil
}
func writeHermesFeishuChannelConfig(confDir string, config dto.AgentFeishuConfig) error {
envPath := path.Join(confDir, ".env")
envMap, err := readHermesEnvMap(envPath)
if err != nil {
return err
}
bot := firstHermesFeishuBot(config.Bots)
if config.GroupPolicy == "allowlist" && bot.DmPolicy == "pairing" {
return buserr.New("ErrHermesFeishuGroupAllowlistRequiresAllowlist")
}
if bot.AppID != "" {
envMap["FEISHU_APP_ID"] = bot.AppID
} else {
delete(envMap, "FEISHU_APP_ID")
}
if bot.AppSecret != "" {
envMap["FEISHU_APP_SECRET"] = bot.AppSecret
} else {
delete(envMap, "FEISHU_APP_SECRET")
}
envMap["FEISHU_DOMAIN"] = "feishu"
envMap["FEISHU_CONNECTION_MODE"] = "websocket"
envMap["FEISHU_GROUP_POLICY"] = config.GroupPolicy
delete(envMap, "FEISHU_ALLOW_ALL_USERS")
delete(envMap, "FEISHU_ALLOWED_USERS")
switch bot.DmPolicy {
case "open":
envMap["FEISHU_ALLOW_ALL_USERS"] = "true"
case "allowlist":
if allow := joinHermesEnvList(bot.AllowFrom); allow != "" {
envMap["FEISHU_ALLOWED_USERS"] = allow
}
}
if config.GroupPolicy == "allowlist" {
if allow := joinHermesEnvList(bot.AllowFrom); allow != "" {
envMap["FEISHU_ALLOWED_USERS"] = allow
}
}
if err := writeHermesEnvMap(envPath, envMap, []string{
"FEISHU_APP_ID",
"FEISHU_APP_SECRET",
"FEISHU_DOMAIN",
"FEISHU_CONNECTION_MODE",
"FEISHU_ALLOW_ALL_USERS",
"FEISHU_ALLOWED_USERS",
"FEISHU_GROUP_POLICY",
}); err != nil {
return err
}
configPath := path.Join(confDir, "config.yaml")
cfg, err := readHermesConfigMap(configPath)
if err != nil {
return err
}
platform := ensureChildMap(ensureChildMap(cfg, "platforms"), "feishu")
platform["enabled"] = config.Enabled && bot.AppID != "" && bot.AppSecret != ""
return writeHermesConfigMap(configPath, cfg)
}
func deleteHermesFeishuChannelConfig(confDir string) error {
if err := deleteHermesEnvKeys(confDir,
"FEISHU_APP_ID",
"FEISHU_APP_SECRET",
"FEISHU_DOMAIN",
"FEISHU_CONNECTION_MODE",
"FEISHU_ALLOW_ALL_USERS",
"FEISHU_ALLOWED_USERS",
"FEISHU_GROUP_POLICY",
"FEISHU_HOME_CHANNEL",
"FEISHU_VERIFICATION_TOKEN",
"FEISHU_ENCRYPT_KEY",
); err != nil {
return err
}
return deleteHermesConfigSections(confDir, nil, []string{"feishu"})
}
func readHermesWeixinChannelConfig(confDir string) (*dto.AgentWeixinConfig, error) {
envMap, err := readHermesEnvMap(path.Join(confDir, ".env"))
if err != nil {
return nil, err
}
return &dto.AgentWeixinConfig{
Enabled: envMap["WEIXIN_ACCOUNT_ID"] != "" || envMap["WEIXIN_TOKEN"] != "",
}, nil
}
func deleteHermesWeixinChannelConfig(confDir string) error {
if err := deleteHermesEnvKeys(confDir,
"WEIXIN_ACCOUNT_ID",
"WEIXIN_TOKEN",
"WEIXIN_BASE_URL",
"WEIXIN_CDN_BASE_URL",
"WEIXIN_DM_POLICY",
"WEIXIN_ALLOW_ALL_USERS",
"WEIXIN_ALLOWED_USERS",
"WEIXIN_GROUP_POLICY",
"WEIXIN_GROUP_ALLOWED_USERS",
"WEIXIN_HOME_CHANNEL",
); err != nil {
return err
}
return deleteHermesConfigSections(confDir, nil, []string{"weixin"})
}
func firstHermesDingTalkBotCredentials(bots []dto.AgentDingTalkBot) (string, string) {
for _, bot := range bots {
if bot.IsDefault || bot.AccountID == "default" {
return bot.ClientID, bot.ClientSecret
}
}
if len(bots) == 0 {
return "", ""
}
return bots[0].ClientID, bots[0].ClientSecret
}
func firstHermesFeishuBot(bots []dto.AgentFeishuBot) dto.AgentFeishuBot {
for _, bot := range bots {
if bot.IsDefault || bot.AccountID == "default" {
return bot
}
}
if len(bots) == 0 {
return dto.AgentFeishuBot{}
}
return bots[0]
}
func buildHermesWeixinLoginArgs(containerName string) []string {
return buildHermesDockerExecCommandArgs(
containerName,
"/opt/hermes/.venv/bin/python",
"-u",
"-c",
hermesWeixinLoginScript,
)
}
+126
View File
@@ -0,0 +1,126 @@
package service
import (
"database/sql"
"fmt"
"math"
"path/filepath"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/constant"
"github.com/1Panel-dev/1Panel/agent/utils/cmd"
"github.com/1Panel-dev/1Panel/agent/utils/files"
)
const hermesSessionListLimit = 100
func (a AgentService) GetHermesChatSessions(req dto.AgentIDReq) ([]dto.AgentHermesChatSessionItem, error) {
agent, install, err := a.loadAgentAndInstall(req.AgentID)
if err != nil {
return nil, err
}
if agent.AgentType != constant.AppHermesAgent {
return nil, fmt.Errorf("%s does not support", agent.AgentType)
}
return listHermesChatSessionsFromStateDB(filepath.Join(install.GetPath(), "data", "state.db"))
}
func (a AgentService) RenameHermesChatSession(req dto.AgentHermesChatSessionRenameReq) error {
agent, install, err := a.loadAgentAndInstall(req.AgentID)
if err != nil {
return err
}
if agent.AgentType != constant.AppHermesAgent {
return fmt.Errorf("%s does not support", agent.AgentType)
}
_, err = cmd.NewCommandMgr(cmd.WithTimeout(20*time.Second)).RunWithStdout(
"docker",
buildHermesDockerExecArgs(install.ContainerName, "sessions", "rename", req.ID, req.Title)...,
)
return err
}
func (a AgentService) DeleteHermesChatSession(req dto.AgentHermesChatSessionDeleteReq) error {
agent, install, err := a.loadAgentAndInstall(req.AgentID)
if err != nil {
return err
}
if agent.AgentType != constant.AppHermesAgent {
return fmt.Errorf("%s does not support", agent.AgentType)
}
_, err = cmd.NewCommandMgr(cmd.WithTimeout(20*time.Second)).RunWithStdout(
"docker",
buildHermesDockerExecArgs(install.ContainerName, "sessions", "delete", req.ID, "--yes")...,
)
return err
}
func listHermesChatSessionsFromStateDB(stateDBPath string) ([]dto.AgentHermesChatSessionItem, error) {
if !files.NewFileOp().Stat(stateDBPath) {
return []dto.AgentHermesChatSessionItem{}, nil
}
db, err := sql.Open("sqlite", stateDBPath)
if err != nil {
return nil, err
}
defer db.Close()
rows, err := db.Query(`
SELECT
s.id,
COALESCE(NULLIF(TRIM(s.title), ''), s.id) AS title,
COALESCE(s.model, '') AS model,
COALESCE(s.message_count, 0) AS message_count,
s.started_at,
COALESCE(MAX(m.timestamp), s.started_at) AS last_active
FROM sessions s
LEFT JOIN messages m ON m.session_id = s.id
WHERE s.source = 'cli'
GROUP BY s.id, title, model, s.message_count, s.started_at
ORDER BY last_active DESC
LIMIT ?
`, hermesSessionListLimit)
if err != nil {
return nil, err
}
defer rows.Close()
items := make([]dto.AgentHermesChatSessionItem, 0, 8)
for rows.Next() {
var item dto.AgentHermesChatSessionItem
var title sql.NullString
var model sql.NullString
var startedAt sql.NullFloat64
var lastActive sql.NullFloat64
if err := rows.Scan(&item.ID, &title, &model, &item.MessageCount, &startedAt, &lastActive); err != nil {
return nil, err
}
item.Title = strings.TrimSpace(title.String)
if item.Title == "" {
item.Title = item.ID
}
item.Model = strings.TrimSpace(model.String)
item.StartedAt = formatHermesSessionTimestamp(startedAt)
item.LastActive = formatHermesSessionTimestamp(lastActive)
items = append(items, item)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
func formatHermesSessionTimestamp(value sql.NullFloat64) string {
if !value.Valid || value.Float64 <= 0 {
return ""
}
seconds, fraction := math.Modf(value.Float64)
return time.Unix(int64(seconds), int64(fraction*float64(time.Second))).UTC().Format(time.RFC3339)
}
+215
View File
@@ -0,0 +1,215 @@
package service
import (
"encoding/json"
"fmt"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/utils/cmd"
)
type hermesSkillsListEntry struct {
Name string `json:"name"`
Description string `json:"description"`
Category string `json:"category"`
}
type hermesSkillsListPayload struct {
Skills []hermesSkillsListEntry `json:"skills"`
}
func listHermesSkills(containerName string) ([]dto.AgentSkillItem, error) {
output, err := runHermesSkillsCommandWithStdout(2*time.Minute, containerName, "list", "--source", "all")
if err != nil {
return nil, err
}
items, err := parseHermesSkillsListOutput(output)
if err != nil {
return nil, err
}
metadata, err := readHermesSkillsListMetadata(containerName)
if err != nil {
return nil, err
}
return mergeHermesSkillsWithMetadata(items, metadata), nil
}
func searchHermesSkills(containerName, source, keyword string) ([]dto.AgentSkillSearchItem, error) {
if source != "official" && source != "skills-sh" {
return nil, fmt.Errorf("unsupported hermes skill source: %s", source)
}
output, err := runHermesSkillsCommandWithStdout(
2*time.Minute,
containerName,
"search",
keyword,
"--source",
source,
"--limit",
"20",
)
if err != nil {
return nil, err
}
return parseHermesSkillSearchOutput(output)
}
func runHermesSkillsCommandWithStdout(timeout time.Duration, containerName string, hermesArgs ...string) (string, error) {
args := []string{"exec", "-e", "COLUMNS=240", "-u", "hermes", containerName, "hermes", "skills"}
args = append(args, hermesArgs...)
return cmd.NewCommandMgr(cmd.WithTimeout(timeout)).RunWithStdout("docker", args...)
}
func readHermesSkillsListMetadata(containerName string) (map[string]hermesSkillsListEntry, error) {
output, err := cmd.NewCommandMgr(cmd.WithTimeout(2*time.Minute)).RunWithStdout(
"docker",
"exec",
"-u",
"hermes",
containerName,
"python",
"-c",
"import sys; sys.path.insert(0, '/opt/hermes'); from tools.skills_tool import skills_list; print(skills_list())",
)
if err != nil {
return nil, err
}
return parseHermesSkillsListMetadataOutput(output)
}
func parseHermesSkillsListOutput(output string) ([]dto.AgentSkillItem, error) {
headers, rows, err := parseHermesTableOutput(output)
if err != nil {
return nil, err
}
items := make([]dto.AgentSkillItem, 0, len(rows))
for _, row := range rows {
item := dto.AgentSkillItem{
Name: row[headers["Name"]],
Category: row[headers["Category"]],
Source: row[headers["Source"]],
Trust: row[headers["Trust"]],
Uninstallable: row[headers["Source"]] != "builtin" && row[headers["Source"]] != "local",
}
items = append(items, item)
}
return items, nil
}
func parseHermesSkillsListMetadataOutput(output string) (map[string]hermesSkillsListEntry, error) {
if strings.TrimSpace(output) == "" {
return map[string]hermesSkillsListEntry{}, nil
}
var payload hermesSkillsListPayload
if err := json.Unmarshal([]byte(output), &payload); err != nil {
return nil, err
}
metadata := make(map[string]hermesSkillsListEntry, len(payload.Skills))
for _, skill := range payload.Skills {
if skill.Name == "" {
continue
}
metadata[skill.Name] = skill
}
return metadata, nil
}
func mergeHermesSkillsWithMetadata(items []dto.AgentSkillItem, metadata map[string]hermesSkillsListEntry) []dto.AgentSkillItem {
for i := range items {
entry, ok := metadata[items[i].Name]
if !ok {
continue
}
items[i].Description = entry.Description
if items[i].Category == "" && entry.Category != "" {
items[i].Category = entry.Category
}
}
return items
}
func parseHermesSkillSearchOutput(output string) ([]dto.AgentSkillSearchItem, error) {
if strings.Contains(output, "No skills found matching your query.") {
return []dto.AgentSkillSearchItem{}, nil
}
headers, rows, err := parseHermesTableOutput(output)
if err != nil {
return nil, err
}
items := make([]dto.AgentSkillSearchItem, 0, len(rows))
for _, row := range rows {
identifier := row[headers["Identifier"]]
items = append(items, dto.AgentSkillSearchItem{
Slug: identifier,
Identifier: identifier,
Name: row[headers["Name"]],
Description: row[headers["Description"]],
Source: row[headers["Source"]],
Trust: row[headers["Trust"]],
})
}
return items, nil
}
func parseHermesTableOutput(output string) (map[string]int, [][]string, error) {
lines := strings.Split(strings.TrimSpace(ansiEscapePattern.ReplaceAllString(output, "")), "\n")
var headers []string
rows := make([][]string, 0)
var current []string
for _, rawLine := range lines {
line := strings.TrimSpace(rawLine)
if (!strings.HasPrefix(line, "│") || !strings.HasSuffix(line, "│")) &&
(!strings.HasPrefix(line, "┃") || !strings.HasSuffix(line, "┃")) {
continue
}
line = strings.ReplaceAll(line, "┃", "│")
parts := strings.Split(line, "│")
if len(parts) < 3 {
continue
}
cols := make([]string, 0, len(parts)-2)
for _, part := range parts[1 : len(parts)-1] {
cols = append(cols, strings.TrimSpace(part))
}
if len(headers) == 0 {
headers = cols
continue
}
if cols[0] != "" {
if current != nil {
rows = append(rows, current)
}
current = cols
continue
}
if current == nil {
continue
}
for i := range cols {
if cols[i] == "" {
continue
}
if current[i] == "" {
current[i] = cols[i]
continue
}
current[i] = current[i] + " " + cols[i]
}
}
if current != nil {
rows = append(rows, current)
}
if len(headers) == 0 {
return nil, nil, fmt.Errorf("hermes skills table not found")
}
headerIndex := make(map[string]int, len(headers))
for i, header := range headers {
headerIndex[header] = i
}
return headerIndex, rows, nil
}
+239
View File
@@ -0,0 +1,239 @@
package service
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/constant"
)
const (
openclawCronJobsRelativePath = "data/conf/cron/jobs.json"
openclawSessionStoreRelativeGlob = "data/conf/agents/*/sessions/sessions.json"
openclawSessionFilesRelativeGlob = "data/conf/agents/*/sessions/*.jsonl"
)
func (a AgentService) GetOverview(req dto.AgentOverviewReq) (*dto.AgentOverview, error) {
agent, install, conf, err := a.loadOpenclawAgentConfig(req.AgentID)
if err != nil {
return nil, err
}
overview := &dto.AgentOverview{
Snapshot: dto.AgentOverviewSnapshot{
ContainerStatus: install.Status,
AppVersion: install.Version,
DefaultModel: extractOpenclawDefaultModel(conf),
ChannelCount: countOpenclawConfiguredChannels(conf),
},
}
if overview.Snapshot.DefaultModel == "" {
overview.Snapshot.DefaultModel = agent.Model
}
if install.Status != constant.StatusRunning {
return overview, nil
}
var wg sync.WaitGroup
wg.Add(3)
go func() {
defer wg.Done()
skillCount, err := loadOpenclawOverviewSkillStats(install.ContainerName)
if err == nil {
overview.Snapshot.SkillCount = skillCount
}
}()
go func() {
defer wg.Done()
sessionCount, err := loadOpenclawOverviewSessionCount(install.GetPath())
if err == nil {
overview.Snapshot.SessionCount = sessionCount
}
}()
go func() {
defer wg.Done()
jobCount, err := loadOpenclawOverviewJobCount(install.GetPath())
if err == nil {
overview.Snapshot.JobCount = jobCount
}
}()
wg.Wait()
return overview, nil
}
func extractOpenclawDefaultModel(conf map[string]interface{}) string {
agents, ok := conf["agents"].(map[string]interface{})
if !ok {
return ""
}
defaults, ok := agents["defaults"].(map[string]interface{})
if !ok {
return ""
}
model, ok := defaults["model"].(map[string]interface{})
if !ok {
return ""
}
primary, _ := model["primary"].(string)
return strings.TrimSpace(primary)
}
func countOpenclawConfiguredChannels(conf map[string]interface{}) int {
channels, ok := conf["channels"].(map[string]interface{})
if !ok {
return 0
}
count := 0
for _, value := range channels {
channel, ok := value.(map[string]interface{})
if !ok || len(channel) == 0 {
continue
}
count++
}
return count
}
func loadOpenclawOverviewSkillStats(containerName string) (int, error) {
output, err := runDockerExecWithStdout(5*time.Minute, containerName, "sh", "-c", "openclaw skills list --json 2>&1")
if err != nil {
return 0, err
}
if strings.TrimSpace(output) == "" {
return 0, nil
}
skills, err := parseOpenclawSkillsList(output)
if err != nil {
return 0, err
}
return len(skills), nil
}
func loadOpenclawOverviewSessionCount(installPath string) (int, error) {
sessionStorePaths, err := filepath.Glob(filepath.Join(installPath, openclawSessionStoreRelativeGlob))
if err != nil {
return 0, err
}
total := 0
for _, sessionStorePath := range sessionStorePaths {
count, err := loadOpenclawSessionCountFromFile(sessionStorePath)
if err != nil {
continue
}
total += count
}
if total > 0 {
return total, nil
}
sessionFiles, err := filepath.Glob(filepath.Join(installPath, openclawSessionFilesRelativeGlob))
if err != nil {
return 0, err
}
return len(sessionFiles), nil
}
func loadOpenclawSessionCountFromFile(sessionStorePath string) (int, error) {
content, err := os.ReadFile(sessionStorePath)
if err != nil {
return 0, err
}
return parseOpenclawSessionCount(string(content))
}
func loadOpenclawOverviewJobCount(installPath string) (int, error) {
content, err := os.ReadFile(filepath.Join(installPath, openclawCronJobsRelativePath))
if err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
return parseOpenclawCronCount(string(content))
}
func parseOpenclawSessionCount(output string) (int, error) {
if strings.TrimSpace(output) == "" {
return 0, nil
}
var payload interface{}
if err := json.Unmarshal([]byte(strings.TrimSpace(output)), &payload); err != nil {
return 0, err
}
return countOpenclawSessions(payload), nil
}
func parseOpenclawCronCount(output string) (int, error) {
if strings.TrimSpace(output) == "" {
return 0, nil
}
var payload interface{}
if err := json.Unmarshal([]byte(strings.TrimSpace(output)), &payload); err != nil {
return 0, err
}
switch value := payload.(type) {
case []interface{}:
return len(value), nil
case map[string]interface{}:
if total, ok := value["total"].(float64); ok {
return int(total), nil
}
if jobs, ok := value["jobs"].([]interface{}); ok {
return len(jobs), nil
}
return len(value), nil
default:
return 0, nil
}
}
func countOpenclawSessions(payload interface{}) int {
switch value := payload.(type) {
case []interface{}:
return len(value)
case map[string]interface{}:
if count, ok := value["count"].(float64); ok {
return int(count)
}
if sessions, ok := value["sessions"]; ok {
return countOpenclawSessions(sessions)
}
count := 0
for _, item := range value {
switch typed := item.(type) {
case map[string]interface{}:
if looksLikeOpenclawSession(typed) {
count++
}
case string:
if strings.HasSuffix(strings.TrimSpace(typed), ".jsonl") {
count++
}
}
}
return count
default:
return 0
}
}
func looksLikeOpenclawSession(item map[string]interface{}) bool {
if _, ok := item["sessionFile"]; ok {
return true
}
if _, ok := item["agentId"]; ok {
return true
}
if _, ok := item["lastActivityTs"]; ok {
return true
}
if _, ok := item["updatedAt"]; ok {
return true
}
return false
}
+338
View File
@@ -0,0 +1,338 @@
package service
import (
"encoding/json"
"fmt"
"regexp"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/task"
"github.com/1Panel-dev/1Panel/agent/constant"
"github.com/1Panel-dev/1Panel/agent/global"
"github.com/1Panel-dev/1Panel/agent/utils/cmd"
)
const clawhubGlobalRegistry = "https://clawhub.com"
const clawhubChinaRegistry = "https://mirror-cn.clawhub.com"
type openclawSkillsList struct {
Skills []openclawSkillListItem `json:"skills"`
}
type openclawSkillListItem struct {
Name string `json:"name"`
Description string `json:"description"`
Source string `json:"source"`
Bundled bool `json:"bundled"`
Disabled bool `json:"disabled"`
}
type openclawSkillInfo struct {
SkillKey string `json:"skillKey"`
}
type skillhubSearchPayload struct {
Skills []dto.AgentSkillSearchItem `json:"skills"`
Results []dto.AgentSkillSearchItem `json:"results"`
}
var clawhubSearchLinePattern = regexp.MustCompile(`^(\S+)\s+(.+?)\s+\(([\d.]+)\)$`)
var ansiEscapePattern = regexp.MustCompile(`\x1b\[[0-9;?]*[ -/]*[@-~]`)
func (a AgentService) ListSkills(req dto.AgentIDReq) ([]dto.AgentSkillItem, error) {
agent, install, err := a.loadAgentAndInstall(req.AgentID)
if err != nil {
return nil, err
}
if err := ensureContainerRunning(install.ContainerName); err != nil {
return nil, err
}
if agent.AgentType == constant.AppHermesAgent {
return listHermesSkills(install.ContainerName)
}
if agent.AgentType != constant.AppOpenclaw {
return nil, fmt.Errorf("%s does not support", agent.AgentType)
}
output, err := runDockerExecWithStdout(5*time.Minute, install.ContainerName, "sh", "-c", "openclaw skills list --json 2>&1")
if err != nil {
return nil, err
}
if len(output) == 0 {
return nil, nil
}
return parseOpenclawSkillsList(output)
}
func (a AgentService) SearchSkills(req dto.AgentSkillSearchReq) ([]dto.AgentSkillSearchItem, error) {
agent, install, err := a.loadAgentAndInstall(req.AgentID)
if err != nil {
return nil, err
}
if err := ensureContainerRunning(install.ContainerName); err != nil {
return nil, err
}
if agent.AgentType == constant.AppHermesAgent {
return searchHermesSkills(install.ContainerName, req.Source, req.Keyword)
}
if agent.AgentType != constant.AppOpenclaw {
return nil, fmt.Errorf("%s does not support", agent.AgentType)
}
output, err := loadOpenclawSkillSearchOutput(install.ContainerName, req.Source, req.Keyword)
if err != nil {
return nil, err
}
if len(output) == 0 {
return nil, nil
}
switch req.Source {
case "skillhub":
return parseSkillhubSearchResult(output)
default:
return parseClawhubSearchResult(output, req.Source), nil
}
}
func (a AgentService) UpdateSkill(req dto.AgentSkillUpdateReq) error {
agent, install, err := a.loadOpenclawAgentAndInstall(req.AgentID)
if err != nil {
return err
}
if err := ensureContainerRunning(install.ContainerName); err != nil {
return err
}
conf, err := readOpenclawConfig(agent.ConfigPath)
if err != nil {
return err
}
skillKey, err := getOpenclawSkillKey(install.ContainerName, req.Name)
if err != nil {
return err
}
setOpenclawSkillEnabled(conf, skillKey, req.Enabled)
return writeOpenclawConfigRaw(agent.ConfigPath, conf)
}
func (a AgentService) InstallSkill(req dto.AgentSkillInstallReq) error {
agent, install, err := a.loadAgentAndInstall(req.AgentID)
if err != nil {
return err
}
if err := ensureContainerRunning(install.ContainerName); err != nil {
return err
}
installTask, err := task.NewTaskWithOps(req.Slug, task.TaskInstall, task.TaskScopeAI, req.TaskID, req.AgentID)
if err != nil {
return err
}
if agent.AgentType == constant.AppHermesAgent {
installTask.AddSubTask("Install Hermes skill", func(t *task.Task) error {
mgr := cmd.NewCommandMgr(cmd.WithTask(*t), cmd.WithContext(t.TaskCtx), cmd.WithTimeout(20*time.Minute))
return mgr.Run("docker", buildHermesDockerExecArgs(install.ContainerName, "skills", "install", req.Slug, "--yes")...)
}, nil)
go func() {
if err := installTask.Execute(); err != nil {
global.LOG.Errorf("install hermes skill failed: %v", err)
}
}()
return nil
}
if agent.AgentType != constant.AppOpenclaw {
return fmt.Errorf("%s does not support", agent.AgentType)
}
installTask.AddSubTask("Install OpenClaw skill", func(t *task.Task) error {
mgr := cmd.NewCommandMgr(cmd.WithTask(*t), cmd.WithContext(t.TaskCtx), cmd.WithTimeout(20*time.Minute))
return mgr.Run("docker", "exec", install.ContainerName, "sh", "-c", buildOpenclawSkillInstallCommand(req.Source, req.Slug))
}, nil)
go func() {
if err := installTask.Execute(); err != nil {
global.LOG.Errorf("install openclaw skill failed: %v", err)
}
}()
return nil
}
func (a AgentService) UninstallSkill(req dto.AgentSkillUninstallReq) error {
agent, install, err := a.loadAgentAndInstall(req.AgentID)
if err != nil {
return err
}
if agent.AgentType != constant.AppHermesAgent {
return fmt.Errorf("%s does not support", agent.AgentType)
}
if err := ensureContainerRunning(install.ContainerName); err != nil {
return err
}
return cmd.NewCommandMgr(cmd.WithTimeout(5*time.Minute)).Run(
"docker",
buildHermesDockerExecCommandArgs(
install.ContainerName,
"sh",
"-lc",
fmt.Sprintf(`printf 'y\n' | %s skills uninstall "$1"`, hermesExecutablePath),
"sh",
req.Name,
)...,
)
}
func parseOpenclawSkillsList(output string) ([]dto.AgentSkillItem, error) {
payloadBytes, err := extractEmbeddedJSON(output)
if err != nil {
return nil, err
}
if len(payloadBytes) == 0 {
return nil, nil
}
var payload openclawSkillsList
if err := json.Unmarshal(payloadBytes, &payload); err != nil {
return nil, err
}
items := make([]dto.AgentSkillItem, 0, len(payload.Skills))
for _, item := range payload.Skills {
items = append(items, dto.AgentSkillItem{
Name: item.Name,
Description: item.Description,
Source: item.Source,
Bundled: item.Bundled,
Disabled: item.Disabled,
})
}
return items, nil
}
func loadOpenclawSkillSearchOutput(containerName, source, keyword string) (string, error) {
switch source {
case "skillhub":
return runDockerExecWithStdout(2*time.Minute, containerName, "skillhub", "search", keyword, "--json")
default:
return runDockerExecWithStdout(
2*time.Minute,
containerName,
"sh",
"-c",
fmt.Sprintf("CLAWHUB_REGISTRY=%q clawhub search %q", resolveClawhubRegistry(source), keyword),
)
}
}
func parseSkillhubSearchResult(output string) ([]dto.AgentSkillSearchItem, error) {
trimmed := strings.TrimSpace(output)
if trimmed == "" {
return nil, nil
}
var list []dto.AgentSkillSearchItem
if err := json.Unmarshal([]byte(trimmed), &list); err == nil {
for i := range list {
list[i].Source = "skillhub"
}
return list, nil
}
var payload skillhubSearchPayload
if err := json.Unmarshal([]byte(trimmed), &payload); err != nil {
return nil, err
}
items := payload.Skills
if len(items) == 0 {
items = payload.Results
}
for i := range items {
items[i].Source = "skillhub"
}
return items, nil
}
func parseClawhubSearchResult(output, source string) []dto.AgentSkillSearchItem {
lines := strings.Split(strings.TrimSpace(output), "\n")
items := make([]dto.AgentSkillSearchItem, 0, len(lines))
for _, line := range lines {
matches := clawhubSearchLinePattern.FindStringSubmatch(strings.TrimSpace(line))
if len(matches) != 4 {
continue
}
items = append(items, dto.AgentSkillSearchItem{
Slug: matches[1],
Name: matches[2],
Score: matches[3],
Source: source,
})
}
return items
}
func buildOpenclawSkillInstallCommand(source, slug string) string {
switch source {
case "clawhub-global", "clawhub-cn":
return fmt.Sprintf(
"mkdir -p %s && CLAWHUB_REGISTRY=%q clawhub --workdir /home/node/.openclaw --dir skills install %q",
openclawManagedSkillsDir,
resolveClawhubRegistry(source),
slug,
)
default:
return fmt.Sprintf(
"mkdir -p %s && skillhub --dir %s install %q",
openclawManagedSkillsDir,
openclawManagedSkillsDir,
slug,
)
}
}
func resolveClawhubRegistry(source string) string {
switch source {
case "clawhub-cn":
return clawhubChinaRegistry
default:
return clawhubGlobalRegistry
}
}
func getOpenclawSkillKey(containerName, name string) (string, error) {
output, err := runDockerExecWithStdout(2*time.Minute, containerName, "sh", "-c", fmt.Sprintf("openclaw skills info %q --json 2>&1", name))
if err != nil {
return "", err
}
return parseOpenclawSkillKey(name, output)
}
func parseOpenclawSkillKey(name, output string) (string, error) {
payloadBytes, err := extractEmbeddedJSON(output)
if err != nil {
return "", err
}
var payload openclawSkillInfo
if err := json.Unmarshal(payloadBytes, &payload); err != nil {
return "", err
}
if payload.SkillKey == "" {
return "", fmt.Errorf("skill %s does not have a skillKey", name)
}
return payload.SkillKey, nil
}
func extractEmbeddedJSON(output string) ([]byte, error) {
trimmed := strings.TrimSpace(ansiEscapePattern.ReplaceAllString(output, ""))
if trimmed == "" {
return nil, nil
}
for i := 0; i < len(trimmed); i++ {
if trimmed[i] != '{' && trimmed[i] != '[' {
continue
}
decoder := json.NewDecoder(strings.NewReader(trimmed[i:]))
var raw json.RawMessage
if err := decoder.Decode(&raw); err == nil {
return raw, nil
}
}
return nil, fmt.Errorf("json payload not found")
}
func setOpenclawSkillEnabled(conf map[string]interface{}, skillKey string, enabled bool) {
skills := ensureChildMap(conf, "skills")
entries := ensureChildMap(skills, "entries")
entry := ensureChildMap(entries, skillKey)
entry["enabled"] = enabled
}
File diff suppressed because it is too large Load Diff
+338
View File
@@ -0,0 +1,338 @@
package service
import (
"context"
"errors"
"fmt"
"net"
"sort"
"strings"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/model"
"github.com/1Panel-dev/1Panel/agent/app/repo"
"github.com/1Panel-dev/1Panel/agent/buserr"
"github.com/1Panel-dev/1Panel/agent/constant"
"gorm.io/gorm"
)
func (a AgentService) BindWebsite(req dto.AgentWebsiteBindReq) error {
agent, err := agentRepo.GetFirst(repo.WithByID(req.AgentID))
if err != nil {
return err
}
if agent.WebsiteID != 0 {
return buserr.New("ErrAgentWebsiteBound")
}
website, err := websiteRepo.GetFirst(repo.WithByID(req.WebsiteID))
if err != nil {
return err
}
if !isBindableAgentWebsiteType(website.Type) {
return buserr.New("ErrAgentWebsiteTypeUnsupported")
}
boundAgent, err := agentRepo.GetFirst(repo.WithByWebsiteID(req.WebsiteID))
if err == nil && boundAgent.ID > 0 {
return buserr.New("ErrAgentWebsiteInUse")
}
if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
return err
}
agent.WebsiteID = req.WebsiteID
if err := agentRepo.Save(agent); err != nil {
return err
}
return ensureOpenclawWebsiteAllowedOrigin(agent, &website)
}
func (a AgentService) UnbindWebsite(req dto.AgentIDReq) error {
agent, err := agentRepo.GetFirst(repo.WithByID(req.AgentID))
if err != nil {
return err
}
if agent.WebsiteID == 0 {
return nil
}
website, err := websiteRepo.GetFirst(repo.WithByID(agent.WebsiteID))
if err != nil {
return err
}
if website.Type == constant.Deployment {
return buserr.New("ErrAgentWebsiteUnbindUnsupported")
}
agent.WebsiteID = 0
return agentRepo.Save(agent)
}
func hydrateAgentWebsiteItems(items []dto.AgentItem) error {
explicitWebsiteMap, err := loadAgentWebsiteMapByID(items)
if err != nil {
return err
}
websiteDomainMap, err := loadAgentWebsiteDomainMapByWebsiteID(items)
if err != nil {
return err
}
fillAgentWebsiteItems(items, explicitWebsiteMap, websiteDomainMap)
return nil
}
func loadAgentWebsiteMapByID(items []dto.AgentItem) (map[uint]model.Website, error) {
websiteIDs := make([]uint, 0, len(items))
for _, item := range items {
if item.WebsiteID > 0 {
websiteIDs = append(websiteIDs, item.WebsiteID)
}
}
if len(websiteIDs) == 0 {
return map[uint]model.Website{}, nil
}
websites, err := websiteRepo.GetBy(repo.WithByIDs(uniqueUintList(websiteIDs)))
if err != nil {
return nil, err
}
websiteMap := make(map[uint]model.Website, len(websites))
for _, website := range websites {
websiteMap[website.ID] = website
}
return websiteMap, nil
}
func loadAgentWebsiteDomainMapByWebsiteID(items []dto.AgentItem) (map[uint][]model.WebsiteDomain, error) {
websiteIDs := make([]uint, 0, len(items))
for _, item := range items {
if item.WebsiteID > 0 {
websiteIDs = append(websiteIDs, item.WebsiteID)
}
}
if len(websiteIDs) == 0 {
return map[uint][]model.WebsiteDomain{}, nil
}
websiteDomains, err := websiteDomainRepo.GetBy(websiteDomainRepo.WithWebsiteIds(uniqueUintList(websiteIDs)), repo.WithOrderAsc("id"))
if err != nil {
return nil, err
}
websiteDomainMap := make(map[uint][]model.WebsiteDomain, len(websiteDomains))
for _, websiteDomain := range websiteDomains {
websiteDomainMap[websiteDomain.WebsiteID] = append(websiteDomainMap[websiteDomain.WebsiteID], websiteDomain)
}
return websiteDomainMap, nil
}
func loadAgentWebsiteResourceName(website model.Website) (string, error) {
websiteDomains, err := websiteDomainRepo.GetBy(websiteDomainRepo.WithWebsiteId(website.ID), repo.WithOrderAsc("id"))
if err != nil {
return "", err
}
if len(websiteDomains) > 0 {
return websiteDomains[0].Domain, nil
}
if website.PrimaryDomain != "" {
return website.PrimaryDomain, nil
}
return fmt.Sprintf("%d", website.ID), nil
}
func fillAgentWebsiteItems(items []dto.AgentItem, explicitWebsiteMap map[uint]model.Website, websiteDomainMap map[uint][]model.WebsiteDomain) {
for index := range items {
if items[index].WebsiteID == 0 {
continue
}
website, ok := explicitWebsiteMap[items[index].WebsiteID]
if !ok {
items[index].WebsiteID = 0
items[index].WebsitePrimaryDomain = ""
items[index].WebsiteType = ""
items[index].WebsiteProtocol = ""
continue
}
items[index].WebsiteType = website.Type
items[index].WebsiteProtocol = website.Protocol
websiteDomains := websiteDomainMap[items[index].WebsiteID]
if len(websiteDomains) == 0 {
items[index].WebsitePrimaryDomain = ""
continue
}
items[index].WebsitePrimaryDomain = websiteDomains[0].Domain
}
}
func uniqueDeploymentWebsiteMapByAppInstall(websites []model.Website) map[uint]model.Website {
websiteMap := make(map[uint]model.Website)
duplicateAppInstallIDs := make(map[uint]struct{})
for _, website := range websites {
if website.AppInstallID == 0 {
continue
}
if _, duplicated := duplicateAppInstallIDs[website.AppInstallID]; duplicated {
continue
}
if _, exists := websiteMap[website.AppInstallID]; exists {
delete(websiteMap, website.AppInstallID)
duplicateAppInstallIDs[website.AppInstallID] = struct{}{}
continue
}
websiteMap[website.AppInstallID] = website
}
return websiteMap
}
func UniqueDeploymentWebsiteMapForMigration(websites []model.Website) map[uint]model.Website {
return uniqueDeploymentWebsiteMapByAppInstall(websites)
}
func uniqueUintList(items []uint) []uint {
itemMap := make(map[uint]struct{}, len(items))
uniq := make([]uint, 0, len(items))
for _, item := range items {
if item == 0 {
continue
}
if _, exists := itemMap[item]; exists {
continue
}
itemMap[item] = struct{}{}
uniq = append(uniq, item)
}
return uniq
}
func isBindableAgentWebsiteType(websiteType string) bool {
return websiteType == constant.Proxy || websiteType == constant.Static
}
func bindDeploymentWebsiteToAgentByAppInstall(website *model.Website) error {
if website.ID == 0 || website.Type != constant.Deployment || website.AppInstallID == 0 {
return nil
}
appInstall, err := appInstallRepo.GetFirst(repo.WithByID(website.AppInstallID))
if err != nil {
return err
}
if appInstall.App.Key != constant.AppOpenclaw && appInstall.App.Key != constant.AppCopaw && appInstall.App.Key != constant.AppHermesAgent {
return nil
}
agent, err := agentRepo.GetFirst(repo.WithByAppInstallID(website.AppInstallID))
if errors.Is(err, gorm.ErrRecordNotFound) {
return nil
}
if err != nil {
return err
}
if agent.WebsiteID != 0 {
return nil
}
agent.WebsiteID = website.ID
if err := agentRepo.Save(agent); err != nil {
return err
}
return ensureOpenclawWebsiteAllowedOrigin(agent, website)
}
func ensureOpenclawWebsiteAllowedOrigin(agent *model.Agent, website *model.Website) error {
if agent == nil || website == nil || agent.AgentType != constant.AppOpenclaw {
return nil
}
origins, err := buildWebsiteAllowedOrigins(website)
if err != nil {
return err
}
if len(origins) == 0 {
return nil
}
install, err := appInstallRepo.GetFirst(repo.WithByID(agent.AppInstallID))
if err != nil {
return err
}
conf, err := readOpenclawConfig(agent.ConfigPath)
if err != nil {
return err
}
allowedOrigins := extractSecurityConfig(conf).AllowedOrigins
allowedOrigins, err = normalizeAllowedOrigins(append(allowedOrigins, origins...))
if err != nil {
return err
}
setSecurityConfig(conf, dto.AgentSecurityConfig{AllowedOrigins: allowedOrigins})
if err := writeOpenclawConfigRaw(agent.ConfigPath, conf); err != nil {
return err
}
if err := syncOpenclawAllowedOriginEnv(&install, allowedOrigins); err != nil {
return err
}
return appInstallRepo.Save(context.Background(), &install)
}
func buildWebsiteAllowedOrigins(website *model.Website) ([]string, error) {
websiteDomains, err := websiteDomainRepo.GetBy(websiteDomainRepo.WithWebsiteId(website.ID), repo.WithOrderAsc("id"))
if err != nil {
return nil, err
}
if len(websiteDomains) == 0 {
return nil, nil
}
defaultHTTPSPort := 443
if strings.EqualFold(website.Protocol, "https") {
nginxInstall, err := getAppInstallByKey(constant.AppOpenresty)
if err == nil && nginxInstall.HttpsPort > 0 {
defaultHTTPSPort = nginxInstall.HttpsPort
}
}
return buildWebsiteAllowedOriginsFromDomains(website, websiteDomains, defaultHTTPSPort)
}
func buildWebsiteAllowedOriginsFromDomains(website *model.Website, websiteDomains []model.WebsiteDomain, defaultHTTPSPort int) ([]string, error) {
if len(websiteDomains) == 0 {
return nil, nil
}
sort.Slice(websiteDomains, func(i, j int) bool {
return websiteDomains[i].ID < websiteDomains[j].ID
})
origins := make([]string, 0, len(websiteDomains))
for _, websiteDomain := range websiteDomains {
origin, err := buildWebsiteDomainOrigin(website.Protocol, websiteDomain, defaultHTTPSPort)
if err != nil {
return nil, err
}
origins = append(origins, origin)
}
return origins, nil
}
func buildWebsiteDomainOrigin(protocol string, websiteDomain model.WebsiteDomain, defaultHTTPSPort int) (string, error) {
origin := strings.ToLower(strings.TrimSpace(protocol)) + "://" + formatWebsiteDomainHost(websiteDomain.Domain)
switch strings.ToLower(strings.TrimSpace(protocol)) {
case "http":
if websiteDomain.Port > 0 && websiteDomain.Port != 80 {
origin = fmt.Sprintf("%s:%d", origin, websiteDomain.Port)
}
case "https":
port := websiteDomain.Port
if !websiteDomain.SSL {
port = defaultHTTPSPort
}
if port > 0 && port != 443 {
origin = fmt.Sprintf("%s:%d", origin, port)
}
}
return normalizeAllowedOrigin(origin)
}
func formatWebsiteDomainHost(domain string) string {
host := strings.Trim(strings.TrimSpace(domain), "[]")
if ip := net.ParseIP(host); ip != nil && strings.Contains(host, ":") {
return "[" + host + "]"
}
return host
}
+130 -4
View File
@@ -1,11 +1,14 @@
package service
import (
"bytes"
"context"
"fmt"
"os"
"os/exec"
"path"
"strings"
"syscall"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
@@ -19,6 +22,7 @@ import (
"github.com/1Panel-dev/1Panel/agent/i18n"
"github.com/1Panel-dev/1Panel/agent/utils/cmd"
"github.com/1Panel-dev/1Panel/agent/utils/common"
"github.com/1Panel-dev/1Panel/agent/utils/re"
"github.com/jinzhu/copier"
)
@@ -107,8 +111,7 @@ func (u *AIToolService) Create(req dto.OllamaModelName) error {
}
go func() {
taskItem.AddSubTask(i18n.GetWithName("OllamaModelPull", req.Name), func(t *task.Task) error {
cmdMgr := cmd.NewCommandMgr(cmd.WithTask(*taskItem), cmd.WithTimeout(time.Hour))
return cmdMgr.Run("docker", "exec", containerName, "ollama", "pull", info.Name)
return runOllamaPullWithProcess(t, containerName, info.Name)
}, nil)
taskItem.AddSubTask(i18n.GetWithName("OllamaModelSize", req.Name), func(t *task.Task) error {
itemSize, err := loadModelSize(info.Name, containerName)
@@ -162,8 +165,7 @@ func (u *AIToolService) Recreate(req dto.OllamaModelName) error {
}
go func() {
taskItem.AddSubTask(i18n.GetWithName("OllamaModelPull", req.Name), func(t *task.Task) error {
cmdMgr := cmd.NewCommandMgr(cmd.WithTask(*taskItem), cmd.WithTimeout(time.Hour))
return cmdMgr.Run("docker", "exec", containerName, "ollama", "pull", req.Name)
return runOllamaPullWithProcess(t, containerName, req.Name)
}, nil)
taskItem.AddSubTask(i18n.GetWithName("OllamaModelSize", req.Name), func(t *task.Task) error {
itemSize, err := loadModelSize(modelInfo.Name, containerName)
@@ -394,3 +396,127 @@ func loadModelSize(name string, containerName string) (string, error) {
}
return "", fmt.Errorf("no such model %s in ollama list, std: %s", name, stdout)
}
func runOllamaPullWithProcess(taskItem *task.Task, containerName, modelName string) error {
ctx, cancel := context.WithTimeout(context.Background(), time.Hour)
defer cancel()
cmdItem := exec.CommandContext(ctx, "docker", "exec", containerName, "ollama", "pull", modelName)
cmdItem.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
writer := &ollamaPullLogWriter{taskItem: taskItem}
cmdItem.Stdout = writer
cmdItem.Stderr = writer
if err := cmdItem.Start(); err != nil {
return fmt.Errorf("failed to start ollama pull %s: %w", modelName, err)
}
waitErr := cmdItem.Wait()
writer.Flush()
if ctx.Err() == context.DeadlineExceeded {
if cmdItem.Process != nil && cmdItem.Process.Pid > 0 {
_ = syscall.Kill(-cmdItem.Process.Pid, syscall.SIGKILL)
}
return buserr.New("ErrCmdTimeout")
}
if waitErr == nil {
return nil
}
if len(strings.TrimSpace(writer.errBuf.String())) > 0 {
return fmt.Errorf("%s", strings.TrimSpace(writer.errBuf.String()))
}
return waitErr
}
type ollamaPullLogWriter struct {
taskItem *task.Task
errBuf bytes.Buffer
}
func (w *ollamaPullLogWriter) Write(p []byte) (n int, err error) {
w.errBuf.Write(p)
for _, segment := range splitOllamaPullSegments(string(p)) {
w.logLine(segment)
}
return len(p), nil
}
func (w *ollamaPullLogWriter) Flush() {}
func (w *ollamaPullLogWriter) logLine(line string) {
if w == nil || w.taskItem == nil {
return
}
line = sanitizeOllamaPullLine(line)
if line == "" {
return
}
if prefix := resolveOllamaPullLogPrefix(line); prefix != "" {
_ = replaceOllamaPullLogLine(prefix, line, w.taskItem)
return
}
w.taskItem.Log(line)
}
func sanitizeOllamaPullLine(line string) string {
line = re.StripAnsiControlSeq(line)
line = strings.TrimSpace(line)
if strings.HasPrefix(line, "pulling manifes") {
return ""
}
return line
}
func splitOllamaPullSegments(chunk string) []string {
if chunk == "" {
return nil
}
chunk = strings.ReplaceAll(chunk, "\r", "\n")
parts := strings.Split(chunk, "\x1b[1G")
segments := make([]string, 0, len(parts))
for _, part := range parts {
for _, line := range strings.Split(part, "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
segments = append(segments, line)
}
}
return segments
}
func resolveOllamaPullLogPrefix(line string) string {
if strings.HasPrefix(line, "pulling ") {
if idx := strings.Index(line, ":"); idx > 0 {
return strings.TrimSpace(line[:idx])
}
fields := strings.Fields(line)
if len(fields) >= 2 {
return strings.TrimSpace(fields[0] + " " + fields[1])
}
}
return ""
}
func replaceOllamaPullLogLine(prefix, newLine string, taskItem *task.Task) error {
if taskItem == nil || taskItem.Task == nil {
return nil
}
data, err := os.ReadFile(taskItem.Task.LogFile)
if err != nil {
return err
}
lines := strings.Split(string(data), "\n")
for idx, line := range lines {
trimmed := strings.TrimSpace(line)
if trimmed == "" {
continue
}
if strings.Contains(trimmed, prefix) {
lines[idx] = time.Now().Format("2006/01/02 15:04:05") + " " + newLine
return os.WriteFile(taskItem.Task.LogFile, []byte(strings.Join(lines, "\n")), os.ModePerm)
}
}
taskItem.Log(newLine)
return nil
}
+3 -1
View File
@@ -15,6 +15,7 @@ import (
"github.com/1Panel-dev/1Panel/agent/utils/email"
"github.com/1Panel-dev/1Panel/agent/utils/xpack"
"github.com/shirou/gopsutil/v4/disk"
"mime"
"sort"
"strings"
"sync"
@@ -483,13 +484,14 @@ func (a AlertService) TestAlertConfig(req dto.AlertConfigTest) (bool, error) {
if username == "" {
username = req.Sender
}
encodedDisplayName := mime.BEncoding.Encode("UTF-8", req.DisplayName)
cfg := email.SMTPConfig{
Host: req.Host,
Port: req.Port,
Sender: req.Sender,
Username: username,
Password: req.Password,
From: fmt.Sprintf(`"%s" <%s>`, req.DisplayName, req.Sender),
From: fmt.Sprintf(`"%s" <%s>`, encodedDisplayName, req.Sender),
Encryption: req.Encryption,
Recipient: req.Recipient,
}
+38 -31
View File
@@ -3,6 +3,12 @@ package service
import (
"encoding/json"
"fmt"
"math"
"sort"
"strconv"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/model"
"github.com/1Panel-dev/1Panel/agent/app/repo"
@@ -18,11 +24,6 @@ import (
"github.com/shirou/gopsutil/v4/load"
"github.com/shirou/gopsutil/v4/mem"
"github.com/shirou/gopsutil/v4/net"
"math"
"sort"
"strconv"
"strings"
"time"
)
const (
@@ -423,25 +424,19 @@ func loadDiskUsage(alert dto.AlertDTO) {
}
if isAlertDue(newDate) {
if strings.Contains(alert.Project, "all") {
err = processAllDisks(alert)
_ = processAllDisks(alert)
} else {
err = processSingleDisk(alert)
_ = processSingleDisk(alert)
}
}
}
func loadPanelLogin(alert dto.AlertDTO) {
count, isAlert, err := alertUtil.CountRecentFailedLoginLogs(alert.Cycle, alert.Count)
alertType := alert.Type
quota := strconv.Itoa(count)
quotaType := strconv.Itoa(int(alert.Cycle))
if err != nil {
global.LOG.Errorf("Failed to count recent failed login logs: %v", err)
}
if isAlert {
alertType = "panelLogin"
quota = strconv.Itoa(count)
quotaType = "panelLogin"
params := []dto.Param{
{
Index: "1",
@@ -454,7 +449,7 @@ func loadPanelLogin(alert dto.AlertDTO) {
Value: "",
},
}
sendAlerts(alert, alertType, quota, quotaType, params)
sendAlerts(alert, "panelLogin", strconv.Itoa(count), "panelLogin", params)
}
whitelist := strings.Split(strings.TrimSpace(alert.AdvancedParams), "\n")
@@ -463,15 +458,13 @@ func loadPanelLogin(alert dto.AlertDTO) {
global.LOG.Errorf("Failed to check recent failed ip login logs: %v", err)
}
if len(records) > 0 {
quota = strings.Join(func() []string {
quota := strings.Join(func() []string {
var ips []string
for _, r := range records {
ips = append(ips, r.IP)
}
return ips
}(), "\n")
alertType = "panelIpLogin"
quotaType = "panelIpLogin"
params := []dto.Param{
{
Index: "1",
@@ -484,22 +477,16 @@ func loadPanelLogin(alert dto.AlertDTO) {
Value: " IP ",
},
}
sendAlerts(alert, alertType, quota, quotaType, params)
sendAlerts(alert, "panelIpLogin", quota, "panelIpLogin", params)
}
}
func loadSSHLogin(alert dto.AlertDTO) {
count, isAlert, err := alertUtil.CountRecentFailedSSHLog(alert.Cycle, alert.Count)
alertType := alert.Type
quota := strconv.Itoa(count)
quotaType := strconv.Itoa(int(alert.Cycle))
if err != nil {
global.LOG.Errorf("Failed to count recent failed ssh login logs: %v", err)
}
if isAlert {
alertType = "sshLogin"
quota = strconv.Itoa(count)
quotaType = "sshLogin"
params := []dto.Param{
{
Index: "1",
@@ -512,7 +499,7 @@ func loadSSHLogin(alert dto.AlertDTO) {
Value: "",
},
}
sendAlerts(alert, alertType, quota, quotaType, params)
sendAlerts(alert, "sshLogin", strconv.Itoa(count), "sshLogin", params)
}
whitelist := strings.Split(strings.TrimSpace(alert.AdvancedParams), "\n")
records, err := alertUtil.FindRecentSuccessLoginNotInWhitelist(30, whitelist)
@@ -520,9 +507,7 @@ func loadSSHLogin(alert dto.AlertDTO) {
global.LOG.Errorf("Failed to check recent failed ip ssh login logs: %v", err)
}
if len(records) > 0 {
quota = strings.Join(records, "\n")
alertType = "sshIpLogin"
quotaType = "sshIpLogin"
quota := strings.Join(records, "\n")
params := []dto.Param{
{
Index: "1",
@@ -535,7 +520,7 @@ func loadSSHLogin(alert dto.AlertDTO) {
Value: " IP ",
},
}
sendAlerts(alert, alertType, quota, quotaType, params)
sendAlerts(alert, "sshIpLogin", quota, "sshIpLogin", params)
}
}
@@ -665,6 +650,28 @@ func sendAlerts(alert dto.AlertDTO, alertType, quota, quotaType string, params [
continue
}
alertUtil.CreateNewAlertTask(quota, alertType, quotaType, m)
case constant.Bark:
todayCount, isValid := canSendAlertToday(alertType, quotaType, alert.SendCount, m)
if !isValid {
continue
}
var create = dto.AlertLogCreate{
Type: alertUtil.GetCronJobType(alert.Type),
AlertId: alert.ID,
Count: todayCount + 1,
}
alertInfo := alert
alertInfo.Type = alertType
create.AlertRule = alertUtil.ProcessAlertRule(alert)
create.AlertDetail = alertUtil.ProcessAlertDetail(alertInfo, quotaType, params, m)
transport := xpack.LoadRequestTransport()
agentInfo, _ := xpack.GetAgentInfo()
alertErr := alertUtil.CreateBarkAlertLog(create, alertInfo, params, transport, agentInfo)
if alertErr != nil {
global.LOG.Infof("%s alert %s push failed, err: %v", alertType, m, alertErr.Error())
continue
}
alertUtil.CreateNewAlertTask(quota, alertType, quotaType, m)
default:
}
}
@@ -894,7 +901,7 @@ func processAllDisks(alert dto.AlertDTO) error {
if err != nil {
errMsg := fmt.Sprintf("disk path %s process failed: %v", item.Path, err)
errMsgs = append(errMsgs, errMsg)
global.LOG.Errorf(errMsg)
global.LOG.Errorf("%s", errMsg)
continue
}
}
@@ -907,7 +914,7 @@ func processAllDisks(alert dto.AlertDTO) error {
func processSingleDisk(alert dto.AlertDTO) error {
err := checkAndCreateDiskAlert(alert, alert.Project)
if err != nil {
global.LOG.Errorf(err.Error())
global.LOG.Errorf("%s", err.Error())
return err
}
return nil
+53
View File
@@ -30,6 +30,8 @@ func (s *AlertSender) Send(quota string, params []dto.Param) {
s.sendSMS(quota, params)
case constant.Email:
s.sendEmail(quota, params)
case constant.Bark:
s.sendBark(quota, params)
case constant.WeCom, constant.DingTalk, constant.FeiShu:
s.sendWebhook(quota, params, method)
}
@@ -45,6 +47,8 @@ func (s *AlertSender) ResourceSend(quota string, params []dto.Param) {
s.sendResourceSMS(quota, params)
case constant.Email:
s.sendResourceEmail(quota, params)
case constant.Bark:
s.sendResourceBark(quota, params)
case constant.WeCom, constant.DingTalk, constant.FeiShu:
s.sendResourceWebhook(quota, params, method)
}
@@ -101,6 +105,31 @@ func (s *AlertSender) sendEmail(quota string, params []dto.Param) {
alertUtil.CreateNewAlertTask(quota, s.alert.Type, s.quotaType, constant.Email)
}
func (s *AlertSender) sendBark(quota string, params []dto.Param) {
totalCount, isValid := s.canSendAlert(constant.Bark)
if !isValid {
return
}
create := dto.AlertLogCreate{
Status: constant.AlertSuccess,
Count: totalCount + 1,
AlertId: s.alert.ID,
Type: s.alert.Type,
AlertRule: alertUtil.ProcessAlertRule(s.alert),
AlertDetail: alertUtil.ProcessAlertDetail(s.alert, quota, params, constant.Bark),
}
transport := xpack.LoadRequestTransport()
agentInfo, _ := xpack.GetAgentInfo()
err := alertUtil.CreateBarkAlertLog(create, s.alert, params, transport, agentInfo)
if err != nil {
global.LOG.Errorf("%s alert bark push failed: %v", s.alert.Type, err)
return
}
alertUtil.CreateNewAlertTask(quota, s.alert.Type, s.quotaType, constant.Bark)
}
func (s *AlertSender) sendWebhook(quota string, params []dto.Param, method string) {
totalCount, isValid := s.canSendAlert(method)
if !isValid {
@@ -171,6 +200,30 @@ func (s *AlertSender) sendResourceEmail(quota string, params []dto.Param) {
alertUtil.CreateNewAlertTask(quota, s.alert.Type, s.quotaType, constant.Email)
}
func (s *AlertSender) sendResourceBark(quota string, params []dto.Param) {
todayCount, isValid := s.canResourceSendAlert(constant.Bark)
if !isValid {
return
}
create := dto.AlertLogCreate{
Status: constant.AlertSuccess,
Count: todayCount + 1,
AlertId: s.alert.ID,
Type: s.alert.Type,
AlertRule: alertUtil.ProcessAlertRule(s.alert),
AlertDetail: alertUtil.ProcessAlertDetail(s.alert, quota, params, constant.Bark),
}
transport := xpack.LoadRequestTransport()
agentInfo, _ := xpack.GetAgentInfo()
if err := alertUtil.CreateBarkAlertLog(create, s.alert, params, transport, agentInfo); err != nil {
global.LOG.Errorf("failed to send Bark alert: %v", err)
return
}
alertUtil.CreateNewAlertTask(quota, s.alert.Type, s.quotaType, constant.Bark)
}
func (s *AlertSender) sendResourceWebhook(quota string, params []dto.Param, method string) {
todayCount, isValid := s.canResourceSendAlert(method)
if !isValid {
+23 -5
View File
@@ -13,6 +13,7 @@ import (
"strconv"
"strings"
"sync"
"time"
"github.com/gin-gonic/gin"
@@ -57,6 +58,10 @@ type IAppService interface {
GetAppDetailByKey(appKey, version string) (response.AppDetailSimpleDTO, error)
}
type appInstallHooks struct {
AfterCopyData func(appInstall *model.AppInstall) error
}
func NewIAppService() IAppService {
return &AppService{}
}
@@ -340,6 +345,10 @@ func (a AppService) GetAppDetailByID(id uint) (*response.AppDetailDTO, error) {
}
func (a AppService) Install(req request.AppInstallCreate, executeScript bool) (appInstall *model.AppInstall, err error) {
return a.installWithHooks(req, executeScript, nil)
}
func (a AppService) installWithHooks(req request.AppInstallCreate, executeScript bool, hooks *appInstallHooks) (appInstall *model.AppInstall, err error) {
if err = docker.CreateDefaultDockerNetwork(); err != nil {
err = buserr.WithDetail("Err1PanelNetworkFailed", err.Error(), nil)
return
@@ -416,7 +425,7 @@ func (a AppService) Install(req request.AppInstallCreate, executeScript bool) (a
}
} else {
if appDetail.DockerCompose == "" {
dockerComposeUrl := fmt.Sprintf("%s/%s/1panel/%s/%s/docker-compose.yml", global.CONF.RemoteURL.AppRepo, global.CONF.Base.Mode, app.Key, appDetail.Version)
dockerComposeUrl := fmt.Sprintf("%s/%s/1panel/%s/%s/docker-compose.yml", global.AppRepoURL(), global.CONF.Base.Mode, app.Key, appDetail.Version)
_, composeRes, err = req_helper.HandleRequest(dockerComposeUrl, http.MethodGet, constant.TimeOut20s)
if err != nil {
return
@@ -516,6 +525,10 @@ func (a AppService) Install(req request.AppInstallCreate, executeScript bool) (a
}
appInstall.Env = string(paramByte)
var maxSort int
global.DB.Model(&model.AppInstall{}).Where("favorite = ?", false).Select("COALESCE(MAX(sort_order),0)").Scan(&maxSort)
appInstall.SortOrder = maxSort + 1
if err = appInstallRepo.Create(context.Background(), appInstall); err != nil {
return
}
@@ -533,6 +546,11 @@ func (a AppService) Install(req request.AppInstallCreate, executeScript bool) (a
if err = copyData(t, app, appDetail, appInstall, req); err != nil {
return err
}
if hooks != nil && hooks.AfterCopyData != nil {
if err = hooks.AfterCopyData(appInstall); err != nil {
return err
}
}
if executeScript {
if err = runScript(t, appInstall, "init"); err != nil {
return err
@@ -559,7 +577,7 @@ func (a AppService) Install(req request.AppInstallCreate, executeScript bool) (a
_ = appInstallRepo.Save(context.Background(), appInstall)
}
installTask.AddSubTask(task.GetTaskName(appInstall.Name, task.TaskInstall, task.TaskScopeApp), installApp, handleAppStatus)
installTask.AddSubTaskWithOps(task.GetTaskName(appInstall.Name, task.TaskInstall, task.TaskScopeApp), installApp, handleAppStatus, 0, time.Hour)
go func() {
if taskErr := installTask.Execute(); taskErr != nil {
@@ -817,7 +835,7 @@ func (a AppService) GetAppUpdate() (*response.AppUpdateRes, error) {
return res, nil
}
versionUrl := fmt.Sprintf("%s/%s/1panel.json.version.txt", global.CONF.RemoteURL.AppRepo, global.CONF.Base.Mode)
versionUrl := fmt.Sprintf("%s/%s/1panel.json.version.txt", global.AppRepoURL(), global.CONF.Base.Mode)
_, versionRes, err := req_helper.HandleRequest(versionUrl, http.MethodGet, constant.TimeOut20s)
if err != nil {
return nil, err
@@ -878,7 +896,7 @@ func getAppFromRepo(downloadPath string) error {
return err
}
if err := fileOp.Decompress(packagePath, global.Dir.ResourceDir, files.SdkZip, ""); err != nil {
if err := fileOp.Decompress(context.Background(), packagePath, global.Dir.ResourceDir, files.SdkZip, ""); err != nil {
return err
}
defer func() {
@@ -889,7 +907,7 @@ func getAppFromRepo(downloadPath string) error {
func getAppList() (*dto.AppList, error) {
list := &dto.AppList{}
if err := getAppFromRepo(fmt.Sprintf("%s/%s/1panel.json.zip", global.CONF.RemoteURL.AppRepo, global.CONF.Base.Mode)); err != nil {
if err := getAppFromRepo(fmt.Sprintf("%s/%s/1panel.json.zip", global.AppRepoURL(), global.CONF.Base.Mode)); err != nil {
return nil, err
}
listFile := filepath.Join(global.Dir.ResourceDir, "1panel.json")
+35 -8
View File
@@ -57,6 +57,7 @@ type IAppInstallService interface {
UpdateAppConfig(req request.AppConfigUpdate) error
GetInstallList() ([]dto.AppInstallInfo, error)
GetAppInstallInfo(appInstallID uint) (*response.AppInstallInfo, error)
UpdateSort(req request.AppInstallSort) error
}
func NewIAppInstalledService() IAppInstallService {
@@ -83,6 +84,7 @@ func (a *AppInstallService) Page(req request.AppInstalledSearch) (int64, []respo
err error
)
opts = append(opts, repo.WithOrderRuleBy("favorite", "descending"))
opts = append(opts, repo.WithOrderRuleBy("sort_order", "ascending"))
if req.Name != "" {
opts = append(opts, repo.WithByLikeName(req.Name))
@@ -300,6 +302,9 @@ func (a *AppInstallService) Operate(req request.AppInstalledOperate) error {
return opNginx(install.ContainerName, constant.NginxReload)
case constant.Favorite:
install.Favorite = req.Favorite
var maxSort int
global.DB.Model(&model.AppInstall{}).Where("favorite = ?", req.Favorite).Select("COALESCE(MAX(sort_order),0)").Scan(&maxSort)
install.SortOrder = maxSort + 1
return appInstallRepo.Save(context.Background(), &install)
default:
return errors.New("operate not support")
@@ -318,17 +323,25 @@ func (a *AppInstallService) UpdateAppConfig(req request.AppConfigUpdate) error {
return appInstallRepo.Save(context.Background(), &installed)
}
func (a *AppInstallService) UpdateSort(req request.AppInstallSort) error {
for _, item := range req.Items {
if err := appInstallRepo.BatchUpdateBy(map[string]interface{}{"sort_order": item.SortOrder}, repo.WithByID(item.InstallID)); err != nil {
return err
}
}
return nil
}
func (a *AppInstallService) Update(req request.AppInstalledUpdate) error {
installed, err := appInstallRepo.GetFirst(repo.WithByID(req.InstallId))
if err != nil {
return err
}
changePort := false
oldInstalled := installed
port, ok := req.Params["PANEL_APP_PORT_HTTP"]
if ok {
portN := int(math.Ceil(port.(float64)))
if portN != installed.HttpPort {
changePort = true
httpPort, err := checkPort("PANEL_APP_PORT_HTTP", req.Params)
if err != nil {
return err
@@ -414,9 +427,17 @@ func (a *AppInstallService) Update(req request.AppInstalledUpdate) error {
installed.Status = constant.StatusRunning
_ = appInstallRepo.Save(context.Background(), &installed)
proxyChanged := hasAppInstallProxyPassChanged(&oldInstalled, &installed)
currentProxy, currentProxyErr := getAppInstallProxyPass(&installed)
website, _ := websiteRepo.GetFirst(websiteRepo.WithAppInstallId(installed.ID))
if changePort && website.ID != 0 && website.Status == constant.StatusRunning {
go func() {
if proxyChanged && website.ID != 0 && currentProxyErr == nil {
website.Proxy = currentProxy
if err := websiteRepo.SaveWithoutCtx(&website); err != nil {
global.LOG.Error(buserr.WithErr("ErrUpdateBuWebsite", err).Error())
}
}
if proxyChanged && website.ID != 0 && website.Status == constant.StatusRunning && currentProxyErr == nil {
go func(website model.Website, proxy string) {
nginxInstall, err := getNginxFull(&website)
if err != nil {
global.LOG.Error(buserr.WithErr("ErrUpdateBuWebsite", err).Error())
@@ -429,7 +450,6 @@ func (a *AppInstallService) Update(req request.AppInstalledUpdate) error {
return
}
server := servers[0]
proxy := fmt.Sprintf("http://127.0.0.1:%d", installed.HttpPort)
server.UpdateRootProxy([]string{proxy})
if err := nginx.WriteConfig(config, nginx.IndentedStyle); err != nil {
@@ -440,7 +460,10 @@ func (a *AppInstallService) Update(req request.AppInstalledUpdate) error {
global.LOG.Error(buserr.WithErr("ErrUpdateBuWebsite", err).Error())
return
}
}()
}(website, currentProxy)
}
if proxyChanged && website.ID != 0 && currentProxyErr != nil {
global.LOG.Error(buserr.WithErr("ErrUpdateBuWebsite", currentProxyErr).Error())
}
return nil
}
@@ -830,6 +853,10 @@ func syncAppInstallStatus(appInstall *model.AppInstall, force bool) error {
return nil
}
func SyncAppInstallStatus(appInstall *model.AppInstall, force bool) error {
return syncAppInstallStatus(appInstall, force)
}
func updateInstallInfoInDB(appKey, appName, param string, value interface{}) error {
if param != "password" && param != "port" && param != "user-password" {
return nil
@@ -847,7 +874,7 @@ func updateInstallInfoInDB(appKey, appName, param string, value interface{}) err
envKey := ""
switch param {
case "password":
if appKey == "mysql" || appKey == "mariadb" || appKey == "postgresql" {
if appKey == "mysql" || appKey == "mariadb" || appKey == "postgresql" || appKey == "mongodb" {
envKey = "PANEL_DB_ROOT_PASSWORD="
} else {
envKey = "PANEL_REDIS_ROOT_PASSWORD="
@@ -888,7 +915,7 @@ func updateInstallInfoInDB(appKey, appName, param string, value interface{}) err
"param": strings.ReplaceAll(appInstall.Param, oldVal, newVal),
"env": strings.ReplaceAll(appInstall.Env, oldVal, newVal),
}, repo.WithByID(appInstall.ID))
if appKey == "mysql" || appKey == "postgresql" {
if appKey == "mysql" || appKey == "postgresql" || appKey == "mongodb" {
return nil
}
}
+175 -66
View File
@@ -7,6 +7,8 @@ import (
"net/http"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
@@ -81,7 +83,7 @@ func (a AppService) createSyncAppStoreTask(sharedCtx **appSyncContext) func(t *t
ctx := &appSyncContext{
task: t,
httpClient: http.Client{Timeout: time.Duration(constant.TimeOut20s) * time.Second, Transport: xpack.LoadRequestTransport()},
baseRemoteUrl: fmt.Sprintf("%s/%s/1panel", global.CONF.RemoteURL.AppRepo, global.CONF.Base.Mode),
baseRemoteUrl: fmt.Sprintf("%s/%s/1panel", global.AppRepoURL(), global.CONF.Base.Mode),
systemVersion: setting.SystemVersion,
settingService: settingService,
list: list,
@@ -127,90 +129,197 @@ func (a AppService) createSyncAppStoreTask(sharedCtx **appSyncContext) func(t *t
}
}
type appWorkItem struct {
appDef dto.AppDefine
app model.App
iconUrl string
hadIcon bool
}
type appWorkResult struct {
appKey string
app model.App
iconStatus int
hadIcon bool
httpFailed bool
}
func (c *appSyncContext) processOneApp(item appWorkItem) appWorkResult {
app := item.app
l := item.appDef
result := appWorkResult{
appKey: l.AppProperty.Key,
hadIcon: item.hadIcon,
}
if item.hadIcon {
status, iconField := c.downloadAppIcon(item.iconUrl, l.AppProperty.Key, app.Icon)
result.iconStatus = status
switch status {
case http.StatusOK:
app.Icon = iconField
case http.StatusNotModified:
default:
result.httpFailed = true
}
}
app.TagsKey = l.AppProperty.Tags
if l.AppProperty.Recommend > 0 {
app.Recommend = l.AppProperty.Recommend
} else {
app.Recommend = 9999
}
app.ReadMe = l.ReadMe
app.LastModified = l.LastModified
versions := l.Versions
detailsMap := getAppDetails(app.Details, versions)
for _, v := range versions {
version := v.Name
detail := detailsMap[version]
versionUrl := fmt.Sprintf("%s/%s/%s", c.baseRemoteUrl, app.Key, version)
paramByte, _ := json.Marshal(v.AppForm)
var appForm dto.AppForm
_ = json.Unmarshal(paramByte, &appForm)
if appForm.SupportVersion > 0 && common.CompareVersion(strconv.FormatFloat(appForm.SupportVersion, 'f', -1, 64), c.systemVersion) {
delete(detailsMap, version)
continue
}
if _, ok := InitTypes[app.Type]; ok {
dockerComposeUrl := fmt.Sprintf("%s/%s", versionUrl, "docker-compose.yml")
_, composeRes, err := req_helper.HandleRequestWithClient(&c.httpClient, dockerComposeUrl, http.MethodGet, constant.TimeOut20s)
if err == nil {
detail.DockerCompose = string(composeRes)
} else {
result.httpFailed = true
}
} else {
detail.DockerCompose = ""
}
detail.Params = string(paramByte)
detail.DownloadUrl = fmt.Sprintf("%s/%s", versionUrl, app.Key+"-"+version+".tar.gz")
detail.DownloadCallBackUrl = v.DownloadCallBackUrl
detail.Update = true
detail.LastModified = v.LastModified
detailsMap[version] = detail
}
newDetails := make([]model.AppDetail, 0, len(detailsMap))
for _, detail := range detailsMap {
newDetails = append(newDetails, detail)
}
app.Details = newDetails
result.app = app
return result
}
func (c *appSyncContext) syncAppIconsAndDetails() error {
total := len(c.list.Apps)
global.LOG.Infof("[AppStore] sync app detail start, total apps: %d", total)
var (
icon200Count = 0
icon304Count = 0
iconFailCount = 0
)
for i, l := range c.list.Apps {
if (i+1)%10 == 0 {
c.task.LogWithProgress(i18n.GetMsgByKey("SyncAppDetail"), i+1, total)
}
workItems := make([]appWorkItem, 0, total)
for _, l := range c.list.Apps {
app, ok := c.appsMap[l.AppProperty.Key]
if !ok {
continue
}
iconUrl, hasPending := c.pendingIcons[l.AppProperty.Key]
if hasPending {
status, iconField := c.downloadAppIcon(iconUrl, l.AppProperty.Key, app.Icon)
switch status {
workItems = append(workItems, appWorkItem{
appDef: l,
app: app,
iconUrl: iconUrl,
hadIcon: hasPending,
})
}
totalWork := len(workItems)
if totalWork == 0 {
return nil
}
const maxWorkers = 4
var (
failFlag atomic.Bool
workCh = make(chan int, maxWorkers)
resultCh = make(chan appWorkResult, maxWorkers)
wg sync.WaitGroup
)
wg.Add(maxWorkers)
for range maxWorkers {
go func() {
defer wg.Done()
for idx := range workCh {
result := c.processOneApp(workItems[idx])
resultCh <- result
if result.httpFailed {
failFlag.Store(true)
}
}
}()
}
go func() { wg.Wait(); close(resultCh) }()
var fed atomic.Int32
go func() {
for i := range workItems {
if failFlag.Load() {
break
}
workCh <- i
fed.Store(int32(i + 1))
}
close(workCh)
}()
var (
completed int
icon200Count int
icon304Count int
iconFailCount int
)
milestones := [4]int{totalWork / 4, totalWork / 2, totalWork * 3 / 4, totalWork}
nextMS := 0
checkProgress := func() {
if nextMS < len(milestones) && completed >= milestones[nextMS] {
c.task.LogWithProgress(i18n.GetMsgByKey("SyncAppDetail"), completed, totalWork)
nextMS++
}
}
applyResult := func(result appWorkResult) {
c.appsMap[result.appKey] = result.app
if result.hadIcon {
switch result.iconStatus {
case http.StatusOK:
app.Icon = iconField
icon200Count++
case http.StatusNotModified:
icon304Count++
default:
global.LOG.Warnf("[AppStore] download icon failed url=%s, appKey=%s", iconUrl, l.AppProperty.Key)
iconFailCount++
}
}
completed++
checkProgress()
}
app.TagsKey = l.AppProperty.Tags
if l.AppProperty.Recommend > 0 {
app.Recommend = l.AppProperty.Recommend
} else {
app.Recommend = 9999
for result := range resultCh {
applyResult(result)
}
fedCount := int(fed.Load())
if failFlag.Load() && fedCount < len(workItems) {
global.LOG.Warnf("[AppStore] HTTP failure detected, falling back to serial for remaining %d apps", len(workItems)-fedCount)
for i := fedCount; i < len(workItems); i++ {
result := c.processOneApp(workItems[i])
applyResult(result)
}
app.ReadMe = l.ReadMe
app.LastModified = l.LastModified
versions := l.Versions
detailsMap := getAppDetails(app.Details, versions)
for _, v := range versions {
version := v.Name
detail := detailsMap[version]
versionUrl := fmt.Sprintf("%s/%s/%s", c.baseRemoteUrl, app.Key, version)
paramByte, _ := json.Marshal(v.AppForm)
var appForm dto.AppForm
_ = json.Unmarshal(paramByte, &appForm)
if appForm.SupportVersion > 0 && common.CompareVersion(strconv.FormatFloat(appForm.SupportVersion, 'f', -1, 64), c.systemVersion) {
delete(detailsMap, version)
continue
}
if _, ok := InitTypes[app.Type]; ok {
dockerComposeUrl := fmt.Sprintf("%s/%s", versionUrl, "docker-compose.yml")
_, composeRes, err := req_helper.HandleRequestWithClient(&c.httpClient, dockerComposeUrl, http.MethodGet, constant.TimeOut20s)
if err == nil {
detail.DockerCompose = string(composeRes)
}
} else {
detail.DockerCompose = ""
}
detail.Params = string(paramByte)
detail.DownloadUrl = fmt.Sprintf("%s/%s", versionUrl, app.Key+"-"+version+".tar.gz")
detail.DownloadCallBackUrl = v.DownloadCallBackUrl
detail.Update = true
detail.LastModified = v.LastModified
detailsMap[version] = detail
}
var newDetails []model.AppDetail
for _, detail := range detailsMap {
newDetails = append(newDetails, detail)
}
app.Details = newDetails
c.appsMap[l.AppProperty.Key] = app
}
global.LOG.Infof("[AppStore] icon download completed - total: %d, success(200): %d, cached(304): %d, failed: %d",
+46 -11
View File
@@ -33,7 +33,6 @@ import (
"github.com/1Panel-dev/1Panel/agent/utils/common"
"github.com/1Panel-dev/1Panel/agent/utils/compose"
"github.com/1Panel-dev/1Panel/agent/utils/docker"
composeV2 "github.com/1Panel-dev/1Panel/agent/utils/docker"
"github.com/1Panel-dev/1Panel/agent/utils/env"
"github.com/1Panel-dev/1Panel/agent/utils/files"
"github.com/1Panel-dev/1Panel/agent/utils/nginx"
@@ -363,7 +362,7 @@ func deleteAppInstall(deleteReq request.AppInstallDelete) error {
if err != nil {
return err
}
images, err := composeV2.GetImagesFromDockerCompose(content, []byte(install.DockerCompose))
images, err := docker.GetImagesFromDockerCompose(content, []byte(install.DockerCompose))
if err != nil {
return err
}
@@ -393,7 +392,7 @@ func deleteAppInstall(deleteReq request.AppInstallDelete) error {
return err
}
appKey := install.App.Key
if appKey == constant.AppOpenclaw {
if isAgentAppKey(appKey) {
_ = agentRepo.DeleteByAppInstallIDWithCtx(ctx, install.ID)
}
@@ -426,6 +425,8 @@ func deleteAppInstall(deleteReq request.AppInstallDelete) error {
switch install.App.Key {
case constant.AppMysql, constant.AppMariaDB, constant.AppMysqlCluster:
_ = mysqlRepo.Delete(ctx, mysqlRepo.WithByMysqlName(install.Name))
case constant.AppMongodb:
_ = mongodbRepo.Delete(ctx, mongodbRepo.WithByMongodbName(install.Name))
case constant.AppPostgresql, constant.AppPostgresqlCluster:
_ = postgresqlRepo.Delete(ctx, postgresqlRepo.WithByPostgresqlName(install.Name))
}
@@ -676,6 +677,7 @@ func upgradeInstall(req request.AppInstallUpgrade) error {
if err != nil {
return err
}
oldVersion := install.Version
detail, err := appDetailRepo.GetFirst(repo.WithByID(req.DetailID))
if err != nil {
return err
@@ -752,7 +754,7 @@ func upgradeInstall(req request.AppInstallUpgrade) error {
if req.DockerCompose != "" {
composeContent = []byte(req.DockerCompose)
}
images, err := composeV2.GetImagesFromDockerCompose(content, composeContent)
images, err := docker.GetImagesFromDockerCompose(content, composeContent)
if err != nil {
return err
}
@@ -802,6 +804,9 @@ func upgradeInstall(req request.AppInstallUpgrade) error {
}
var newCompose string
if err = migrateOpenclawProtocolUpgrade(&install, oldVersion, detail.Version); err != nil {
return err
}
if req.DockerCompose == "" {
newCompose, err = getUpgradeCompose(install, detail)
if err != nil {
@@ -911,7 +916,7 @@ func getContainerNames(install model.AppInstall) ([]string, error) {
if err != nil {
return nil, err
}
project, err := composeV2.GetComposeProject(install.Name, install.GetPath(), []byte(install.DockerCompose), []byte(envStr), true)
project, err := docker.GetComposeProject(install.Name, install.GetPath(), []byte(install.DockerCompose), []byte(envStr), true)
if err != nil {
return nil, err
}
@@ -1029,7 +1034,7 @@ func downloadApp(app model.App, appDetail model.AppDetail, appInstall *model.App
}
return
}
if err = fileOp.Decompress(filePath, appResourceDir, files.SdkTarGz, ""); err != nil {
if err = fileOp.Decompress(context.Background(), filePath, appResourceDir, files.SdkTarGz, ""); err != nil {
if logger == nil {
global.LOG.Errorf("decompress app[%s] error %v", app.Name, err)
} else {
@@ -1133,7 +1138,7 @@ func runScript(task *task.Task, appInstall *model.AppInstall, operate string) er
}
func checkContainerNameIsExist(containerName, appDir string) (bool, error) {
client, err := composeV2.NewDockerClient()
client, err := docker.NewDockerClient()
if err != nil {
return false, err
}
@@ -1169,7 +1174,7 @@ func upApp(task *task.Task, appInstall *model.AppInstall, pullImages bool) error
if err != nil {
return err
}
images, err := composeV2.GetImagesFromDockerCompose(envByte, []byte(appInstall.DockerCompose))
images, err := docker.GetImagesFromDockerCompose(envByte, []byte(appInstall.DockerCompose))
if err != nil {
return err
}
@@ -1613,6 +1618,7 @@ func handleInstalled(appInstallList []model.AppInstall, updated, sync, checkUpda
Document: installed.App.Document,
},
Favorite: installed.Favorite,
SortOrder: installed.SortOrder,
Container: installed.ContainerName,
ServiceName: strings.ToLower(installed.ServiceName),
}
@@ -2242,10 +2248,39 @@ func isEditCompose(installed model.AppInstall) bool {
if rawCompose == "" || err != nil {
return false
}
if rawCompose != installed.DockerCompose {
return true
equal, err := composeEqualExceptImage(rawCompose, installed.DockerCompose)
if err != nil {
return false
}
return !equal
}
func composeEqualExceptImage(expected, current string) (bool, error) {
expectedCompose := make(map[string]interface{})
if err := yaml.Unmarshal([]byte(expected), &expectedCompose); err != nil {
return false, err
}
currentCompose := make(map[string]interface{})
if err := yaml.Unmarshal([]byte(current), &currentCompose); err != nil {
return false, err
}
removeComposeServiceImages(expectedCompose)
removeComposeServiceImages(currentCompose)
return reflect.DeepEqual(expectedCompose, currentCompose), nil
}
func removeComposeServiceImages(composeMap map[string]interface{}) {
services, ok := composeMap["services"].(map[string]interface{})
if !ok {
return
}
for _, service := range services {
serviceMap, ok := service.(map[string]interface{})
if !ok {
continue
}
delete(serviceMap, "image")
}
return false
}
func getAppVersions(key string, details []model.AppDetail) []string {
+8
View File
@@ -44,10 +44,13 @@ type IBackupService interface {
MysqlBackup(db dto.CommonBackup) error
PostgresqlBackup(db dto.CommonBackup) error
MongodbBackup(db dto.CommonBackup) error
MysqlRecover(db dto.CommonRecover) error
PostgresqlRecover(db dto.CommonRecover) error
MongodbRecover(db dto.CommonRecover) error
MysqlRecoverByUpload(req dto.CommonRecover) error
PostgresqlRecoverByUpload(req dto.CommonRecover) error
MongodbRecoverByUpload(req dto.CommonRecover) error
RedisBackup(db dto.CommonBackup) error
RedisRecover(db dto.CommonRecover) error
@@ -57,6 +60,11 @@ type IBackupService interface {
AppBackup(db dto.CommonBackup) (*model.BackupRecord, error)
AppRecover(req dto.CommonRecover) error
ContainerBackup(req dto.CommonBackup) error
ContainerRecover(req dto.CommonRecover) error
ComposeBackup(req dto.CommonBackup) error
ComposeRecover(req dto.CommonRecover) error
}
func NewIBackupService() IBackupService {
+153 -39
View File
@@ -11,6 +11,7 @@ import (
"time"
"github.com/1Panel-dev/1Panel/agent/app/repo"
"github.com/subosito/gotenv"
"github.com/1Panel-dev/1Panel/agent/app/task"
"github.com/1Panel-dev/1Panel/agent/i18n"
@@ -179,6 +180,9 @@ func handleAppRecover(install *model.AppInstall, parentTask *task.Task, recoverF
return err
}
tmpPath := strings.ReplaceAll(recoverFile, ".tar.gz", "")
if err := fileOp.TarGzExtractPro(tmpPath+"/app.tar.gz", tmpPath, ""); err != nil {
return err
}
defer func() {
_, _ = compose.Up(install.GetComposePath())
_ = os.RemoveAll(strings.ReplaceAll(recoverFile, ".tar.gz", ""))
@@ -187,19 +191,26 @@ func handleAppRecover(install *model.AppInstall, parentTask *task.Task, recoverF
if !fileOp.Stat(tmpPath+"/app.json") || !fileOp.Stat(tmpPath+"/app.tar.gz") {
return errors.New(i18n.GetMsgByKey("AppBackupFileIncomplete"))
}
var oldInstall model.AppInstall
var backupInstall model.AppInstall
appJson, err := os.ReadFile(tmpPath + "/app.json")
if err != nil {
return err
}
if err := json.Unmarshal(appJson, &oldInstall); err != nil {
if err := json.Unmarshal(appJson, &backupInstall); err != nil {
return fmt.Errorf("unmarshal app.json failed, err: %v", err)
}
if oldInstall.App.Key != install.App.Key || oldInstall.Name != install.Name {
if backupInstall.App.Key != install.App.Key || backupInstall.Name != install.Name {
return errors.New(i18n.GetMsgByKey("AppAttributesNotMatch"))
}
newEnvFile := ""
backupEnvMap, err := getEnvMapByPath(path.Join(tmpPath, install.Name, ".env"))
if err != nil {
return err
}
installedEnvMap, err := getEnvMapByPath(install.GetEnvPath())
if err != nil {
return err
}
var mergedEnvContent string
resources, _ := appInstallResourceRepo.GetBy(appInstallResourceRepo.WithAppInstallId(install.ID))
for _, resource := range resources {
var database model.Database
@@ -225,11 +236,20 @@ func handleAppRecover(install *model.AppInstall, parentTask *task.Task, recoverF
if err != nil {
return err
}
newDB, err := reCreatePostgresqlDB(db.ID, database, backupEnvMap)
if err != nil {
return err
}
backupInstall.Env, mergedEnvContent, err = buildRecoverEnv(backupInstall.Env, backupEnvMap, installedEnvMap)
if err != nil {
return err
}
_ = appInstallResourceRepo.BatchUpdateBy(map[string]interface{}{"resource_id": newDB.ID}, repo.WithByID(resource.ID))
taskName := task.GetTaskName(db.Name, task.TaskRecover, task.TaskScopeDatabase)
t.LogStart(taskName)
if err := handlePostgresqlRecover(dto.CommonRecover{
Name: database.Name,
DetailName: db.Name,
Name: newDB.PostgresqlName,
DetailName: newDB.Name,
File: fmt.Sprintf("%s/%s.sql.gz", tmpPath, install.Name),
}, parentTask, true); err != nil {
t.LogFailedWithErr(taskName, err)
@@ -241,15 +261,11 @@ func handleAppRecover(install *model.AppInstall, parentTask *task.Task, recoverF
if err != nil {
return err
}
newDB, envMap, err := reCreateDB(db.ID, database, oldInstall.Env)
newDB, err := reCreateDB(db.ID, database, backupEnvMap)
if err != nil {
return err
}
oldHost := fmt.Sprintf("\"PANEL_DB_HOST\":\"%v\"", envMap["PANEL_DB_HOST"].(string))
newHost := fmt.Sprintf("\"PANEL_DB_HOST\":\"%v\"", database.Address)
oldInstall.Env = strings.ReplaceAll(oldInstall.Env, oldHost, newHost)
envMap["PANEL_DB_HOST"] = database.Address
newEnvFile, err = coverEnvJsonToStr(oldInstall.Env)
backupInstall.Env, mergedEnvContent, err = buildRecoverEnv(backupInstall.Env, backupEnvMap, installedEnvMap)
if err != nil {
return err
}
@@ -284,22 +300,22 @@ func handleAppRecover(install *model.AppInstall, parentTask *task.Task, recoverF
t.LogSuccess(deCompressName)
_ = fileOp.DeleteDir(backPath)
if len(newEnvFile) != 0 {
if len(mergedEnvContent) != 0 {
envPath := fmt.Sprintf("%s/%s/.env", install.GetAppPath(), install.Name)
file, err := os.OpenFile(envPath, os.O_WRONLY|os.O_TRUNC, 0640)
if err != nil {
return err
}
defer file.Close()
_, _ = file.WriteString(newEnvFile)
_, _ = file.WriteString(mergedEnvContent)
}
oldInstall.ID = install.ID
oldInstall.Status = constant.StatusRunning
oldInstall.AppId = install.AppId
oldInstall.AppDetailId = install.AppDetailId
oldInstall.App.ID = install.AppId
if err := appInstallRepo.Save(context.Background(), &oldInstall); err != nil {
backupInstall.ID = install.ID
backupInstall.Status = constant.StatusRunning
backupInstall.AppId = install.AppId
backupInstall.AppDetailId = install.AppDetailId
backupInstall.App.ID = install.AppId
if err := appInstallRepo.Save(context.Background(), &backupInstall); err != nil {
global.LOG.Errorf("save db app install failed, err: %v", err)
return err
}
@@ -373,33 +389,131 @@ func doAppBackup(install *model.AppInstall, parentTask *task.Task, backupDir, fi
return nil
}
func reCreateDB(dbID uint, database model.Database, oldEnv string) (*model.DatabaseMysql, map[string]interface{}, error) {
func reCreateDB(dbID uint, database model.Database, envMap map[string]interface{}) (*model.DatabaseMysql, error) {
mysqlService := NewIMysqlService()
ctx := context.Background()
_ = mysqlService.Delete(ctx, dto.MysqlDBDelete{ID: dbID, Database: database.Name, Type: database.Type, DeleteBackup: false, ForceDelete: true})
envMap := make(map[string]interface{})
if err := json.Unmarshal([]byte(oldEnv), &envMap); err != nil {
return nil, envMap, err
}
oldName, _ := envMap["PANEL_DB_NAME"].(string)
oldUser, _ := envMap["PANEL_DB_USER"].(string)
oldPassword, _ := envMap["PANEL_DB_USER_PASSWORD"].(string)
dbInfo := getDBCreateInfoFromEnv(envMap, "utf8mb4")
createDB, err := mysqlService.Create(context.Background(), dto.MysqlDBCreate{
Name: oldName,
Name: dbInfo.Name,
From: database.From,
Database: database.Name,
Format: "utf8mb4",
Username: oldUser,
Password: oldPassword,
Format: dbInfo.Format,
Username: dbInfo.User,
Password: dbInfo.Password,
Permission: "%",
})
cronjobs, _ := cronjobRepo.List(cronjobRepo.WithByDbName(fmt.Sprintf("%v", dbID)))
for _, job := range cronjobs {
_ = cronjobRepo.Update(job.ID, map[string]interface{}{"db_name": fmt.Sprintf("%v", createDB.ID)})
}
if err != nil {
return nil, envMap, err
return nil, err
}
return createDB, envMap, nil
updateCronjobsDBRef(dbID, createDB.ID)
return createDB, nil
}
func reCreatePostgresqlDB(dbID uint, database model.Database, envMap map[string]interface{}) (*model.DatabasePostgresql, error) {
postgresqlService := NewIPostgresqlService()
_ = postgresqlService.Delete(context.Background(), dto.PostgresqlDBDelete{
ID: dbID,
Type: database.Type,
Database: database.Name,
DeleteBackup: false,
ForceDelete: true,
})
dbInfo := getDBCreateInfoFromEnv(envMap, "UTF8")
createDB, err := postgresqlService.Create(context.Background(), dto.PostgresqlDBCreate{
Name: dbInfo.Name,
From: database.From,
Database: database.Name,
Format: dbInfo.Format,
Username: dbInfo.User,
Password: dbInfo.Password,
SuperUser: true,
})
if err != nil {
return nil, err
}
updateCronjobsDBRef(dbID, createDB.ID)
return createDB, nil
}
type dbRecreateInfo struct {
Name string
User string
Password string
Format string
}
func getDBCreateInfoFromEnv(envMap map[string]interface{}, defaultFormat string) dbRecreateInfo {
name, _ := envMap["PANEL_DB_NAME"].(string)
user, _ := envMap["PANEL_DB_USER"].(string)
password, _ := envMap["PANEL_DB_USER_PASSWORD"].(string)
format, _ := envMap["format"].(string)
if len(format) == 0 {
format = defaultFormat
}
return dbRecreateInfo{
Name: name,
User: user,
Password: password,
Format: format,
}
}
func updateCronjobsDBRef(oldDBID, newDBID uint) {
cronjobs, _ := cronjobRepo.List(cronjobRepo.WithByDbName(fmt.Sprintf("%v", oldDBID)))
for _, job := range cronjobs {
_ = cronjobRepo.Update(job.ID, map[string]interface{}{"db_name": fmt.Sprintf("%v", newDBID)})
}
}
func buildRecoverEnv(appEnv string, backupEnvMap, installedEnvMap map[string]interface{}) (string, string, error) {
currentHostVal, hasCurrentHost := installedEnvMap["PANEL_DB_HOST"]
if hasCurrentHost && fmt.Sprintf("%v", currentHostVal) != "" {
backupHost := fmt.Sprintf("\"PANEL_DB_HOST\":\"%v\"", backupEnvMap["PANEL_DB_HOST"])
currentHost := fmt.Sprintf("\"PANEL_DB_HOST\":\"%v\"", currentHostVal)
appEnv = strings.ReplaceAll(appEnv, backupHost, currentHost)
if _, ok := backupEnvMap["CASDOOR_DATASOURCE_NAME"]; ok {
backupEnvMap["CASDOOR_DATASOURCE_NAME"] = strings.ReplaceAll(
fmt.Sprintf("%v", backupEnvMap["CASDOOR_DATASOURCE_NAME"]),
fmt.Sprintf("%v", backupEnvMap["PANEL_DB_HOST"]),
fmt.Sprintf("%v", currentHostVal),
)
}
backupEnvMap["PANEL_DB_HOST"] = currentHostVal
}
newEnvMap := make(map[string]string, len(backupEnvMap))
handleMap(backupEnvMap, newEnvMap)
mergedEnvContent, err := gotenv.Marshal(newEnvMap)
if err != nil {
return appEnv, "", err
}
return appEnv, mergedEnvContent, nil
}
func getEnvMapByPath(envPath string) (map[string]interface{}, error) {
envMap := make(map[string]interface{})
envFile, err := os.ReadFile(envPath)
if err != nil {
return envMap, err
}
lines := strings.Split(string(envFile), "\n")
for _, line := range lines {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") {
continue
}
parts := strings.SplitN(line, "=", 2)
if len(parts) == 2 {
value := strings.TrimSpace(parts[1])
if len(value) >= 2 {
if (value[0] == '"' && value[len(value)-1] == '"') || (value[0] == '\'' && value[len(value)-1] == '\'') {
value = value[1 : len(value)-1]
}
}
envMap[parts[0]] = value
}
}
return envMap, nil
}
+707
View File
@@ -0,0 +1,707 @@
package service
import (
"context"
"encoding/json"
"fmt"
"io/fs"
"os"
"path"
"path/filepath"
"sort"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/model"
"github.com/1Panel-dev/1Panel/agent/app/repo"
"github.com/1Panel-dev/1Panel/agent/app/task"
"github.com/1Panel-dev/1Panel/agent/buserr"
"github.com/1Panel-dev/1Panel/agent/constant"
"github.com/1Panel-dev/1Panel/agent/global"
"github.com/1Panel-dev/1Panel/agent/i18n"
"github.com/1Panel-dev/1Panel/agent/utils/common"
"github.com/1Panel-dev/1Panel/agent/utils/compose"
dockerUtils "github.com/1Panel-dev/1Panel/agent/utils/docker"
"github.com/1Panel-dev/1Panel/agent/utils/files"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/client"
)
type composeBackupFile struct {
OriginalPath string `json:"originalPath"`
FileName string `json:"fileName"`
RelativePath string `json:"relativePath,omitempty"`
BackupPath string `json:"backupPath"`
}
type composeBackupMeta struct {
ComposeName string `json:"composeName"`
ComposePath string `json:"composePath"`
CreatedAt string `json:"createdAt"`
Files []composeBackupFile `json:"files"`
Containers []string `json:"containers"`
}
type composeBackupContext struct {
req dto.CommonBackup
composeName string
composePath string
composeFiles []string
composeDir string
fileOp files.FileOp
dockerClient *client.Client
stopped bool
backupDir string
fileName string
filePath string
tmpDir string
meta composeBackupMeta
}
type composeRecoverContext struct {
req dto.CommonRecover
fileOp files.FileOp
tmpDir string
meta composeBackupMeta
composeName string
targetDir string
composePath string
enqueued bool
}
func (u *BackupService) ComposeBackup(req dto.CommonBackup) error {
timeNow := time.Now().Format(constant.DateTimeSlimLayout) + common.RandStrAndNum(5)
fileName := req.FileName
if fileName == "" {
fileName = fmt.Sprintf("%s_%s.tar.gz", req.Name, timeNow)
}
if !strings.HasSuffix(fileName, ".tar.gz") {
fileName += ".tar.gz"
}
itemDir := fmt.Sprintf("compose/%s", req.Name)
backupDir := path.Join(global.Dir.LocalBackupDir, itemDir)
record := &model.BackupRecord{
Type: req.Type,
Name: req.Name,
SourceAccountIDs: "1",
DownloadAccountID: 1,
FileDir: itemDir,
FileName: fileName,
TaskID: req.TaskID,
Status: constant.StatusWaiting,
Description: req.Description,
}
if err := backupRepo.CreateRecord(record); err != nil {
global.LOG.Errorf("save compose backup record failed, err: %v", err)
return err
}
if err := handleComposeBackup(req, nil, record.ID, backupDir, fileName); err != nil {
backupRepo.UpdateRecordByMap(record.ID, map[string]interface{}{"status": constant.StatusFailed, "message": err.Error()})
return err
}
return nil
}
func (u *BackupService) ComposeRecover(req dto.CommonRecover) error {
return handleComposeRecover(req, nil)
}
func handleComposeBackup(req dto.CommonBackup, parentTask *task.Task, recordID uint, backupDir, fileName string) error {
composeCtx, err := newComposeBackupContext(req, backupDir, fileName)
if err != nil {
return err
}
containerNames, err := loadComposeContainerNames(composeCtx)
if err != nil {
composeCtx.close()
return err
}
backupTask := parentTask
if backupTask == nil {
backupTask, err = task.NewTaskWithOps(composeCtx.composeName, task.TaskBackup, task.TaskScopeBackup, req.TaskID, 1)
if err != nil {
return err
}
}
if req.StopBefore {
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeBackupStop"), func(t *task.Task) error {
return stepStopComposeForBackup(composeCtx)
}, func(t *task.Task) {
_ = stepStartComposeAfterBackup(composeCtx)
}, 3, time.Hour)
}
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeBackupPrepare"), func(t *task.Task) error { return stepPrepareComposeBackup(composeCtx) }, nil, 3, time.Hour)
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeBackupFiles"), func(t *task.Task) error { return stepBackupComposeFiles(composeCtx) }, nil, 3, time.Hour)
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeBackupContainers"), func(t *task.Task) error { return nil }, nil, 3, time.Hour)
for _, containerName := range containerNames {
backupFileName := fmt.Sprintf("%s.tar.gz", sanitizeComposeFileName(containerName))
backupFile := path.Join(composeCtx.tmpDir, "containers", backupFileName)
if err := handleContainerBackup(containerName, backupTask, 0, path.Dir(backupFile), path.Base(backupFile), "", "", false); err != nil {
return err
}
composeCtx.meta.Containers = append(composeCtx.meta.Containers, path.Join("containers", backupFileName))
}
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeBackupMeta"), func(t *task.Task) error { return stepWriteComposeBackupMeta(composeCtx) }, nil, 3, time.Hour)
backupTask.AddSubTaskWithOps(task.GetTaskName(composeCtx.composeName, task.TaskBackup, task.TaskScopeBackup), func(t *task.Task) error {
return stepPackComposeBackup(composeCtx)
}, nil, 3, time.Hour)
if req.StopBefore {
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeBackupStart"), func(t *task.Task) error {
return stepStartComposeAfterBackup(composeCtx)
}, nil, 3, time.Hour)
}
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeBackupCleanup"), func(t *task.Task) error {
composeCtx.close()
return nil
}, nil, 0, time.Hour)
if parentTask != nil {
return nil
}
go func() {
defer composeCtx.close()
if err := backupTask.Execute(); err != nil {
backupRepo.UpdateRecordByMap(recordID, map[string]interface{}{"status": constant.StatusFailed, "message": err.Error()})
return
}
backupRepo.UpdateRecordByMap(recordID, map[string]interface{}{"status": constant.StatusSuccess})
}()
return nil
}
func loadComposeContainerNames(composeCtx *composeBackupContext) ([]string, error) {
options := container.ListOptions{All: true}
options.Filters = filters.NewArgs(filters.Arg("label", composeProjectLabel+"="+composeCtx.composeName))
containers, err := composeCtx.dockerClient.ContainerList(context.Background(), options)
if err != nil {
return nil, err
}
names := make([]string, 0, len(containers))
for _, item := range containers {
if len(item.Names) == 0 {
continue
}
names = append(names, strings.TrimPrefix(item.Names[0], "/"))
}
sort.Strings(names)
return names, nil
}
func handleComposeRecover(req dto.CommonRecover, parentTask *task.Task) error {
var recoverCtx *composeRecoverContext
recoverTask := parentTask
var err error
if recoverTask == nil {
if isImportRecover(req) {
taskName := i18n.GetMsgByKey("TaskImport") + i18n.GetMsgByKey("Compose")
recoverTask, err = task.NewTask(taskName, task.TaskImport, task.TaskScopeBackup, req.TaskID, 1)
if err != nil {
return err
}
} else {
taskName := req.Name
if taskName == "" {
taskName = "compose"
}
recoverTask, err = task.NewTaskWithOps(taskName, task.TaskRecover, task.TaskScopeBackup, req.TaskID, 1)
if err != nil {
return err
}
}
}
timeout := loadRecoverTimeout(req.Timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeRecoverPrepare"), func(t *task.Task) error {
ctx, err := newComposeRecoverContext(req)
if err != nil {
return err
}
recoverCtx = ctx
if err := stepPrepareComposeRecover(recoverCtx); err != nil {
recoverCtx.close()
recoverCtx = nil
return err
}
return nil
}, func(t *task.Task) {
if recoverCtx != nil {
recoverCtx.close()
recoverCtx = nil
}
}, 3, timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeRecoverExtract"), func(t *task.Task) error { return stepExtractComposeRecover(recoverCtx) }, nil, 3, timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeRecoverMeta"), func(t *task.Task) error {
if err := stepLoadComposeRecoverMeta(recoverCtx); err != nil {
return err
}
t.Log(i18n.GetMsgWithMap("ComposeRecoverMetaLogName", map[string]interface{}{
"name": recoverCtx.composeName,
}))
t.Log(i18n.GetMsgWithMap("ComposeRecoverMetaLogPath", map[string]interface{}{
"backupPath": recoverCtx.meta.ComposePath,
"targetDir": recoverCtx.targetDir,
}))
t.Log(i18n.GetMsgWithMap("ComposeRecoverMetaLogCount", map[string]interface{}{
"files": len(recoverCtx.meta.Files),
"containers": len(recoverCtx.meta.Containers),
}))
return nil
}, nil, 3, timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeRecoverFiles"), func(t *task.Task) error { return stepRestoreComposeFiles(recoverCtx) }, nil, 3, timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeRecoverContainers"), func(t *task.Task) error {
if recoverCtx.enqueued {
return nil
}
containerItems := make([]string, 0, len(recoverCtx.meta.Containers))
for _, item := range recoverCtx.meta.Containers {
backupItem := item
filePath, err := safeJoinWithinBase(recoverCtx.tmpDir, backupItem)
if err != nil {
return fmt.Errorf("invalid container backup path %q, err: %v", backupItem, err)
}
if !recoverCtx.fileOp.Stat(filePath) {
return fmt.Errorf("container backup file not found: %s", backupItem)
}
containerItems = append(containerItems, backupItem)
}
for _, backupItem := range containerItems {
filePath, err := safeJoinWithinBase(recoverCtx.tmpDir, backupItem)
if err != nil {
return fmt.Errorf("invalid container backup path %q, err: %v", backupItem, err)
}
containerLabel := strings.TrimSuffix(path.Base(backupItem), ".tar.gz")
containerReq := recoverCtx.req
containerReq.Type = "container"
containerReq.Name = containerLabel
containerReq.DetailName = ""
containerReq.File = filePath
containerReq.Secret = ""
if err := handleContainerRecover(containerReq, recoverTask); err != nil {
return err
}
}
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeRecoverRecord"), func(t *task.Task) error {
return stepSaveComposeRecord(recoverCtx)
}, nil, 3, timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ComposeRecoverCleanup"), func(t *task.Task) error {
if recoverCtx != nil {
recoverCtx.close()
recoverCtx = nil
}
return nil
}, nil, 0, timeout)
recoverCtx.enqueued = true
return nil
}, nil, 3, timeout)
if parentTask != nil {
return nil
}
go func() {
_ = recoverTask.Execute()
}()
return nil
}
func newComposeBackupContext(req dto.CommonBackup, backupDir, fileName string) (*composeBackupContext, error) {
if req.Name == "" {
return nil, fmt.Errorf("compose name is required")
}
dockerClient, err := dockerUtils.NewDockerClient()
if err != nil {
return nil, err
}
composePath, composeFiles, err := loadComposePathAndFiles(req.Name, dockerClient)
if err != nil {
_ = dockerClient.Close()
return nil, err
}
filePath := path.Join(backupDir, fileName)
tmpDir := path.Join(path.Dir(filePath), strings.TrimSuffix(path.Base(filePath), ".tar.gz"))
ctx := &composeBackupContext{
req: req,
composeName: req.Name,
composePath: composePath,
composeFiles: composeFiles,
composeDir: path.Dir(composeFiles[0]),
fileOp: files.NewFileOp(),
dockerClient: dockerClient,
backupDir: backupDir,
fileName: fileName,
filePath: filePath,
tmpDir: tmpDir,
meta: composeBackupMeta{
ComposeName: req.Name,
ComposePath: composePath,
CreatedAt: time.Now().Format(constant.DateTimeLayout),
Files: make([]composeBackupFile, 0),
Containers: make([]string, 0),
},
}
return ctx, nil
}
func loadComposePathAndFiles(composeName string, dockerClient *client.Client) (string, []string, error) {
composeRecord, _ := composeRepo.GetRecord(repo.WithByName(composeName))
if composeRecord.ID == 0 {
composeRecord, _ = composeRepo.GetRecord(repo.WithByName(strings.ToLower(composeName)))
}
composePath := composeRecord.Path
if composePath == "" {
options := container.ListOptions{All: true}
options.Filters = filters.NewArgs(filters.Arg("label", composeProjectLabel))
list, err := dockerClient.ContainerList(context.Background(), options)
if err != nil {
return "", nil, err
}
if len(list) == 0 {
return "", nil, fmt.Errorf("compose %s not found", composeName)
}
var targetContainer *container.Summary
for i := range list {
if strings.EqualFold(list[i].Labels[composeProjectLabel], composeName) {
targetContainer = &list[i]
break
}
}
if targetContainer == nil {
return "", nil, fmt.Errorf("compose %s not found", composeName)
}
config := targetContainer.Labels[composeConfigLabel]
workdir := targetContainer.Labels[composeWorkdirLabel]
if len(config) != 0 && len(workdir) != 0 && strings.Contains(config, workdir) {
composePath = config
} else {
composePath = workdir
}
}
composeFiles := normalizeComposeFiles(composePath)
if len(composeFiles) == 0 {
return "", nil, fmt.Errorf("compose file not found for %s", composeName)
}
return composePath, composeFiles, nil
}
func normalizeComposeFiles(composePath string) []string {
items := strings.Split(composePath, ",")
result := make([]string, 0)
seen := make(map[string]struct{})
for _, item := range items {
item = strings.TrimSpace(item)
if item == "" {
continue
}
stat, err := os.Stat(item)
if err == nil && stat.IsDir() {
item = path.Join(item, "docker-compose.yml")
}
if _, err := os.Stat(item); err != nil {
continue
}
if _, ok := seen[item]; ok {
continue
}
seen[item] = struct{}{}
result = append(result, item)
}
sort.Strings(result)
return result
}
func (c *composeBackupContext) close() {
if c.dockerClient != nil {
_ = c.dockerClient.Close()
c.dockerClient = nil
}
if c.tmpDir != "" {
_ = os.RemoveAll(c.tmpDir)
c.tmpDir = ""
}
}
func stepPrepareComposeBackup(composeCtx *composeBackupContext) error {
if err := os.MkdirAll(composeCtx.backupDir, os.ModePerm); err != nil {
return fmt.Errorf("mkdir %s failed, err: %v", composeCtx.backupDir, err)
}
_ = os.RemoveAll(composeCtx.tmpDir)
if err := os.MkdirAll(path.Join(composeCtx.tmpDir, "compose_files"), os.ModePerm); err != nil {
return err
}
if err := os.MkdirAll(path.Join(composeCtx.tmpDir, "containers"), os.ModePerm); err != nil {
return err
}
return nil
}
func stepStopComposeForBackup(composeCtx *composeBackupContext) error {
if composeCtx.stopped {
return nil
}
options := container.ListOptions{All: false}
options.Filters = filters.NewArgs(filters.Arg("label", composeProjectLabel+"="+composeCtx.composeName))
runningList, err := composeCtx.dockerClient.ContainerList(context.Background(), options)
if err != nil {
return err
}
if len(runningList) == 0 {
return nil
}
if stdout, err := compose.Operate(composeCtx.composePath, "stop"); err != nil {
return fmt.Errorf("docker-compose stop failed, std: %s, err: %v", stdout, err)
}
composeCtx.stopped = true
return nil
}
func stepStartComposeAfterBackup(composeCtx *composeBackupContext) error {
if !composeCtx.stopped {
return nil
}
if stdout, err := compose.Up(composeCtx.composePath); err != nil {
return fmt.Errorf("docker-compose up failed, std: %s, err: %v", stdout, err)
}
composeCtx.stopped = false
return nil
}
func stepBackupComposeFiles(composeCtx *composeBackupContext) error {
for i, filePath := range composeCtx.composeFiles {
backupName := fmt.Sprintf("%02d_%s", i, path.Base(filePath))
backupPath := path.Join(composeCtx.tmpDir, "compose_files", backupName)
content, err := os.ReadFile(filePath)
if err != nil {
return err
}
if err := composeCtx.fileOp.SaveFile(backupPath, string(content), fs.ModePerm); err != nil {
return err
}
relativePath := path.Base(filePath)
if composeCtx.composeDir != "" {
rel, relErr := filepath.Rel(composeCtx.composeDir, filePath)
if relErr == nil {
rel = filepath.ToSlash(rel)
if rel != "" && rel != "." && !strings.HasPrefix(rel, "../") {
relativePath = rel
}
}
}
composeCtx.meta.Files = append(composeCtx.meta.Files, composeBackupFile{
OriginalPath: filePath,
FileName: path.Base(filePath),
RelativePath: relativePath,
BackupPath: path.Join("compose_files", backupName),
})
}
if len(composeCtx.composeFiles) != 0 {
envPath := path.Join(path.Dir(composeCtx.composeFiles[0]), ".env")
if composeCtx.fileOp.Stat(envPath) {
envContent, err := os.ReadFile(envPath)
if err != nil {
return err
}
if err := composeCtx.fileOp.SaveFile(path.Join(composeCtx.tmpDir, "compose_files", ".env"), string(envContent), fs.ModePerm); err != nil {
return err
}
}
}
return nil
}
func stepWriteComposeBackupMeta(composeCtx *composeBackupContext) error {
metaBytes, err := json.MarshalIndent(composeCtx.meta, "", " ")
if err != nil {
return err
}
return composeCtx.fileOp.SaveFile(path.Join(composeCtx.tmpDir, "compose_meta.json"), string(metaBytes), fs.ModePerm)
}
func stepPackComposeBackup(composeCtx *composeBackupContext) error {
return composeCtx.fileOp.TarGzCompressPro(true, composeCtx.tmpDir, composeCtx.filePath, composeCtx.req.Secret, "")
}
func newComposeRecoverContext(req dto.CommonRecover) (*composeRecoverContext, error) {
tmpDir := path.Join(path.Dir(req.File), strings.TrimSuffix(path.Base(req.File), ".tar.gz"))
ctx := &composeRecoverContext{
req: req,
fileOp: files.NewFileOp(),
tmpDir: tmpDir,
meta: composeBackupMeta{
Files: make([]composeBackupFile, 0),
Containers: make([]string, 0),
},
}
return ctx, nil
}
func (c *composeRecoverContext) close() {
if c.tmpDir != "" {
_ = os.RemoveAll(c.tmpDir)
}
}
func stepPrepareComposeRecover(recoverCtx *composeRecoverContext) error {
if !recoverCtx.fileOp.Stat(recoverCtx.req.File) {
return buserr.WithName("ErrFileNotFound", recoverCtx.req.File)
}
_ = os.RemoveAll(recoverCtx.tmpDir)
return nil
}
func stepExtractComposeRecover(recoverCtx *composeRecoverContext) error {
return recoverCtx.fileOp.TarGzExtractPro(recoverCtx.req.File, path.Dir(recoverCtx.req.File), recoverCtx.req.Secret)
}
func stepLoadComposeRecoverMeta(recoverCtx *composeRecoverContext) error {
metaPath := path.Join(recoverCtx.tmpDir, "compose_meta.json")
if !recoverCtx.fileOp.Stat(metaPath) {
return fmt.Errorf("compose_meta.json not found in backup file")
}
metaBytes, err := os.ReadFile(metaPath)
if err != nil {
return err
}
if err := json.Unmarshal(metaBytes, &recoverCtx.meta); err != nil {
return fmt.Errorf("unmarshal compose_meta.json failed, err: %v", err)
}
recoverCtx.composeName = strings.TrimSpace(recoverCtx.req.Name)
if recoverCtx.composeName == "" {
recoverCtx.composeName = strings.TrimSpace(recoverCtx.meta.ComposeName)
}
if recoverCtx.composeName == "" {
return fmt.Errorf("compose name not found in recover request or backup file")
}
recoverCtx.targetDir = resolveComposeRecoverTargetDir(recoverCtx.meta, recoverCtx.composeName)
return nil
}
func resolveComposeRecoverTargetDir(meta composeBackupMeta, composeName string) string {
composePath := strings.TrimSpace(meta.ComposePath)
if composePath != "" {
items := strings.Split(composePath, ",")
for _, item := range items {
p := strings.TrimSpace(item)
if p == "" {
continue
}
ext := strings.ToLower(path.Ext(p))
if ext == ".yml" || ext == ".yaml" {
return path.Dir(p)
}
return p
}
}
return path.Join(global.Dir.DataDir, "docker/compose", composeName)
}
func safeJoinWithinBase(baseDir, name string) (string, error) {
base := filepath.Clean(baseDir)
candidate := strings.TrimSpace(name)
candidate = strings.ReplaceAll(candidate, "\\", "/")
candidate = filepath.Clean(filepath.FromSlash(candidate))
if candidate == "" || candidate == "." {
return "", fmt.Errorf("invalid path: empty")
}
if filepath.IsAbs(candidate) {
return "", fmt.Errorf("invalid path %q: absolute path is not allowed", name)
}
if candidate == ".." || strings.HasPrefix(candidate, ".."+string(filepath.Separator)) {
return "", fmt.Errorf("invalid path %q: path escapes base directory", name)
}
resolved := filepath.Clean(filepath.Join(base, candidate))
rel, err := filepath.Rel(base, resolved)
if err != nil {
return "", fmt.Errorf("resolve path %q failed, err: %v", name, err)
}
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return "", fmt.Errorf("invalid path %q: path escapes base directory", name)
}
return resolved, nil
}
func stepRestoreComposeFiles(recoverCtx *composeRecoverContext) error {
if recoverCtx.targetDir != "" {
_ = os.RemoveAll(recoverCtx.targetDir)
}
if err := os.MkdirAll(recoverCtx.targetDir, os.ModePerm); err != nil {
return err
}
restored := make([]string, 0, len(recoverCtx.meta.Files))
for _, item := range recoverCtx.meta.Files {
backupPath, err := safeJoinWithinBase(recoverCtx.tmpDir, item.BackupPath)
if err != nil {
return fmt.Errorf("invalid compose backup path %q, err: %v", item.BackupPath, err)
}
if !recoverCtx.fileOp.Stat(backupPath) {
continue
}
targetName := item.FileName
if item.RelativePath != "" {
targetName = item.RelativePath
}
if targetName == "" {
targetName = path.Base(item.OriginalPath)
}
if targetName == "" {
targetName = "docker-compose.yml"
}
targetPath, err := safeJoinWithinBase(recoverCtx.targetDir, targetName)
if err != nil {
return fmt.Errorf("invalid compose target path %q, err: %v", targetName, err)
}
if err := os.MkdirAll(path.Dir(targetPath), os.ModePerm); err != nil {
return err
}
content, err := os.ReadFile(backupPath)
if err != nil {
return err
}
if err := recoverCtx.fileOp.SaveFile(targetPath, string(content), fs.ModePerm); err != nil {
return err
}
restored = append(restored, targetPath)
}
envPath := path.Join(recoverCtx.tmpDir, "compose_files", ".env")
if recoverCtx.fileOp.Stat(envPath) {
envContent, err := os.ReadFile(envPath)
if err != nil {
return err
}
if err := recoverCtx.fileOp.SaveFile(path.Join(recoverCtx.targetDir, ".env"), string(envContent), fs.ModePerm); err != nil {
return err
}
}
if len(restored) == 0 {
defaultPath := path.Join(recoverCtx.targetDir, "docker-compose.yml")
if !recoverCtx.fileOp.Stat(defaultPath) {
return fmt.Errorf("compose file not found in backup data")
}
restored = append(restored, defaultPath)
}
sort.Strings(restored)
recoverCtx.composePath = strings.Join(restored, ",")
return nil
}
func stepSaveComposeRecord(recoverCtx *composeRecoverContext) error {
if recoverCtx.composePath == "" {
recoverCtx.composePath = path.Join(recoverCtx.targetDir, "docker-compose.yml")
}
recordName := strings.ToLower(recoverCtx.composeName)
record, _ := composeRepo.GetRecord(repo.WithByName(recordName))
if record.ID == 0 {
return composeRepo.CreateRecord(&model.Compose{Name: recordName, Path: recoverCtx.composePath})
}
return composeRepo.UpdateRecord(recordName, map[string]interface{}{"path": recoverCtx.composePath})
}
func sanitizeComposeFileName(in string) string {
name := strings.TrimSpace(in)
name = strings.ReplaceAll(name, "/", "_")
name = strings.ReplaceAll(name, ":", "_")
if name == "" {
return "container"
}
return name
}
+914
View File
@@ -0,0 +1,914 @@
package service
import (
"context"
"encoding/json"
"fmt"
"io/fs"
"os"
"path"
"sort"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/model"
"github.com/1Panel-dev/1Panel/agent/app/task"
"github.com/1Panel-dev/1Panel/agent/buserr"
"github.com/1Panel-dev/1Panel/agent/constant"
"github.com/1Panel-dev/1Panel/agent/global"
"github.com/1Panel-dev/1Panel/agent/i18n"
"github.com/1Panel-dev/1Panel/agent/utils/common"
dockerUtils "github.com/1Panel-dev/1Panel/agent/utils/docker"
"github.com/1Panel-dev/1Panel/agent/utils/files"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/mount"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/volume"
"github.com/docker/docker/client"
)
type containerBackupMeta struct {
ContainerName string `json:"containerName"`
ContainerID string `json:"containerID"`
CreatedAt string `json:"createdAt"`
Image string `json:"image"`
HostConfig *container.HostConfig `json:"hostConfig,omitempty"`
Config *container.Config `json:"config,omitempty"`
Mounts []containerMountBackup `json:"mounts"`
}
type containerMountBackup struct {
Type string `json:"type"`
Name string `json:"name,omitempty"`
Source string `json:"source,omitempty"`
Destination string `json:"destination"`
Mode string `json:"mode,omitempty"`
RW bool `json:"rw"`
Propagation string `json:"propagation,omitempty"`
BackupPath string `json:"backupPath,omitempty"`
Status string `json:"status"`
Message string `json:"message,omitempty"`
}
type containerBackupContext struct {
containerName string
backupDir string
fileName string
secret string
filePath string
tmpDir string
mountRoot string
wasRunning bool
stopped bool
fileOp files.FileOp
inspectInfo container.InspectResponse
meta containerBackupMeta
}
type containerRecoverContext struct {
req dto.CommonRecover
targetName string
fileOp files.FileOp
client *client.Client
tmpDir string
meta containerBackupMeta
inspectInfo container.InspectResponse
shouldStart bool
createdContainerID string
}
func (u *BackupService) ContainerBackup(req dto.CommonBackup) error {
timeNow := time.Now().Format(constant.DateTimeSlimLayout) + common.RandStrAndNum(5)
fileName := req.FileName
if fileName == "" {
fileName = fmt.Sprintf("%s_%s.tar.gz", req.Name, timeNow)
}
if !strings.HasSuffix(fileName, ".tar.gz") {
fileName += ".tar.gz"
}
itemDir := fmt.Sprintf("container/%s", req.Name)
backupDir := path.Join(global.Dir.LocalBackupDir, itemDir)
record := &model.BackupRecord{
Type: req.Type,
Name: req.Name,
SourceAccountIDs: "1",
DownloadAccountID: 1,
FileDir: itemDir,
FileName: fileName,
TaskID: req.TaskID,
Status: constant.StatusWaiting,
Description: req.Description,
}
if err := backupRepo.CreateRecord(record); err != nil {
global.LOG.Errorf("save backup record failed, err: %v", err)
return err
}
if err := handleContainerBackup(req.Name, nil, record.ID, backupDir, fileName, req.TaskID, req.Secret, req.StopBefore); err != nil {
backupRepo.UpdateRecordByMap(record.ID, map[string]interface{}{"status": constant.StatusFailed, "message": err.Error()})
return err
}
return nil
}
func (u *BackupService) ContainerRecover(req dto.CommonRecover) error {
return handleContainerRecover(req, nil)
}
func handleContainerBackup(containerName string, parentTask *task.Task, recordID uint, backupDir, fileName, taskID, secret string, stopBefore bool) error {
var (
err error
backupTask *task.Task
)
backupCtx, err := newContainerBackupContext(containerName, backupDir, fileName, secret)
if err != nil {
return err
}
backupTask = parentTask
if backupTask == nil {
backupTask, err = task.NewTaskWithOps(containerName, task.TaskBackup, task.TaskScopeBackup, taskID, 1)
if err != nil {
return err
}
}
if stopBefore {
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerBackupStop"), func(t *task.Task) error {
return stepStopContainerForBackup(backupCtx)
}, func(t *task.Task) {
_ = stepStartContainerAfterBackup(backupCtx)
}, 3, time.Hour)
}
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerBackupPrepare"), func(t *task.Task) error {
t.Logf("------------------ %s ------------------", containerName)
return stepPrepareContainerBackup(backupCtx)
}, nil, 3, time.Hour)
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerBackupInspect"), func(t *task.Task) error { return stepBackupContainerInspect(backupCtx) }, nil, 3, time.Hour)
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerBackupMounts"), func(t *task.Task) error { return stepBackupContainerMounts(backupCtx) }, nil, 3, time.Hour)
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerBackupMeta"), func(t *task.Task) error { return stepWriteContainerMeta(backupCtx) }, nil, 3, time.Hour)
backupTask.AddSubTaskWithOps(task.GetTaskName(containerName, task.TaskBackup, task.TaskScopeBackup), func(t *task.Task) error { return stepPackContainerBackup(backupCtx) }, nil, 3, time.Hour)
if stopBefore {
backupTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerBackupStart"), func(t *task.Task) error {
return stepStartContainerAfterBackup(backupCtx)
}, nil, 3, time.Hour)
}
if parentTask != nil {
return nil
}
go func() {
defer backupCtx.close()
if err := backupTask.Execute(); err != nil {
backupRepo.UpdateRecordByMap(recordID, map[string]interface{}{"status": constant.StatusFailed, "message": err.Error()})
return
}
backupRepo.UpdateRecordByMap(recordID, map[string]interface{}{"status": constant.StatusSuccess})
}()
return nil
}
func handleContainerRecover(req dto.CommonRecover, parentTask *task.Task) error {
var (
err error
recoverTask *task.Task
recoverCtx *containerRecoverContext
)
recoverTask = parentTask
if recoverTask == nil {
if isImportRecover(req) {
taskName := i18n.GetMsgByKey("TaskImport") + i18n.GetMsgByKey("Container")
recoverTask, err = task.NewTask(taskName, task.TaskImport, task.TaskScopeBackup, req.TaskID, 1)
if err != nil {
return err
}
} else {
recoverTask, err = task.NewTaskWithOps("container", task.TaskRecover, task.TaskScopeBackup, req.TaskID, 1)
if err != nil {
return err
}
}
}
timeout := loadRecoverTimeout(req.Timeout)
logName := strings.TrimSpace(req.Name)
if logName == "" && req.File != "" {
logName = strings.TrimSuffix(path.Base(req.File), ".tar.gz")
}
if logName == "" {
logName = "container"
}
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerRecoverPrepare"), func(t *task.Task) error {
ctx, err := newContainerRecoverContext(req)
if err != nil {
return err
}
recoverCtx = ctx
t.Logf("------------------ %s ------------------", logName)
if err := stepPrepareContainerRecover(recoverCtx); err != nil {
recoverCtx.close()
recoverCtx = nil
return err
}
return nil
}, func(t *task.Task) {
if recoverCtx != nil {
recoverCtx.close()
recoverCtx = nil
}
}, 3, timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerRecoverExtract"), func(t *task.Task) error { return stepExtractContainerRecover(recoverCtx) }, nil, 3, timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerRecoverParse"), func(t *task.Task) error { return stepLoadContainerRecoverData(recoverCtx) }, nil, 3, timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerRecoverCreate"), func(t *task.Task) error { return stepRecreateContainer(recoverCtx, t) }, nil, 3, timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerRecoverMounts"), func(t *task.Task) error { return stepRestoreContainerMounts(recoverCtx) }, nil, 3, timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerRecoverStart"), func(t *task.Task) error { return stepStartRecoveredContainer(recoverCtx) }, nil, 3, timeout)
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("ContainerRecoverCleanup"), func(t *task.Task) error {
if recoverCtx != nil {
recoverCtx.close()
recoverCtx = nil
}
return nil
}, nil, 0, timeout)
if parentTask != nil {
return nil
}
go func() {
_ = recoverTask.Execute()
}()
return nil
}
func loadRecoverTimeout(timeout int) time.Duration {
switch timeout {
case -1:
return 0
case 0:
return 3 * time.Hour
default:
return time.Duration(timeout) * time.Second
}
}
func isImportRecover(req dto.CommonRecover) bool {
return req.BackupRecordID == 0
}
func newContainerBackupContext(containerName, backupDir, fileName, secret string) (*containerBackupContext, error) {
dockerClient, err := dockerUtils.NewDockerClient()
if err != nil {
return nil, err
}
defer func() {
_ = dockerClient.Close()
}()
inspectInfo, err := dockerClient.ContainerInspect(context.Background(), containerName)
if err != nil {
return nil, err
}
filePath := path.Join(backupDir, fileName)
tmpDir := path.Join(path.Dir(filePath), strings.TrimSuffix(path.Base(filePath), ".tar.gz"))
backupCtx := &containerBackupContext{
containerName: containerName,
backupDir: backupDir,
fileName: fileName,
secret: secret,
filePath: filePath,
tmpDir: tmpDir,
mountRoot: path.Join(tmpDir, "mounts"),
wasRunning: inspectInfo.State != nil && inspectInfo.State.Running,
fileOp: files.NewFileOp(),
inspectInfo: inspectInfo,
meta: containerBackupMeta{
ContainerName: containerName,
ContainerID: inspectInfo.ID,
CreatedAt: time.Now().Format(constant.DateTimeLayout),
Image: inspectInfo.Config.Image,
HostConfig: inspectInfo.HostConfig,
Config: inspectInfo.Config,
Mounts: make([]containerMountBackup, 0),
},
}
return backupCtx, nil
}
func newContainerRecoverContext(req dto.CommonRecover) (*containerRecoverContext, error) {
dockerClient, err := dockerUtils.NewDockerClient()
if err != nil {
return nil, err
}
tmpDir := path.Join(path.Dir(req.File), strings.TrimSuffix(path.Base(req.File), ".tar.gz"))
ctx := &containerRecoverContext{
req: req,
targetName: req.Name,
fileOp: files.NewFileOp(),
client: dockerClient,
tmpDir: tmpDir,
meta: containerBackupMeta{
Mounts: make([]containerMountBackup, 0),
},
}
return ctx, nil
}
func (c *containerBackupContext) close() {
if c.tmpDir != "" {
_ = os.RemoveAll(c.tmpDir)
}
}
func (c *containerRecoverContext) close() {
if c.client != nil {
_ = c.client.Close()
}
if c.tmpDir != "" {
_ = os.RemoveAll(c.tmpDir)
}
}
func stepPrepareContainerBackup(backupCtx *containerBackupContext) error {
if err := os.MkdirAll(backupCtx.backupDir, os.ModePerm); err != nil {
return fmt.Errorf("mkdir %s failed, err: %v", backupCtx.backupDir, err)
}
_ = os.RemoveAll(backupCtx.tmpDir)
if err := os.MkdirAll(backupCtx.mountRoot, os.ModePerm); err != nil {
return err
}
return nil
}
func stepStopContainerForBackup(backupCtx *containerBackupContext) error {
if !backupCtx.wasRunning || backupCtx.stopped {
return nil
}
dockerClient, err := dockerUtils.NewDockerClient()
if err != nil {
return err
}
defer func() {
_ = dockerClient.Close()
}()
if err := dockerClient.ContainerStop(context.Background(), backupCtx.inspectInfo.ID, container.StopOptions{}); err != nil {
return err
}
backupCtx.stopped = true
return nil
}
func stepStartContainerAfterBackup(backupCtx *containerBackupContext) error {
if !backupCtx.stopped {
return nil
}
dockerClient, err := dockerUtils.NewDockerClient()
if err != nil {
return err
}
defer func() {
_ = dockerClient.Close()
}()
if err := dockerClient.ContainerStart(context.Background(), backupCtx.inspectInfo.ID, container.StartOptions{}); err != nil {
return err
}
backupCtx.stopped = false
return nil
}
func stepBackupContainerInspect(backupCtx *containerBackupContext) error {
inspectBytes, err := json.MarshalIndent(backupCtx.inspectInfo, "", " ")
if err != nil {
return err
}
if err := backupCtx.fileOp.SaveFile(path.Join(backupCtx.tmpDir, "inspect.json"), string(inspectBytes), fs.ModePerm); err != nil {
return err
}
if backupCtx.inspectInfo.NetworkSettings != nil {
networkBytes, err := json.MarshalIndent(backupCtx.inspectInfo.NetworkSettings, "", " ")
if err != nil {
return err
}
if err := backupCtx.fileOp.SaveFile(path.Join(backupCtx.tmpDir, "network.json"), string(networkBytes), fs.ModePerm); err != nil {
return err
}
}
return nil
}
func stepBackupContainerMounts(backupCtx *containerBackupContext) error {
var (
dockerClient *client.Client
clientErr error
)
ensureClient := func() (*client.Client, error) {
if dockerClient != nil || clientErr != nil {
return dockerClient, clientErr
}
dockerClient, clientErr = dockerUtils.NewDockerClient()
return dockerClient, clientErr
}
defer func() {
if dockerClient != nil {
_ = dockerClient.Close()
}
}()
for i, item := range backupCtx.inspectInfo.Mounts {
mountMeta := containerMountBackup{
Type: string(item.Type),
Name: item.Name,
Source: item.Source,
Destination: item.Destination,
Mode: item.Mode,
RW: item.RW,
Propagation: string(item.Propagation),
Status: "skipped",
}
mountDirName := fmt.Sprintf("%02d_%s", i, sanitizeContainerMountName(item.Destination))
mountDir := path.Join(backupCtx.mountRoot, mountDirName)
mountMeta.BackupPath = path.Join("mounts", mountDirName, "data")
switch item.Type {
case mount.TypeBind, mount.TypeVolume:
if item.Source == "" {
mountMeta.Message = "empty source"
backupCtx.meta.Mounts = append(backupCtx.meta.Mounts, mountMeta)
continue
}
sourceInfo, statErr := os.Stat(item.Source)
if statErr != nil {
mountMeta.Message = statErr.Error()
backupCtx.meta.Mounts = append(backupCtx.meta.Mounts, mountMeta)
continue
}
dataDir := path.Join(mountDir, "data")
if err := os.MkdirAll(dataDir, os.ModePerm); err != nil {
return err
}
if sourceInfo.IsDir() {
if err := backupCtx.fileOp.CopyDirWithNewName(item.Source, dataDir, "."); err != nil {
return err
}
} else {
if err := backupCtx.fileOp.CopyFile(item.Source, dataDir); err != nil {
return err
}
}
if item.Type == mount.TypeVolume && item.Name != "" {
cli, err := ensureClient()
if err != nil {
return err
}
volumeInfo, volumeErr := cli.VolumeInspect(context.Background(), item.Name)
if volumeErr == nil {
volumeBytes, volumeMarshalErr := json.MarshalIndent(volumeInfo, "", " ")
if volumeMarshalErr == nil {
_ = backupCtx.fileOp.SaveFile(path.Join(mountDir, "volume.json"), string(volumeBytes), fs.ModePerm)
}
}
}
mountMeta.Status = "backed_up"
default:
mountMeta.Message = "mount type not supported for data backup"
}
backupCtx.meta.Mounts = append(backupCtx.meta.Mounts, mountMeta)
}
return nil
}
func stepWriteContainerMeta(backupCtx *containerBackupContext) error {
metaBytes, err := json.MarshalIndent(backupCtx.meta, "", " ")
if err != nil {
return err
}
if err := backupCtx.fileOp.SaveFile(path.Join(backupCtx.tmpDir, "meta.json"), string(metaBytes), fs.ModePerm); err != nil {
return err
}
return nil
}
func stepPackContainerBackup(backupCtx *containerBackupContext) error {
if err := backupCtx.fileOp.TarGzCompressPro(true, backupCtx.tmpDir, backupCtx.filePath, backupCtx.secret, ""); err != nil {
return err
}
return nil
}
func stepPrepareContainerRecover(recoverCtx *containerRecoverContext) error {
if !recoverCtx.fileOp.Stat(recoverCtx.req.File) {
return buserr.WithName("ErrFileNotFound", recoverCtx.req.File)
}
_ = os.RemoveAll(recoverCtx.tmpDir)
return nil
}
func stepExtractContainerRecover(recoverCtx *containerRecoverContext) error {
return recoverCtx.fileOp.TarGzExtractPro(recoverCtx.req.File, path.Dir(recoverCtx.req.File), recoverCtx.req.Secret)
}
func stepLoadContainerRecoverData(recoverCtx *containerRecoverContext) error {
if err := loadContainerRecoverMeta(recoverCtx); err != nil {
return err
}
if err := loadContainerRecoverInspect(recoverCtx); err != nil {
return err
}
if recoverCtx.targetName == "" {
recoverCtx.targetName = strings.TrimPrefix(recoverCtx.inspectInfo.Name, "/")
}
if recoverCtx.targetName == "" {
recoverCtx.targetName = recoverCtx.meta.ContainerName
}
if recoverCtx.targetName == "" {
return fmt.Errorf("container name not found in recover request or backup file")
}
if recoverCtx.inspectInfo.Config == nil {
recoverCtx.inspectInfo.Config = recoverCtx.meta.Config
}
if recoverCtx.inspectInfo.HostConfig == nil {
recoverCtx.inspectInfo.HostConfig = recoverCtx.meta.HostConfig
}
if recoverCtx.inspectInfo.Config == nil {
return fmt.Errorf("container config not found in backup file")
}
if recoverCtx.inspectInfo.HostConfig == nil {
recoverCtx.inspectInfo.HostConfig = &container.HostConfig{}
}
recoverCtx.shouldStart = recoverCtx.inspectInfo.State != nil && recoverCtx.inspectInfo.State.Running
return nil
}
func loadContainerRecoverMeta(recoverCtx *containerRecoverContext) error {
metaPath := path.Join(recoverCtx.tmpDir, "meta.json")
if !recoverCtx.fileOp.Stat(metaPath) {
return nil
}
metaBytes, err := os.ReadFile(metaPath)
if err != nil {
return err
}
if err := json.Unmarshal(metaBytes, &recoverCtx.meta); err != nil {
return fmt.Errorf("unmarshal meta.json failed, err: %v", err)
}
return nil
}
func loadContainerRecoverInspect(recoverCtx *containerRecoverContext) error {
inspectPath := path.Join(recoverCtx.tmpDir, "inspect.json")
if !recoverCtx.fileOp.Stat(inspectPath) {
return fmt.Errorf("inspect.json not found in backup file")
}
inspectBytes, err := os.ReadFile(inspectPath)
if err != nil {
return err
}
if err := json.Unmarshal(inspectBytes, &recoverCtx.inspectInfo); err != nil {
return fmt.Errorf("unmarshal inspect.json failed, err: %v", err)
}
return nil
}
func stepRecreateContainer(recoverCtx *containerRecoverContext, taskItem *task.Task) error {
ctx := context.Background()
if err := ensureContainerRecoverNetworks(recoverCtx); err != nil {
return err
}
if err := ensureContainerRecoverVolumes(recoverCtx); err != nil {
return err
}
config := cloneContainerConfig(recoverCtx.inspectInfo.Config)
hostConfig := cloneContainerHostConfig(recoverCtx.inspectInfo.HostConfig)
if config.Image == "" {
config.Image = recoverCtx.meta.Image
}
if config.Image == "" {
return fmt.Errorf("container image not found in backup file")
}
if !checkImageExist(recoverCtx.client, config.Image) {
if err := pullImages(taskItem, recoverCtx.client, config.Image); err != nil {
return err
}
}
if _, err := recoverCtx.client.ContainerInspect(ctx, recoverCtx.targetName); err == nil {
if err := recoverCtx.client.ContainerRemove(ctx, recoverCtx.targetName, container.RemoveOptions{Force: true, RemoveVolumes: false}); err != nil {
return err
}
} else if !client.IsErrNotFound(err) {
return err
}
networkConfig, extraNetworks := buildContainerRecoverNetworkConfig(recoverCtx.inspectInfo.NetworkSettings, hostConfig)
removeBridgeDriverIPAM(recoverCtx.client, networkConfig, extraNetworks)
createRes, err := recoverCtx.client.ContainerCreate(ctx, config, hostConfig, networkConfig, nil, recoverCtx.targetName)
if err != nil {
return err
}
recoverCtx.createdContainerID = createRes.ID
extraNames := make([]string, 0, len(extraNetworks))
for name := range extraNetworks {
extraNames = append(extraNames, name)
}
sort.Strings(extraNames)
for _, item := range extraNames {
if err := recoverCtx.client.NetworkConnect(ctx, item, recoverCtx.createdContainerID, extraNetworks[item]); err != nil {
return err
}
}
return nil
}
func removeBridgeDriverIPAM(cli *client.Client, primary *network.NetworkingConfig, extras map[string]*network.EndpointSettings) {
if primary != nil {
removeBridgeDriverIPAMFromEndpoints(cli, primary.EndpointsConfig)
}
removeBridgeDriverIPAMFromEndpoints(cli, extras)
}
func removeBridgeDriverIPAMFromEndpoints(cli *client.Client, endpoints map[string]*network.EndpointSettings) {
for netName, endpoint := range endpoints {
if endpoint == nil || endpoint.IPAMConfig == nil {
continue
}
info, err := cli.NetworkInspect(context.Background(), netName, network.InspectOptions{})
if err != nil {
continue
}
if info.Driver == "bridge" {
endpoint.IPAMConfig = nil
}
}
}
func ensureContainerRecoverNetworks(recoverCtx *containerRecoverContext) error {
if recoverCtx.inspectInfo.NetworkSettings == nil {
return nil
}
for netName := range recoverCtx.inspectInfo.NetworkSettings.Networks {
if netName == "" || netName == "bridge" || netName == "host" || netName == "none" {
continue
}
if _, err := recoverCtx.client.NetworkInspect(context.Background(), netName, network.InspectOptions{}); err != nil {
if !client.IsErrNotFound(err) {
return err
}
if _, err := recoverCtx.client.NetworkCreate(context.Background(), netName, network.CreateOptions{Driver: "bridge"}); err != nil {
return err
}
}
}
return nil
}
func ensureContainerRecoverVolumes(recoverCtx *containerRecoverContext) error {
for _, item := range recoverCtx.meta.Mounts {
if item.Type != string(mount.TypeVolume) || item.Name == "" {
continue
}
if _, err := recoverCtx.client.VolumeInspect(context.Background(), item.Name); err == nil {
continue
} else if !client.IsErrNotFound(err) {
return err
}
createOptions := volume.CreateOptions{Name: item.Name}
if item.BackupPath != "" {
volumeMetaPath := path.Join(recoverCtx.tmpDir, path.Dir(item.BackupPath), "volume.json")
if recoverCtx.fileOp.Stat(volumeMetaPath) {
volumeBytes, readErr := os.ReadFile(volumeMetaPath)
if readErr != nil {
return readErr
}
var volumeInfo volume.Volume
if unmarshalErr := json.Unmarshal(volumeBytes, &volumeInfo); unmarshalErr != nil {
return unmarshalErr
}
if volumeInfo.Driver != "" {
createOptions.Driver = volumeInfo.Driver
}
if len(volumeInfo.Options) != 0 {
createOptions.DriverOpts = volumeInfo.Options
}
if len(volumeInfo.Labels) != 0 {
createOptions.Labels = volumeInfo.Labels
}
}
}
if _, err := recoverCtx.client.VolumeCreate(context.Background(), createOptions); err != nil {
return err
}
}
return nil
}
func buildContainerRecoverNetworkConfig(networkSettings *container.NetworkSettings, hostConfig *container.HostConfig) (*network.NetworkingConfig, map[string]*network.EndpointSettings) {
extraNetworks := make(map[string]*network.EndpointSettings)
if hostConfig != nil {
networkMode := string(hostConfig.NetworkMode)
if networkMode == "host" || networkMode == "none" {
return nil, extraNetworks
}
}
if networkSettings == nil || len(networkSettings.Networks) == 0 {
return nil, extraNetworks
}
primaryName := ""
if hostConfig != nil {
networkMode := string(hostConfig.NetworkMode)
if networkMode != "" && networkMode != "default" && networkMode != "bridge" {
if _, ok := networkSettings.Networks[networkMode]; ok {
primaryName = networkMode
}
}
}
if primaryName == "" {
if _, ok := networkSettings.Networks["bridge"]; ok {
primaryName = "bridge"
} else {
names := make([]string, 0, len(networkSettings.Networks))
for name := range networkSettings.Networks {
names = append(names, name)
}
sort.Strings(names)
if len(names) > 0 {
primaryName = names[0]
}
}
}
config := &network.NetworkingConfig{EndpointsConfig: make(map[string]*network.EndpointSettings)}
for name, endpoint := range networkSettings.Networks {
if name == "host" || name == "none" {
continue
}
endpointSetting := &network.EndpointSettings{Aliases: append([]string(nil), endpoint.Aliases...), MacAddress: endpoint.MacAddress}
if endpoint.IPAMConfig != nil {
endpointSetting.IPAMConfig = &network.EndpointIPAMConfig{
IPv4Address: endpoint.IPAMConfig.IPv4Address,
IPv6Address: endpoint.IPAMConfig.IPv6Address,
}
} else if endpoint.IPAddress != "" || endpoint.GlobalIPv6Address != "" {
endpointSetting.IPAMConfig = &network.EndpointIPAMConfig{
IPv4Address: endpoint.IPAddress,
IPv6Address: endpoint.GlobalIPv6Address,
}
}
if name == primaryName {
config.EndpointsConfig[name] = endpointSetting
} else {
extraNetworks[name] = endpointSetting
}
}
if len(config.EndpointsConfig) == 0 {
return nil, extraNetworks
}
return config, extraNetworks
}
func cloneContainerConfig(config *container.Config) *container.Config {
if config == nil {
return &container.Config{}
}
item := *config
if len(config.Env) != 0 {
item.Env = append([]string(nil), config.Env...)
}
if len(config.Cmd) != 0 {
item.Cmd = append([]string(nil), config.Cmd...)
}
if len(config.Entrypoint) != 0 {
item.Entrypoint = append([]string(nil), config.Entrypoint...)
}
if len(config.Labels) != 0 {
labels := make(map[string]string, len(config.Labels))
for key, val := range config.Labels {
labels[key] = val
}
item.Labels = labels
}
if len(config.Volumes) != 0 {
volumes := make(map[string]struct{}, len(config.Volumes))
for key, val := range config.Volumes {
volumes[key] = val
}
item.Volumes = volumes
}
return &item
}
func cloneContainerHostConfig(hostConfig *container.HostConfig) *container.HostConfig {
if hostConfig == nil {
return &container.HostConfig{}
}
item := *hostConfig
if len(hostConfig.Binds) != 0 {
item.Binds = append([]string(nil), hostConfig.Binds...)
}
if len(hostConfig.DNS) != 0 {
item.DNS = append([]string(nil), hostConfig.DNS...)
}
if len(hostConfig.ExtraHosts) != 0 {
item.ExtraHosts = append([]string(nil), hostConfig.ExtraHosts...)
}
if len(hostConfig.Mounts) != 0 {
item.Mounts = append([]mount.Mount(nil), hostConfig.Mounts...)
}
return &item
}
func stepRestoreContainerMounts(recoverCtx *containerRecoverContext) error {
currentContainer := recoverCtx.createdContainerID
if currentContainer == "" {
currentContainer = recoverCtx.targetName
}
currentInspect, err := recoverCtx.client.ContainerInspect(context.Background(), currentContainer)
if err != nil {
return err
}
currentMounts := make(map[string]container.MountPoint, len(currentInspect.Mounts))
for _, item := range currentInspect.Mounts {
currentMounts[item.Destination] = item
}
for _, item := range recoverCtx.meta.Mounts {
if item.Status != "backed_up" || item.BackupPath == "" || !item.RW {
continue
}
backupPath := path.Join(recoverCtx.tmpDir, item.BackupPath)
if !recoverCtx.fileOp.Stat(backupPath) {
continue
}
sourcePath := item.Source
if currentMount, ok := currentMounts[item.Destination]; ok {
if currentMount.Source != "" {
sourcePath = currentMount.Source
}
if item.Type == string(mount.TypeVolume) && item.Name == "" {
item.Name = currentMount.Name
}
}
if sourcePath == "" && item.Type == string(mount.TypeVolume) && item.Name != "" {
volumeInfo, volumeErr := recoverCtx.client.VolumeInspect(context.Background(), item.Name)
if volumeErr != nil {
return volumeErr
}
sourcePath = volumeInfo.Mountpoint
}
if sourcePath == "" {
continue
}
if err := restoreContainerMountData(recoverCtx.fileOp, backupPath, sourcePath); err != nil {
return err
}
}
return nil
}
func restoreContainerMountData(fileOp files.FileOp, backupPath, sourcePath string) error {
if sourcePath == "/" {
return fmt.Errorf("invalid mount source path /")
}
entries, err := os.ReadDir(backupPath)
if err != nil {
return err
}
if len(entries) == 1 && !entries[0].IsDir() && entries[0].Name() == path.Base(sourcePath) {
if err := os.MkdirAll(path.Dir(sourcePath), os.ModePerm); err != nil {
return err
}
_ = os.RemoveAll(sourcePath)
if err := fileOp.CopyFile(path.Join(backupPath, entries[0].Name()), path.Dir(sourcePath)); err != nil {
return err
}
return nil
}
_ = os.RemoveAll(sourcePath)
if err := os.MkdirAll(sourcePath, os.ModePerm); err != nil {
return err
}
if err := fileOp.CopyDirWithNewName(backupPath, sourcePath, "."); err != nil {
return err
}
return nil
}
func stepStartRecoveredContainer(recoverCtx *containerRecoverContext) error {
if !recoverCtx.shouldStart {
return nil
}
containerID := recoverCtx.createdContainerID
if containerID == "" {
containerID = recoverCtx.targetName
}
return recoverCtx.client.ContainerStart(context.Background(), containerID, container.StartOptions{})
}
func sanitizeContainerMountName(in string) string {
name := strings.TrimSpace(in)
name = strings.Trim(name, "/")
name = strings.ReplaceAll(name, "/", "_")
name = strings.ReplaceAll(name, ":", "_")
if name == "" {
return "root"
}
return name
}
+588
View File
@@ -0,0 +1,588 @@
package service
import (
"bytes"
"context"
"fmt"
"io"
"net/url"
"os"
"os/exec"
"path"
"regexp"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/model"
"github.com/1Panel-dev/1Panel/agent/app/repo"
"github.com/1Panel-dev/1Panel/agent/app/task"
"github.com/1Panel-dev/1Panel/agent/buserr"
"github.com/1Panel-dev/1Panel/agent/constant"
"github.com/1Panel-dev/1Panel/agent/global"
"github.com/1Panel-dev/1Panel/agent/i18n"
"github.com/1Panel-dev/1Panel/agent/utils/cmd"
"github.com/1Panel-dev/1Panel/agent/utils/common"
"github.com/1Panel-dev/1Panel/agent/utils/files"
dockerImage "github.com/docker/docker/api/types/image"
dockerClient "github.com/docker/docker/client"
)
const opensslSaltedHeader = "Salted__"
func (u *BackupService) MongodbBackup(req dto.CommonBackup) error {
timeNow := time.Now().Format(constant.DateTimeSlimLayout)
itemDir := fmt.Sprintf("database/%s/%s/%s", req.Type, req.Name, req.DetailName)
targetDir := path.Join(global.Dir.LocalBackupDir, itemDir)
fileName := fmt.Sprintf("%s_%s.gz", req.DetailName, timeNow+common.RandStrAndNum(5))
record := &model.BackupRecord{
Type: req.Type,
Name: req.Name,
DetailName: req.DetailName,
SourceAccountIDs: "1",
DownloadAccountID: 1,
FileDir: itemDir,
FileName: fileName,
TaskID: req.TaskID,
Status: constant.StatusWaiting,
Description: req.Description,
}
if err := backupRepo.CreateRecord(record); err != nil {
global.LOG.Errorf("save backup record failed, err: %v", err)
return err
}
if err := handleMongodbBackup(req, nil, record.ID, targetDir, fileName); err != nil {
backupRepo.UpdateRecordByMap(record.ID, map[string]interface{}{"status": constant.StatusFailed, "message": err.Error()})
return err
}
return nil
}
func (u *BackupService) MongodbRecover(req dto.CommonRecover) error {
return handleMongodbRecover(req, nil, false)
}
func (u *BackupService) MongodbRecoverByUpload(req dto.CommonRecover) error {
return handleMongodbRecover(req, nil, false)
}
func handleMongodbBackup(req dto.CommonBackup, parentTask *task.Task, recordID uint, targetDir, fileName string) error {
dbItem, err := mongodbRepo.Get(repo.WithByName(req.DetailName), mongodbRepo.WithByMongodbName(req.Name))
if err != nil {
return err
}
itemName := fmt.Sprintf("%s[%s] - %s", req.Name, req.Type, req.DetailName)
backupTask := parentTask
if backupTask == nil {
backupTask, err = task.NewTaskWithOps(itemName, task.TaskBackup, task.TaskScopeBackup, req.TaskID, dbItem.ID)
if err != nil {
return err
}
}
itemHandler := func(t *task.Task) error {
return doMongodbBackup(req.Name, req.Type, req.DetailName, targetDir, fileName, req.Secret, t)
}
if parentTask != nil {
return itemHandler(parentTask)
}
backupTask.AddSubTaskWithOps(
task.GetTaskName(itemName, task.TaskBackup, task.TaskScopeBackup),
func(t *task.Task) error { return itemHandler(t) },
nil,
0,
3*time.Hour,
)
go func() {
if err := backupTask.Execute(); err != nil {
backupRepo.UpdateRecordByMap(recordID, map[string]interface{}{"status": constant.StatusFailed, "message": err.Error()})
return
}
backupRepo.UpdateRecordByMap(recordID, map[string]interface{}{"status": constant.StatusSuccess})
}()
return nil
}
func handleMongodbRecover(req dto.CommonRecover, parentTask *task.Task, isRollback bool) error {
dbItem, err := mongodbRepo.Get(repo.WithByName(req.DetailName), mongodbRepo.WithByMongodbName(req.Name))
if err != nil {
return err
}
itemName := fmt.Sprintf("%s[%s] - %s", req.Name, req.Type, req.DetailName)
recoverTask := parentTask
if recoverTask == nil {
recoverTask, err = task.NewTaskWithOps(itemName, task.TaskRecover, task.TaskScopeBackup, req.TaskID, dbItem.ID)
if err != nil {
return err
}
}
recoverDatabase := func(t *task.Task) error {
fileOp := files.NewFileOp()
if !fileOp.Stat(req.File) {
return buserr.WithName("ErrFileNotFound", req.File)
}
restoreFile, cleanup, err := prepareMongodbBackupFileForRestore(req.File, req.Secret, t)
if err != nil {
return err
}
defer cleanup()
isOk := false
if !isRollback {
rollbackFile := path.Join(
global.Dir.TmpDir,
fmt.Sprintf("database/%s/%s_%s.gz", req.Type, req.DetailName, time.Now().Format(constant.DateTimeSlimLayout)),
)
if err := doMongodbBackup(req.Name, req.Type, req.DetailName, path.Dir(rollbackFile), path.Base(rollbackFile), "", t); err != nil {
return fmt.Errorf("backup mongodb db %s for rollback before recover failed, err: %v", req.DetailName, err)
}
defer func() {
if !isOk {
global.LOG.Info("recover failed, start to rollback now")
if err := doMongodbRestore(req.Name, req.Type, req.DetailName, rollbackFile, t); err != nil {
global.LOG.Errorf("rollback mongodb db %s from %s failed, err: %v", req.DetailName, rollbackFile, err)
} else {
global.LOG.Infof("rollback mongodb db %s from %s successful", req.DetailName, rollbackFile)
}
}
_ = os.RemoveAll(rollbackFile)
}()
}
if req.DropAllCollections {
if err := clearMongodbDatabase(req.Name, req.Type, req.DetailName, t); err != nil {
return err
}
}
if err := doMongodbRestore(req.Name, req.Type, req.DetailName, restoreFile, t); err != nil {
global.LOG.Errorf("recover mongodb db %s from %s failed, err: %v", req.DetailName, restoreFile, err)
return err
}
isOk = true
return nil
}
if parentTask != nil {
return recoverDatabase(parentTask)
}
var timeout time.Duration
switch req.Timeout {
case -1:
timeout = 0
case 0:
timeout = 3 * time.Hour
default:
timeout = time.Duration(req.Timeout) * time.Second
}
recoverTask.AddSubTaskWithOps(i18n.GetMsgByKey("TaskRecover"), recoverDatabase, nil, 0, timeout)
go func() {
_ = recoverTask.Execute()
}()
return nil
}
func doMongodbBackup(database, dbType, dbName, targetDir, fileName, secret string, taskItem *task.Task) error {
dbItem, err := mongodbRepo.Get(repo.WithByName(dbName), mongodbRepo.WithByMongodbName(database))
if err == nil && dbItem.From == constant.AppResourceRemote {
if err := doRemoteMongodbBackup(database, dbName, targetDir, fileName, taskItem); err != nil {
return err
}
if len(secret) != 0 {
return files.OpensslEncrypt(path.Join(targetDir, fileName), secret)
}
return nil
}
appInfo, err := appInstallRepo.LoadBaseInfo(dbType, database)
if err != nil {
return err
}
if appInfo.ContainerName == "" {
return fmt.Errorf("mongodb container not found for database %s", database)
}
if err := os.MkdirAll(targetDir, constant.DirPerm); err != nil {
return err
}
targetFile := path.Join(targetDir, fileName)
containerFile := path.Join("/tmp", fileName)
defer func() {
_ = cmd.NewCommandMgr().Run("docker", "exec", appInfo.ContainerName, "rm", "-f", containerFile)
}()
uri := buildMongodbDumpURI(appInfo.UserName, appInfo.Password, dbName)
cmdMgr := mongodbCmdMgr(taskItem)
if err := cmdMgr.Run(
"docker",
"exec",
appInfo.ContainerName,
"mongodump",
"--uri="+uri,
"--archive="+containerFile,
"--gzip",
); err != nil {
return err
}
if err := cmdMgr.Run("docker", "cp", fmt.Sprintf("%s:%s", appInfo.ContainerName, containerFile), targetFile); err != nil {
return err
}
if len(secret) != 0 {
return files.OpensslEncrypt(targetFile, secret)
}
return nil
}
func doMongodbRestore(database, dbType, dbName, sourceFile string, taskItem *task.Task) error {
dbItem, err := mongodbRepo.Get(repo.WithByName(dbName), mongodbRepo.WithByMongodbName(database))
if err == nil && dbItem.From == constant.AppResourceRemote {
return doRemoteMongodbRestore(database, dbName, sourceFile, taskItem)
}
appInfo, err := appInstallRepo.LoadBaseInfo(dbType, database)
if err != nil {
return err
}
if appInfo.ContainerName == "" {
return fmt.Errorf("mongodb container not found for database %s", database)
}
containerFile := path.Join("/tmp", fmt.Sprintf("1panel-mongodb-restore-%s.gz", common.RandStrAndNum(8)))
defer func() {
_ = cmd.NewCommandMgr().Run("docker", "exec", appInfo.ContainerName, "rm", "-f", containerFile)
}()
cmdMgr := mongodbCmdMgr(taskItem)
if err := cmdMgr.Run("docker", "cp", sourceFile, fmt.Sprintf("%s:%s", appInfo.ContainerName, containerFile)); err != nil {
return err
}
uri := buildMongodbRestoreURI(appInfo.UserName, appInfo.Password)
if err := cmdMgr.Run(
"docker",
"exec",
appInfo.ContainerName,
"mongorestore",
"--uri="+uri,
"--nsInclude="+buildMongodbNamespace(sourceFile, dbName),
"--nsFrom="+buildMongodbNamespace(sourceFile, dbName),
"--nsTo="+dbName+".*",
"--archive="+containerFile,
"--gzip",
"--drop",
); err != nil {
return err
}
return nil
}
func prepareMongodbBackupFileForRestore(filePath, secret string, taskItem *task.Task) (string, func(), error) {
isEncrypted, err := isOpenSSLEncryptedMongodbBackup(filePath)
if err != nil {
return "", nil, err
}
if !isEncrypted {
return filePath, func() {}, nil
}
if secret == "" {
return "", nil, buserr.New("ErrBadDecrypt")
}
if err := files.OpensslDecrypt(filePath, secret); err != nil {
return "", nil, err
}
restoreFile := path.Join(path.Dir(filePath), "tmp_"+path.Base(filePath))
taskItem.LogWithStatus(i18n.GetMsgByKey("Decrypt"), nil)
return restoreFile, func() { _ = os.Remove(restoreFile) }, nil
}
func isOpenSSLEncryptedMongodbBackup(filePath string) (bool, error) {
file, err := os.Open(filePath)
if err != nil {
return false, err
}
defer file.Close()
header := make([]byte, len(opensslSaltedHeader))
n, err := io.ReadFull(file, header)
if err != nil {
if err == io.EOF || err == io.ErrUnexpectedEOF {
return false, nil
}
return false, err
}
return n == len(opensslSaltedHeader) && string(header) == opensslSaltedHeader, nil
}
func clearMongodbDatabase(database, dbType, dbName string, taskItem *task.Task) error {
dbItem, err := mongodbRepo.Get(repo.WithByName(dbName), mongodbRepo.WithByMongodbName(database))
if err == nil && dbItem.From == constant.AppResourceRemote {
return clearRemoteMongodbDatabase(database, dbName, taskItem)
}
appInfo, err := appInstallRepo.LoadBaseInfo(dbType, database)
if err != nil {
return err
}
if appInfo.ContainerName == "" {
return fmt.Errorf("mongodb container not found for database %s", database)
}
logRemoteMongodbStep(taskItem, fmt.Sprintf("clear local mongodb database %s before restore", dbName))
uri := buildMongodbRestoreURI(appInfo.UserName, appInfo.Password)
return mongodbCmdMgr(taskItem).Run(
"docker",
"exec",
appInfo.ContainerName,
"mongosh",
uri,
"--quiet",
"--eval",
fmt.Sprintf(`db.getSiblingDB(%q).dropDatabase()`, dbName),
)
}
func clearRemoteMongodbDatabase(database, dbName string, taskItem *task.Task) error {
info, err := loadRemoteMongodbConnection(database)
if err != nil {
return err
}
client, ctx, cancel, err := newRemoteMongodbClient(info)
if err != nil {
return err
}
defer cancel()
defer client.Disconnect(context.Background())
logRemoteMongodbStep(taskItem, fmt.Sprintf("clear remote mongodb database %s before restore", dbName))
if err := client.Database(dbName).Drop(ctx); err != nil {
return fmt.Errorf("drop mongodb database %s failed, err: %v", dbName, err)
}
return nil
}
func buildMongodbDumpURI(username, password, dbName string) string {
return (&url.URL{
Scheme: "mongodb",
User: url.UserPassword(username, password),
Host: "127.0.0.1:27017",
Path: "/" + dbName,
RawQuery: "authSource=admin",
}).String()
}
func buildMongodbRestoreURI(username, password string) string {
return (&url.URL{
Scheme: "mongodb",
User: url.UserPassword(username, password),
Host: "127.0.0.1:27017",
Path: "/",
RawQuery: "authSource=admin",
}).String()
}
func mongodbCmdMgr(taskItem *task.Task) *cmd.CommandHelper {
if taskItem == nil {
return cmd.NewCommandMgr(cmd.WithTimeout(3 * time.Hour))
}
return cmd.NewCommandMgr(cmd.WithTimeout(3*time.Hour), cmd.WithTask(*taskItem))
}
func doRemoteMongodbBackup(database, dbName, targetDir, fileName string, taskItem *task.Task) error {
info, err := loadRemoteMongodbConnection(database)
if err != nil {
return err
}
imageTag, err := ensureMongodbImage(database, taskItem)
if err != nil {
return err
}
logRemoteMongodbImage(taskItem, "backup", database, dbName, imageTag, info)
logRemoteMongodbStep(taskItem, fmt.Sprintf("local image %s is ready, start backup", imageTag))
if err := os.MkdirAll(targetDir, constant.DirPerm); err != nil {
return err
}
targetFile, err := os.OpenFile(path.Join(targetDir, fileName), os.O_RDWR|os.O_CREATE|os.O_TRUNC, constant.DirPerm)
if err != nil {
return fmt.Errorf("open file %s failed, err: %v", path.Join(targetDir, fileName), err)
}
defer func() { _ = targetFile.Close() }()
backupCmd := exec.Command(
"docker",
"run",
"--rm",
"--net=host",
"-i",
imageTag,
"mongodump",
"--uri="+buildRemoteMongodbURI(info),
"--db="+dbName,
"--archive",
"--gzip",
)
backupCmd.Stdout = targetFile
var stderr bytes.Buffer
backupCmd.Stderr = &stderr
if err := backupCmd.Run(); err != nil {
return fmt.Errorf("handle backup mongodb database failed, err: %s", strings.TrimSpace(stderr.String()))
}
return nil
}
func doRemoteMongodbRestore(database, dbName, sourceFile string, taskItem *task.Task) error {
info, err := loadRemoteMongodbConnection(database)
if err != nil {
return err
}
imageTag, err := ensureMongodbImage(database, taskItem)
if err != nil {
return err
}
logRemoteMongodbImage(taskItem, "restore", database, dbName, imageTag, info)
logRemoteMongodbStep(taskItem, fmt.Sprintf("local image %s is ready, start restore", imageTag))
fi, err := os.Open(sourceFile)
if err != nil {
return err
}
defer func() { _ = fi.Close() }()
restoreCmd := exec.Command(
"docker",
"run",
"--rm",
"--net=host",
"-i",
imageTag,
"mongorestore",
"--uri="+buildRemoteMongodbURI(info),
"--nsInclude="+buildMongodbNamespace(sourceFile, dbName),
"--nsFrom="+buildMongodbNamespace(sourceFile, dbName),
"--nsTo="+dbName+".*",
"--archive",
"--gzip",
"--drop",
)
restoreCmd.Stdin = fi
var stderr bytes.Buffer
restoreCmd.Stderr = &stderr
if err := restoreCmd.Run(); err != nil {
return fmt.Errorf("handle recover mongodb database failed, err: %s", strings.TrimSpace(stderr.String()))
}
return nil
}
func ensureMongodbImage(database string, taskItem *task.Task) (string, error) {
imageTag, exists, err := loadMongodbImageTag(database)
if err != nil {
return "", err
}
logRemoteMongodbStep(taskItem, fmt.Sprintf("check local image %s", imageTag))
if exists {
logRemoteMongodbStep(taskItem, fmt.Sprintf("local image %s exists", imageTag))
return imageTag, nil
}
logRemoteMongodbStep(taskItem, fmt.Sprintf("local image %s not found, start docker pull", imageTag))
if err := mongodbCmdMgr(taskItem).Run("docker", "pull", imageTag); err != nil {
return "", err
}
logRemoteMongodbStep(taskItem, fmt.Sprintf("docker pull %s finished", imageTag))
return imageTag, nil
}
func loadMongodbImageTag(database string) (string, bool, error) {
databaseInfo, err := databaseRepo.Get(repo.WithByName(database))
if err != nil {
return "", false, err
}
cli, err := dockerClient.NewClientWithOpts(dockerClient.FromEnv, dockerClient.WithAPIVersionNegotiation())
if err != nil {
return "", false, err
}
defer cli.Close()
images, err := cli.ImageList(context.Background(), dockerImage.ListOptions{})
if err != nil {
return "", false, err
}
imagePrefix := "mongo:" + loadMongodbImageMajor(databaseInfo.Version)
for _, image := range images {
for _, tag := range image.RepoTags {
if strings.HasPrefix(tag, imagePrefix) {
return tag, true, nil
}
}
}
return imagePrefix, false, nil
}
func buildMongodbNamespace(sourceFile, targetDB string) string {
sourceDB := loadMongodbBackupDBName(sourceFile, targetDB)
return sourceDB + ".*"
}
func loadMongodbBackupDBName(sourceFile, defaultDB string) string {
baseName := path.Base(sourceFile)
if strings.HasSuffix(baseName, ".gz") {
baseName = strings.TrimSuffix(baseName, ".gz")
}
// Encrypted backups are restored from a decrypted temp file like tmp_<original>.
// Strip the temp prefix before deriving the original database name.
baseName = strings.TrimPrefix(baseName, "tmp_")
patterns := []*regexp.Regexp{
regexp.MustCompile(`^1panel_mongodb_(.+)_\d{14}[A-Za-z0-9]*$`),
regexp.MustCompile(`^db_(.+)_\d{14}[A-Za-z0-9]*$`),
regexp.MustCompile(`^(.+)_\d{14}[A-Za-z0-9]*$`),
}
for _, pattern := range patterns {
matches := pattern.FindStringSubmatch(baseName)
if len(matches) == 2 && len(matches[1]) != 0 {
return matches[1]
}
}
return defaultDB
}
func loadMongodbImageMajor(version string) string {
switch {
case strings.HasPrefix(version, "6"):
return "6"
case strings.HasPrefix(version, "7"):
return "7"
default:
return "8"
}
}
func logRemoteMongodbImage(taskItem *task.Task, action, database, dbName, imageTag string, info mongodbConnectionInfo) {
message := fmt.Sprintf(
"use local docker image %s to %s remote mongodb %s/%s via %s:%d",
imageTag,
action,
database,
dbName,
info.Address,
info.Port,
)
global.LOG.Info(message)
if taskItem != nil {
taskItem.Log(message)
}
}
func logRemoteMongodbStep(taskItem *task.Task, message string) {
global.LOG.Info(message)
if taskItem != nil {
taskItem.Log(message)
}
}
+2 -1
View File
@@ -1,6 +1,7 @@
package service
import (
"context"
"fmt"
"os"
"path"
@@ -263,7 +264,7 @@ func loadSqlFile(file string) (string, error) {
_ = os.RemoveAll(dstDir)
return "", err
}
if err := archiver.Extract(file, dstDir, ""); err != nil {
if err := archiver.Extract(context.Background(), file, dstDir, ""); err != nil {
_ = os.RemoveAll(dstDir)
return "", err
}
+487 -45
View File
@@ -1,7 +1,9 @@
package service
import (
"archive/tar"
"bufio"
"bytes"
"context"
"encoding/base64"
"encoding/json"
@@ -11,6 +13,7 @@ import (
"net/url"
"os"
"os/exec"
"path"
"path/filepath"
"sort"
"strconv"
@@ -40,10 +43,10 @@ import (
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/api/types/volume"
"github.com/docker/docker/client"
"github.com/docker/docker/pkg/stdcopy"
"github.com/docker/go-connections/nat"
"github.com/gin-gonic/gin"
v1 "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/shirou/gopsutil/v4/cpu"
"github.com/shirou/gopsutil/v4/mem"
)
@@ -69,7 +72,6 @@ type IContainerService interface {
ComposeLogClean(req dto.ComposeLogClean) error
ContainerCreate(req dto.ContainerOperate, inThread bool) error
ContainerCreateByCommand(req dto.ContainerCreateByCommand) error
ContainerUpdate(req dto.ContainerOperate) error
ContainerUpgrade(req dto.ContainerUpgrade) error
ContainerInfo(req dto.OperationWithName) (*dto.ContainerOperate, error)
@@ -91,6 +93,12 @@ type IContainerService interface {
Prune(req dto.ContainerPrune) error
LoadUsers(req dto.OperationWithName) []string
ListContainerFiles(req dto.ContainerFileReq) ([]dto.ContainerFileInfo, error)
UploadContainerFile(req dto.ContainerFileReq, fileName string, fileSize int64, file io.Reader) error
GetContainerFileContent(req dto.ContainerFileReq) (*dto.ContainerFileContent, error)
GetContainerFileSize(req dto.ContainerFileReq) (int64, error)
DeleteContainerFile(req dto.ContainerFileBatchDeleteReq) error
DownloadContainerFile(req dto.ContainerFileReq) (io.ReadCloser, string, string, error)
StreamLogs(ctx *gin.Context, params dto.StreamLog)
}
@@ -104,7 +112,7 @@ func (u *ContainerService) Page(req dto.PageContainer) (int64, interface{}, erro
if err != nil {
return 0, nil, err
}
defer client.Close()
defer func() { _ = client.Close() }()
options := container.ListOptions{All: true}
if len(req.Filters) != 0 {
options.Filters = filters.NewArgs()
@@ -302,42 +310,6 @@ func (u *ContainerService) ContainerListStats() ([]dto.ContainerListStats, error
return datas, nil
}
func (u *ContainerService) ContainerCreateByCommand(req dto.ContainerCreateByCommand) error {
if cmd.CheckIllegal(req.Command) {
return buserr.New("ErrCmdIllegal")
}
if !strings.HasPrefix(strings.TrimSpace(req.Command), "docker run ") {
return errors.New("error command format")
}
containerName := ""
commands := strings.Split(req.Command, " ")
for index, val := range commands {
if val == "--name" && len(commands) > index+1 {
containerName = commands[index+1]
}
}
if !strings.Contains(req.Command, " -d ") {
req.Command = strings.ReplaceAll(req.Command, "docker run", "docker run -d")
}
if len(containerName) == 0 {
containerName = fmt.Sprintf("1Panel-%s-%s", common.RandStr(5), common.RandStrAndNum(4))
req.Command += fmt.Sprintf(" --name %s", containerName)
}
taskItem, err := task.NewTaskWithOps(containerName, task.TaskCreate, task.TaskScopeContainer, req.TaskID, 1)
if err != nil {
global.LOG.Errorf("new task for create container failed, err: %v", err)
return err
}
go func() {
taskItem.AddSubTask(i18n.GetWithName("ContainerCreate", containerName), func(t *task.Task) error {
cmdMgr := cmd.NewCommandMgr(cmd.WithTask(*taskItem), cmd.WithTimeout(5*time.Minute))
return cmdMgr.RunBashC(req.Command)
}, nil)
_ = taskItem.Execute()
}()
return nil
}
func (u *ContainerService) Inspect(req dto.InspectReq) (string, error) {
client, err := docker.NewDockerClient()
if err != nil {
@@ -621,6 +593,16 @@ func (u *ContainerService) ContainerInfo(req dto.OperationWithName) (*dto.Contai
data.ExposedPorts = loadContainerPortForInfo(exposePorts)
data.Hostname = oldContainer.Config.Hostname
data.DNS = oldContainer.HostConfig.DNS
for _, item := range oldContainer.HostConfig.ExtraHosts {
parts := strings.SplitN(item, ":", 2)
if len(parts) != 2 || len(parts[0]) == 0 || len(parts[1]) == 0 {
continue
}
data.ExtraHosts = append(data.ExtraHosts, dto.ExtraHost{
Hostname: parts[0],
IP: parts[1],
})
}
data.DomainName = oldContainer.Config.Domainname
data.Cmd = oldContainer.Config.Cmd
@@ -1094,14 +1076,18 @@ func (u *ContainerService) DownloadContainerLogs(containerType, container, since
}
stdout, err := dockerCmd.StdoutPipe()
if err != nil {
_ = dockerCmd.Process.Signal(syscall.SIGTERM)
return err
}
dockerCmd.Stderr = dockerCmd.Stdout
if err := dockerCmd.Start(); err != nil {
_ = dockerCmd.Process.Signal(syscall.SIGTERM)
return err
}
defer func() {
if dockerCmd.Process != nil {
_ = dockerCmd.Process.Kill()
_ = dockerCmd.Wait()
}
}()
tempFile, err := os.CreateTemp("", "cmd_output_*.txt")
if err != nil {
@@ -1113,7 +1099,7 @@ func (u *ContainerService) DownloadContainerLogs(containerType, container, since
global.LOG.Errorf("os.Remove() failed: %v", err)
}
}()
errCh := make(chan error)
errCh := make(chan error, 1)
go func() {
scanner := bufio.NewScanner(stdout)
var ansiRegex = re.GetRegex(re.AnsiEscapePattern)
@@ -1136,8 +1122,8 @@ func (u *ContainerService) DownloadContainerLogs(containerType, container, since
if err != nil {
global.LOG.Errorf("Error: %v", err)
}
case <-time.After(3 * time.Second):
global.LOG.Errorf("Timeout reached")
case <-time.After(40 * time.Second):
global.LOG.Errorf("Download container logs timeout reached")
}
info, _ := tempFile.Stat()
@@ -1181,7 +1167,7 @@ func (u *ContainerService) ContainerStats(id string) (*dto.ContainerStats, error
func (u *ContainerService) LoadUsers(req dto.OperationWithName) []string {
var users []string
std, err := cmd.RunDefaultWithStdoutBashCf("docker exec %s cat /etc/passwd", req.Name)
std, err := cmd.NewCommandMgr(cmd.WithTimeout(20*time.Second)).RunWithStdout("docker", "exec", req.Name, "cat", "/etc/passwd")
if err != nil {
return users
}
@@ -1194,6 +1180,455 @@ func (u *ContainerService) LoadUsers(req dto.OperationWithName) []string {
return users
}
func (u *ContainerService) ListContainerFiles(req dto.ContainerFileReq) ([]dto.ContainerFileInfo, error) {
if len(req.Path) == 0 {
req.Path = "/"
}
cli, err := docker.NewDockerClient()
if err != nil {
return nil, err
}
defer cli.Close()
ctx := context.Background()
stat, err := cli.ContainerStatPath(ctx, req.ContainerID, req.Path)
if err != nil {
return nil, normalizeContainerFileError(err)
}
isDir := stat.Mode.IsDir()
isLink := stat.Mode&os.ModeSymlink != 0
if isLink && !isDir {
linkDir, linkErr := isContainerDir(cli, req.ContainerID, req.Path)
if linkErr == nil {
isDir = linkDir
}
}
if !isDir {
return []dto.ContainerFileInfo{toContainerFileInfo(req.Path, stat, isDir)}, nil
}
output, err := runContainerCommand(cli, req.ContainerID, []string{"ls", "-1A", "--", req.Path})
if err != nil {
return nil, normalizeContainerFileError(err)
}
lines := strings.Split(strings.TrimSpace(output), "\n")
files := make([]dto.ContainerFileInfo, 0, len(lines))
for _, line := range lines {
name := strings.TrimSpace(line)
if len(name) == 0 || name == "." || name == ".." {
continue
}
childPath := req.Path
if childPath == "/" {
childPath = "/" + name
} else {
childPath = strings.TrimSuffix(childPath, "/") + "/" + name
}
childStat, statErr := cli.ContainerStatPath(ctx, req.ContainerID, childPath)
if statErr != nil {
continue
}
childIsDir := childStat.Mode.IsDir()
if childStat.Mode&os.ModeSymlink != 0 && !childIsDir {
linkDir, linkErr := isContainerDir(cli, req.ContainerID, childPath)
if linkErr == nil {
childIsDir = linkDir
}
}
files = append(files, toContainerFileInfo(childPath, childStat, childIsDir))
}
sort.Slice(files, func(i, j int) bool {
if files[i].IsDir != files[j].IsDir {
return files[i].IsDir
}
return strings.ToLower(files[i].Name) < strings.ToLower(files[j].Name)
})
return files, nil
}
func (u *ContainerService) DeleteContainerFile(req dto.ContainerFileBatchDeleteReq) error {
for _, item := range req.Paths {
if strings.TrimSpace(item) == "/" {
return buserr.New("ErrPathNotDelete")
}
}
cli, err := docker.NewDockerClient()
if err != nil {
return err
}
defer cli.Close()
command := []string{"rm", "-rf", "--"}
command = append(command, req.Paths...)
_, err = runContainerCommand(cli, req.ContainerID, command)
return err
}
func (u *ContainerService) UploadContainerFile(req dto.ContainerFileReq, fileName string, fileSize int64, file io.Reader) error {
if len(req.Path) == 0 {
req.Path = "/"
}
safeName := path.Base(fileName)
if safeName == "." || safeName == "/" || len(safeName) == 0 {
return buserr.New("ErrInvalidChar")
}
cli, err := docker.NewDockerClient()
if err != nil {
return err
}
defer cli.Close()
ctx := context.Background()
stat, err := cli.ContainerStatPath(ctx, req.ContainerID, req.Path)
if err != nil {
if _, mkErr := runContainerCommand(cli, req.ContainerID, []string{"mkdir", "-p", "--", req.Path}); mkErr != nil {
return mkErr
}
stat, err = cli.ContainerStatPath(ctx, req.ContainerID, req.Path)
if err != nil {
return err
}
}
if !stat.Mode.IsDir() {
return fmt.Errorf("path %s is not directory", req.Path)
}
pipeReader, pipeWriter := io.Pipe()
writeErr := make(chan error, 1)
go func() {
tw := tar.NewWriter(pipeWriter)
header := &tar.Header{
Name: safeName,
Mode: 0644,
Size: fileSize,
ModTime: time.Now(),
}
if err := tw.WriteHeader(header); err != nil {
_ = tw.Close()
_ = pipeWriter.CloseWithError(err)
writeErr <- err
return
}
if _, err := io.Copy(tw, file); err != nil {
_ = tw.Close()
_ = pipeWriter.CloseWithError(err)
writeErr <- err
return
}
if err := tw.Close(); err != nil {
_ = pipeWriter.CloseWithError(err)
writeErr <- err
return
}
_ = pipeWriter.Close()
writeErr <- nil
}()
err = cli.CopyToContainer(ctx, req.ContainerID, req.Path, pipeReader, container.CopyToContainerOptions{
CopyUIDGID: true,
})
if err != nil {
_ = pipeReader.CloseWithError(err)
_ = pipeWriter.CloseWithError(err)
<-writeErr
return err
}
if err := <-writeErr; err != nil {
return err
}
return nil
}
func (u *ContainerService) GetContainerFileContent(req dto.ContainerFileReq) (*dto.ContainerFileContent, error) {
if len(req.Path) == 0 {
return nil, buserr.New("ErrInvalidChar")
}
cli, err := docker.NewDockerClient()
if err != nil {
return nil, err
}
defer cli.Close()
stat, err := cli.ContainerStatPath(context.Background(), req.ContainerID, req.Path)
if err != nil {
return nil, normalizeContainerFileError(err)
}
if stat.Mode.IsDir() {
return nil, fmt.Errorf("path %s is directory", req.Path)
}
content := &dto.ContainerFileContent{Size: stat.Size}
headBytes, err := runContainerCommandRaw(cli, req.ContainerID, []string{"head", "-c", "4096", "--", req.Path})
if err != nil {
return nil, err
}
if bytes.IndexByte(headBytes, 0) >= 0 {
content.IsBinary = true
return content, nil
}
const inlinePreviewMax = 512 * 1024
if stat.Size <= inlinePreviewMax {
raw, err := runContainerCommandRaw(cli, req.ContainerID, []string{"cat", "--", req.Path})
if err != nil {
return nil, err
}
content.Content = string(raw)
return content, nil
}
raw, err := runContainerCommandRaw(cli, req.ContainerID, []string{"tail", "-n", "300", "--", req.Path})
if err != nil {
return nil, err
}
content.Content = string(raw)
content.Truncated = true
return content, nil
}
func (u *ContainerService) GetContainerFileSize(req dto.ContainerFileReq) (int64, error) {
if len(req.Path) == 0 {
return 0, buserr.New("ErrInvalidChar")
}
cli, err := docker.NewDockerClient()
if err != nil {
return 0, err
}
defer cli.Close()
stat, err := cli.ContainerStatPath(context.Background(), req.ContainerID, req.Path)
if err != nil {
return 0, normalizeContainerFileError(err)
}
if !stat.Mode.IsDir() {
return stat.Size, nil
}
output, err := runContainerCommand(cli, req.ContainerID, []string{"du", "-sb", "--", req.Path})
if err != nil {
return 0, err
}
parts := strings.Fields(output)
if len(parts) == 0 {
return 0, fmt.Errorf("invalid du output")
}
size, err := strconv.ParseInt(parts[0], 10, 64)
if err != nil {
return 0, err
}
return size, nil
}
func (u *ContainerService) DownloadContainerFile(req dto.ContainerFileReq) (io.ReadCloser, string, string, error) {
if len(req.Path) == 0 {
req.Path = "/"
}
cli, err := docker.NewDockerClient()
if err != nil {
return nil, "", "", err
}
ctx := context.Background()
stat, err := cli.ContainerStatPath(ctx, req.ContainerID, req.Path)
if err != nil {
_ = cli.Close()
return nil, "", "", normalizeContainerFileError(err)
}
fileName := stat.Name
if len(fileName) == 0 {
fileName = "container-file"
}
if stat.Mode.IsDir() {
if _, err := runContainerCommand(cli, req.ContainerID, []string{"sh", "-c", "command -v tar >/dev/null 2>&1"}); err != nil {
_ = cli.Close()
return nil, "", "", fmt.Errorf("tar command not found in container")
}
targetPath := path.Clean(req.Path)
parentPath := path.Dir(targetPath)
targetName := path.Base(targetPath)
if parentPath == "." || parentPath == "" {
parentPath = "/"
}
tarStream, err := runContainerCommandStream(cli, req.ContainerID, []string{
"tar", "-czf", "-", "-C", parentPath, "--", targetName,
})
if err != nil {
_ = cli.Close()
return nil, "", "", err
}
if !strings.HasSuffix(fileName, ".tar.gz") {
fileName += ".tar.gz"
}
return &closeHookReader{
ReadCloser: tarStream,
onClose: cli.Close,
}, fileName, "application/gzip", nil
}
fileStream, err := runContainerCommandStream(cli, req.ContainerID, []string{"cat", "--", req.Path})
if err != nil {
_ = cli.Close()
return nil, "", "", err
}
return &closeHookReader{
ReadCloser: fileStream,
onClose: cli.Close,
}, fileName, "application/octet-stream", nil
}
func normalizeContainerFileError(err error) error {
if err == nil {
return nil
}
message := strings.ToLower(err.Error())
if strings.Contains(message, "no such file or directory") || strings.Contains(message, "not found") {
return buserr.New("ErrPathNotFound")
}
return err
}
func runContainerCommand(cli *client.Client, containerID string, command []string) (string, error) {
raw, err := runContainerCommandRaw(cli, containerID, command)
if err != nil {
return "", err
}
return strings.TrimSpace(string(raw)), nil
}
type closeHookReader struct {
io.ReadCloser
onClose func() error
}
func (r *closeHookReader) Close() error {
var closeErr error
if r.ReadCloser != nil {
closeErr = r.ReadCloser.Close()
}
if r.onClose != nil {
if err := r.onClose(); err != nil && closeErr == nil {
closeErr = err
}
}
return closeErr
}
func runContainerCommandRaw(cli *client.Client, containerID string, command []string) ([]byte, error) {
ctx := context.Background()
resp, err := cli.ContainerExecCreate(ctx, containerID, container.ExecOptions{
Cmd: command,
AttachStdout: true,
AttachStderr: true,
})
if err != nil {
return nil, err
}
hijack, err := cli.ContainerExecAttach(ctx, resp.ID, container.ExecAttachOptions{})
if err != nil {
return nil, err
}
defer hijack.Close()
raw, err := io.ReadAll(hijack.Reader)
if err != nil {
return nil, err
}
var stdout bytes.Buffer
var stderr bytes.Buffer
if _, err := stdcopy.StdCopy(&stdout, &stderr, bytes.NewReader(raw)); err != nil {
return nil, err
}
output := strings.TrimSpace(stdout.String())
errorOutput := strings.TrimSpace(stderr.String())
info, err := cli.ContainerExecInspect(ctx, resp.ID)
if err != nil {
return nil, err
}
if info.ExitCode != 0 {
if len(errorOutput) != 0 {
return nil, fmt.Errorf("%s", errorOutput)
}
if len(output) == 0 {
return nil, fmt.Errorf("container command failed with exit code %d", info.ExitCode)
}
return nil, fmt.Errorf("%s", output)
}
return stdout.Bytes(), nil
}
func runContainerCommandStream(cli *client.Client, containerID string, command []string) (io.ReadCloser, error) {
ctx := context.Background()
resp, err := cli.ContainerExecCreate(ctx, containerID, container.ExecOptions{
Cmd: command,
AttachStdout: true,
AttachStderr: true,
})
if err != nil {
return nil, err
}
hijack, err := cli.ContainerExecAttach(ctx, resp.ID, container.ExecAttachOptions{})
if err != nil {
return nil, err
}
pipeReader, pipeWriter := io.Pipe()
go func() {
defer hijack.Close()
var stderr bytes.Buffer
_, copyErr := stdcopy.StdCopy(pipeWriter, &stderr, hijack.Reader)
if copyErr != nil {
_ = pipeWriter.CloseWithError(copyErr)
return
}
info, inspectErr := cli.ContainerExecInspect(ctx, resp.ID)
if inspectErr != nil {
_ = pipeWriter.CloseWithError(inspectErr)
return
}
if info.ExitCode != 0 {
msg := strings.TrimSpace(stderr.String())
if len(msg) == 0 {
msg = fmt.Sprintf("container command failed with exit code %d", info.ExitCode)
}
_ = pipeWriter.CloseWithError(fmt.Errorf("%s", msg))
return
}
_ = pipeWriter.Close()
}()
return pipeReader, nil
}
func toContainerFileInfo(filePath string, stat container.PathStat, isDir bool) dto.ContainerFileInfo {
name := stat.Name
if len(name) == 0 {
items := strings.Split(strings.TrimSuffix(filePath, "/"), "/")
name = items[len(items)-1]
}
isLink := stat.Mode&os.ModeSymlink != 0
return dto.ContainerFileInfo{
Name: name,
Path: filePath,
IsDir: isDir,
IsLink: isLink,
LinkTo: stat.LinkTarget,
Size: stat.Size,
Mode: stat.Mode.String(),
ModTime: stat.Mtime.Format(constant.DateTimeLayout),
}
}
func isContainerDir(cli *client.Client, containerID, targetPath string) (bool, error) {
_, err := runContainerCommand(cli, containerID, []string{
"sh", "-c", "[ -d \"$1\" ]", "sh", targetPath,
})
if err != nil {
return false, err
}
return true, nil
}
func stringsToMap(list []string) map[string]string {
var labelMap = make(map[string]string)
for _, label := range list {
@@ -1488,6 +1923,13 @@ func loadConfigInfo(isCreate bool, req dto.ContainerOperate, oldContainer *conta
hostConf.Binds = []string{}
hostConf.Mounts = []mount.Mount{}
hostConf.DNS = req.DNS
hostConf.ExtraHosts = []string{}
for _, item := range req.ExtraHosts {
if len(item.Hostname) == 0 || len(item.IP) == 0 {
continue
}
hostConf.ExtraHosts = append(hostConf.ExtraHosts, fmt.Sprintf("%s:%s", item.Hostname, item.IP))
}
config.Volumes = make(map[string]struct{})
for _, volume := range req.Volumes {
item := mount.Mount{
+29 -2
View File
@@ -135,6 +135,7 @@ func (u *ContainerService) PageCompose(req dto.SearchWithPage) (int64, interface
for key, value := range mergedMap {
value.Name = key
value.ComposeFileExists = composeFileExists(value.Workdir, value.ConfigFile)
records = append(records, value)
}
if len(req.Info) != 0 {
@@ -164,6 +165,29 @@ func (u *ContainerService) PageCompose(req dto.SearchWithPage) (int64, interface
return int64(total), listItem, nil
}
func composeFileExists(workdir, configFile string) bool {
workdir = strings.TrimSpace(workdir)
configFile = strings.TrimSpace(configFile)
if configFile == "" {
return false
}
for _, item := range strings.Split(configFile, ",") {
file := strings.TrimSpace(item)
if file == "" {
continue
}
if !filepath.IsAbs(file) && workdir != "" {
file = filepath.Join(workdir, file)
}
file = filepath.Clean(file)
info, err := os.Stat(file)
if err == nil && !info.IsDir() {
return true
}
}
return false
}
func (u *ContainerService) TestCompose(req dto.ComposeCreate) (bool, error) {
if cmd.CheckIllegal(req.Path) {
return false, buserr.New("ErrCmdIllegal")
@@ -383,8 +407,11 @@ func (u *ContainerService) loadPath(req *dto.ComposeCreate) error {
}
func removeContainerForCompose(composeName, composePath string) error {
if stdout, err := compose.Operate(composePath, "down"); err != nil {
return errors.New(stdout)
if _, err := os.Stat(composePath); err == nil {
if stdout, err := compose.Operate(composePath, "down"); err != nil {
return errors.New(stdout)
}
return nil
}
var options container.ListOptions
options.All = true
+9
View File
@@ -287,6 +287,15 @@ func (u *CronjobService) Import(req []dto.CronjobTrans) error {
}
dbIDs = append(dbIDs, fmt.Sprintf("%v", dbItem.ID))
}
} else if cronjob.DBType == constant.AppMongodb {
for _, db := range item.DBNames {
dbItem, err := mongodbRepo.Get(mongodbRepo.WithByMongodbName(db.Name), repo.WithByName(db.DetailName))
if err != nil {
hasNotFound = true
continue
}
dbIDs = append(dbIDs, fmt.Sprintf("%v", dbItem.ID))
}
} else {
for _, db := range item.DBNames {
dbItem, err := mysqlRepo.Get(mysqlRepo.WithByMysqlName(db.Name), repo.WithByName(db.DetailName))
+46 -1
View File
@@ -165,7 +165,20 @@ func (u *CronjobService) handleDatabase(cronjob model.Cronjob, startTime time.Ti
record.DownloadAccountID, record.SourceAccountIDs = cronjob.DownloadAccountID, cronjob.SourceAccountIDs
backupDir := path.Join(global.Dir.LocalBackupDir, fmt.Sprintf("tmp/database/%s/%s/%s", dbInfo.DBType, record.Name, dbInfo.Name))
record.FileName = simplifiedFileName(fmt.Sprintf("db_%s_%s.sql.gz", dbInfo.Name, startTime.Format(constant.DateTimeSlimLayout)+common.RandStrAndNum(5)))
switch dbInfo.DBType {
case constant.AppMongodb:
record.FileName = simplifiedFileName(fmt.Sprintf(
"db_%s_%s.gz",
dbInfo.Name,
startTime.Format(constant.DateTimeSlimLayout)+common.RandStrAndNum(5),
))
default:
record.FileName = simplifiedFileName(fmt.Sprintf(
"db_%s_%s.sql.gz",
dbInfo.Name,
startTime.Format(constant.DateTimeSlimLayout)+common.RandStrAndNum(5),
))
}
if cronjob.DBType == "mysql" || cronjob.DBType == "mariadb" || cronjob.DBType == "mysql-cluster" {
if err := doMysqlBackup(dbInfo, backupDir, record.FileName, cronjob.Secret); err != nil {
if retry < int(cronjob.RetryTimes) || !cronjob.IgnoreErr {
@@ -177,6 +190,17 @@ func (u *CronjobService) handleDatabase(cronjob model.Cronjob, startTime time.Ti
return nil
}
}
} else if cronjob.DBType == constant.AppMongodb {
if err := doMongodbBackup(dbInfo.Database, dbInfo.DBType, dbInfo.Name, backupDir, record.FileName, cronjob.Secret, task); err != nil {
if retry < int(cronjob.RetryTimes) || !cronjob.IgnoreErr {
retry++
return err
} else {
task.Log(i18n.GetMsgWithDetail("IgnoreBackupErr", err.Error()))
cleanAccountMap(accountMap)
return nil
}
}
} else {
if err := doPostgresqlBackup(dbInfo, backupDir, record.FileName, cronjob.Secret, taskItem); err != nil {
if retry < int(cronjob.RetryTimes) || !cronjob.IgnoreErr {
@@ -404,6 +428,18 @@ func loadDbsForJob(cronjob model.Cronjob) []DatabaseHelper {
Args: strings.Split(cronjob.Args, ","),
})
}
} else if cronjob.DBType == constant.AppMongodb {
databaseService := NewIDatabaseService()
mongodbItems, _ := databaseService.LoadItems(cronjob.DBType)
for _, mongodb := range mongodbItems {
dbs = append(dbs, DatabaseHelper{
ID: mongodb.ID,
DBType: cronjob.DBType,
Database: mongodb.Database,
Name: mongodb.Name,
Args: strings.Split(cronjob.Args, ","),
})
}
} else {
pgItems, _ := postgresqlRepo.List()
for _, pg := range pgItems {
@@ -430,6 +466,15 @@ func loadDbsForJob(cronjob model.Cronjob) []DatabaseHelper {
Name: mysqlItem.Name,
Args: strings.Split(cronjob.Args, ","),
})
} else if cronjob.DBType == constant.AppMongodb {
mongodbItem, _ := mongodbRepo.Get(repo.WithByID(uint(itemID)))
dbs = append(dbs, DatabaseHelper{
ID: mongodbItem.ID,
DBType: cronjob.DBType,
Database: mongodbItem.MongodbName,
Name: mongodbItem.Name,
Args: strings.Split(cronjob.Args, ","),
})
} else {
pgItem, _ := postgresqlRepo.Get(repo.WithByID(uint(itemID)))
dbs = append(dbs, DatabaseHelper{
+11 -3
View File
@@ -284,7 +284,7 @@ func (u *DashboardService) LoadAppLauncher(ctx *gin.Context) ([]dto.AppLauncher,
return data, err
}
showList, err := launcherRepo.ListName()
showList, _ := launcherRepo.ListName()
defaultList, err := appRepo.GetTopRecommend()
if err != nil {
return data, nil
@@ -361,6 +361,9 @@ func (u *DashboardService) LoadQuickOptions() []dto.QuickJump {
_ = copier.Copy(&item, &quick)
list = append(list, item)
}
sort.Slice(list, func(i, j int) bool {
return list[i].Recommend < list[j].Recommend
})
return list
}
func (u *DashboardService) ChangeQuick(req dto.ChangeQuicks) error {
@@ -612,6 +615,9 @@ func loadQuickJump(base *dto.DashboardBase) {
website, _ := websiteRepo.GetBy()
base.WebsiteNumber = len(website)
agents, _ := agentRepo.List()
base.AgentNumber = len(agents)
postgresqlDbs, _ := postgresqlRepo.List()
mysqlDbs, _ := mysqlRepo.List()
base.DatabaseNumber = len(mysqlDbs) + len(postgresqlDbs)
@@ -625,6 +631,8 @@ func loadQuickJump(base *dto.DashboardBase) {
quicks := launcherRepo.ListQuickJump(false)
for i := 0; i < len(quicks); i++ {
switch quicks[i].Name {
case "Agent":
quicks[i].Detail = fmt.Sprintf("%d", base.AgentNumber)
case "Website":
quicks[i].Detail = fmt.Sprintf("%d", base.WebsiteNumber)
case "Database":
@@ -638,7 +646,7 @@ func loadQuickJump(base *dto.DashboardBase) {
_ = copier.Copy(&item, quicks[i])
base.QuickJumps = append(base.QuickJumps, item)
}
sort.Slice(quicks, func(i, j int) bool {
return quicks[i].Recommend < quicks[j].Recommend
sort.Slice(base.QuickJumps, func(i, j int) bool {
return base.QuickJumps[i].Recommend < base.QuickJumps[j].Recommend
})
}
+47 -1
View File
@@ -90,7 +90,7 @@ func (u *DatabaseService) LoadItems(dbType string) ([]dto.DatabaseItem, error) {
dbs, err := databaseRepo.GetList(databaseRepo.WithTypeList(dbType))
var datas []dto.DatabaseItem
for _, db := range dbs {
if dbType == constant.AppPostgresql || dbType == constant.AppPostgresqlCluster {
if db.Type == constant.AppPostgresql || db.Type == constant.AppPostgresqlCluster {
items, _ := postgresqlRepo.List(postgresqlRepo.WithByPostgresqlName(db.Name))
for _, item := range items {
var dItem dto.DatabaseItem
@@ -100,6 +100,16 @@ func (u *DatabaseService) LoadItems(dbType string) ([]dto.DatabaseItem, error) {
dItem.Database = db.Name
datas = append(datas, dItem)
}
} else if db.Type == constant.AppMongodb {
items, _ := mongodbRepo.List(mongodbRepo.WithByMongodbName(db.Name))
for _, item := range items {
var dItem dto.DatabaseItem
if err := copier.Copy(&dItem, &item); err != nil {
continue
}
dItem.Database = db.Name
datas = append(datas, dItem)
}
} else {
items, _ := mysqlRepo.List(mysqlRepo.WithByMysqlName(db.Name))
for _, item := range items {
@@ -138,6 +148,13 @@ func (u *DatabaseService) CheckDatabase(req dto.DatabaseCreate) bool {
Password: req.Password,
Timeout: req.Timeout,
})
case constant.AppMongodb:
client, ctx, cancel, connectErr := newRemoteMongodbClient(mongodbConnectionInfoFromCreate(req))
if connectErr == nil {
defer cancel()
defer client.Disconnect(ctx)
}
err = connectErr
case "mysql", "mariadb":
_, err = mysql.NewMysqlClient(client.DBInfo{
From: "remote",
@@ -195,6 +212,13 @@ func (u *DatabaseService) Create(req dto.DatabaseCreate) error {
}); err != nil {
return err
}
case constant.AppMongodb:
client, ctx, cancel, err := newRemoteMongodbClient(mongodbConnectionInfoFromCreate(req))
if err != nil {
return err
}
defer cancel()
defer client.Disconnect(ctx)
case "mysql", "mariadb":
if _, err := mysql.NewMysqlClient(client.DBInfo{
From: "remote",
@@ -265,6 +289,10 @@ func (u *DatabaseService) Delete(req dto.DatabaseDelete) error {
if err := mysqlRepo.Delete(context.Background(), mysqlRepo.WithByMysqlName(db.Name)); err != nil && !req.ForceDelete {
return err
}
} else if db.Type == constant.AppMongodb {
if err := mongodbRepo.Delete(context.Background(), mongodbRepo.WithByMongodbName(db.Name)); err != nil && !req.ForceDelete {
return err
}
} else {
if err := postgresqlRepo.Delete(context.Background(), postgresqlRepo.WithByPostgresqlName(db.Name)); err != nil && !req.ForceDelete {
return err
@@ -297,6 +325,24 @@ func (u *DatabaseService) Update(req dto.DatabaseUpdate) error {
}); err != nil {
return err
}
case constant.AppMongodb:
client, ctx, cancel, err := newRemoteMongodbClient(mongodbConnectionInfoFromCreate(dto.DatabaseCreate{
Address: req.Address,
Port: req.Port,
Username: req.Username,
Password: req.Password,
Timeout: req.Timeout,
SSL: req.SSL,
RootCert: req.RootCert,
ClientKey: req.ClientKey,
ClientCert: req.ClientCert,
SkipVerify: req.SkipVerify,
}))
if err != nil {
return err
}
defer cancel()
defer client.Disconnect(ctx)
case "mysql", "mariadb":
if _, err := mysql.NewMysqlClient(client.DBInfo{
From: "remote",
+5
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"os"
"path"
"path/filepath"
"strings"
"github.com/1Panel-dev/1Panel/agent/app/dto"
@@ -43,6 +44,10 @@ func (u *DBCommonService) LoadBaseInfo(req dto.OperationWithNameAndType) (*dto.D
func (u *DBCommonService) LoadDatabaseFile(req dto.OperationWithNameAndType) (string, error) {
filePath := ""
safeName := filepath.Base(req.Name)
if safeName != req.Name || strings.Contains(safeName, "..") {
return "", buserr.New("ErrInvalidParams")
}
switch req.Type {
case "mysql-cluster-conf":
filePath = path.Join(global.Dir.DataDir, fmt.Sprintf("apps/mysql-cluster/%s/conf/my.cnf", req.Name))
+948
View File
@@ -0,0 +1,948 @@
package service
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/app/model"
"github.com/1Panel-dev/1Panel/agent/app/repo"
"github.com/1Panel-dev/1Panel/agent/buserr"
"github.com/1Panel-dev/1Panel/agent/constant"
"github.com/1Panel-dev/1Panel/agent/global"
"github.com/1Panel-dev/1Panel/agent/utils/cmd"
"github.com/1Panel-dev/1Panel/agent/utils/encrypt"
"github.com/jinzhu/copier"
"go.mongodb.org/mongo-driver/bson"
)
type MongodbService struct{}
type IMongodbService interface {
SearchWithPage(search dto.MongodbDBSearch) (int64, interface{}, error)
Create(ctx context.Context, req dto.MongodbDBCreate) (*model.DatabaseMongodb, error)
LoadFromRemote(req dto.MongodbLoadDB) error
UpdateDescription(req dto.UpdateDescription) error
BindUser(req dto.MongodbBind) error
ChangePassword(req dto.MongodbPassword) error
ChangeRootPassword(req dto.ChangeDBInfo) error
LoadPrivileges(req dto.MongodbPrivilegesLoad) (string, error)
ChangePrivileges(req dto.MongodbPrivileges) error
DeleteCheck(req dto.MongodbDBDeleteCheck) ([]dto.DBResource, error)
Delete(ctx context.Context, req dto.MongodbDBDelete) error
}
func NewIMongodbService() IMongodbService {
return &MongodbService{}
}
func (u *MongodbService) SearchWithPage(search dto.MongodbDBSearch) (int64, interface{}, error) {
total, mongodbs, err := mongodbRepo.Page(
search.Page,
search.PageSize,
mongodbRepo.WithByMongodbName(search.Database),
repo.WithByLikeName(search.Info),
repo.WithOrderRuleBy(search.OrderBy, search.Order),
)
var dtoMongodbs []dto.MongodbDBInfo
for _, mongodb := range mongodbs {
var item dto.MongodbDBInfo
if err := copier.Copy(&item, &mongodb); err != nil {
return 0, nil, buserr.WithDetail("ErrStructTransform", err.Error(), nil)
}
dtoMongodbs = append(dtoMongodbs, item)
}
return total, dtoMongodbs, err
}
func (u *MongodbService) UpdateDescription(req dto.UpdateDescription) error {
return mongodbRepo.Update(req.ID, map[string]interface{}{"description": req.Description})
}
func (u *MongodbService) Create(ctx context.Context, req dto.MongodbDBCreate) (*model.DatabaseMongodb, error) {
if cmd.CheckIllegal(req.Name, req.Username, req.Password) {
return nil, buserr.New("ErrCmdIllegal")
}
if !isSupportedMongodbPrivilege(req.Permission) {
return nil, buserr.New("ErrCmdIllegal")
}
mongodb, _ := mongodbRepo.Get(repo.WithByName(req.Name), mongodbRepo.WithByMongodbName(req.Database), repo.WithByFrom(req.From))
if mongodb.ID != 0 {
return nil, buserr.New("ErrRecordExist")
}
var createItem model.DatabaseMongodb
if err := copier.Copy(&createItem, &req); err != nil {
return nil, buserr.WithDetail("ErrStructTransform", err.Error(), nil)
}
createItem.MongodbName = req.Database
if err := runMongodbCreate(req.From, req.Database, req.Name, req.Username, req.Password, req.Permission); err != nil {
return nil, err
}
global.LOG.Infof("create mongodb database %s successful", req.Name)
if err := mongodbRepo.Create(ctx, &createItem); err != nil {
return nil, err
}
return &createItem, nil
}
func (u *MongodbService) LoadFromRemote(req dto.MongodbLoadDB) error {
databases, err := mongodbRepo.List(mongodbRepo.WithByMongodbName(req.Database))
if err != nil {
return err
}
datas, err := loadMongodbDatabases(req)
if err != nil {
return err
}
global.LOG.Infof("sync mongodb databases from %s:%s, found %d items", req.From, req.Database, len(datas))
deleteList := append([]model.DatabaseMongodb(nil), databases...)
for _, data := range datas {
hasOld := false
for i := 0; i < len(databases); i++ {
if strings.EqualFold(databases[i].Name, data.Name) && strings.EqualFold(databases[i].MongodbName, req.Database) {
hasOld = true
updateMap := map[string]interface{}{"is_delete": false}
if len(data.Username) != 0 {
updateMap["username"] = data.Username
}
_ = mongodbRepo.Update(databases[i].ID, updateMap)
for j := 0; j < len(deleteList); j++ {
if deleteList[j].ID == databases[i].ID {
deleteList = append(deleteList[:j], deleteList[j+1:]...)
break
}
}
break
}
}
if !hasOld {
createItem := model.DatabaseMongodb{
Name: data.Name,
From: req.From,
MongodbName: req.Database,
Username: data.Username,
Password: "",
Description: "",
}
if err := mongodbRepo.Create(context.Background(), &createItem); err != nil {
return err
}
}
}
for _, delItem := range deleteList {
_ = mongodbRepo.Update(delItem.ID, map[string]interface{}{"is_delete": true})
}
return nil
}
func (u *MongodbService) BindUser(req dto.MongodbBind) error {
if cmd.CheckIllegal(req.Database, req.Name, req.Username, req.Password) {
return buserr.New("ErrCmdIllegal")
}
dbItem, err := mongodbRepo.Get(mongodbRepo.WithByMongodbName(req.Database), repo.WithByName(req.Name))
if err != nil {
return err
}
if err := bindMongodbUser(dbItem.From, dbItem.MongodbName, dbItem.Name, req.Username, req.Password); err != nil {
return err
}
pass, err := encrypt.StringEncrypt(req.Password)
if err != nil {
return fmt.Errorf("encrypt mongodb db %s password failed, err: %v", req.Name, err)
}
return mongodbRepo.Update(dbItem.ID, map[string]interface{}{
"username": req.Username,
"password": pass,
})
}
func (u *MongodbService) ChangePassword(req dto.MongodbPassword) error {
if cmd.CheckIllegal(req.Database, req.Name, req.Password) {
return buserr.New("ErrCmdIllegal")
}
dbItem, err := mongodbRepo.Get(mongodbRepo.WithByMongodbName(req.Database), repo.WithByName(req.Name))
if err != nil {
return err
}
if dbItem.Username == "" {
return buserr.New("ErrRecordNotFound")
}
if err := updateMongodbPassword(dbItem.From, dbItem.MongodbName, dbItem.Name, dbItem.Username, req.Password); err != nil {
return err
}
pass, err := encrypt.StringEncrypt(req.Password)
if err != nil {
return fmt.Errorf("encrypt mongodb db %s password failed, err: %v", req.Name, err)
}
return mongodbRepo.Update(dbItem.ID, map[string]interface{}{"password": pass})
}
func (u *MongodbService) ChangeRootPassword(req dto.ChangeDBInfo) error {
if cmd.CheckIllegal(req.Value) {
return buserr.New("ErrCmdIllegal")
}
if req.From != constant.AppResourceLocal {
return buserr.New("ErrRecordNotFound")
}
appInfo, err := appInstallRepo.LoadBaseInfo(req.Type, req.Database)
if err != nil {
return err
}
if appInfo.UserName == "" {
return buserr.New("ErrRecordNotFound")
}
if err := updateMongodbPassword(req.From, req.Database, "admin", appInfo.UserName, req.Value); err != nil {
return err
}
if err := updateInstallInfoInDB(req.Type, req.Database, "password", req.Value); err != nil {
return err
}
remote, err := databaseRepo.Get(repo.WithByName(req.Database))
if err != nil {
return err
}
pass, err := encrypt.StringEncrypt(req.Value)
if err != nil {
return fmt.Errorf("encrypt mongodb root password failed, err: %v", err)
}
_ = databaseRepo.Update(remote.ID, map[string]interface{}{"password": pass})
return nil
}
func (u *MongodbService) DeleteCheck(req dto.MongodbDBDeleteCheck) ([]dto.DBResource, error) {
var res []dto.DBResource
db, err := mongodbRepo.Get(repo.WithByID(req.ID))
if err != nil {
return res, err
}
if db.From == "local" {
app, err := appInstallRepo.LoadBaseInfo(req.Type, req.Database)
if err != nil {
return res, err
}
apps, _ := appInstallResourceRepo.GetBy(appInstallResourceRepo.WithLinkId(app.ID), appInstallResourceRepo.WithResourceId(db.ID))
for _, app := range apps {
appInstall, _ := appInstallRepo.GetFirst(repo.WithByID(app.AppInstallId))
if appInstall.ID != 0 {
res = append(res, dto.DBResource{
Type: constant.TypeApp,
Name: appInstall.Name,
})
}
}
} else {
apps, _ := appInstallResourceRepo.GetBy(appInstallResourceRepo.WithResourceId(db.ID), appRepo.WithKey(req.Type))
for _, app := range apps {
appInstall, _ := appInstallRepo.GetFirst(repo.WithByID(app.AppInstallId))
if appInstall.ID != 0 {
res = append(res, dto.DBResource{
Type: constant.TypeApp,
Name: appInstall.Name,
})
}
}
}
return res, nil
}
func (u *MongodbService) LoadPrivileges(req dto.MongodbPrivilegesLoad) (string, error) {
dbItem, err := mongodbRepo.Get(mongodbRepo.WithByMongodbName(req.Database), repo.WithByName(req.Name))
if err != nil {
return "", err
}
return loadMongodbPrivilege(dbItem.From, dbItem.MongodbName, dbItem.Name, req.Username)
}
func (u *MongodbService) ChangePrivileges(req dto.MongodbPrivileges) error {
if cmd.CheckIllegal(req.Database, req.Username) {
return buserr.New("ErrCmdIllegal")
}
if !isSupportedMongodbPrivilege(req.Permission) {
return buserr.New("ErrCmdIllegal")
}
dbItem, err := mongodbRepo.Get(mongodbRepo.WithByMongodbName(req.Database), repo.WithByName(req.Name))
if err != nil {
return err
}
return updateMongodbPrivilege(dbItem.From, dbItem.MongodbName, dbItem.Name, req.Username, req.Permission)
}
func (u *MongodbService) Delete(ctx context.Context, req dto.MongodbDBDelete) error {
db, err := mongodbRepo.Get(repo.WithByID(req.ID))
if err != nil && !req.ForceDelete {
return err
}
if err := runMongodbDelete(db.From, req.Database, db.Name); err != nil && !req.ForceDelete {
return err
}
if req.DeleteBackup {
uploadDir := filepath.Join(global.Dir.DataDir, fmt.Sprintf("uploads/database/%s/%s/%s", req.Type, req.Database, db.Name))
if _, err := os.Stat(uploadDir); err == nil {
_ = os.RemoveAll(uploadDir)
}
backupDir := filepath.Join(global.Dir.LocalBackupDir, fmt.Sprintf("database/%s/%s/%s", req.Type, req.Database, db.Name))
if _, err := os.Stat(backupDir); err == nil {
_ = os.RemoveAll(backupDir)
}
_ = backupRepo.DeleteRecord(ctx, repo.WithByType(req.Type), repo.WithByName(req.Database), repo.WithByDetailName(db.Name))
global.LOG.Infof("delete mongodb database %s-%s backups successful", req.Database, db.Name)
}
_ = mongodbRepo.Delete(ctx, repo.WithByID(db.ID))
return nil
}
func runMongodbCreate(from, database, dbName, username, password, permission string) error {
if from == constant.AppResourceRemote {
return runRemoteMongodbCreate(database, dbName, username, password, permission)
}
script, err := buildMongodbCreateScript(dbName, username, password, permission)
if err != nil {
return err
}
return runMongodbAdminScript(database, script)
}
func runMongodbDelete(from, database, dbName string) error {
if from == constant.AppResourceRemote {
return runRemoteMongodbDelete(database, dbName)
}
script, err := buildMongodbDeleteScript(dbName)
if err != nil {
return err
}
return runMongodbAdminScript(database, script)
}
func bindMongodbUser(from, connectionName, dbName, username, password string) error {
if from == constant.AppResourceRemote {
return bindRemoteMongodbUser(connectionName, dbName, username, password)
}
script, err := buildMongodbBindUserScript(dbName, username, password)
if err != nil {
return err
}
return runMongodbAdminScript(connectionName, script)
}
func updateMongodbPassword(from, connectionName, dbName, username, password string) error {
if from == constant.AppResourceRemote {
return updateRemoteMongodbPasswordOnly(connectionName, dbName, username, password)
}
script, err := buildMongodbPasswordScript(dbName, username, password)
if err != nil {
return err
}
return runMongodbAdminScript(connectionName, script)
}
func runMongodbAdminScript(database, script string) error {
appInfo, err := appInstallRepo.LoadBaseInfo(constant.AppMongodb, database)
if err != nil {
return err
}
if appInfo.ContainerName == "" {
return fmt.Errorf("mongodb container not found for database %s", database)
}
return cmd.NewCommandMgr().Run(
"docker",
"exec",
appInfo.ContainerName,
"mongosh",
buildMongodbRestoreURI(appInfo.UserName, appInfo.Password),
"--quiet",
"--eval",
script,
)
}
func runMongodbAdminScriptWithStdout(database, script string) (string, error) {
appInfo, err := appInstallRepo.LoadBaseInfo(constant.AppMongodb, database)
if err != nil {
return "", err
}
if appInfo.ContainerName == "" {
return "", fmt.Errorf("mongodb container not found for database %s", database)
}
return cmd.NewCommandMgr().RunWithStdout(
"docker",
"exec",
appInfo.ContainerName,
"mongosh",
buildMongodbRestoreURI(appInfo.UserName, appInfo.Password),
"--quiet",
"--eval",
script,
)
}
func buildMongodbCreateScript(dbName, username, password, permission string) (string, error) {
dbNameJSON, err := json.Marshal(dbName)
if err != nil {
return "", err
}
usernameJSON, err := json.Marshal(username)
if err != nil {
return "", err
}
passwordJSON, err := json.Marshal(password)
if err != nil {
return "", err
}
permissionJSON, err := json.Marshal(permission)
if err != nil {
return "", err
}
return strings.TrimSpace(fmt.Sprintf(`
const dbName = %s;
const userName = %s;
const password = %s;
const permission = %s;
const targetDb = db.getSiblingDB(dbName);
targetDb.createCollection("_init");
targetDb.createUser({
user: userName,
pwd: password,
roles: [{ role: permission, db: dbName }]
});
`, dbNameJSON, usernameJSON, passwordJSON, permissionJSON)), nil
}
func buildMongodbDeleteScript(dbName string) (string, error) {
dbNameJSON, err := json.Marshal(dbName)
if err != nil {
return "", err
}
return strings.TrimSpace(fmt.Sprintf(`
const dbName = %s;
const targetDb = db.getSiblingDB(dbName);
const dropUsersResult = targetDb.runCommand({ dropAllUsersFromDatabase: 1 });
if (!dropUsersResult || dropUsersResult.ok !== 1) {
throw new Error("failed to drop users from " + dbName);
}
const dropDatabaseResult = targetDb.runCommand({ dropDatabase: 1 });
if (!dropDatabaseResult || dropDatabaseResult.ok !== 1) {
throw new Error("failed to drop database " + dbName);
}
`, dbNameJSON)), nil
}
func buildMongodbBindUserScript(dbName, username, password string) (string, error) {
dbNameJSON, err := json.Marshal(dbName)
if err != nil {
return "", err
}
usernameJSON, err := json.Marshal(username)
if err != nil {
return "", err
}
passwordJSON, err := json.Marshal(password)
if err != nil {
return "", err
}
return strings.TrimSpace(fmt.Sprintf(`
const dbName = %s;
const userName = %s;
const password = %s;
const targetDb = db.getSiblingDB(dbName);
const userInfo = targetDb.runCommand({
usersInfo: userName,
showCredentials: false,
showCustomData: false
});
if (!userInfo || userInfo.ok !== 1) {
throw new Error("failed to load mongodb user " + userName);
}
const roles = [{ role: "readWrite", db: dbName }];
if (Array.isArray(userInfo.users) && userInfo.users.length > 0) {
const result = targetDb.runCommand({
updateUser: userName,
pwd: password,
roles: roles
});
if (!result || result.ok !== 1) {
throw new Error("failed to update mongodb user " + userName);
}
} else {
const result = targetDb.runCommand({
createUser: userName,
pwd: password,
roles: roles
});
if (!result || result.ok !== 1) {
throw new Error("failed to create mongodb user " + userName);
}
}
`, dbNameJSON, usernameJSON, passwordJSON)), nil
}
func buildMongodbPasswordScript(dbName, username, password string) (string, error) {
dbNameJSON, err := json.Marshal(dbName)
if err != nil {
return "", err
}
usernameJSON, err := json.Marshal(username)
if err != nil {
return "", err
}
passwordJSON, err := json.Marshal(password)
if err != nil {
return "", err
}
return strings.TrimSpace(fmt.Sprintf(`
const dbName = %s;
const userName = %s;
const password = %s;
const targetDb = db.getSiblingDB(dbName);
const result = targetDb.runCommand({
updateUser: userName,
pwd: password
});
if (!result || result.ok !== 1) {
throw new Error("failed to update mongodb user password " + userName);
}
`, dbNameJSON, usernameJSON, passwordJSON)), nil
}
type mongodbSyncItem struct {
Name string `json:"name"`
Username string `json:"username"`
}
func loadMongodbDatabases(req dto.MongodbLoadDB) ([]mongodbSyncItem, error) {
if req.From == constant.AppResourceRemote {
return loadRemoteMongodbDatabases(req.Database)
}
return loadLocalMongodbDatabases(req.Database)
}
func loadLocalMongodbDatabases(database string) ([]mongodbSyncItem, error) {
script := strings.TrimSpace(`
const systemDbs = new Set(["admin", "config", "local"]);
const result = db.adminCommand({ listDatabases: 1, nameOnly: false });
if (!result || result.ok !== 1) {
throw new Error("failed to list mongodb databases");
}
const items = result.databases
.filter(item => !systemDbs.has(item.name))
.map(item => ({ name: item.name, username: "" }));
print("__1panel_json_begin__");
print(JSON.stringify(items));
print("__1panel_json_end__");
`)
stdout, err := runMongodbAdminScriptWithStdout(database, script)
if err != nil {
return nil, err
}
items := make([]mongodbSyncItem, 0)
jsonResult, err := extractMongodbJSONOutput(stdout)
if err != nil {
return nil, err
}
if err := json.Unmarshal(jsonResult, &items); err != nil {
return nil, err
}
return items, nil
}
func runRemoteMongodbCreate(database, dbName, username, password, permission string) error {
info, err := loadRemoteMongodbConnection(database)
if err != nil {
return err
}
client, ctx, cancel, err := newRemoteMongodbClient(info)
if err != nil {
return err
}
defer cancel()
defer client.Disconnect(ctx)
targetDB := client.Database(dbName)
if err := targetDB.CreateCollection(ctx, "_init"); err != nil {
return err
}
return targetDB.RunCommand(ctx, bson.D{
{Key: "createUser", Value: username},
{Key: "pwd", Value: password},
{Key: "roles", Value: bson.A{
bson.D{{Key: "role", Value: permission}, {Key: "db", Value: dbName}},
}},
}).Err()
}
func runRemoteMongodbDelete(database, dbName string) error {
info, err := loadRemoteMongodbConnection(database)
if err != nil {
return err
}
client, ctx, cancel, err := newRemoteMongodbClient(info)
if err != nil {
return err
}
defer cancel()
defer client.Disconnect(ctx)
targetDB := client.Database(dbName)
if err := targetDB.RunCommand(ctx, bson.D{{Key: "dropAllUsersFromDatabase", Value: 1}}).Err(); err != nil {
return err
}
return targetDB.RunCommand(ctx, bson.D{{Key: "dropDatabase", Value: 1}}).Err()
}
func bindRemoteMongodbUser(connectionName, dbName, username, password string) error {
info, err := loadRemoteMongodbConnection(connectionName)
if err != nil {
return err
}
client, ctx, cancel, err := newRemoteMongodbClient(info)
if err != nil {
return err
}
defer cancel()
defer client.Disconnect(ctx)
targetDB := client.Database(dbName)
var userInfo struct {
Users []struct{} `bson:"users"`
}
if err := targetDB.RunCommand(ctx, bson.D{
{Key: "usersInfo", Value: username},
{Key: "showCredentials", Value: false},
{Key: "showCustomData", Value: false},
}).Decode(&userInfo); err != nil {
return err
}
if len(userInfo.Users) > 0 {
return targetDB.RunCommand(ctx, bson.D{
{Key: "updateUser", Value: username},
{Key: "pwd", Value: password},
{Key: "roles", Value: bson.A{
bson.D{{Key: "role", Value: "readWrite"}, {Key: "db", Value: dbName}},
}},
}).Err()
}
return targetDB.RunCommand(ctx, bson.D{
{Key: "createUser", Value: username},
{Key: "pwd", Value: password},
{Key: "roles", Value: bson.A{
bson.D{{Key: "role", Value: "readWrite"}, {Key: "db", Value: dbName}},
}},
}).Err()
}
func updateRemoteMongodbPasswordOnly(connectionName, dbName, username, password string) error {
info, err := loadRemoteMongodbConnection(connectionName)
if err != nil {
return err
}
client, ctx, cancel, err := newRemoteMongodbClient(info)
if err != nil {
return err
}
defer cancel()
defer client.Disconnect(ctx)
targetDB := client.Database(dbName)
return targetDB.RunCommand(ctx, bson.D{
{Key: "updateUser", Value: username},
{Key: "pwd", Value: password},
}).Err()
}
func loadRemoteMongodbDatabases(database string) ([]mongodbSyncItem, error) {
info, err := loadRemoteMongodbConnection(database)
if err != nil {
return nil, err
}
client, ctx, cancel, err := newRemoteMongodbClient(info)
if err != nil {
return nil, err
}
defer cancel()
defer client.Disconnect(ctx)
adminDB := client.Database("admin")
var listResult struct {
Databases []struct {
Name string `bson:"name"`
} `bson:"databases"`
}
if err := adminDB.RunCommand(ctx, bson.D{
{Key: "listDatabases", Value: 1},
{Key: "nameOnly", Value: false},
}).Decode(&listResult); err != nil {
return nil, err
}
systemDbs := map[string]struct{}{
"admin": {},
"config": {},
"local": {},
}
items := make([]mongodbSyncItem, 0, len(listResult.Databases))
for _, databaseItem := range listResult.Databases {
if _, ok := systemDbs[databaseItem.Name]; ok {
continue
}
items = append(items, mongodbSyncItem{
Name: databaseItem.Name,
Username: "",
})
}
return items, nil
}
func loadMongodbPrivilege(from, connectionName, dbName, username string) (string, error) {
if from == constant.AppResourceRemote {
return loadRemoteMongodbPrivilege(connectionName, dbName, username)
}
return loadLocalMongodbPrivilege(connectionName, dbName, username)
}
func updateMongodbPrivilege(from, connectionName, dbName, username, permission string) error {
if from == constant.AppResourceRemote {
return updateRemoteMongodbPrivilege(connectionName, dbName, username, permission)
}
return updateLocalMongodbPrivilege(connectionName, dbName, username, permission)
}
func loadLocalMongodbPrivilege(connectionName, dbName, username string) (string, error) {
databaseJSON, err := json.Marshal(dbName)
if err != nil {
return "", err
}
usernameJSON, err := json.Marshal(username)
if err != nil {
return "", err
}
script := strings.TrimSpace(fmt.Sprintf(`
const dbName = %s;
const userName = %s;
const result = db.getSiblingDB(dbName).runCommand({
usersInfo: userName,
showCredentials: false,
showCustomData: false
});
if (!result || result.ok !== 1) {
throw new Error("failed to load mongodb user privileges");
}
const roles = Array.isArray(result.users) && result.users.length > 0 ? result.users[0].roles || [] : [];
const permissions = roles.filter(role => role.db === dbName).map(role => role.role);
print("__1panel_json_begin__");
print(JSON.stringify(permissions));
print("__1panel_json_end__");
`, databaseJSON, usernameJSON))
stdout, err := runMongodbAdminScriptWithStdout(connectionName, script)
if err != nil {
return "", err
}
var permissions []string
jsonResult, err := extractMongodbJSONOutput(stdout)
if err != nil {
return "", err
}
if err := json.Unmarshal(jsonResult, &permissions); err != nil {
return "", err
}
return normalizeMongodbPrivilege(permissions), nil
}
func updateLocalMongodbPrivilege(connectionName, dbName, username, permission string) error {
databaseJSON, err := json.Marshal(dbName)
if err != nil {
return err
}
usernameJSON, err := json.Marshal(username)
if err != nil {
return err
}
permissionJSON, err := json.Marshal(permission)
if err != nil {
return err
}
script := strings.TrimSpace(fmt.Sprintf(`
const dbName = %s;
const userName = %s;
const permission = %s;
const targetDb = db.getSiblingDB(dbName);
const userInfo = targetDb.runCommand({
usersInfo: userName,
showCredentials: false,
showCustomData: false
});
if (!userInfo || userInfo.ok !== 1) {
throw new Error("failed to load mongodb user privileges");
}
if (!Array.isArray(userInfo.users) || userInfo.users.length === 0) {
throw new Error("mongodb user not found: " + userName);
}
const roles = (userInfo.users[0].roles || []).filter(role => role.db !== dbName);
roles.push({ role: permission, db: dbName });
const result = targetDb.runCommand({
updateUser: userName,
roles: roles
});
if (!result || result.ok !== 1) {
throw new Error("failed to update mongodb user privileges");
}
`, databaseJSON, usernameJSON, permissionJSON))
return runMongodbAdminScript(connectionName, script)
}
func loadRemoteMongodbPrivilege(connectionName, dbName, username string) (string, error) {
info, err := loadRemoteMongodbConnection(connectionName)
if err != nil {
return "", err
}
client, ctx, cancel, err := newRemoteMongodbClient(info)
if err != nil {
return "", err
}
defer cancel()
defer client.Disconnect(ctx)
targetDB := client.Database(dbName)
var result struct {
Users []struct {
Roles []struct {
Role string `bson:"role"`
DB string `bson:"db"`
} `bson:"roles"`
} `bson:"users"`
}
if err := targetDB.RunCommand(ctx, bson.D{
{Key: "usersInfo", Value: username},
{Key: "showCredentials", Value: false},
{Key: "showCustomData", Value: false},
}).Decode(&result); err != nil {
return "", err
}
permissions := make([]string, 0)
if len(result.Users) > 0 {
for _, role := range result.Users[0].Roles {
if role.DB == dbName {
permissions = append(permissions, role.Role)
}
}
}
return normalizeMongodbPrivilege(permissions), nil
}
func updateRemoteMongodbPrivilege(connectionName, dbName, username, permission string) error {
info, err := loadRemoteMongodbConnection(connectionName)
if err != nil {
return err
}
client, ctx, cancel, err := newRemoteMongodbClient(info)
if err != nil {
return err
}
defer cancel()
defer client.Disconnect(ctx)
targetDB := client.Database(dbName)
var userInfo struct {
Users []struct {
Roles []struct {
Role string `bson:"role"`
DB string `bson:"db"`
} `bson:"roles"`
} `bson:"users"`
}
if err := targetDB.RunCommand(ctx, bson.D{
{Key: "usersInfo", Value: username},
{Key: "showCredentials", Value: false},
{Key: "showCustomData", Value: false},
}).Decode(&userInfo); err != nil {
return err
}
if len(userInfo.Users) == 0 {
return fmt.Errorf("mongodb user %s not found in database %s", username, dbName)
}
roles := make(bson.A, 0, len(userInfo.Users[0].Roles)+1)
for _, role := range userInfo.Users[0].Roles {
if role.DB == dbName {
continue
}
roles = append(roles, bson.D{{Key: "role", Value: role.Role}, {Key: "db", Value: role.DB}})
}
roles = append(roles, bson.D{{Key: "role", Value: permission}, {Key: "db", Value: dbName}})
return targetDB.RunCommand(ctx, bson.D{
{Key: "updateUser", Value: username},
{Key: "roles", Value: roles},
}).Err()
}
func normalizeMongodbPrivilege(permissions []string) string {
roleMap := map[string]struct{}{
"dbOwner": {},
"read": {},
"readWrite": {},
"userAdmin": {},
}
for _, permission := range permissions {
if _, ok := roleMap[permission]; ok {
return permission
}
}
return ""
}
func isSupportedMongodbPrivilege(permission string) bool {
switch permission {
case "dbOwner", "read", "readWrite", "userAdmin":
return true
default:
return false
}
}
func extractMongodbJSONOutput(stdout string) ([]byte, error) {
const (
beginMark = "__1panel_json_begin__"
endMark = "__1panel_json_end__"
)
output := strings.TrimSpace(stdout)
if output == "" {
return nil, fmt.Errorf("empty mongodb command output")
}
beginIndex := strings.Index(output, beginMark)
endIndex := strings.LastIndex(output, endMark)
if beginIndex >= 0 && endIndex > beginIndex {
content := strings.TrimSpace(output[beginIndex+len(beginMark) : endIndex])
if content != "" {
return []byte(content), nil
}
}
lines := strings.Split(output, "\n")
for i := len(lines) - 1; i >= 0; i-- {
line := strings.TrimSpace(lines[i])
if line == "" {
continue
}
return []byte(line), nil
}
return nil, fmt.Errorf("mongodb command output does not contain json result")
}
+8 -8
View File
@@ -109,7 +109,7 @@ func (u *DeviceService) CheckDNS(key, value string) (bool, error) {
if err != nil {
return false, err
}
defer conn.Close()
defer func() { _ = conn.Close() }()
return true, nil
}
@@ -132,7 +132,7 @@ func (u *DeviceService) Update(key, value string) error {
if cmd.CheckIllegal(value) {
return buserr.New("ErrCmdIllegal")
}
if err := cmd.RunDefaultBashCf("%s hostnamectl set-hostname %s", cmd.SudoHandleCmd(), value); err != nil {
if err := cmd.NewCommandMgr(cmd.WithTimeout(20*time.Second)).Run("hostnamectl", "set-hostname", value); err != nil {
return err
}
_, _ = psutil.HOST.GetHostInfo(true)
@@ -245,7 +245,7 @@ func (u *DeviceService) UpdateSwap(req dto.SwapHelper) error {
}
cmdMgr := cmd.NewCommandMgr(cmd.WithTask(*taskItem))
if !req.IsNew {
if err := cmdMgr.RunBashCf("%s swapoff %s", cmd.SudoHandleCmd(), req.Path); err != nil {
if err := cmdMgr.Run("swapoff", req.Path); err != nil {
return fmt.Errorf("handle swapoff %s failed, %v", req.Path, err)
}
}
@@ -256,22 +256,22 @@ func (u *DeviceService) UpdateSwap(req dto.SwapHelper) error {
return operateSwapWithFile(true, req)
}
taskItem.LogStart(i18n.GetMsgByKey("CreateSwap"))
if err := cmdMgr.RunBashCf("%s dd if=/dev/zero of=%s bs=1024 count=%d", cmd.SudoHandleCmd(), req.Path, req.Size); err != nil {
if err := cmdMgr.Run("dd", "if=/dev/zero", fmt.Sprintf("of=%s", req.Path), "bs=1024", fmt.Sprintf("count=%d", req.Size)); err != nil {
return fmt.Errorf("handle dd %s failed, %v", req.Path, err)
}
taskItem.Log("chmod 0600 " + req.Path)
if err := cmdMgr.RunBashCf("%s chmod 0600 %s", cmd.SudoHandleCmd(), req.Path); err != nil {
if err := cmdMgr.Run("chmod", "0600", req.Path); err != nil {
return fmt.Errorf("handle chmod 0600 %s failed, %v", req.Path, err)
}
taskItem.LogStart(i18n.GetMsgByKey("FormatSwap"))
if err := cmdMgr.RunBashCf("%s mkswap -f %s", cmd.SudoHandleCmd(), req.Path); err != nil {
if err := cmdMgr.Run("mkswap", "-f", req.Path); err != nil {
return fmt.Errorf("handle mkswap -f %s failed, %v", req.Path, err)
}
taskItem.LogStart(i18n.GetMsgByKey("EnableSwap"))
if err := cmdMgr.RunBashCf("%s swapon %s", cmd.SudoHandleCmd(), req.Path); err != nil {
_ = cmdMgr.RunBashCf("%s swapoff %s", cmd.SudoHandleCmd(), req.Path)
if err := cmdMgr.Run("swapon", req.Path); err != nil {
_ = cmdMgr.Run("swapoff", req.Path)
return fmt.Errorf("handle swapoff %s failed, %v", req.Path, err)
}
return operateSwapWithFile(false, req)
+39 -21
View File
@@ -49,6 +49,7 @@ func (u *DeviceService) Scan() dto.CleanData {
Size: uint64(originalSize),
IsCheck: originalSize > 0,
IsRecommend: true,
CanDelete: false,
Type: "1panel_original",
Children: loadTreeWithDir(true, "1panel_original", originalPath, fileOp),
})
@@ -68,6 +69,7 @@ func (u *DeviceService) Scan() dto.CleanData {
Size: rollbackSize,
IsCheck: rollbackSize > 0,
IsRecommend: true,
CanDelete: false,
Type: "rollback",
Children: rollBackTree,
})
@@ -105,11 +107,7 @@ func (u *DeviceService) Clean(req []dto.Clean) {
case "tmp_backup":
dropFileOrDir(path.Join(global.Dir.LocalBackupDir, "tmp"))
case "unknown_backup":
if strings.HasPrefix(item.Name, path.Join(global.Dir.LocalBackupDir, "log/website")) {
dropFileOrDir(item.Name)
} else {
dropFile(item.Name)
}
dropFileOrDir(item.Name)
case "rollback":
dropFileOrDir(path.Join(global.Dir.BaseDir, rollbackPath, "app"))
@@ -269,6 +267,7 @@ func loadUpgradeTree(fileOp fileUtils.FileOp) dto.CleanTree {
Size: uint64(upgradeSize),
IsCheck: false,
IsRecommend: true,
CanDelete: false,
Type: "upgrade",
Children: loadTreeWithDir(true, "upgrade", upgradePath, fileOp),
}
@@ -302,6 +301,7 @@ func loadAgentPackage(fileOp fileUtils.FileOp) dto.CleanTree {
Label: "agent_packages",
IsCheck: false,
IsRecommend: true,
CanDelete: false,
Type: "agent",
}
files, _ := os.ReadDir(pathItem)
@@ -316,6 +316,7 @@ func loadAgentPackage(fileOp fileUtils.FileOp) dto.CleanTree {
Size: uint64(itemSize),
IsCheck: true,
IsRecommend: true,
CanDelete: true,
Type: "agent",
})
} else {
@@ -333,6 +334,7 @@ func loadAgentPackage(fileOp fileUtils.FileOp) dto.CleanTree {
Size: uint64(itemSize.Size()),
IsCheck: !isCurrentVersion,
IsRecommend: true,
CanDelete: true,
Type: "agent",
})
}
@@ -353,6 +355,7 @@ func loadBackupTree(fileOp fileUtils.FileOp) []dto.CleanTree {
Size: uint64(tmpSize),
IsCheck: tmpSize != 0,
IsRecommend: true,
CanDelete: true,
Type: "tmp_backup",
})
backupRecords, _ := backupRepo.ListRecord()
@@ -391,6 +394,7 @@ func loadUnknownApps(fileOp fileUtils.FileOp, recordMap map[string][]string) dto
Label: "unknown_app",
IsCheck: false,
IsRecommend: false,
CanDelete: false,
Name: backupPath,
Type: "unknown_backup",
}
@@ -419,6 +423,7 @@ func loadUnknownDbs(fileOp fileUtils.FileOp, recordMap map[string][]string) dto.
Name: backupPath,
IsCheck: false,
IsRecommend: false,
CanDelete: false,
Type: "unknown_backup",
}
_ = loadFileOrDirWithExclude(fileOp, 0, backupPath, &treeData, excludePaths)
@@ -442,6 +447,7 @@ func loadUnknownWebsites(fileOp fileUtils.FileOp, recordMap map[string][]string)
Name: backupPath,
IsCheck: false,
IsRecommend: false,
CanDelete: false,
Type: "unknown_backup",
}
_ = loadFileOrDirWithExclude(fileOp, 0, backupPath, &treeData, excludePaths)
@@ -460,6 +466,7 @@ func loadUnknownSnapshot(fileOp fileUtils.FileOp) dto.CleanTree {
Name: backupPath,
IsCheck: false,
IsRecommend: false,
CanDelete: false,
Type: "unknown_backup",
}
entries, _ := os.ReadDir(backupPath)
@@ -473,6 +480,7 @@ func loadUnknownSnapshot(fileOp fileUtils.FileOp) dto.CleanTree {
Label: entry.Name(),
IsCheck: false,
IsRecommend: false,
CanDelete: true,
Name: childPath,
Type: "unknown_backup",
}
@@ -499,6 +507,7 @@ func loadUnknownWebsiteLog(fileOp fileUtils.FileOp) dto.CleanTree {
Label: "unknown_website_log",
IsCheck: false,
IsRecommend: true,
CanDelete: false,
Type: "unknown_backup",
}
dir := path.Join(global.Dir.LocalBackupDir, "log/website")
@@ -522,6 +531,7 @@ func loadUnknownWebsiteLog(fileOp fileUtils.FileOp) dto.CleanTree {
Label: dirName,
IsCheck: true,
IsRecommend: true,
CanDelete: true,
Name: dirPath,
Type: "unknown_backup",
Size: uint64(itemSize),
@@ -552,6 +562,7 @@ func loadFileOrDirWithExclude(fileOp fileUtils.FileOp, index uint, dir string, r
Label: entry.Name(),
IsCheck: false,
IsRecommend: false,
CanDelete: true,
Name: childPath,
Type: "unknown_backup",
}
@@ -564,18 +575,19 @@ func loadFileOrDirWithExclude(fileOp fileUtils.FileOp, index uint, dir string, r
for _, child := range childNode.Children {
childNode.Size += child.Size
}
rootTree.Size += childNode.Size
} else {
itemSize, _ := fileOp.GetDirSize(childPath)
childNode.Size = uint64(itemSize)
rootTree.Size += childNode.Size
}
if childNode.Size == 0 {
continue
}
} else {
info, _ := entry.Info()
childNode.Size = uint64(info.Size())
rootTree.Size += childNode.Size
}
rootTree.Size += childNode.Size
rootTree.Children = append(rootTree.Children, childNode)
}
return nil
@@ -633,6 +645,7 @@ func loadDownloadTree(fileOp fileUtils.FileOp) []dto.CleanTree {
Label: "app_tmp_download",
IsCheck: true,
IsRecommend: true,
CanDelete: false,
Type: "app_tmp_download",
Name: "apps",
}
@@ -653,19 +666,19 @@ func loadLogTree(fileOp fileUtils.FileOp) []dto.CleanTree {
for _, file := range list1 {
size += file.Size
}
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "system_log", Size: size, Children: list1, Type: "system_log", IsRecommend: true})
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "system_log", Size: size, Children: list1, Type: "system_log", IsRecommend: true, CanDelete: false})
path2 := path.Join(global.Dir.TaskDir)
list2 := loadTreeWithDir(false, "task_log", path2, fileOp)
size2, _ := fileOp.GetDirSize(path2)
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "task_log", Size: uint64(size2), Children: list2, Type: "task_log"})
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "task_log", Size: uint64(size2), Children: list2, Type: "task_log", CanDelete: false})
websiteLogList := loadWebsiteLogTree(fileOp)
logTotalSize := uint64(0)
for _, websiteLog := range websiteLogList {
logTotalSize += websiteLog.Size
}
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "website_log", Size: logTotalSize, Children: websiteLogList, Type: "website_log", IsRecommend: false})
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "website_log", Size: logTotalSize, Children: websiteLogList, Type: "website_log", IsRecommend: false, CanDelete: false})
return treeData
}
@@ -679,11 +692,12 @@ func loadWebsiteLogTree(fileOp fileUtils.FileOp) []dto.CleanTree {
for _, website := range websites {
size3, _ := fileOp.GetDirSize(path.Join(GetSiteDir(website.Alias), "log"))
res = append(res, dto.CleanTree{
ID: uuid.NewString(),
Label: website.PrimaryDomain,
Size: uint64(size3),
Type: "website_log",
Name: website.Alias,
ID: uuid.NewString(),
Label: website.PrimaryDomain,
Size: uint64(size3),
Type: "website_log",
Name: website.Alias,
CanDelete: true,
})
}
return res
@@ -733,6 +747,7 @@ func loadAppTmpDownloadTree(fileOp fileUtils.FileOp) []dto.CleanTree {
appTree.Name = appKey
appTree.IsRecommend = true
appTree.IsCheck = true
appTree.CanDelete = false
for _, version := range missingVersions {
versionPath := filepath.Join(appPath, version)
size, _ := fileOp.GetDirSize(versionPath)
@@ -743,6 +758,7 @@ func loadAppTmpDownloadTree(fileOp fileUtils.FileOp) []dto.CleanTree {
Size: uint64(size),
IsCheck: true,
IsRecommend: true,
CanDelete: true,
Type: "app_tmp_download_version",
Name: path.Join(appKey, version),
})
@@ -770,7 +786,7 @@ func loadContainerTree() []dto.CleanTree {
imageSize += uint64(file.Size)
}
}
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "container_images", Size: imageSize, Children: nil, Type: "images", IsRecommend: true})
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "container_images", Size: imageSize, Children: nil, Type: "images", IsRecommend: true, CanDelete: true})
containerSize := uint64(0)
for _, file := range diskUsage.Containers {
@@ -778,7 +794,7 @@ func loadContainerTree() []dto.CleanTree {
containerSize += uint64(file.SizeRw)
}
}
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "container_containers", Size: containerSize, Children: nil, Type: "containers", IsRecommend: true})
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "container_containers", Size: containerSize, Children: nil, Type: "containers", IsRecommend: true, CanDelete: true})
volumeSize := uint64(0)
for _, file := range diskUsage.Volumes {
@@ -786,7 +802,7 @@ func loadContainerTree() []dto.CleanTree {
volumeSize += uint64(file.UsageData.Size)
}
}
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "container_volumes", Size: volumeSize, IsCheck: volumeSize > 0, Children: nil, Type: "volumes", IsRecommend: true})
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "container_volumes", Size: volumeSize, IsCheck: volumeSize > 0, Children: nil, Type: "volumes", IsRecommend: true, CanDelete: true})
var buildCacheTotalSize int64
for _, cache := range diskUsage.BuildCache {
@@ -794,7 +810,7 @@ func loadContainerTree() []dto.CleanTree {
buildCacheTotalSize += cache.Size
}
}
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "build_cache", Size: uint64(buildCacheTotalSize), IsCheck: buildCacheTotalSize > 0, Type: "build_cache", IsRecommend: true})
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: "build_cache", Size: uint64(buildCacheTotalSize), IsCheck: buildCacheTotalSize > 0, Type: "build_cache", IsRecommend: true, CanDelete: true})
return treeData
}
@@ -804,7 +820,7 @@ func loadTreeWithCheck(treeData []dto.CleanTree, pathItem, treeType string, file
return treeData
}
list := loadTreeWithAllFile(true, pathItem, treeType, pathItem, fileOp)
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: treeType, Size: uint64(size), IsCheck: size > 0, Children: list, Type: treeType, IsRecommend: true})
treeData = append(treeData, dto.CleanTree{ID: uuid.NewString(), Label: treeType, Size: uint64(size), IsCheck: size > 0, Children: list, Type: treeType, IsRecommend: true, CanDelete: false})
return treeData
}
@@ -831,6 +847,7 @@ func loadTreeWithDir(isCheck bool, treeType, pathItem string, fileOp fileUtils.F
Name: strings.TrimPrefix(file.Name(), "/"),
IsCheck: isCheck,
IsRecommend: isCheck,
CanDelete: true,
}
lists = append(lists, item)
}
@@ -878,6 +895,7 @@ func loadTreeWithAllFile(isCheck bool, originalPath, treeType, pathItem string,
Name: name,
IsCheck: isCheck,
IsRecommend: isCheck,
CanDelete: true,
}
if file.IsDir() {
item.Children = loadTreeWithAllFile(isCheck, originalPath, treeType, path.Join(pathItem, file.Name()), fileOp)
+10 -9
View File
@@ -2,13 +2,14 @@ package service
import (
"fmt"
"os"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto"
"github.com/1Panel-dev/1Panel/agent/buserr"
"github.com/1Panel-dev/1Panel/agent/global"
"github.com/1Panel-dev/1Panel/agent/utils/cmd"
"os"
"strings"
"time"
"github.com/1Panel-dev/1Panel/agent/app/dto/request"
"github.com/1Panel-dev/1Panel/agent/app/dto/response"
@@ -65,19 +66,19 @@ func (s *DiskService) PartitionDisk(req request.DiskPartitionRequest) (string, e
}
cmdMgr := cmd.NewCommandMgr(cmd.WithTimeout(10 * time.Second))
if err := cmdMgr.RunBashC(fmt.Sprintf("partprobe %s", req.Device)); err != nil {
if err := cmdMgr.Run("partprobe", req.Device); err != nil {
return "", buserr.WithErr("PartitionDiskErr", err)
}
if err := cmdMgr.RunBashC(fmt.Sprintf("parted -s %s mklabel gpt", req.Device)); err != nil {
if err := cmdMgr.Run("parted", "-s", req.Device, "mklabel", "gpt"); err != nil {
return "", buserr.WithErr("PartitionDiskErr", err)
}
if err := cmdMgr.RunBashC(fmt.Sprintf("parted -s %s mkpart primary 1MiB 100%%", req.Device)); err != nil {
if err := cmdMgr.Run("parted", "-s", req.Device, "mkpart", "primary", "1MiB", "100%"); err != nil {
return "", buserr.WithErr("PartitionDiskErr", err)
}
if err := cmdMgr.RunBashC(fmt.Sprintf("partprobe %s", req.Device)); err != nil {
if err := cmdMgr.Run("partprobe", req.Device); err != nil {
return "", buserr.WithErr("PartitionDiskErr", err)
}
partition := req.Device + "1"
@@ -133,7 +134,7 @@ func (s *DiskService) MountDisk(req request.DiskMountRequest) error {
}
cmdMgr := cmd.NewCommandMgr(cmd.WithTimeout(1 * time.Minute))
if err := cmdMgr.RunBashC(fmt.Sprintf("mount -t %s %s %s", req.Filesystem, req.Device, req.MountPoint)); err != nil {
if err := cmdMgr.Run("mount", "-t", req.Filesystem, req.Device, req.MountPoint); err != nil {
return buserr.WithErr("MountDiskErr", err)
}
if req.AutoMount {
@@ -156,7 +157,7 @@ func (s *DiskService) UnmountDisk(req request.DiskUnmountRequest) error {
if !isPointMounted(req.MountPoint) {
return buserr.New("MountDiskErr")
}
if err := cmd.RunDefaultBashC(fmt.Sprintf("umount -f %s", req.MountPoint)); err != nil {
if err := cmd.NewCommandMgr(cmd.WithTimeout(20*time.Second)).Run("umount", "-f", req.MountPoint); err != nil {
return buserr.WithErr("MountDiskErr", err)
}
if err := removeFromFstab(req.MountPoint); err != nil {
+9 -6
View File
@@ -12,14 +12,16 @@ var (
appInstallResourceRepo = repo.NewIAppInstallResourceRpo()
appIgnoreUpgradeRepo = repo.NewIAppIgnoreUpgradeRepo()
aiRepo = repo.NewIAiRepo()
mcpServerRepo = repo.NewIMcpServerRepo()
tensorrtLLMRepo = repo.NewITensorRTLLMRepo()
agentRepo = repo.NewIAgentRepo()
agentAccountRepo = repo.NewIAgentAccountRepo()
aiRepo = repo.NewIAiRepo()
mcpServerRepo = repo.NewIMcpServerRepo()
tensorrtLLMRepo = repo.NewITensorRTLLMRepo()
agentRepo = repo.NewIAgentRepo()
agentAccountRepo = repo.NewIAgentAccountRepo()
agentAccountModelRepo = repo.NewIAgentAccountModelRepo()
mysqlRepo = repo.NewIMysqlRepo()
postgresqlRepo = repo.NewIPostgresqlRepo()
mongodbRepo = repo.NewIMongodbRepo()
databaseRepo = repo.NewIDatabaseRepo()
imageRepoRepo = repo.NewIImageRepoRepo()
@@ -48,7 +50,8 @@ var (
runtimeRepo = repo.NewIRunTimeRepo()
phpExtensionsRepo = repo.NewIPHPExtensionsRepo()
favoriteRepo = repo.NewIFavoriteRepo()
favoriteRepo = repo.NewIFavoriteRepo()
fileShareRepo = repo.NewIFileShareRepo()
taskRepo = repo.NewITaskRepo()
+455 -33
View File
@@ -2,6 +2,7 @@ package service
import (
"bufio"
"bytes"
"context"
"encoding/base64"
"encoding/json"
@@ -39,8 +40,10 @@ import (
"golang.org/x/text/transform"
"github.com/1Panel-dev/1Panel/agent/global"
"github.com/1Panel-dev/1Panel/agent/utils/cmd"
"github.com/1Panel-dev/1Panel/agent/utils/common"
"github.com/1Panel-dev/1Panel/agent/utils/files"
terminalai "github.com/1Panel-dev/1Panel/agent/utils/terminal/ai"
"github.com/pkg/errors"
)
@@ -55,7 +58,9 @@ type IFileService interface {
Delete(op request.FileDelete) error
BatchDelete(op request.FileBatchDelete) error
Compress(c request.FileCompress) error
StopCompress(taskID string) error
DeCompress(c request.FileDeCompress) error
StopDeCompress(taskID string) error
GetContent(op request.FileContentReq) (response.FileInfo, error)
GetPreviewContent(op request.FileContentReq) (response.FileInfo, error)
SaveContent(edit request.FileEdit) error
@@ -78,10 +83,7 @@ type IFileService interface {
ConvertLog(req dto.PageInfo) (int64, []response.FileConvertLog, error)
BatchGetRemarks(req request.FileRemarkBatch) map[string]string
SetRemark(req request.FileRemarkUpdate) error
}
var filteredPaths = []string{
"/.1panel_clash",
AISearch(req request.FileAISearch) (*response.FileAISearchResult, error)
}
const (
@@ -106,10 +108,29 @@ func (f *FileService) GetFileList(op request.FileOption) (response.FileInfo, err
if err != nil {
return fileInfo, err
}
shareMap, err := NewIFileShareService().SharePathCodeMap()
if err != nil {
return fileInfo, err
}
applyFileShares(info, shareMap)
fileInfo.FileInfo = *info
return fileInfo, nil
}
func applyFileShares(info *files.FileInfo, shareMap map[string]string) {
if info == nil {
return
}
if code, ok := shareMap[info.Path]; ok {
info.ShareCode = code
} else {
info.ShareCode = ""
}
for _, item := range info.Items {
applyFileShares(item, shareMap)
}
}
func (f *FileService) SearchUploadWithPage(req request.SearchUploadWithPage) (int64, interface{}, error) {
var (
files []response.UploadInfo
@@ -170,14 +191,7 @@ func (f *FileService) GetFileTree(op request.FileOption) ([]response.FileTree, e
}
func shouldFilterPath(path string) bool {
cleanedPath := filepath.Clean(path)
for _, filteredPath := range filteredPaths {
cleanedFilteredPath := filepath.Clean(filteredPath)
if cleanedFilteredPath == cleanedPath || strings.HasPrefix(cleanedPath, cleanedFilteredPath+"/") {
return true
}
}
return false
return files.ShouldFilterSensitivePath(path)
}
func (f *FileService) buildFileTree(node *response.FileTree, items []*files.FileInfo, op request.FileOption, level int) error {
@@ -219,8 +233,12 @@ func (f *FileService) buildChildNode(childNode *response.FileTree, fileInfo *fil
return f.buildFileTree(childNode, subInfo.Items, op, level-1)
}
func hasInvalidFileName(fullPath string) bool {
return files.IsInvalidChar(filepath.Base(fullPath))
}
func (f *FileService) Create(op request.FileCreate) error {
if files.IsInvalidChar(op.Path) {
if hasInvalidFileName(op.Path) {
return buserr.New("ErrInvalidChar")
}
fo := files.NewFileOp()
@@ -263,7 +281,7 @@ func (f *FileService) Create(op request.FileCreate) error {
func (f *FileService) Delete(op request.FileDelete) error {
if op.IsDir {
excludeDir := global.Dir.DataDir
if filepath.Base(op.Path) == ".1panel_clash" || op.Path == excludeDir {
if path.Base(op.Path) == ".1panel_clash" || op.Path == excludeDir {
return buserr.New("ErrPathNotDelete")
}
}
@@ -272,16 +290,37 @@ func (f *FileService) Delete(op request.FileDelete) error {
if recycleBinStatus.Value == "Disable" {
op.ForceDelete = true
}
var historyTargets []string
if op.ForceDelete {
if op.IsDir {
return fo.DeleteDir(op.Path)
} else {
return fo.DeleteFile(op.Path)
var err error
historyTargets, err = f.collectPermanentDeleteTargets(op.Path, op.IsDir)
if err != nil {
return err
}
}
if op.ForceDelete {
var err error
if op.IsDir {
err = fo.DeleteDir(op.Path)
} else {
err = fo.DeleteFile(op.Path)
}
if err != nil {
return err
}
if err := cleanupTrashInfoByEntryPath(op.Path); err != nil {
global.LOG.Warnf("cleanup trashinfo failed for %s: %v", op.Path, err)
}
f.cleanupPermanentDeleteHistory(historyTargets)
return nil
}
info, _ := fo.Fs.Stat(op.Path)
if info == nil || files.IsSymlink(info.Mode()) {
return os.Remove(op.Path)
if err := os.Remove(op.Path); err != nil {
return err
}
f.cleanupPermanentDeleteHistory([]string{op.Path})
return nil
}
if err := NewIRecycleBinService().Create(request.RecycleBinCreate{SourcePath: op.Path}); err != nil {
@@ -294,15 +333,27 @@ func (f *FileService) BatchDelete(op request.FileBatchDelete) error {
fo := files.NewFileOp()
if op.IsDir {
for _, file := range op.Paths {
targets, err := f.collectPermanentDeleteTargets(file, true)
if err != nil {
return err
}
if err := fo.DeleteDir(file); err != nil {
return err
}
if err := cleanupTrashInfoByEntryPath(file); err != nil {
global.LOG.Warnf("cleanup trashinfo failed for %s: %v", file, err)
}
f.cleanupPermanentDeleteHistory(targets)
}
} else {
for _, file := range op.Paths {
if err := fo.DeleteFile(file); err != nil {
return err
}
if err := cleanupTrashInfoByEntryPath(file); err != nil {
global.LOG.Warnf("cleanup trashinfo failed for %s: %v", file, err)
}
f.cleanupPermanentDeleteHistory([]string{file})
}
}
return nil
@@ -310,7 +361,10 @@ func (f *FileService) BatchDelete(op request.FileBatchDelete) error {
func (f *FileService) ChangeMode(op request.FileCreate) error {
fo := files.NewFileOp()
return fo.ChmodR(op.Path, op.Mode, op.Sub)
if err := fo.ChmodR(op.Path, op.Mode, op.Sub); err != nil {
return err
}
return nil
}
func (f *FileService) BatchChangeModeAndOwner(op request.FileRoleReq) error {
@@ -329,9 +383,44 @@ func (f *FileService) BatchChangeModeAndOwner(op request.FileRoleReq) error {
return nil
}
func (f *FileService) collectPermanentDeleteTargets(targetPath string, isDir bool) ([]string, error) {
if !isDir {
return []string{targetPath}, nil
}
var targets []string
if err := filepath.WalkDir(targetPath, func(currentPath string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d == nil || d.IsDir() {
return nil
}
targets = append(targets, currentPath)
return nil
}); err != nil {
return nil, err
}
return targets, nil
}
func (f *FileService) cleanupPermanentDeleteHistory(targets []string) {
if len(targets) == 0 {
return
}
for _, target := range targets {
if err := historyService.DeleteRelatedHistory(target); err != nil {
global.LOG.Warnf("cleanup file history failed for %s: %v", target, err)
}
}
}
func (f *FileService) ChangeOwner(req request.FileRoleUpdate) error {
fo := files.NewFileOp()
return fo.ChownR(req.Path, req.User, req.Group, req.Sub)
if err := fo.ChownR(req.Path, req.User, req.Group, req.Sub); err != nil {
return err
}
return nil
}
func (f *FileService) Compress(c request.FileCompress) error {
@@ -339,7 +428,76 @@ func (f *FileService) Compress(c request.FileCompress) error {
if !c.Replace && fo.Stat(filepath.Join(c.Dst, c.Name)) {
return buserr.New("ErrFileIsExist")
}
return fo.Compress(c.Files, c.Dst, c.Name, files.CompressType(c.Type), c.Secret)
if err := preflightCompressTool(files.CompressType(c.Type)); err != nil {
return err
}
taskItem, err := task.NewTask(c.Name, task.TaskExec, task.TaskScopeTask, c.TaskID, 1)
if err != nil {
return err
}
go func() {
taskItem.AddSubTask(c.Name, func(t *task.Task) error {
t.LogStart(c.Name)
compressType := files.CompressType(c.Type)
dstFile := filepath.Join(c.Dst, c.Name)
success := false
defer func() {
if !success {
_ = os.Remove(dstFile)
}
}()
if err := fo.Compress(t.TaskCtx, c.Files, c.Dst, c.Name, compressType, c.Secret, nil); err != nil {
return err
}
info, err := os.Stat(dstFile)
if err != nil {
return err
}
if info.Size() == 0 {
return fmt.Errorf("compressed file not generated: %s", dstFile)
}
success = true
return nil
}, nil)
_ = taskItem.Execute()
}()
return nil
}
func preflightCompressTool(compressType files.CompressType) error {
switch compressType {
case files.TarGz, files.Rar, files.X7z:
_, err := files.NewShellArchiver(compressType)
return err
default:
return nil
}
}
func preflightDecompressTool(decompressType files.CompressType) error {
switch decompressType {
case files.Tar, files.Zip, files.TarGz, files.Rar, files.X7z:
_, err := files.NewExtractShellArchiver(decompressType)
return err
default:
return nil
}
}
func (f *FileService) StopCompress(taskID string) error {
if cancel, ok := global.TaskCtxMap[taskID]; ok {
cancel()
return nil
}
return buserr.New("TaskNotFound")
}
func (f *FileService) StopDeCompress(taskID string) error {
if cancel, ok := global.TaskCtxMap[taskID]; ok {
cancel()
return nil
}
return buserr.New("TaskNotFound")
}
func (f *FileService) DeCompress(c request.FileDeCompress) error {
@@ -347,7 +505,49 @@ func (f *FileService) DeCompress(c request.FileDeCompress) error {
if c.Type == "tar" && len(c.Secret) != 0 {
c.Type = "tar.gz"
}
return fo.Decompress(c.Path, c.Dst, files.CompressType(c.Type), c.Secret)
if err := preflightDecompressTool(files.CompressType(c.Type)); err != nil {
return err
}
taskItem, err := task.NewTask(c.Path, task.TaskExec, task.TaskScopeTask, c.TaskID, 1)
if err != nil {
return err
}
go func() {
taskItem.AddSubTask(c.Path, func(t *task.Task) error {
t.LogStart(c.Path)
dstExisted := fo.Stat(c.Dst)
parentDir := filepath.Dir(c.Dst)
if !fo.Stat(parentDir) {
if err := fo.CreateDir(parentDir, constant.DirPerm); err != nil {
return err
}
}
tempDst, err := os.MkdirTemp(parentDir, ".decompress-*")
if err != nil {
return err
}
success := false
defer func() {
_ = os.RemoveAll(tempDst)
if !success && !dstExisted {
_ = os.RemoveAll(c.Dst)
}
}()
if err := fo.Decompress(t.TaskCtx, c.Path, tempDst, files.CompressType(c.Type), c.Secret); err != nil {
return err
}
if err := fo.CreateDir(c.Dst, constant.DirPerm); err != nil {
return err
}
if err := cmd.NewCommandMgr(cmd.WithContext(t.TaskCtx)).RunBashCf("cp -rfp '%s'/. '%s'", tempDst, c.Dst); err != nil {
return err
}
success = true
return nil
}, nil)
_ = taskItem.Execute()
}()
return nil
}
func (f *FileService) GetContent(op request.FileContentReq) (response.FileInfo, error) {
@@ -472,15 +672,36 @@ func (f *FileService) SaveContent(edit request.FileEdit) error {
}
fo := files.NewFileOp()
return fo.WriteFile(edit.Path, strings.NewReader(edit.Content), info.FileMode)
oldContent, _ := os.ReadFile(edit.Path)
if bytes.Equal(oldContent, []byte(edit.Content)) {
return nil
}
if err := fo.WriteFile(edit.Path, strings.NewReader(edit.Content), info.FileMode); err != nil {
return err
}
if err := historyService.RecordSave(edit.Path, oldContent, info.FileMode); err != nil {
global.LOG.Warnf("record file save history failed for %s: %v", edit.Path, err)
}
return nil
}
func (f *FileService) ChangeName(req request.FileRename) error {
if files.IsInvalidChar(req.NewName) {
if hasInvalidFileName(req.NewName) {
return buserr.New("ErrInvalidChar")
}
fo := files.NewFileOp()
return fo.Rename(req.OldName, req.NewName)
info, _ := files.NewFileInfo(files.FileOption{Path: req.OldName, Expand: false})
content, _ := os.ReadFile(req.OldName)
if err := fo.Rename(req.OldName, req.NewName); err != nil {
return err
}
if info != nil && !info.IsDir {
if histErr := historyService.RecordOperation(fileHistoryOpRename, req.OldName, content, info.FileMode, req.OldName, req.NewName); histErr != nil {
global.LOG.Warnf("record file rename history failed for %s: %v", req.OldName, histErr)
}
}
return nil
}
func (f *FileService) Wget(w request.FileWget) (string, error) {
@@ -502,6 +723,23 @@ func (f *FileService) MvFile(m request.FileMove) error {
return buserr.New("ErrMovePathFailed")
}
}
type moveSnapshot struct {
path string
content []byte
mode os.FileMode
isDir bool
}
snapshots := make([]moveSnapshot, 0, len(m.OldPaths))
for _, oldPath := range m.OldPaths {
content, _ := os.ReadFile(oldPath)
mode := os.FileMode(0640)
isDir := false
if info, err := files.NewFileInfo(files.FileOption{Path: oldPath, Expand: false}); err == nil {
mode = info.FileMode
isDir = info.IsDir
}
snapshots = append(snapshots, moveSnapshot{path: oldPath, content: content, mode: mode, isDir: isDir})
}
var errs []error
if m.Type == "cut" {
if len(m.CoverPaths) > 0 {
@@ -512,7 +750,18 @@ func (f *FileService) MvFile(m request.FileMove) error {
}
}
}
return fo.Cut(m.OldPaths, m.NewPath, m.Name, m.Cover)
if err := fo.Cut(m.OldPaths, m.NewPath, m.Name, m.Cover); err != nil {
return err
}
for _, snapshot := range snapshots {
if !snapshot.isDir {
targetPath := buildHistoryMoveTargetPath(m.NewPath, m.Name, snapshot.path, len(m.OldPaths))
if histErr := historyService.RecordOperation(fileHistoryOpMove, snapshot.path, snapshot.content, snapshot.mode, snapshot.path, targetPath); histErr != nil {
global.LOG.Warnf("record file move history failed for %s: %v", snapshot.path, histErr)
}
}
}
return nil
}
if m.Type == "copy" {
for _, src := range m.OldPaths {
@@ -541,6 +790,16 @@ func (f *FileService) MvFile(m request.FileMove) error {
return nil
}
func buildHistoryMoveTargetPath(dst, name, sourcePath string, sourceCount int) string {
if strings.TrimSpace(dst) == "" {
return sourcePath
}
if strings.TrimSpace(name) != "" && sourceCount == 1 {
return filepath.Join(dst, name)
}
return filepath.Join(dst, filepath.Base(sourcePath))
}
func (f *FileService) FileDownload(d request.FileDownload) (string, error) {
filePath := d.Paths[0]
if d.Compress {
@@ -549,7 +808,7 @@ func (f *FileService) FileDownload(d request.FileDownload) (string, error) {
return "", err
}
fo := files.NewFileOp()
if err := fo.Compress(d.Paths, tempPath, d.Name, files.CompressType(d.Type), ""); err != nil {
if err := fo.Compress(context.Background(), d.Paths, tempPath, d.Name, files.CompressType(d.Type), "", nil); err != nil {
return "", err
}
filePath = filepath.Join(tempPath, d.Name)
@@ -592,6 +851,12 @@ func (f *FileService) DepthDirSize(req request.DirSizeReq) ([]response.DepthDirS
func (f *FileService) ReadLogByLine(req request.FileReadByLineReq) (*response.FileLineContent, error) {
logFilePath := ""
taskStatus := ""
if len(req.Name) != 0 {
safeName := path.Base(req.Name)
if safeName != req.Name || strings.Contains(safeName, "..") {
return nil, buserr.New("ErrInvalidParams")
}
}
switch req.Type {
case constant.TypeWebsite:
website, err := websiteRepo.GetFirst(repo.WithByID(req.ID))
@@ -649,9 +914,9 @@ func (f *FileService) ReadLogByLine(req request.FileReadByLineReq) (*response.Fi
logFilePath = taskModel.LogFile
taskStatus = taskModel.Status
case "mysql-slow-logs":
logFilePath = path.Join(global.Dir.DataDir, fmt.Sprintf("apps/mysql/%s/data/1Panel-slow.log", req.Name))
logFilePath = path.Join(global.Dir.DataDir, "apps", "mysql", req.Name, "data", "1Panel-slow.log")
case "mariadb-slow-logs":
logFilePath = path.Join(global.Dir.DataDir, fmt.Sprintf("apps/mariadb/%s/db/data/1Panel-slow.log", req.Name))
logFilePath = path.Join(global.Dir.DataDir, "apps", "mariadb", req.Name, "db", "data", "1Panel-slow.log")
case "php-fpm-slow-logs":
php, err := runtimeRepo.GetFirst(context.Background(), repo.WithByID(req.ID))
if err != nil {
@@ -666,8 +931,7 @@ func (f *FileService) ReadLogByLine(req request.FileReadByLineReq) (*response.Fi
}
logFilePath, _ = ini_conf.GetIniValue(configPath, "supervisord", "logfile")
case constant.Supervisor:
logDir := path.Join(global.Dir.DataDir, "tools", "supervisord", "log")
logFilePath = path.Join(logDir, req.Name)
logFilePath = path.Join(global.Dir.DataDir, "tools", "supervisord", "log", req.Name)
}
file, err := os.Open(logFilePath)
@@ -686,7 +950,7 @@ func (f *FileService) ReadLogByLine(req request.FileReadByLineReq) (*response.Fi
logFileRes *dto.LogFileRes
)
if stat.Size() > files.MaxReadFileSize {
lines, err = files.TailFromEnd(logFilePath, req.PageSize)
lines, _ = files.TailFromEnd(logFilePath, req.PageSize)
isEndOfFile = true
scope = "tail"
} else {
@@ -1014,3 +1278,161 @@ func (f *FileService) ConvertLog(req dto.PageInfo) (total int64, data []response
return total, data, nil
}
func (f *FileService) AISearch(req request.FileAISearch) (*response.FileAISearchResult, error) {
root := filepath.Clean(strings.TrimSpace(req.Path))
if root == "" {
return nil, buserr.WithDetail("ErrInvalidParams", "path is required", nil)
}
query := strings.TrimSpace(req.Query)
if query == "" {
return nil, buserr.WithDetail("ErrInvalidParams", "query is required", nil)
}
st, err := os.Stat(root)
if err != nil {
if os.IsNotExist(err) {
return nil, buserr.New("ErrPathNotFound")
}
return nil, err
}
if !st.IsDir() {
return nil, buserr.New("ErrPathNotFound")
}
maxItems := req.MaxItems
if maxItems <= 0 {
maxItems = files.DefaultFileAIMaxItems
}
if maxItems > 2000 {
maxItems = 2000
}
containSub := true
if req.ContainSub != nil {
containSub = *req.ContainSub
}
searchOpts, err := files.MergeContentSearchOptions(
req.MatchCase, req.WholeWord, req.UseRegex,
req.Extensions,
req.MinSize, req.MaxSize,
req.ModifiedAfter, req.ModifiedBefore,
req.MaxScanFiles,
req.MaxFileBytes,
req.MaxHitsPerFile, req.MaxTotalHits,
req.ContentHitsPromptMaxBytes,
req.LlmMaxOutputTokens,
)
if err != nil {
return nil, buserr.WithDetail("ErrInvalidParams", err.Error(), nil)
}
matchFn, err := files.NewContentLineMatcher(query, searchOpts)
if err != nil {
return nil, buserr.WithDetail("ErrFileAISearchBadPattern", err.Error(), nil)
}
cfg, timeout, err := terminalai.LoadFileAIRuntimeConfig()
aiEnabled := err == nil
if err != nil && !errors.Is(err, os.ErrNotExist) {
return nil, err
}
items, truncated, err := files.CollectDirInventory(root, containSub, maxItems)
if err != nil {
return nil, err
}
preFiltered := false
llmItems := items
qLower := strings.ToLower(query)
if len(llmItems) > 0 && query != "" {
filtered := make([]files.AISearchInventoryItem, 0, len(llmItems))
for _, it := range llmItems {
rel := strings.TrimSpace(it.RelPath)
if rel == "" {
continue
}
if !req.UseRegex && !req.MatchCase && !req.WholeWord && strings.Contains(strings.ToLower(rel), qLower) {
filtered = append(filtered, it)
}
}
if len(filtered) >= 8 {
llmItems = filtered
preFiltered = true
}
}
start := time.Now()
contentHits, scannedFiles, hitsTrunc := files.SearchFileAIContentHits(root, llmItems, searchOpts, matchFn)
hitsDTO := make([]response.FileAIContentHit, 0, len(contentHits))
for _, h := range contentHits {
hitsDTO = append(hitsDTO, response.FileAIContentHit{Path: h.Path, Line: h.Line, Text: h.Text})
}
matchDesc := searchOpts.ContentMatchDescription()
result := &response.FileAISearchResult{
Hits: hitsDTO,
ContentScannedFiles: scannedFiles,
ContentHitsTruncated: hitsTrunc,
Truncated: truncated,
PreFiltered: preFiltered,
ItemCount: len(llmItems),
}
if len(llmItems) == 0 {
if aiEnabled {
result.Mode = "ai"
result.Summary = i18n.GetMsgByKey("FileAISearchEmptyDir")
if result.Summary == "" || result.Summary == "FileAISearchEmptyDir" {
result.Summary = "No files or directories found under this path (or all entries were filtered)."
}
} else {
result.Mode = "grep"
result.Summary = ""
}
result.Duration = time.Since(start).Round(time.Millisecond).String()
return result, nil
}
if !aiEnabled {
result.Mode = "grep"
result.Summary = ""
result.Duration = time.Since(start).Round(time.Millisecond).String()
return result, nil
}
result.Mode = "ai"
clientTimeout := timeout
if clientTimeout < 30*time.Second {
clientTimeout = 90 * time.Second
}
if clientTimeout > 5*time.Minute {
clientTimeout = 5 * time.Minute
}
runCtx, cancel := context.WithTimeout(context.Background(), timeout+time.Minute)
defer cancel()
llmMaxOut := searchOpts.LlmMaxOutputTokens
summary, usage, err := files.RunFileAISearchLLM(runCtx, cfg, clientTimeout, root, query, req.ResponseLanguage, llmItems, truncated, preFiltered, contentHits, scannedFiles, hitsTrunc, matchDesc, searchOpts.ContentHitsPromptMaxBytes, llmMaxOut)
if err != nil {
result.Mode = "grep"
result.Summary = ""
result.Duration = time.Since(start).Round(time.Millisecond).String()
if errors.Is(err, context.DeadlineExceeded) || strings.Contains(strings.ToLower(err.Error()), "timeout") {
return result, nil
}
return result, nil
}
result.Summary = summary
result.PromptTokens = usage.PromptTokens
result.CompletionTokens = usage.CompletionTokens
result.TotalTokens = usage.TotalTokens
if result.TotalTokens == 0 {
result.TotalTokens = usage.PromptTokens + usage.CompletionTokens
}
result.Duration = time.Since(start).Round(time.Millisecond).String()
return result, nil
}

Some files were not shown because too many files have changed in this diff Show More