Commit Graph
27 Commits
Author SHA1 Message Date
Subhra264 8482a913b3 fix: show savedview and report resources in role permissions and fix list_objects_for_user for _all_{org} (#3284)
For enterprise in rbac, `GET` methods for `reports` and `savedviews`
consists of `{org_id}/<reports|savedviews>/{id}` format, i.e. `url_len`
is 3. This PR corrects how the fga object string is extracted (which
should be `report:{report_name}` or `savedview:{view_id}`) from the url
path with the above format with `GET` method.
2024-04-23 15:09:09 +05:30
Huaijin 0ec3285e1a fix: add order by field to select when need (#3259) 2024-04-16 18:04:55 +08:00
Subhra264 e9e8160391 feat: add audit trails for enterprise version (#3211) 2024-04-16 11:40:32 +05:30
Subhra264 08d12f8571 feat: introduce reports in backend and replace TRIGGERS with infra::scheduler (#3086)
## Reports
Both `ZO_CHROME_ENABLED` and `ZO_SMTP_ENABLED` ENV variables need to be
`true` for reports to work.

**Report APIs -**
```
POST /api/{org_id}/reports
PUT /api/{org_id}/reports/{report_name}
GET /api/{org_id}/reports
GET /api/{org_id}/reports/{report_name}
DELETE /api/{org_id}/reports/{report_name}
PUT /api/{org_id}/reports/{report_name}/enable?value=<true|false>
```

**Report body**
```
{
  "createdAt": "2024-03-23T09:46:32.913574782Z",
  "dashboards": [
    {
      "dashboard": "7177215109754781699",
      "folder": "default",
      "tabs": [
        "default"
      ],
      "timerange": {
        "from": 0,
        "period": "1h",
        "to": 0,
        "type": "relative"
      },
      "variables": []
    }
  ],
  "description": "Report description",
  "destinations": [
    {
      "email": "mail@gmail.com"
    }
  ],
  "enabled": false,
  "frequency": {
    "cron": "",
    "interval": 1,
    "type": "weeks"
  },
  "lastEditedBy": "root@example.com",
  "mediaType": "pdf",
  "message": "Email message",
  "name": "report_name",
  "orgId": "default",
  "owner": "root@example.com",
  "password": "ComplexPass#123",
  "start": 1711187192913571,
  "timezone": "Asia/Calcutta",
  "timezoneOffset": 330,
  "title": "Email subject",
  "user": "root@example.com"
}
```

**ENVs**
| ENV |  Description | 
|------|---------------|
| `ZO_CHROME_ENABLED` | Must be `true` for reports, Default - `false` |
| `ZO_CHROME_PATH` | If chrome is enabled, custom `chrome` executable
path can be specified. If not specified, it looks for chrome executable
in default locations. If still not found, it automatically downloads a
good known version of `chromium`. |
| `ZO_CHROME_CHECK_DEFAULT_PATH` | If `false`, it does not look for
chromium in default locations, Default is `true`. |
| `ZO_CHROME_DOWNLOAD_PATH` | If chromium can not be found in default
locations and also `ZO_CHROME_PATH` is not specified, it downloads the
system specific chromium in the given path. Default is `./download`
(gitignored). `chromium` is downloaded for the first time only,
afterwords, `chromium` is fetched from the given path. If there is any
error regarding download of `chromium`, delete the download folder as it
might be in a bad state. |
| `ZO_CHROME_NO_SANDBOX` | If `true`, it launches chromium in
`no-sandbox` environment. Default is `false` |
| `ZO_CHROME_SLEEP_SECS` | Specify the number of seconds the
`service::dashboards::reports::generate_report()` will wait to load all
the data in the headless chromium dashboard. Default is `20` seconds. |
| `ZO_CHROME_WINDOW_WIDTH` | Specifies the width of the headless
chromium browser. Default is `1370` |
| `ZO_CHROME_WINDOW_HEIGHT` | Specifies the height of the headless
chromium browser. Default is `730` |

## Scheduler

Currently `Scheduler` is implemented for only `Sqlite`, `Postgres` and
`MySql`.
**API**
```rust
pub trait Scheduler: Sync + Send + 'static {
    async fn create_table(&self) -> Result<()>;
    async fn create_table_index(&self) -> Result<()>;
    async fn push(&self, trigger: Trigger) -> Result<()>;
    async fn delete(&self, org: &str, module: TriggerModule, key: &str) -> Result<()>;
    async fn update_status(
        &self,
        org: &str,
        module: TriggerModule,
        key: &str,
        status: TriggerStatus,
        retries: i32,
    ) -> Result<()>;
    async fn update_trigger(&self, trigger: Trigger) -> Result<()>;
    async fn pull(
        &self,
        concurrency: i64,
        alert_timeout: i64,
        report_timeout: i64,
    ) -> Result<Vec<Trigger>>;
    async fn clean_complete(&self, interval: u64);
    async fn watch_timeout(&self, interval: u64);
    async fn len_module(&self, module: TriggerModule) -> usize;
    async fn len(&self) -> usize;
    async fn is_empty(&self) -> bool;
    async fn clear(&self) -> Result<()>;
}
```

**ENVs**
| ENV |  Description | 
|------|---------------|
| `ZO_ALERT_SCHEDULE_CONCURRENCY` | The number of `scheduled_jobs` the
`alert_manager` can process at a time. Default is `5` |
| `ZO_ALERT_SCHEDULE_TIMEOUT` | Visibility timeout for alerts. Default
is `90` seconds |
| `ZO_REPORT_SCHEDULE_TIMEOUT` | Visibility timeout for reports. Default
is `300` seconds |
| `ZO_SCHEDULER_MAX_RETRIES` | Maximum number of retries that will be
made for a `scheduled_job`. Default is `3` |
| `ZO_SCHEDULER_CLEAN_INTERVAL` | Time interval after which the
`Scheduler` will clean up the queue. Default is `30` seconds |
| `ZO_SCHEDULER_WATCH_INTERVAL` | Time interval after which the
`Scheduler` will watch for timeout for `scheduled_jobs`. Default is `30`
seconds. |

**Workflow**

![openobserve_scheduler_workflow](https://github.com/openobserve/openobserve/assets/59690052/68beb43b-d04f-49dc-9006-a30942acd99b)
2024-03-25 18:24:14 +05:30
Ashish Kolhe 5bc9d93e23 fix: list should return only allowed objects (#2785) 2024-02-26 22:08:54 +05:30
Ashish Kolheandomkark06 ead43c9873 fix: add delete for roles & groups (#2643)
Co-authored-by: omkark06 <omkar@zinclabs.io>
2024-02-03 19:28:37 +05:30
Hengfei Yang 1b0b8c2bf5 chore: update format (#2144)
fixed #2102
2023-12-07 10:10:45 +05:30
Prabhat Sharma 5db4ca191a change license to AGPL 2023-11-26 18:34:59 -08:00
Hengfei Yang ca52eca9d8 feat: delete API should return 200, 404 or 500 (#1924) 2023-11-10 21:38:34 +08:00
Ankur Srivastava 76d7678acd fix: fixes the create-folder docs in #1649 (#1654) 2023-10-15 21:20:35 -05:00
Ashish Kolhe 84fc09ef0b fix: default folder rename issue (#1585)
We create default folder , when we first receive call to create a
dashboard in default folder. If an API call tries to create a default
folder , it should not be allowed
2023-09-29 12:32:31 +05:30
Ashish Kolhe 47bea0abb9 feat: folders for dashboards (#1559)
Adds support for folder for dashboards, which helps user organise
dashboards in folders.

We should support folders for dashboards like grafana
:https://grafana.com/docs/grafana-cloud/infrastructure-as-code/grizzly/dashboards-folders-datasources/#add-a-folder

Provides API's for :
https://github.com/openobserve/openobserve/issues/1053
2023-09-22 15:54:10 +05:30
Ashish Kolhe c0114e36f6 feat: dashboard versioning (#1464) 2023-09-06 15:45:44 +05:30
Hengfei Yang af521fde54 fix: update license (#1404) 2023-08-21 15:37:50 +08:00
Ankur Srivastava e2dd047bf5 refactor: move infra and meta inside common (#1105)
This is the first step towards splitting the repo into smaller sub
projects.
2023-07-06 12:32:07 +05:30
Hengfei Yang e5a7803d98 feat: metrics API (#686)
* feat: add prometheus API

* fix: unit test
2023-05-04 18:10:10 +08:00
Ashish Kolhe 3d3b09182d fix: update API desc (#677) 2023-05-03 07:03:44 +05:30
Ashish Kolhe 23fa397493 fix: shorten api names (#622)
* fix: shorten api names

* build fixes
2023-04-24 17:33:38 +05:30
Valeriy V. Vorotyntsev 18152f1d6d chore: let Serde fill omitted Dashboard fields (#549) 2023-04-12 23:53:39 +08:00
Valeriy V. Vorotyntsev 6f4d3262c1 docs: generate OpenAPI spec for dashboard APIs (#542) 2023-04-12 12:17:49 +03:00
Valeriy V. Vorotyntsev 4232acaee1 feat: add update_dashboard API endpoint 2023-04-08 00:00:43 +03:00
Valeriy V. Vorotyntsev 5d6cdbcbc8 feat(create_dashboard): remove dashboard_id parameter
Part of #501.
2023-04-08 00:00:43 +03:00
Valeriy V. Vorotyntsev 4f34126174 refactor(dashboards): name -> dashboard_id
Substitute `name` parameter with `dashboard_id`.

Part of #501.
2023-04-08 00:00:43 +03:00
Valeriy V. Vorotyntsev 7006c035ac chore: rename save_dashboard to create_dashboard
Rationale:

1. New name is more precise --- this API creates a dashboard.
2. I'm going to add an API for updating a dashboard. The verb
   "save" is ambiguous --- it is as applicable to "create" as it
   is to "update".

Part of #501.
2023-04-08 00:00:43 +03:00
e289762604 fix: typed dashboard structs (#466)
* WIP: introduce `DashboardXxx`; use it for `save_dashboard`

* service::db::dashboard::list: handle errors, don't unwrap

* [WIP] Update `service::db::dashboard::list`

* XXX The code compiles, but the Dashboards page shows nothing

* web: repair 'dashboards' page

Don't `JSON.parse` backend's response, it's JSON already.

* [WIP] Repair the "view dashboard" page

Saving panel(s) is still broken.

* meta::dashboards.rs: define `Panel` struct

* Fix `Panel` struct definition

* Use `crate::common::json` instead of `serde_json`

* remove json stringify for the dashboard

* Fixed dashboards

* WIP: introduce `DashboardXxx`; use it for `save_dashboard`

* service::db::dashboard::list: handle errors, don't unwrap

* [WIP] Update `service::db::dashboard::list`

* XXX The code compiles, but the Dashboards page shows nothing

* web: repair 'dashboards' page

Don't `JSON.parse` backend's response, it's JSON already.

* [WIP] Repair the "view dashboard" page

Saving panel(s) is still broken.

* meta::dashboards.rs: define `Panel` struct

* Fix `Panel` struct definition

* Use `crate::common::json` instead of `serde_json`

* remove json stringify for the dashboard

* Fixed dashboards

* Added all aggregate functions

* FIXED TESTS

* fixed integration tests

* Updated coverage threshold

---------

Co-authored-by: ktx-keshavi <keshavi.bhalani@kiara.tech>
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
2023-03-31 17:03:34 +05:30
Hengfei Yang 4f9b682d33 style: add License Header (#48) 2023-02-13 11:15:23 +08:00
oasisk 147362cb40 zinc-observe initial commit 2023-02-02 09:03:11 +05:30