openobserve

starred/openobserve

Fork 0

mirror of https://github.com/openobserve/openobserve.git synced 2026-11-04 08:09:27 +00:00

Files

T

History

Yashodhan JoshiandYashodhan Joshi f6bc5ef6f6 fix: consider all fts fields to ignore for distinct values (#8601 )

### **User description**
Consider all fts fields including env specified fts fields for ignoring
in distinct field setup


___

### **PR Type**
Bug fix


___

### **Description**
- Exclude all FTS fields from distinct setup

- Use global and per-stream FTS sources

- Add config constant checks for FTS fields

- Centralize FTS retrieval via schema helper


___

### Diagram Walkthrough


```mermaid
flowchart LR
  CFG["SQL_FULL_TEXT_SEARCH_FIELDS (config)"]
  SCH["get_stream_setting_fts_fields (schema)"]
  MIG["migration/dashboards.rs"]
  SVCDB["service/dashboards/mod.rs"]
  SVCSTR["service/stream.rs"]

  CFG -- "global FTS list" --> MIG
  SCH -- "resolve stream FTS" --> MIG
  SCH -- "resolve stream FTS" --> SVCDB
  SCH -- "resolve stream FTS" --> SVCSTR

  MIG -- "skip FTS in distinct" --> Distinct["Distinct setup"]
  SVCDB -- "skip FTS in distinct" --> Distinct
  SVCSTR -- "skip FTS in distinct" --> Distinct
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Bug
fix</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>dashboards.rs</strong><dd><code>Ignore global and
stream FTS in migration distincts</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

src/migration/dashboards.rs

<ul><li>Import <code>SQL_FULL_TEXT_SEARCH_FIELDS</code> and
<code>get_stream_setting_fts_fields</code><br> <li> Skip adding distinct
fields if in global or stream FTS<br> <li> Preserve existing reserved
field checks</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8601/files#diff-eba7ddaafafa0efbc35474fd19af320f0cce754dc6048bf4ce3d9ff3f22358b1">+6/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>mod.rs</strong><dd><code>Use unified FTS set when
updating dashboard distincts</code>&nbsp; &nbsp; &nbsp; &nbsp;
</dd></summary>
<hr>

src/service/dashboards/mod.rs

<ul><li>Import <code>SQL_FULL_TEXT_SEARCH_FIELDS</code> and schema FTS
helper<br> <li> Use <code>get_stream_setting_fts_fields</code> to filter
FTS<br> <li> Continue ignoring reserved fields like timestamp and
count</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8601/files#diff-6ae9e22ced40c50675dde5348b8d696f6559655bbbded915d7e6e88bf64b71a6">+9/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>stream.rs</strong><dd><code>Centralize FTS filtering in
stream settings update</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></summary>
<hr>

src/service/stream.rs

<ul><li>Import <code>get_stream_setting_fts_fields</code><br> <li>
Precompute FTS set from settings<br> <li> Skip distinct additions if in
FTS or newly added FTS</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8601/files#diff-41452a7fb29b4634437174fd6aecf6a9050dbfdef41b9077a21d59af3d79e9ed">+5/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

</details>

___

---------

Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>

2025-09-23 19:36:49 +05:30

dashboards.rs

fix: consider all fts fields to ignore for distinct values (#8601 )

2025-09-23 19:36:49 +05:30

file_list.rs

chore: upgrade rust to 2025-05-20 (#7316 )

2025-07-01 21:25:45 +08:00

meta.rs

feat: remove etcd (#8466 )

2025-09-16 19:02:47 +08:00

mod.rs

fix: shutting down DDL connection (#8398 )