Commit Graph
11 Commits
Author SHA1 Message Date
13b521651b fix(python-client): return at most size bytes from S3BufferedReader.read (#10623)
* feat: add unit tests for S3BufferedReader.read and improve read method implementation

* feat: refactor S3BufferedReader.read method and add unit tests for its functionality

* feat: implement peek() on S3BufferedReader with buffered reads

* fix(python-client): keep the read(size) contract and trim the test surface

Drop the duplicated `TestS3BufferedReaderRead` class from
`python-client/tests/wmill_client_test.py`: CI runs `pytest tests/` from
`python-client/wmill`, so that legacy manual harness never executes, and the
same assertions already live in `python-client/wmill/tests/test_s3_reader.py`.

Narrow that file to the four behaviours a future change could break, and make
the `bytes_generator` guard actually call `bytes_generator`.

Align `peek()` with `io.BufferedReader.peek`, which does at most one read on
the underlying stream, rather than looping until `size` bytes are buffered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(python-client): hold read1 to one underlying read

read1 forwarded to read, so read1(-1) drained the whole object — the same
unbounded buffering this branch removes from read. Now that a buffer exists,
read1 can honour its own contract: fill only when the buffer is empty, then
serve from it.

Also treat read(None) as read(-1), per the BufferedReader contract, and pin
that read(0) does not pull from the stream: that holds only because the
drain sentinel is a negative size, and widening it to any falsy size would
reintroduce whole-file buffering.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(python-client): return from read1(0) without touching the stream

A zero-length read has nothing to serve, so pulling a chunk to satisfy it
both wastes a round trip and advances the stream. Guard it ahead of the
fill, and pin it with a chunk source that counts pulls.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Tushar <tusharanshu18@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 23:04:27 +02:00
centdix 31e002ad41 feat(ai): standardize and improve system prompts (#7346)
* init

* test in frontend

* copy files

* use in cli

* better

* add desc to sdks

* better

* fix ts parsing

* add docs to ts client

* add docs to python client

* use script prompt in frontend

* regen

* use in flow

* rm

* use in cli, create AGENTS.md instead of cursor rules

* remove apply

* better

* better

* simplify cli

* more docs

* cleaning

* update readme

* generate cli file

* better folder names

* fix ts

* fix multiline
2025-12-12 17:26:19 +00:00
David P. Kleinschmidt 359b34ef08 Use backwards-compatible union type syntax in Python client (#6277) 2025-07-25 15:25:08 +00:00
HugoCasa 23d624aa23 fix: error handling for S3 file loading in py and ts clients (#6124) 2025-07-02 21:28:13 +00:00
Ruben Fiszel 0e7a25c0bd feat: support multiple object storage + parquet_csv + polars -> datafusion" (#3853)
* multiple storage

* all

* rm symlinks

* all

* all
2024-05-31 11:59:51 +02:00
Guillaume Bouvignies 42740b0924 fix: Improve python writeS3File perf (#3182) 2024-02-08 17:57:42 +01:00
Guillaume Bouvignies 81263a9502 fix: Python buffered reader (#3136)
* fix: Python buffered reader

* use bytes generator only for buferred reader
2024-02-02 10:23:04 +01:00
Ruben Fiszel 5362e903d6 fix(python): fix python reader 2024-02-02 01:21:20 +01:00
Ruben Fiszel cdec1c22ad fix(python): fix python reader 2024-02-02 01:12:01 +01:00
0003673e8d feat: migrate s3 client to object_store (#3116)
* feat: migrate s3 client to object_store

* remove multipart from open API

* Remove multipart in favor of a single stream

* progress report

* add progress reader on server side

* small nit fix

* fix read chunk

* Fix TS and python SDK

* Fix download button

* Fix download button object viewer

* fix list

* Better errors

* export loadS3FileContent

* revert changes SDK

* fix browser

* small file list unavailable fix

* Old endpoints throws informative error messages

* Typescript SDK uses raw fetch

* update python SDK

* Error if uploaded file > 50Mb

* revert python SDL changes

* Update python SDK method docs

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-02-01 16:03:30 +01:00
Guillaume Bouvignies 2e6835f610 feat: Download s3 file as stream in Python and TS (#3099) 2024-01-29 14:49:08 +01:00