Struct FeatherWriter

pub struct FeatherWriter { /* private fields */ }

Manages multiple FeatherBuffers and handles encoding, rotation, and flushing to the object store.

The write() method is the single entry point for clients: they supply a data value (of generic type T) and the manager encodes it (using T’s metadata via EncodeToRecordBatch), routes it by CatalogPathPrefix, and writes it to the appropriate FileWriter. When a writer’s buffer is full or rotation criteria are met, its contents are flushed to the object store and it is replaced.

Implementations

impl FeatherWriter

pub fn new( base_path: String, store: Arc<dyn ObjectStore>, clock: Rc<RefCell<dyn Clock>>, rotation_config: RotationConfig, included_types: Option<HashSet<String>>, per_instrument_types: Option<HashSet<String>>, ) -> Self

Creates a new FileWriterManager instance.

pub async fn write<T>(&mut self, data: T) -> Result<(), Box<dyn Error>>

where T: EncodeToRecordBatch + CatalogPathPrefix + 'static,

Writes a single data value. This is the user entry point. The data is encoded into a RecordBatch and written to the appropriate FileWriter. If the writer’s buffer reaches capacity or meets rotation criteria (based on the rotation configuration), the FileWriter is flushed to the object store and replaced.

pub async fn flush(&mut self) -> Result<(), Box<dyn Error>>

Flushes all active FeatherBuffers by writing any remaining buffered bytes to the object store.

Note: This is not called automatically and must be called by the client. It is expected that no other writes are performed after this.