Struct ChainBuilder 

pub struct ChainBuilder<DF: RecFamily, CurF: RecFamily, D, P> { /* private fields */ }

Fluent builder for one pipeline’s operator chain. DF is the deserializer’s record family; CurF the family at the current end of the chain (changed by map_rec and flat_map, and by map for owned payloads).

Implementations

impl<DF: RecFamily, CurF: RecFamily, D, P> ChainBuilder<DF, CurF, D, P>

pub fn with_metrics( self, pipeline: impl Into<String>, component: impl Into<String>, ) -> Self

Enable framework metrics for every stage of this chain. Must be called before any stage is added so all stages get handles.

Panics

Panics if stages were already added.

pub fn deser_error_policy(self, policy: ErrorPolicy) -> Self

Error policy for the deserializer stage (default: Skip).

pub fn map_rec<NF, G>(self, f: G) -> ChainBuilder<DF, NF, D, MapPart<P, G>>

where NF: RecFamily, G: for<'buf> MapFn<CurF::Rec<'buf>, NF::Rec<'buf>>,

Transform each record into family NF. For borrowing families pass a fn item (see the module docs); for owned payloads ChainBuilder::map is more ergonomic.

pub fn try_map_rec<NF, G, E>( self, f: G, policy: ErrorPolicy, ) -> ChainBuilder<DF, NF, D, TryMapPart<P, G>>

where NF: RecFamily, G: for<'buf> TryMapFn<CurF::Rec<'buf>, NF::Rec<'buf>, E>, E: Display,

Fallibly transform each record into family NF with a per-stage ErrorPolicy. For borrowing families pass a fn item.

pub fn filter<Pr>(self, p: Pr) -> ChainBuilder<DF, CurF, D, FilterPart<P, Pr>>

where Pr: for<'buf> FnMut(&CurF::Rec<'buf>) -> bool,

Keep only records whose payload satisfies the predicate.

pub fn inspect<G>(self, f: G) -> ChainBuilder<DF, CurF, D, InspectPart<P, G>>

where G: for<'buf> FnMut(&CurF::Rec<'buf>),

Observe each record’s payload without transforming it.

pub fn flat_map<OutF, G>( self, g: G, ) -> ChainBuilder<DF, OutF, D, FlatMapPart<OutF, P, G>>

where OutF: RecFamily, G: for<'buf> FnMut(CurF::Rec<'buf>, &mut Emitter<'_, OutF>),

Expand each record into 0..N records of family OutF through a stack-borrowed Emitter.

pub fn sink<E, R>( self, encoder: E, router: R, cfg: ChunkConfig, queues: ShardQueues, budget: Arc<InflightBudget>, ) -> SinkedChain<DF, CurF, D, P, E, R>

Terminate the chain into a sink: records are routed by router, encoded by encoder on the pipeline thread, and handed to the sink workers through queues.

impl<DF: RecFamily, T: Send + 'static, D, P> ChainBuilder<DF, Owned<T>, D, P>

Closure-friendly transforms for chains whose current records are owned.

pub fn map<U, G>(self, f: G) -> ChainBuilder<DF, Owned<U>, D, MapPart<P, G>>

where U: Send + 'static, G: FnMut(T) -> U,

Transform each record’s payload. Bare closures infer; the output type may differ (the chain’s family becomes Owned<U>).

pub fn try_map<U, G, E>( self, f: G, policy: ErrorPolicy, ) -> ChainBuilder<DF, Owned<U>, D, TryMapPart<P, G>>

where U: Send + 'static, G: FnMut(T) -> Result<U, E>, E: Display,

Fallibly transform each record’s payload with a per-stage ErrorPolicy.

Trait Implementations

impl<DF: Clone + RecFamily, CurF: Clone + RecFamily, D: Clone, P: Clone> Clone for ChainBuilder<DF, CurF, D, P>

fn clone(&self) -> ChainBuilder<DF, CurF, D, P>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<DF: Debug + RecFamily, CurF: Debug + RecFamily, D: Debug, P: Debug> Debug for ChainBuilder<DF, CurF, D, P>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.