Skip to main content

PartitionTracker

Struct PartitionTracker 

Source
pub struct PartitionTracker { /* private fields */ }
Expand description

Contiguity tracker for one partition within one assignment epoch.

Sequence numbers start at 0 and must be registered contiguously; a gap is a driver bug and panics. Resolutions arrive out of order (sharded sinks ack whenever their flushes complete).

use etl_core::checkpoint::{AckStatus, PartitionTracker};

let mut t = PartitionTracker::new();
t.register(0, 99); // batch 0 covers offsets ..=99
t.register(1, 199);
let _ = t.resolve(1, AckStatus::Delivered); // out of order
assert_eq!(t.advance(), None); // batch 0 still pending
let _ = t.resolve(0, AckStatus::Delivered);
assert_eq!(t.advance(), Some(200)); // committable position

Implementations§

Source§

impl PartitionTracker

Source

pub fn new() -> Self

A fresh tracker expecting its first registration at sequence 0.

Source

pub fn register(&mut self, seq: u64, last_offset: i64)

Register a newly issued batch. Sequences must be contiguous.

Registering after a stall is legal (batches already in flight when the failure surfaced still arrive); they will never advance the watermark.

§Panics

Panics on a sequence gap — the driver issued batches out of order, which would silently corrupt watermark accounting.

Source

pub fn resolve(&mut self, seq: u64, status: AckStatus) -> ResolveOutcome

Apply one resolution. Out-of-order and cross-batch interleavings are expected; duplicates and unknown sequences are reported (and debug_asserted) rather than corrupting state.

Source

pub fn advance(&mut self) -> Option<i64>

Pop the contiguous delivered prefix and return the new committable offset (one past the last delivered batch), or None if the watermark did not move.

If the batch at the head has failed, the tracker stalls permanently: everything delivered before the failure is still reported (commit up to the failure is correct), but nothing past it ever will be.

Source

pub fn stalled(&self) -> bool

Whether a failed batch has reached the head and blocked the watermark. Permanent once set.

Source

pub fn stalled_since(&self) -> Option<Instant>

When the stall was first observed, for watermark-age alerting.

Source

pub fn stalled_seq(&self) -> Option<u64>

Sequence number of the failed batch that caused the stall.

Source

pub fn pending(&self) -> usize

Batches issued but not yet advanced through the watermark. This is the backpressure trigger: it bounds tracker memory and replay size.

Trait Implementations§

Source§

impl Debug for PartitionTracker

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for PartitionTracker

Source§

fn default() -> PartitionTracker

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more
§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

§

impl<T> Pointable for T

§

const ALIGN: usize

The alignment of pointer.
§

type Init = T

The type for initializers.
§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V

§

impl<T> WithSubscriber for T

§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

§

impl<A, B, T> HttpServerConnExec<A, B> for T
where B: Body,

§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,