The use case included a pre filled queue with ~30k messages and multiple(10) consumers which dequeued a small amount(again 10) of messages before disconnecting and immediately reconnecting. In this case, a prefetch value of 10 is ideal, but with the default prefetch value of 1000, the broker is busy dispatching messages to the consumer long after it has decided to quit. In addition, with asyncDispath, while dispatch to a consumer is instigated by the broker, the actual delivery is delegated, to the broker transport connection worker thread. This means that the delivery attempts back up on the individual transport connections rather than slowing down the broker.
The destination inflight count is a measure of the number of messages that have been dispatched by the broker but not yet acknowledged by any consumer. On each dispatch completion by the worker thread, the inflight value is incremented. The decrementing normally happens on a message acknowledge. In the event of a consumer closure with unconsumed messages, the remaining value is decremented when the consumer closes.
This is the crux. From the broker perspective, on consumer closure, it has dispatched 1000 messages and got an ack for 10 so it needs to decrement the inflight by
The good news is that this is perfectly fine, the books are kept in balance and there is eventual consistency. In addition, using either of a prefetch value of 10 or asyncDispatch=false ensures that the negative values do not occur as the broker is kept directly in step with message delivery to the consumer. In general though, the an appropriate value of prefetch is the correct solution if it is known in advance that a consumer will do work in batches.