Skip to content

[bug] pulsar-perf consume, do not fail in case of reading data older than 10 days - #17160

Merged
eolivelli merged 2 commits into
apache:masterfrom
eolivelli:fix/pulsarperf-consume-old-data
Aug 25, 2022
Merged

eolivelli merged 2 commits into
apache:masterfrom
eolivelli:fix/pulsarperf-consume-old-data

Conversation

@eolivelli

Copy link
Copy Markdown
Contributor

Motivation

When you read old data using pulsar-perf consume you see this error:

022-08-18T12:10:36,794+0200 [pulsar-external-listener-3-1] ERROR org.apache.pulsar.client.impl.ConsumerBase - [persistent://public/default/test][b] Message listener error in processing message: 47:1532:-1:0
java.lang.ArrayIndexOutOfBoundsException: value outside of histogram covered range. Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1987201 out of bounds for length 1835008
at org.HdrHistogram.AbstractHistogram.handleRecordException(AbstractHistogram.java:492) ~[HdrHistogram-2.1.9.jar:2.1.9]
at org.HdrHistogram.AbstractHistogram.recordSingleValue(AbstractHistogram.java:482) ~[HdrHistogram-2.1.9.jar:2.1.9]
at org.HdrHistogram.AbstractHistogram.recordValue(AbstractHistogram.java:397) ~[HdrHistogram-2.1.9.jar:2.1.9]
at org.HdrHistogram.Recorder.recordValue(Recorder.java:100) ~[HdrHistogram-2.1.9.jar:2.1.9]
at org.apache.pulsar.testclient.PerformanceConsumer.lambda$main$613f18c5$1(PerformanceConsumer.java:323) ~[pulsar-testclient.jar:2.11.0-SNAPSHOT]
at org.apache.pulsar.client.impl.ConsumerBase.callMessageListener(ConsumerBase.java:1084) ~[pulsar-client-original.jar:2.11.0-SNAPSHOT]
at org.apache.pulsar.client.impl.ConsumerBase.lambda$triggerListener$10(ConsumerBase.java:1057) ~

Modifications

Consider latencies out of bound as the maximum latency

@eolivelli eolivelli added this to the 2.12.0 milestone Aug 18, 2022
@eolivelli eolivelli self-assigned this Aug 18, 2022
@eolivelli
eolivelli merged commit b58439e into apache:master Aug 25, 2022
@eolivelli
eolivelli deleted the fix/pulsarperf-consume-old-data branch August 25, 2022 08:16
Nicklee007 pushed a commit to Nicklee007/pulsar that referenced this pull request Aug 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants