Skip to content

Netty ChannelManager.tryToOfferChannelToPool discard chunks after offering to pool #834

@maxpower-artvandelay

Description

@maxpower-artvandelay

Hello, the Netty ChannelManager(AHC 1.9.11) seems to have an issue on tryToOfferChannelToPool.

public final void tryToOfferChannelToPool(Channel channel, boolean keepAlive, String partition) {
    if (channel.isConnected() && keepAlive && channel.isReadable()) {
        LOGGER.debug("Adding key: {} for channel {}", partition, channel);
        channelPool.offer(channel, partition);
        if (maxConnectionsPerHostEnabled)
            channelId2KeyPool.putIfAbsent(channel.getId(), partition);
        Channels.setDiscard(channel);
    } else {
        // not offered
        closeChannel(channel);
    }
}

Looks like channelPool.offer should happen after Channels.setDiscard, otherwise the channel becomes available for selection while the previous request cleanup is still happening. This will cause a "java.util.concurrent.TimeoutException: Request timed out" under certain loads.

This is the snippet I used to reproduce the issue:

final AsyncCompletionHandler<Response> handler = new AsyncCompletionHandler<Response>() {
    @Override
    public void onThrowable(Throwable t) {
        System.out.println(t.toString());
    }

    @Override
    public Response onCompleted(Response response) throws Exception {
        return response;
    }
};

Request request = httpClient.prepareGet("http://...").build();
for (int i = 0; i < 500; ++i) {
    httpClient.executeRequest(request, handler);
}

I think the fix should be something similar to:

Channels.setDiscard(channel);
channelPool.offer(channel, partition);

instead of :

channelPool.offer(channel, partition);
Channels.setDiscard(channel);

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions