Python 3.12+ breaks backwards compatibility for logging QueueHandler with queue.SimpleQueue

[spacemanspiff2007]

Bug report

Bug description:

I think this is related to #119819

import logging.config
from queue import SimpleQueue

q = SimpleQueue()

config = {
    'version': 1,
    'handlers': {
        'sink': {
            'class': 'logging.handlers.QueueHandler',
            'queue': q,
        },
    },
    'root': {
        'handlers': ['sink'],
    },
}
logging.config.dictConfig(config)

ValueError: Unable to configure handler 'sink'

SimpleQueue does not implement the full Queue interfaces thus both isinstance(obj, queue.Queue) and the queue interface check fails.
Since this has been working on 3.8 - <3.12 I think the queue interface check is checking for methods that are not used at all and should be adjusted accordingly.

cpython/Lib/logging/config.py

Lines 500 to 525 in 2518918

	def _is_queue_like_object(obj):
	"""Check that *obj* implements the Queue API."""
	if isinstance(obj, queue.Queue):
	return True
	# defer importing multiprocessing as much as possible
	from multiprocessing.queues import Queue as MPQueue
	if isinstance(obj, MPQueue):
	return True
	# Depending on the multiprocessing start context, we cannot create
	# a multiprocessing.managers.BaseManager instance 'mm' to get the
	# runtime type of mm.Queue() or mm.JoinableQueue() (see gh-119819).
	#
	# Since we only need an object implementing the Queue API, we only
	# do a protocol check, but we do not use typing.runtime_checkable()
	# and typing.Protocol to reduce import time (see gh-121723).
	#
	# Ideally, we would have wanted to simply use strict type checking
	# instead of a protocol-based type checking since the latter does
	# not check the method signatures.
	queue_interface = [
	'empty', 'full', 'get', 'get_nowait',
	'put', 'put_nowait', 'join', 'qsize',
	'task_done',
	]
	return all(callable(getattr(obj, method, None))
	for method in queue_interface)

Tested with 3.12.6

CPython versions tested on:

3.12

Operating systems tested on:

Windows

Linked PRs

• gh-124653: relax (again) detection of queue API for logging handlers  #124897
• [3.13] gh-124653: Relax (again) detection of queue API for logging handlers (GH-124897) #125059
• [3.12] gh-124653: Relax (again) detection of queue API for logging handlers (GH-124897) #125060

[YvesDup]

Except if I miss something, SimpleQueue has never had a put_nowait method, the one used in the QueueHandler and QueueListener classes (Just see here) . So using this class in the logging QueueHandler always failed.

[spacemanspiff2007]

As I've written this has been working on 3.8 - 3.11. I'm not sure exactly when it broke.
SimpleQueue actually implements put_nowait.

I skimmed over the file you linked and it seems the only required interface methods are put_nowait and get.
The interface check should reflect that.
Additionally get has to raise queue.Empty if the queue is empty but there is no way to check that ...

[YvesDup]

My apologies for my mistaken: I was thinking you talked about multiprocessing.SimpleQueue not about queue.SimpleQueue. So first of all, I suggess to modify title issue (prefixing SimpleQueuewith the queue module).

And then, I agree with you the queue_interface list of the _is_queue_like_object method is too restrictive. At least join and task_done methods must be removed. full and emptymethods (or exceptions) have to exam too.

Perhars it will be nice to ask for a python core dev advice.

[YvesDup]

@picnixz as GH-122154 author, and introducing is_queue_like_object in file logging/config.py, could you take a look to this issue please ?

[picnixz]

Urgh, this issue will haunt me all my life :') According to the docs, it required a "Queue API", but I didn't consider that SimpleQueue was... too simple enough and did not entirely conform to what Queue did. Ok, So I think we should only care about two methods that are those given in SimpleQueue.

[YvesDup]

So I think we should only care about two methods that are those given in SimpleQueue.

What do you think about adding queue.simpleQueue at L502, as below:

if isinstance(obj, (queue.Queue, queue.SimpleQueue)): 
         return True 

[picnixz]

It's correct (and that's what I also did), but since we really don't know whether people could use objects or not I just relaxed the interface (and I think it's fine like this).

[YvesDup]

@spacemanspiff2007 good catch !!