queue for thread of Python

queue

https://docs.python.org/3.7/library/queue.html#queue.Queue

支持多生产者和多消费者。

为线程通信设计。

实现三种类型的队列：

（1）FIFO

（2）LIFO

（3）优先队列

还有一个简单队列，是FIFO的一种易用性特例。

The queue module implements multi-producer, multi-consumer queues.

It is especially useful in threaded programming when information must be exchanged safely between multiple threads.

The Queue class in this module implements all the required locking semantics.

It depends on the availability of thread support in Python; see the threading module.

 

The module implements three types of queue, which differ only in the order in which the entries are retrieved.

In a FIFO queue, the first tasks added are the first retrieved. In a LIFO queue, the most recently added entry is the first retrieved (operating like a stack).

With a priority queue, the entries are kept sorted (using the heapq module) and the lowest valued entry is retrieved first.

 

Internally, those three types of queues use locks to temporarily block competing threads; however, they are not designed to handle reentrancy within a thread.

In addition, the module implements a “simple” FIFO queue type, SimpleQueue, whose specific implementation provides additional guarantees in exchange for the smaller functionality.

 

优先队列

https://pymotw.com/3/queue/index.html#priority-queue

Sometimes the processing order of the items in a queue needs to be based on characteristics of those items, rather than just the order they are created or added to the queue. For example, print jobs from the payroll department may take precedence over a code listing that a developer wants to print. PriorityQueue uses the sort order of the contents of the queue to decide which item to retrieve.

#queue_priority.py

import functools
import queue
import threading


@functools.total_ordering
class Job:

    def __init__(self, priority, description):
        self.priority = priority
        self.description = description
        print('New job:', description)
        return

    def __eq__(self, other):
        try:
            return self.priority == other.priority
        except AttributeError:
            return NotImplemented

    def __lt__(self, other):
        try:
            return self.priority < other.priority
        except AttributeError:
            return NotImplemented


q = queue.PriorityQueue()

q.put(Job(3, 'Mid-level job'))
q.put(Job(10, 'Low-level job'))
q.put(Job(1, 'Important job'))


def process_job(q):
    while True:
        next_job = q.get()
        print('Processing job:', next_job.description)
        q.task_done()


workers = [
    threading.Thread(target=process_job, args=(q,)),
    threading.Thread(target=process_job, args=(q,)),
]
for w in workers:
    w.setDaemon(True)
    w.start()

q.join()

 

functools.total_ordering -- 类全序工具


给类定义全序关系。

让所有此类的实例都可以进行比较。

 

https://docs.python.org/3.7/library/functools.html#functools.total_ordering

 

Given a class defining one or more rich comparison ordering methods, this class decorator supplies the rest. This simplifies the effort involved in specifying all of the possible rich comparison operations:

The class must define one of __lt__(), __le__(), __gt__(), or __ge__(). In addition, the class should supply an __eq__() method.

 

@total_ordering
class Student:
    def _is_valid_operand(self, other):
        return (hasattr(other, "lastname") and
                hasattr(other, "firstname"))
    def __eq__(self, other):
        if not self._is_valid_operand(other):
            return NotImplemented
        return ((self.lastname.lower(), self.firstname.lower()) ==
                (other.lastname.lower(), other.firstname.lower()))
    def __lt__(self, other):
        if not self._is_valid_operand(other):
            return NotImplemented
        return ((self.lastname.lower(), self.firstname.lower()) <
                (other.lastname.lower(), other.firstname.lower()))

 

https://www.geeksforgeeks.org/python-functools-total_ordering/

er_none

edit

play_arrow

brightness_4
from functools import total_ordering 
  
@total_ordering
class num: 
      
    def __init__(self, value): 
        self.value = value 
          
    def __lt__(self, other): 
        return self.value < other.value 
        
    def __eq__(self, other): 
          
        # Changing the functionality 
        # of equality operator 
        return self.value != other.value 
          
# Driver code 
print(num(2) < num(3)) 
print(num(2) > num(3)) 
print(num(3) == num(3)) 
print(num(3) == num(5)) 

 

Total order -- 全序定义

在偏序的基础上，添加任意元素的可比较性。

 

https://en.wikipedia.org/wiki/Total_order

For each (non-strict) total order ≤ there is an associated asymmetric (hence irreflexive) transitive semiconnex relation <, called a strict total order or strict semiconnex order,^[2] which can be defined in two equivalent ways:

• a < b if a ≤ b and a ≠ b
• a < b if not b ≤ a (i.e., < is the inverse of the complement of ≤)

Properties:

• The relation is transitive: a < b and b < c implies a < c.
• The relation is trichotomous: exactly one of a < b, b < a and a = b is true.
• The relation is a strict weak order, where the associated equivalence is equality.

We can work the other way and start by choosing < as a transitive trichotomous binary relation; then a total order ≤ can be defined in two equivalent ways:

• a ≤ b if a < b or a = b
• a ≤ b if not b < a

Two more associated orders are the complements ≥ and >, completing the quadruple {<, >, ≤, ≥}.

We can define or explain the way a set is totally ordered by any of these four relations; the notation implies whether we are talking about the non-strict or the strict total order.

a {\displaystyle a} precedes b {\displaystyle b} and b {\displaystyle b} precedes a {\displaystyle a} .^[7]:325 A relation having the connex property means that any pair of elements in the set of the relation are comparable under the relation. This also means that the set can be diagrammed as a line of elements, giving it the name linear.^[7]:330 The connex property also implies reflexivity, i.e., a ≤ a. Therefore, a total order is also a (special case of a) partial order, as, for a partial order, the connex property is replaced by the weaker reflexivity property. An extension of a given partial order to a total order is called a linear extension of that partial order.

 

https://mathworld.wolfram.com/PartialOrder.html

https://www.wolframalpha.com/input/?i=totally+ordered+set&assumption=%22ClashPrefs%22+-%3E+%7B%22MathWorld%22%2C+%22TotallyOrderedSet%22%7D

A total order (or "totally ordered set, " or "linearly ordered set") is a set plus a relation on the set (called a total order) that satisfies the conditions for a partial order plus an additional condition known as the comparability condition. A relation <= is a total order on a set S ("<= totally orders S") if the following properties hold.
1. Reflexivity: a<=a for all a element S.
2. Antisymmetry: a<=b and b<=a implies a = b.
3. Transitivity: a<=b and b<=c implies a<=c.
4. Comparability (trichotomy law): For any a, b element S, either a<=b or b<=a.
The first three are the axioms of a partial order, while addition of the trichotomy law defines a total order.
Every finite totally ordered set is well ordered. Any two totally ordered sets with k elements (for k a nonnegative integer) are order isomorphic, and therefore have the same order type (which is also an ordinal number).

 

 

FIFO -- Queue

https://github.com/fanqingsong/code_snippet/blob/master/python/thread_queue/queque.py

import threading
import queue


def do_work(item):
    print('now doing work')
    print(item)


def worker():
    while True:
        item = q.get()
        if item is None:
            break
        do_work(item)
        q.task_done()

q = queue.Queue()

num_worker_threads = 3


threads = []
for i in range(num_worker_threads):
    t = threading.Thread(target=worker)
    t.start()
    threads.append(t)


for item in range(3):
    q.put(item)

# block until all tasks are done
q.join()

# stop workers
for i in range(num_worker_threads):
    q.put(None)


for t in threads:
    t.join()