Day 27

Day 27

IPC interprocess communication

IPC definition

Exchange of data between one process and another, either within the same computer or over a network. It implies a protocol that guarantees a response to a request.

Why do we use IPC

There are numerous reasons for providing an environment or situation which allows process co-operation:

• Information sharing: Since some users may be interested in the same piece of information (for example, a shared file), you must provide a situation for allowing concurrent access to that information.

• Computation speedup: If you want a particular work to run fast, you must break it into sub-tasks where each of them will get executed in parallel with the other tasks. Note that such a speed-up can be attained only when the computer has compound or various processing elements like CPUs or I/O channels.

• Modularity:模块性 You may want to build the system in a modular way by dividing the system functions into split processes or threads.

• Convenience: Even a single user may work on many tasks at a time. For example, a user may be editing, formatting, printing, and compiling in parallel.

Methods in Interprocess Communication

These are the methods in IPC:

1. Pipes (Same Process) – This allows flow of data in one direction only. Analogous/əˈnæləɡəs/ 类似的 to simplex systems (Keyboard). Data from the output is usually buffered until input process receives it which must have a common origin.

2. Names Pipes (Different Processes) – This is a pipe with a specific name it can be used in processes that don’t have a shared common process origin. E.g. is FIFO where the details written to a pipe is first named.

3. Message Queuing – This allows messages to be passed between processes using either a single queue or several message queue. This is managed by system kernel 内核these messages are coordinated using an API.

4. Semaphores – This is used in solving problems associated with synchronization and to avoid race condition. These are integer values which are greater than or equal to 0.

5. Shared memory – This allows the interchange of data through a defined area of memory. Semaphore values have to be obtained before data can get access to shared memory.

6. Sockets – This method is mostly used to communicate over a network between a client and a server. It allows for a standard connection which is computer and OS independent.

Queue

https://www.programmersought.com/article/4063765736/#_4

The queue here can't communicate in multiple processes, we need multiprocessing under the module. At the same time, due to the perfect encapsulation of Python, its implementation principle can be said to be completely transparent to the programmer, and the user can use it as an ordinary queue. Just like the demo of the producer/consumer below, the two are connected through the queue.

Pipe

Queue is suitable for most scenarios, and has to be considered in order to satisfy the universality, so it appears to be "heavy." Pipe is lighter and faster. It is used like a socket in Socket programming.recv()withsend()Implement the communication mechanism.

In fact, look at the source code of Pipe will find thatPipe()Method returns twoConnection() The instance, that is to say the two objects returned are exactly the same (but the id is not the same), except that we use different variable names to make a distinction.

send()The method can send data continuously, it can be seen that it sends the data to a container, and recv()The method is to take data from this container, when there is no data in the container,recv()Will block the current process. have to be aware of is:Recv can't take the data out of the same object send.

Value/Array

multiprocessing.Valuewithmultiprocessing.ArrayThe implementation is based on memory sharing,

• For the object of Value, you need to pass.valueGet the attribute value;

• The first parameter in the Array represents: the type of element stored in the array;

• If a string is needed, it is implemented by Array instead of Value.

Array()The second parameter issize_or_initializer, indicating that the incoming parameter can beLength of the array,orInitialization value. The Array here is an authentic array数组, not a list in Python. People with C language experience should be able to understand immediately.

Thread

https://realpython.com/intro-to-python-threading/

A thread is a separate flow of execution. This means that your program will have two things happening at once. But for most Python 3 implementations the different threads do not actually execute at the same time: they merely appear to.

It’s tempting to think of threading as having two (or more) different processors running on your program, each one doing an independent task at the same time. That’s almost right. The threads may be running on different processors, but they will only be running one at a time.

Getting multiple tasks running simultaneously requires a non-standard implementation of Python, writing some of your code in a different language, or using multiprocessing which comes with some extra overhead.

Because of the way CPython implementation of Python works, threading may not speed up all tasks. This is due to interactions with the GIL that essentially limit one Python thread to run at a time.

Tasks that spend much of their time waiting for external events are generally good candidates for threading. Problems that require heavy CPU computation and spend little time waiting for external events might not run faster at all.

This is true for code written in Python and running on the standard CPython implementation. If your threads are written in C they have the ability to release the GIL and run concurrently. If you are running on a different Python implementation, check with the documentation too see how it handles threads.

If you are running a standard Python implementation, writing in only Python, and have a CPU-bound problem, you should check out the multiprocessing module instead.

Architecting your program to use threading can also provide gains in design clarity. Most of the examples you’ll learn about in this tutorial are not necessarily going to run faster because they use threads. Using threading in them helps to make the design cleaner and easier to reason about.

So, let’s stop talking about threading and start using it!

Starting a Thread

Now that you’ve got an idea of what a thread is, let’s learn how to make one. The Python standard library provides threading, which contains most of the primitives you’ll see in this article. Thread, in this module, nicely encapsulates threads, providing a clean interface to work with them.

To start a separate thread, you create a Thread instance and then tell it to .start():

When you create a Thread, you pass it a function and a list containing the arguments to that function. In this case, you’re telling the Thread to run thread_function() and to pass it 1 as an argument.

For this article, you’ll use sequential integers as names for your threads. There is threading.get_ident(), which returns a unique name for each thread, but these are usually neither short nor easily readable.

thread_function() itself doesn’t do much. It simply logs some messages with a time.sleep() in between them.

Daemon Threads

In computer science, a daemon is a process that runs in the background.

Python threading has a more specific meaning for daemon. A daemon thread will shut down immediately when the program exits. One way to think about these definitions is to consider the daemon thread a thread that runs in the background without worrying about shutting it down.

If a program is running Threads that are not daemons, then the program will wait for those threads to complete before it terminates. Threads that are daemons, however, are just killed wherever they are when the program is exiting.

Let’s look a little more closely at the output of your program above. The last two lines are the interesting bit. When you run the program, you’ll notice that there is a pause (of about 2 seconds) after __main__ has printed its all done message and before the thread is finished.

This pause is Python waiting for the non-daemonic thread to complete. When your Python program ends, part of the shutdown process is to clean up the threading routine.

If you look at the source for Python threading, you’ll see that threading._shutdown() walks through all of the running threads and calls .join() on every one that does not have the daemon flag set.

So your program waits to exit because the thread itself is waiting in a sleep. As soon as it has completed and printed the message, .join() will return and the program can exit.

Frequently, this behavior is what you want, but there are other options available to us. Let’s first repeat the program with a daemon thread. You do that by changing how you construct the Thread, adding the daemon=True flag:

join() a Thread

To tell one thread to wait for another thread to finish, you call .join(). If you uncomment that line, the main thread will pause and wait for the thread x to complete running.

Did you test this on the code with the daemon thread or the regular thread? It turns out that it doesn’t matter. If you .join() a thread, that statement will wait until either kind of thread is finished.