FAQ Database Discussion Community


Multi-threading and queuing

java,multithreading,queue,threadpool
I kinda got the hang of threads but now I'm confused. I don't quite understand queues at all. I created a thread pool executor which initializes a set amount of threads and each of those threads input a username(string) on a website to check if its available or not. So...

Multi-threaded loop through collection with C#

c#,multithreading,threadpool,asyncfileupload
I have seen lots of examples on how to do this (read through lots of threads on this site), but I'm having a very hard time getting something that works without locking up the UI thread AND allows me to control the number of threads executing at a time. I...

Whats the best way to asynchronously return a result (as a struct) that hasn't been fully “set up” (or processed) yet

c,asynchronous,concurrency,pthreads,threadpool
Alright, I honestly have tried looking up "Asynchronous Functions in C" (Results are for C# exclusively), but I get nothing for C. So I'm going to ask it here, but if there are better, already asked questions on StackExchange or what-have-you, please direct me to them. So I'm teaching myself...

Number of available threads in threadpool

c#,.net,multithreading,threadpool,threadpoolexecutor
In a Threadpool, I have set maximum number of threads.I want number of worker threads are available. i tried. ThreadPool.GetAvailableThreads(out x, out y); Using this, I got number of available threads (x) which was only decreasing.There was no increase in a number of available threads (x) even after their task...

iterating over methods - Python

python,multithreading,methods,parallel-processing,threadpool
I've created the following threading utility, inside my class Domain: def __run_threads(self, targets): with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor: future_job = { executor.submit(target): target for target in targets } for future in concurrent.futures.as_completed(future_job): try: data = future.result() except Exception as exc: self.log.exception(exc) else: self.log.info("Data: %s" % data) that's it, I may want...

Shared variable among various threads in Python

python,multithreading,threadpool
I've requirement to calculate the total number of post requests sent to a server. My script uses a thread per JSON file which contains post data. Below is the rough code snippet. statistics = 0 def load_from_file(some_arguments, filename): data_list = json.loads(open(filename).read()) url = address + getUrl(filename, config) for data in...

Threadpool Deadlock: designing against or detecting

java,threadpool,deadlock
I hope this isn't overly broad; my question is "How do I design a service with multiple Threadpools which can't deadlock itself?". I own a web service which fans out to 100s of threads on a single user request, to perform data aggregation with low latency. There are a number...

Memory leak in MediaPlayer

c#,multithreading,garbage-collection,threadpool
Can someone please explain why the following program runs out of memory? class Program { private static void ThreadRoutine() { System.Windows.Media.MediaPlayer player = new System.Windows.Media.MediaPlayer(); } static void Main(string[] args) { Thread aThread; int iteration = 1; while (true) { aThread = new Thread(ThreadRoutine); aThread.Start(); aThread.Join(); Console.WriteLine("Iteration: " + iteration++);...

Multiprocessing pool 'apply_async' only seems to call function once

python,multithreading,multiprocessing,threadpool
I've been following the docs to try to understand multiprocessing pools. I came up with this: import time from multiprocessing import Pool def f(a): print 'f(' + str(a) + ')' return True t = time.time() pool = Pool(processes=10) result = pool.apply_async(f, (1,)) print result.get() pool.close() print ' [i] Time elapsed...

Is it necessary to end a ThreadPool on Application.Exit()?

c#,thread-safety,threadpool,cancellation
I have a separate BackgroundWorker and ThreadPool that does methods in the background. (BackgroundWorker for UI feedback) ThreadPool: ThreadPool.QueueUserWorkItem(foo, null); I understand that I need to safely terminate my BackgroundWorker using a flag considering that it gives me an Exception if ever I close it without one. However, the ThreadPool...

Exception error using C++ Thread Pool Library (CTPL)

c++,multithreading,exception,visual-studio-2013,threadpool
I am trying to use the C++ Thread Pool Library (CTPL), which pleasantly consists in a single header file. For this, I created a very simple test program. Here is the code : #include <iostream> #include "ctpl_stl.h" void myFunction(int threadID_0) { std::this_thread::sleep_for(std::chrono::milliseconds(100)); } int main() { ctpl::thread_pool p(2); // We...

Play Framework and Threadpools

playframework,threadpool
Is there a limit on the number of threads that Play framework exposes to my application? Is the thread size inside Play app so darn precious? What if I use a thread pool that I create by myself in the application and not use the one provided by Play. Is...

How to delegate threads to multiple servers and keep a track in which thread is on which server using Java

java,multithreading,delegates,threadpool
We have a solution which runs on 13 servers. I wish to know how do we explicitly delegate threads to different servers and keep a track of which thread was on which server.

Set only maximum worker threads in ThreadPool.SetMaxThreads c#

c#,multithreading,threadpool
ThreadPool.SetMaxThreads(int workerThreads, int completionPortThreads) The SetMaxThreads method requires both worker threads and completion port threads to be set. Is there any way where I can set only max worker threads and let the Threadpool decide the completion port threads. If there is no solution, is there a recommended value for...

Python code not executing in order? MySQLdb UPDATE commits in unexpected order

python,python-2.7,threadpool,mysql-python
I've got a Python 2.7 script I'm working on that retrieves rows from a MySQL table, loops through the data to process it, then is supposed to do the following things in this order: UPDATE the table rows we just got previously to set a locked value in each row...

Unordered map in shared_ptr SIGSEGV

c++,lambda,threadpool,shared-ptr,c++14
I'm building server for speech recognizer. I'm using thread pooling to serve clients. I needed to create unordered map to save instance of recognizer for each client. So I create this_ std::shared_ptr<std::unordered_map<int ,SOnlineSpeechRecognizerI *>> user_recognizers; std::mutex rec_mutex; So on client connection I create instance of recognizer and I need to...

interaction between Queue and pool of Threads?

c++,multithreading,pthreads,threadpool
I am creating a C++ application that receives data from an external provider. The amount of data per second can be very big. I have thought of using a thread that pushes the data into a Queue. then a pool of threads will fetch data from that Queue. my questions...

Java - How to log metrics from a ThreadPool?

java,logging,threadpool,guice
I wrapped a ThreadPoolExecutor in an implementation of ExecutorService of my own, just to send it any filesystem writing task, so they would be treated sequencially and one-by-one. (No need to harass this poor disk writing head.) The wrapper comes in handy by: allowing me to Inject this ThreadPool as...

AsyncTask, Thread or Services to run long shell command

android,multithreading,shell,android-asynctask,threadpool
In my current project , I have to run 5 differents binaries and show the output in different textView , those binaries give continuous output and will be run for a long time ( let's say 3/4h min ) therefore what is the best way to implement those ? AsyncTask...

Embedding Jetty 9 and customizing Socket Address, Port and ThreadPool?

jetty,threadpool
I have previously used Jetty 8.1.14 as embedded web server in my application. Now I am trying to upgrade to version 9.2.10. With Jetty 8, it was possible to specify the Host Address and Port using the setters in the "SelectChannelConnector" or "SslSelectChannelConnector", and also the ThreadPool as a constructor...

C++ Simple Thread Pool

c++,threadpool
I am trying to implement a simple thread pool in c++ as follows: class worker { public: worker(); thread mThread; private: void run(); }; worker::worker() { (this->mThread = thread(&worker::run, this)).detach(); } class threadpool { public: threadpool(int size); void addTask(); private: vector<worker> workers; }; But when I add the constructor of...

In the ForkJoinPool for Scala's “Future”, why is the ID for a worker always an odd number?

java,multithreading,scala,threadpool,fork-join
Here is the codes: import scala.concurrent._ import ExecutionContext.Implicits.global import scala.concurrent.duration._ val is = 1 to 100 toList def db = s"${Thread.currentThread}" def f(i: Int) = Future { println(db) ; 2 * i } val theFuture = Future.traverse(is)(f _) Await.result(theFuture, 10.seconds) I runned it many times, and the result looks like...

C++, How to implement Thread pool for tasks when each worker thread has to do few different tasks

c++,multithreading,threadpool
I am analyzing a video stream. For each new image (frame) i do the following 3 tasks sequentially: Reduce the size of the image Detect faces Track the 4 most important faces in the image In order to speed this up on a 4-cpu machine I use 4 worker threads....

Python thread pools - tasks that create subtasks and wait on them

python,multithreading,concurrency,threadpool,concurrent.futures
Say I have a thread pool executor with max. 10 threads, and I submit a task to it which itself creates another task and in turn waits for it to complete, recursively until I reach a depth of 11. Example code in Python: import concurrent.futures e = concurrent.futures.ThreadPoolExecutor(max_workers=10) def task(depth):...

GThreadPool is not creating more than N threads

multithreading,threadpool,glib
I'm trying to understand why I cannot start more than 4 threads when using a GThreadPool. pool = g_thread_pool_new ((GFunc)func, (gpointer)&values, g_get_num_processors (), TRUE, NULL); g_thread_pool_push (pool, val, NULL); g_thread_pool_push (pool, val, NULL); g_thread_pool_push (pool, val, NULL); g_thread_pool_push (pool, val, NULL); g_thread_pool_push (pool, val, NULL); The function g_get_num_processors () give...

ConcurrentModificationException Error in ThreadPoolTaskExecutor

java,multithreading,spring-boot,threadpool
I create ThreadPoolTaskExecutor using configuration annotation like this public class AsyncConfiguration implements AsyncConfigurer, EnvironmentAware { private final Logger log = LoggerFactory.getLogger(AsyncConfiguration.class); private RelaxedPropertyResolver propertyResolver; @Override public void setEnvironment(Environment environment) { this.propertyResolver = new RelaxedPropertyResolver(environment, "async."); } @Override @Bean public Executor getAsyncExecutor() { log.debug("Creating Async Task Executor"); ThreadPoolTaskExecutor...

PHP pthreads, in pool tasks, ob_flush and flush cause crash

php,apache,pthreads,threadpool,flush
$p = new Pool(10); for ($i = 0; i<1000; i++){ $tasks[i] = new workerThread($i); } foreach ($tasks as $task) { $p->submit($task); } // shutdown will wait for current queue to be completed $p->shutdown(); // garbage collection check / read results $p->collect(function($checkingTask){ return ($checkingTask->isGarbage); }); class workerThread extends Collectable {...

Non blocking way of adding a work item to array or list

c++,c++11,threadpool,future,lock-free
Edit: I now have finished my queue (overcoming the problem described below, and more). For those interested it can be found here. I'd be happy to hear any remarks:). Please note the queue isn't just a work item queue, but rather a template container which of course could be instantiated...

Is my understanding of the C# threadpool correct?

c#,.net,multithreading,threadpool
I'm reading Essential C# 5.0 which says, The thread pool also assumes that all the work will be relatively short-running (that is, consuming milliseconds or seconds of processor time, not hours or days). By making this assumption, it can ensure that each processor is working full out on a task,...

Schedulers in Rxcpp

c++,threadpool,system.reactive,ppl
I'm trying to figure out the scheduling model in the C++ version of Rx. Knowing the C# version where there is a simple interface with one Schedule method; The C++ version seems rather complex, with stuff like scheduler, worker, and coordination. One major missing piece for me is an implementation...

Async-Await vs ThreadPool vs MultiThreading on High-Performance Sockets (C10k Solutions?)

c#,multithreading,sockets,asynchronous,threadpool
I'm really confused about async-awaits, pools and threads. The main problem starts with this question: "What can I do when I have to handle 10k socket I/O?" (aka The C10k Problem). First, I tried to make a custom pooling architecture with threads that uses one main Queue and multiple Threads...

thread pool error trapping

vb.net,threadpool
Checking on a best practice here. I am calling a ThreadPool to run an Async process, specifically sending an SSL email with an attachment which seems to take a good long while. This is just test code, I thought I would drop this in a Try Catch just incase something...

Java sequential implementation is 4 times faster than parallel implementation

java,multithreading,parallel-processing,threadpool,executorservice
I created a really simple scenario where I recognized a really weird behavior which I cant understand. Under following link I created an sequential implementation: http://ideone.com/B8JYeA Basically there are several big arrays with fixed size. The algorithm iterates through them and changes the value. for(int i = 0; i <...

construction of thread using lambda expression

multithreading,c++11,lambda,threadpool
Can somebody explain where a thread is created on the lambda function below? What is the technique used? Can somebody recommend a reference to understand the semantics? I posted the full code now: class ThreadPool { public: ThreadPool(size_t); template<class F, class... Args> auto enqueue(F&& f, Args&&... args) ->std::future<typename std::result_of<F(Args...)>::type>; ~ThreadPool();...

ThreadPoolExecutor application does not Finish

java,concurrency,threadpool,threadpoolexecutor
This super simple application prints "Hello" but does not finish. I see absolutely no reason why this should be. JavaDoc, section finalization, says that A pool that is no longer referenced in a program AND has no remaining threads will be shutdown automatically. tpe is clearly not referenced, that means...

How to get results from the CompletableFuture

java,multithreading,threadpool,executorservice,countdownlatch
each of "CompletableFuture.runAsync" mentioned in the code below does some calculations, an i want to get the results each time i call "CompletableFuture.runAsync". or in other words, i want each of "future0,future1,future2,future3" to contain the result of each call to "CompletableFuture.runAsync" respectively how can i do that. *Update: my requirements...

Android : Run A task in a separate thread outside application scope

android,threadpool,remote-process
I am trying to do specific task in a separate thread in android using Thread Pool executor with the Max thread size of 5 making sure there can be five parallel task would be run at a time. But problem with this approach is When I close my application the...

Creating a thread pool in one function call and using it from another function call

c++,multithreading,dll,threadpool
I have a Fortran program that calls a C++ dll to do some mathematical operations on 10000 sets of data. The data sets are totally independent from each other. I was planning to create a thread pool and then send tasks to it. However, the call to the dll will...

ExecutorService workStealingPool and cancel method

java,threadpool,executorservice
Can you think about any reason why this code doesn't work and always outputs "finished", but the second example works without any problems. I'm using latest JDK (8u45). public static class MyClass implements Runnable { @Override public void run() { try { Thread.sleep(2000); } catch (InterruptedException ex) { System.out.println("Interrupted"); return;...

Is it possible to have a set of thread pools that share threads from a large thread pool instead of creating new thread?

java,multithreading,concurrency,threadpool,executorservice
Is it possible to have a set of thread pools that share threads from a large thread pool instead of creating new thread? In our RESTful API application, a request may involve several parallel tasks. To improve the performance, we want to execute parallel tasks in a thread pool which...

Custom Future objects

java,multithreading,concurrency,threadpool
I'd like to create custom Future objects. The following code works fine ThreadPoolExecutor mExecutor; Future<?> f = mExecutor.submit(new DownloadRunnable(task, itemId)); I'd like to take the return value of submit and assign it to MyFuture object, with additional calls. I've made the following changes, and getting a cast exception... Any suggestions?...

Why i can't start thread in thread pool?

c#,multithreading,threadpool
I just created function of thread: static void ThreadMethod {} And try this: ThreadPool.QueueUserWorkItem(ThreadMethod); But QueueUserWorkItem requests WaitCallback object. It's just example from my book and looks like it must work this way. What am i missing?...

Waiting for ThreadPool to finish

c#,multithreading,threadpool,streamwriter
I developed an application that saves 10000 lines of lorem ipsum in .txt file on the disk. I used StreamWriter to actually write these lines, and ThreadPool to distribute the computational process to the available threads/cores. The saving is very fast, even with 1 million lines, but the problem is,...

What is the impact of libraries that maintain their own threads pool?

concurrency,threadpool,guava,hystrix
I'm using Google Guava and Netflix Hystrix libraries in our project. Each library comes with it is own pool of threads that you can configure. Thats made think of the impact of that. I mean each library is maintaining its own pool of threads and of course, each hardware has...

Context Switches on Sleeping/Waiting Threads

multithreading,threadpool,nonblocking,context-switch,context-switching
I'm trying to understand how operating systems handle context switching in different models to better understand why NIO performance is better in cases of large peaks in the number of requests. Apart from the fact that there may be a limit to the number of threads, I'm curious how blocking...

How to use LinkedBlockingQueue to run tasks

java,threadpool,blockingqueue
I am currently trying to get my code to print what is in the method of simpleTask 20 times using the code in simpleTesting. The idea is that simpleTesting adds 20 instances of simpleTask to a queue then they are taken from the queue in simplePoolThread. What should happen is...

Memory leak when using .NET ThreadPool

.net,memory-leaks,c#-3.0,threadpool,queueuserworkitem
I am using System.Threading.ThreadPool.QueueUserWorkItem() to run my tasks. When running my application, I noticed it is taking a lot of memory and I took a dump file using the task manager. Investigting it, I found that many threads were stuck in the following stack: ntdll!NtWaitForSingleObject+a KERNELBASE!WaitForSingleObjectEx+79 mscorwks!CLREvent::WaitEx+174 mscorwks!CLREvent::WaitEx+63 mscorwks!ThreadpoolMgr::SafeWait+7b mscorwks!ThreadpoolMgr::WorkerThreadStart+11a...

Java thread pool blocking queue, execute always last few

java,multithreading,threadpool,threadpoolexecutor
I have several tasks that are being created by some event. I want to execute the last few tasks(suppose 6) always. I am using a fixed thread pool. But the problem I am facing is that, it uses a blocking queue internally. I want to dequeue the tasks from the...

Multitenented Parallel Processing

java,multithreading,tomcat,parallel-processing,threadpool
I have a tomcat server with a handful of fairly long-running operations that I feel could be greatly sped up if broken up and processed in parallel. What is the best practice as far as this is concerned? I am concerned that if I just create some threadpool that is...

CSV rowcount breaks ThreadPool

python,csv,threadpool
I am having a quite strange problem with CSV's and threadpool in Python 2.7 Whenever I try to count the amount of rows in the CSV my thread won't start. Uncommented the code works without any problems. import csv from multiprocessing.pool import ThreadPool ifile = open('unzipped/49619.txt', "rb") reader = csv.reader(ifile,...

Why ThreadPool start only one thread in time?

c#,multithreading,threadpool
I start threads exactly like book says: for (int i = 1; i <= 4; i++) { ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadMethod), i); } ThreadMethod looks like: static void ThreadMethod(object input) { Console.WriteLine(input + " thread started"); //do some stuff for, like, 400 milliseconds Console.WriteLine(input + " thread completed"); } In some reason...