Summary
K3 Target adaptors process trades concurrently by default. However, some Target adaptors may have problems processing data when using multiple threads. If this occurs, clients are able to process each trade one at a time by running the route on a single thread.
Context
Here is quick explanation of how threads work in K3 routes.
1. A single thread starts the route and goes upstream to ingest data
2. Data is sent to a thread pool (1)
3. If that thread pool is fully occupied, data is placed on the thread pool queue (2)
4. If the thread pool queue fills up, then the thread-pool-caller (original thread) will actually execute to the loader downstream (3). This is the default behavior of threads in the jvm.
5. If a repush is executed on the route and the thread-pool is occupied, another thread is spawned to handle this request (4)
Solution
In order to hack past the fail safe mechanisms (2/3) above to ensure only a single queue gets leveraged end to end:
First set these two default parameters in the route config:
core-thread-pool-size 1
max-thread-pool-size 1
max-db-pool-size 3
The max-db-pool-size
must be at least 2 greater than the number for the max-thread-pool-size
Finally, this additional parameter needs to be enabled in the route config:
thread-pool-queue-size 1000
This parameter represents the number of messages that can be held in the thread-pool-queue. It must be set sufficiently large such that the pool-queue does not overrun.
There is no magic number here, but a rule of thumb is to ensure all data for a single day could exist on the queue at the same time.
You do not want to simply make this number 9999999 as that will consume memory should all that space get occupied.