Linux and Perfomance: Reducing latency spikes by tuning the CPU scheduler

Reducing latency spikes by tuning the CPU scheduler: "It turns out that this effect can be observed when scylla runs in a different cgroup from the java process, but not when they run in the same cgroup. The reason for this effect is yet to be discovered.



 To check to which group given process belongs to, one can run:

 cat /proc/$PID/cgroup

The cgroup subsystem named cpu is relevant for scheduling.



 A process can be forced under a certain cgroup at run time with cgclassify. For example, to force all scylla and java (scylla-jmx) processes under the root group (-w is important for the change to affect all threads):

 sudo cgclassify -g cpu:/ `pgrep -w scylla`

sudo cgclassify -g cpu:/ `pgrep -w java`



Linux has a feature called autogroup which will automatically associate each session with a different cgroup. Processes started in those sessions will inherit group affinity. Due to latency issues mentioned above, the scylla package will disable autogroup (See this commit).



 Forcing scylla and scylla-jmx under the same group and applying scheduler settings has the desired effect of reducing scheduling delays. Now the maximum delay is 1.8 ms:"



'via Blog this'