You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 05:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rackeslot8s.osadl.org (updated Fri Nov 28, 2025 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2517069912910,1288cyclictest7050452sleep300:14:563
2517069912660,1262cyclictest6406102sleep323:34:573
2517069912610,1258cyclictest3926072sleep320:59:563
6780622230,1sleep10-21swapper/123:55:181
25170699181,13cyclictest673239-21taskset23:55:123
190898180,7rtkit-daemon298094-21apt-get19:45:111
25170699176,8cyclictest304101-21turbostat19:50:003
251706991714,1cyclictest0-21swapper/322:00:153
190898170,3rtkit-daemon437222-21ssh21:28:530
251706991614,1cyclictest0-21swapper/323:00:003
251706991613,1cyclictest0-21swapper/323:28:003
251706991613,1cyclictest0-21swapper/323:20:173
251706991613,1cyclictest0-21swapper/321:10:243
251706991613,1cyclictest0-21swapper/320:10:003
251706991613,1cyclictest0-21swapper/320:00:153
25170699161,12cyclictest616229-21turbostat23:19:593
25170699161,11cyclictest648677-21turbostat23:40:013
25170699161,11cyclictest478562-21taskset21:55:143
5128842151,7sleep2514935-21kthreadcore22:15:182
25170699158,4cyclictest6003362sleep323:10:133
25170699157,6cyclictest354876-21turbostat20:30:003
25170699156,7cyclictest421402-21turbostat21:24:563
25170699155,7cyclictest541ktimers/319:40:003
251706991512,1cyclictest0-21swapper/319:15:243
251706991512,1cyclictest0-21swapper/300:04:493
25170699151,11cyclictest559611-21taskset22:45:153
25170699151,11cyclictest543228-21taskset22:35:143
25170699151,11cyclictest444327-21taskset21:31:423
25170699151,10cyclictest511049-21taskset22:15:143
7454072141,4sleep2745423-21latency_hist00:39:592
25170699149,3cyclictest4534212sleep321:39:113
25170699145,7cyclictest688956-21turbostat00:09:553
25170699145,6cyclictest4773312sleep321:53:273
25170699141,9cyclictest524323-21taskset22:20:233
251706991413,0cyclictest0-21swapper/322:30:083
251706991413,0cyclictest0-21swapper/322:30:083
251706991412,1cyclictest0-21swapper/323:41:333
251706991412,1cyclictest0-21swapper/320:49:333
251706991412,1cyclictest0-21swapper/319:32:033
25170699141,11cyclictest251856-21taskset19:10:023
251706991411,1cyclictest0-21swapper/322:05:203
251706991411,1cyclictest0-21swapper/319:58:393
25170699141,10cyclictest737271-21taskset00:34:433
25170699141,10cyclictest721622-21taskset00:25:143
25170699141,10cyclictest657095-21taskset23:45:103
25170699141,10cyclictest591052-21taskset23:03:393
25170699141,10cyclictest557140-21chrt22:40:433
25170699141,10cyclictest532614-21taskset22:25:243
25170699141,10cyclictest461804-21taskset21:44:533
25170699141,10cyclictest414130-21taskset21:15:153
25170699141,10cyclictest404681-21taskset21:05:253
25170699141,10cyclictest380376-21taskset20:50:153
25170699141,10cyclictest367138-21chrt20:35:363
25170699141,10cyclictest360142-21chrt20:30:193
25170699140,11cyclictest738604-21taskset00:35:153
25170699140,11cyclictest599700-21taskset23:09:563
25170699140,11cyclictest367748-21taskset20:40:013
190898140,4rtkit-daemon541381-21ssh22:31:380
190898140,4rtkit-daemon541381-21ssh22:31:380
4448012130,10sleep1444809-21ssh21:32:531
3440172131,9sleep0345150-21grep20:20:200
3440172131,9sleep0345150-21grep20:20:200
25170699131,9cyclictest572694-21taskset22:50:203
25170699131,9cyclictest271822-21taskset19:20:243
251706991312,0cyclictest0-21swapper/321:45:483
251706991312,0cyclictest0-21swapper/300:15:453
251706991311,1cyclictest0-21swapper/321:27:463
251706991311,1cyclictest0-21swapper/319:28:303
25170699131,10cyclictest720127-21taskset00:22:213
25170699131,10cyclictest672467-21taskset23:54:003
25170699131,10cyclictest297566-21taskset19:41:553
25170699130,10cyclictest3929052sleep321:00:183
25170699130,10cyclictest342545-21taskset20:20:023
25170699130,10cyclictest342545-21taskset20:20:023
190898130,9rtkit-daemon735442-21ssh00:30:350
4520582121,9sleep1452062-21ssh21:36:021
25170699122,7cyclictest3296842sleep320:10:133
25170699121,9cyclictest310159-21taskset19:52:453
25170699120,9cyclictest335968-21chrt20:15:013
25170699120,10cyclictest502325-21taskset22:10:013
190898120,7rtkit-daemon454372-21grep21:40:140
190898120,4rtkit-daemon477864-21ssh21:54:450
6794112111,7sleep2679419-21ssh23:56:322
6243062111,7sleep0624415-21apt-key23:25:010
5911002110,8sleep1591106-21ssh23:03:461
5169882111,7sleep0516996-21ssh22:16:200
4867702110,9sleep2487057-21sed22:00:152
3985622110,1sleep0310290-21kworker/0:1+mm_percpu_wq21:00:350
2515612110,6sleep1251590-21munin-run19:10:011
190898110,4rtkit-daemon81364-21packagekitd19:52:011
7364802101,6sleep2736486-21ssh00:32:552
6795832100,7sleep0679591-21ssh23:56:550
6425872100,3sleep2644535-21kthreadcore23:35:172
6218182100,1sleep2616557-21/usr/sbin/munin23:20:222
5563992100,7sleep0556477-21/usr/sbin/munin22:40:210
4782852100,3sleep0478289-21ssh21:55:080
2455892101,5sleep30-21swapper/319:05:143
190898100,7rtkit-daemon667-21snmpd21:12:101
190898100,6rtkit-daemon468271-21ssh21:46:050
190898100,6rtkit-daemon340380-21cut20:15:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional