You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 16:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Mon Apr 06, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
801699250,24cyclictest5603-21ssh11:51:093
801699250,24cyclictest5352-21apt-get10:48:563
801599250,24cyclictest2157-21diskmemload09:30:132
8011992525,0cyclictest0-21swapper/111:43:391
8011992523,1cyclictest0-21swapper/109:38:331
801199250,1cyclictest19504-21ssh10:29:451
8009992523,1cyclictest15973-21ssh12:01:200
801699240,23cyclictest2157-21diskmemload12:06:583
801699240,23cyclictest0-21swapper/312:33:223
801699240,23cyclictest0-21swapper/310:56:263
801699240,23cyclictest0-21swapper/309:18:233
801699240,23cyclictest0-21swapper/309:14:053
801699240,1cyclictest15448-21sh10:25:583
801599242,21cyclictest2157-21diskmemload12:22:372
8015992420,1cyclictest0-21swapper/210:34:232
8015992419,2cyclictest0-21swapper/209:15:382
801599241,22cyclictest28-21ksoftirqd/211:48:572
801599240,23cyclictest21349-21ssh12:38:582
801599240,23cyclictest0-21swapper/210:05:012
801599240,23cyclictest0-21swapper/209:26:452
801599240,1cyclictest28-21ksoftirqd/211:04:302
801599240,0cyclictest0-21swapper/208:23:552
8011992422,1cyclictest622-21gdbus09:55:001
8011992422,1cyclictest25116-21cron12:10:011
8011992422,1cyclictest0-21swapper/110:44:541
801199240,24cyclictest0-21swapper/110:49:381
801199240,24cyclictest0-21swapper/110:15:351
801199240,23cyclictest29985-21sh10:40:011
801199240,23cyclictest2638-21ssh09:10:301
801199240,23cyclictest2157-21diskmemload09:34:111
801199240,23cyclictest0-21swapper/112:03:451
801199240,23cyclictest0-21swapper/111:12:571
801199240,23cyclictest0-21swapper/109:16:461
8009992423,1cyclictest31975-21ssh10:10:130
8009992421,2cyclictest14860-21sh11:59:570
800999240,24cyclictest0-21swapper/010:57:080
800999240,23cyclictest20763-21perf07:35:000
800999240,23cyclictest18141-21ssh11:31:590
800999240,1cyclictest0-21swapper/011:18:330
8016992323,0cyclictest2157-21diskmemload12:15:553
8016992323,0cyclictest0-21swapper/310:15:073
8016992321,1cyclictest313-21systemd-journal10:44:563
801699230,23cyclictest0-21swapper/312:35:473
801699230,23cyclictest0-21swapper/311:56:263
801699230,22cyclictest30906-21sh11:13:133
801699230,22cyclictest30607-21kworker/u8:009:41:343
801699230,22cyclictest28867-21ssh09:35:453
801699230,22cyclictest28860-21ssh11:42:443
801699230,22cyclictest23801-21ssh11:37:493
801699230,22cyclictest22996-21sh10:02:153
801699230,22cyclictest22391-21ssh11:04:033
801699230,22cyclictest18764-21sh11:33:043
801699230,0cyclictest2157-21diskmemload12:21:233
801699230,0cyclictest2157-21diskmemload11:20:363
8015992323,0cyclictest16257-21apt-get12:33:542
8015992323,0cyclictest0-21swapper/212:10:352
8015992323,0cyclictest0-21swapper/211:56:542
8015992323,0cyclictest0-21swapper/210:02:042
8015992321,1cyclictest0-21swapper/212:16:012
8015992320,1cyclictest0-21swapper/211:11:412
8015992320,0cyclictest0-21swapper/210:24:512
801599231,1cyclictest28-21ksoftirqd/209:58:062
801599230,23cyclictest0-21swapper/211:50:432
801599230,22cyclictest2157-21diskmemload10:17:212
801599230,22cyclictest16733-21munin-plugin-st11:30:012
801599230,1cyclictest2157-21diskmemload09:40:432
801599230,0cyclictest28-21ksoftirqd/211:05:392
801599230,0cyclictest2157-21diskmemload12:08:152
801599230,0cyclictest0-21swapper/212:27:512
801599230,0cyclictest0-21swapper/212:01:572
8011992323,0cyclictest0-21swapper/109:42:461
8011992323,0cyclictest0-21swapper/109:27:431
8011992321,1cyclictest0-21swapper/111:16:471
8011992320,2cyclictest6608-21sh11:52:481
8011992320,2cyclictest313-21systemd-journal08:45:011
8011992320,2cyclictest24816-21sh11:06:431
801199230,2cyclictest2157-21diskmemload09:48:561
801199230,23cyclictest0-21swapper/112:26:141
801199230,23cyclictest0-21swapper/111:25:091
801199230,22cyclictest9803-21sh10:52:551
801199230,22cyclictest26833-21ssh12:12:231
801199230,22cyclictest19759-21rm11:01:491
801199230,22cyclictest14911-21cron12:00:011
801199230,22cyclictest0-21swapper/110:14:531
801199230,22cyclictest0-21swapper/109:55:361
801199230,22cyclictest0-21swapper/108:40:001
801199230,1cyclictest2157-21diskmemload11:47:321
801199230,1cyclictest0-21swapper/111:24:311
801199230,1cyclictest0-21swapper/110:55:021
801199230,0cyclictest0-21swapper/112:19:561
801199230,0cyclictest0-21swapper/111:32:271
801199230,0cyclictest0-21swapper/110:31:251
801199230,0cyclictest0-21swapper/109:21:301
8009992323,0cyclictest0-21swapper/011:44:480
8009992323,0cyclictest0-21swapper/010:45:510
8009992323,0cyclictest0-21swapper/010:39:060
800999231,1cyclictest0-21swapper/011:24:210
800999230,23cyclictest0-21swapper/009:44:020
800999230,22cyclictest8170-21rm10:50:200
800999230,22cyclictest7374-21ssh09:46:490
800999230,22cyclictest3493-21rm12:20:280
800999230,22cyclictest32474-21ssh11:45:270
800999230,22cyclictest23026-21ssh10:02:170
800999230,22cyclictest23021-21perf09:30:010
800999230,22cyclictest20058-21ssh11:02:210
800999230,22cyclictest14974-21sh10:25:170
800999230,22cyclictest13542-21ssh09:20:510
800999230,22cyclictest0-21swapper/012:05:080
800999230,22cyclictest0-21swapper/011:08:150
800999230,1cyclictest16800-21ssh09:55:320
800999230,1cyclictest13135-21cron07:20:010
800999230,0cyclictest0-21swapper/009:35:020
8016992222,0cyclictest0-21swapper/310:37:503
8016992222,0cyclictest0-21swapper/309:51:393
8016992220,1cyclictest32667-21ssh11:45:443
8016992220,1cyclictest30624-21sh10:09:163
8016992220,1cyclictest0-21swapper/307:50:003
801699220,2cyclictest1693-21ssh10:13:403
801699220,22cyclictest0-21swapper/312:11:413
801699220,22cyclictest0-21swapper/307:13:543
801699220,21cyclictest20117-21ssh10:30:173
801699220,21cyclictest13019-21sh09:20:043
801699220,21cyclictest0-21swapper/311:15:503
801699220,21cyclictest0-21swapper/311:05:233
801699220,21cyclictest0-21swapper/309:59:363
801699220,21cyclictest0-21swapper/307:38:553
801699220,21cyclictest0-21swapper/307:28:553
801699220,1cyclictest6674-21ssh09:45:463
801699220,1cyclictest0-21swapper/310:50:193
801699220,0cyclictest0-21swapper/312:25:313
8015992222,0cyclictest0-21swapper/210:45:192
8015992221,0cyclictest0-21swapper/210:57:482
8015992220,1cyclictest812-21runrttasks10:12:152
8015992220,1cyclictest13270-21ssh09:20:272
8015992220,1cyclictest0-21swapper/211:41:152
8015992220,1cyclictest0-21swapper/211:36:302
8015992220,1cyclictest0-21swapper/211:30:052
8015992220,1cyclictest0-21swapper/211:21:182
8015992220,1cyclictest0-21swapper/210:36:312
801599220,1cyclictest0-21swapper/210:52:382
801599220,1cyclictest0-21swapper/209:37:172
801599220,1cyclictest0-21swapper/208:53:542
801599220,0cyclictest0-21swapper/210:41:332
801599220,0cyclictest0-21swapper/209:50:392
8011992221,1cyclictest0-21swapper/112:21:581
8011992220,1cyclictest2157-21diskmemload10:00:301
801199220,21cyclictest23291-21cron07:40:001
801199220,21cyclictest0-21swapper/112:35:121
801199220,21cyclictest0-21swapper/112:30:021
801199220,21cyclictest0-21swapper/111:36:131
801199220,21cyclictest0-21swapper/110:22:521
801199220,21cyclictest0-21swapper/110:06:341
801199220,21cyclictest0-21swapper/108:18:551
801199220,1cyclictest15570-21latency_hist07:25:011
8009992220,2cyclictest28394-21sh10:07:360
8009992220,1cyclictest8482-21ssh12:25:200
8009992220,1cyclictest5524-21ssh11:51:020
800999221,21cyclictest62550irq/127-eno109:17:120
800999220,22cyclictest0-21swapper/012:11:130
800999220,21cyclictest32203-21munin-plugin-st09:05:010
800999220,21cyclictest30507-21ssh12:15:020
800999220,21cyclictest29773-21idleruntime-cro09:00:010
800999220,21cyclictest13740-21apt-get07:23:540
800999220,21cyclictest13231-21ssh09:53:030
800999220,21cyclictest13090-21apt-get08:28:540
800999220,21cyclictest0-21swapper/011:36:140
800999220,21cyclictest0-21swapper/008:05:010
800999220,1cyclictest27393-21munin-run08:55:010
800999220,1cyclictest24990-21perf08:50:010
800999220,1cyclictest2157-21diskmemload12:30:410
800999220,1cyclictest20852-21ssh12:38:140
800999220,1cyclictest0-21swapper/010:41:090
800999220,0cyclictest0-21swapper/010:18:040
800999220,0cyclictest0-21swapper/009:31:500
8016992120,1cyclictest9828-21ssh10:20:113
8016992119,1cyclictest0-21swapper/308:40:003
801699210,21cyclictest0-21swapper/312:00:023
801699210,20cyclictest0-21swapper/307:53:553
801699210,0cyclictest30349-21apt-get09:03:533
801699210,0cyclictest26845-1kworker/u9:011:29:233
801699210,0cyclictest2157-21diskmemload09:30:543
801699210,0cyclictest0-21swapper/309:28:363
8015992119,1cyclictest23336-21sendmail-msp07:40:002
801599210,21cyclictest0-21swapper/209:03:532
801599210,20cyclictest28685-21apt-get07:53:552
801599210,0cyclictest2157-21diskmemload09:45:152
801599210,0cyclictest0-21swapper/207:33:552
8011992120,0cyclictest1074-21apt-get08:03:541
801199210,20cyclictest0-21swapper/107:48:541
801199210,0cyclictest25539-21apt-get08:53:541
8009992121,0cyclictest0-21swapper/009:11:050
8009992121,0cyclictest0-21swapper/008:23:540
8009992120,1cyclictest28685-21apt-get07:53:550
800999210,21cyclictest0-21swapper/010:20:340
800999210,0cyclictest8283-21apt-get08:18:550
800999210,0cyclictest2157-21diskmemload10:30:120
800999210,0cyclictest0-21swapper/011:25:430
800999210,0cyclictest0-21swapper/007:48:540
8016992020,0cyclictest0-21swapper/308:28:543
8016992020,0cyclictest0-21swapper/308:13:543
8015992020,0cyclictest0-21swapper/208:33:552
8015992020,0cyclictest0-21swapper/208:28:532
8015992020,0cyclictest0-21swapper/207:28:552
8015992018,1cyclictest26286-21apt-get07:48:542
8015992018,1cyclictest0-21swapper/208:50:012
8015992018,1cyclictest0-21swapper/208:10:012
8011992020,0cyclictest16529-21apt-get07:28:551
8011992020,0cyclictest0-21swapper/107:53:551
8011992019,1cyclictest0-21swapper/107:10:021
801199200,0cyclictest0-21swapper/107:58:551
801199200,0cyclictest0-21swapper/107:58:551
8009992020,0cyclictest0-21swapper/008:38:560
8009992020,0cyclictest0-21swapper/008:08:550
800999200,1cyclictest0-21swapper/009:09:000
800999200,0cyclictest0-21swapper/008:33:550
8016991919,0cyclictest0-21swapper/308:33:553
8016991918,1cyclictest18927-21apt-get07:33:553
8016991917,1cyclictest456-21dump-pmu-power08:00:013
8016991917,1cyclictest456-21dump-pmu-power08:00:013
8015991919,0cyclictest0-21swapper/207:58:562
8015991919,0cyclictest0-21swapper/207:58:562
8011991919,0cyclictest0-21swapper/108:48:561
8011991917,1cyclictest0-21swapper/109:00:011
8009991919,0cyclictest20299-21apt-get08:43:540
8009991919,0cyclictest0-21swapper/008:13:550
8015991818,0cyclictest0-21swapper/207:13:542
8011991818,0cyclictest0-21swapper/108:23:551
8011991817,1cyclictest15490-21apt-get08:33:551
8016991717,0cyclictest0-21swapper/307:43:553
801699170,16cyclictest15582-21apt07:25:013
801699170,16cyclictest0-21swapper/308:48:563
8011991717,0cyclictest0-21swapper/108:28:541
8011991715,1cyclictest7719-21cat08:15:011
8011991714,2cyclictest13195-21basename07:20:011
8009991717,0cyclictest21424-21apt-get07:38:550
8009991717,0cyclictest0-21swapper/007:28:540
8009991716,0cyclictest23894-21apt-get07:43:550
8016991616,0cyclictest0-21swapper/308:53:543
8016991616,0cyclictest0-21swapper/308:18:553
8016991616,0cyclictest0-21swapper/308:03:543
8016991616,0cyclictest0-21swapper/307:18:553
801699160,9cyclictest313-21systemd-journal08:45:013
8015991616,0cyclictest0-21swapper/208:38:562
8015991616,0cyclictest0-21swapper/208:03:542
8015991616,0cyclictest0-21swapper/207:43:552
8015991614,1cyclictest29797-21perf09:00:012
8015991614,1cyclictest22207-21munin-run08:45:012
801199160,0cyclictest151rcu_preempt08:08:571
801599153,11cyclictest751-21/usr/sbin/munin09:08:562
801599151,0cyclictest28-21ksoftirqd/208:10:012
8009991515,0cyclictest31081-21apt-get07:58:550
8009991515,0cyclictest31081-21apt-get07:58:550
8009991515,0cyclictest0-21swapper/007:13:540
8016991414,0cyclictest0-21swapper/308:20:143
801699140,14cyclictest0-21swapper/309:06:443
801699140,14cyclictest0-21swapper/308:55:073
801699140,0cyclictest0-21swapper/308:05:343
8015991414,0cyclictest0-21swapper/207:20:402
801599140,14cyclictest0-21swapper/207:15:232
8011991414,0cyclictest0-21swapper/109:05:031
801199140,0cyclictest0-21swapper/109:00:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional