You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 02:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Fri Jul 03, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2927399270,26cyclictest0-21swapper/110:34:171
2927399270,1cyclictest10394-21munin-run10:25:011
2928099260,2cyclictest6979-21idleruntime-cro09:25:003
2927399260,25cyclictest0-21swapper/111:56:151
2927399260,25cyclictest0-21swapper/111:46:341
2927399260,25cyclictest0-21swapper/111:16:161
2927399260,22cyclictest22610-21diskmemload11:33:031
2927399260,0cyclictest0-21swapper/112:38:511
2927299260,2cyclictest4591-21sh09:21:440
29280992523,1cyclictest0-21swapper/312:37:343
29276992522,3cyclictest0-21swapper/209:29:272
29276992520,4cyclictest0-21swapper/209:16:162
2927699250,24cyclictest0-21swapper/212:20:272
29273992522,3cyclictest0-21swapper/109:46:311
29273992521,3cyclictest27285-21ssh11:08:531
2927299250,24cyclictest22610-21diskmemload12:01:260
29280992423,1cyclictest0-21swapper/309:11:003
2928099242,21cyclictest28839-21ssh09:15:383
29280992422,1cyclictest22610-21diskmemload11:09:223
2928099240,23cyclictest30987-21perf08:20:013
2928099240,23cyclictest29418-21ssh10:12:193
2928099240,23cyclictest22610-21diskmemload12:15:293
2928099240,23cyclictest0-21swapper/310:47:013
2928099240,23cyclictest0-21swapper/309:54:483
2928099240,23cyclictest0-21swapper/309:46:553
2928099240,1cyclictest0-21swapper/311:34:353
2928099240,1cyclictest0-21swapper/311:23:153
2928099240,0cyclictest0-21swapper/310:23:113
29276992424,0cyclictest0-21swapper/209:54:242
29276992423,1cyclictest0-21swapper/210:36:562
29276992422,1cyclictest16956-21sh11:00:002
29276992422,0cyclictest0-21swapper/210:18:292
2927699240,24cyclictest19010-21rm10:31:422
2927699240,24cyclictest0-21swapper/210:02:252
2927699240,23cyclictest22610-21diskmemload12:33:332
2927699240,23cyclictest13036-21ssh11:25:202
2927699240,23cyclictest0-21swapper/211:30:262
2927699240,23cyclictest0-21swapper/211:20:112
2927699240,22cyclictest313-21systemd-journal08:15:012
29273992422,2cyclictest0-21swapper/110:00:151
29273992422,2cyclictest0-21swapper/109:43:271
29273992422,2cyclictest0-21swapper/109:43:271
29273992421,3cyclictest0-21swapper/109:55:431
29273992421,2cyclictest0-21swapper/109:15:581
29273992420,3cyclictest0-21swapper/110:26:471
29273992419,1cyclictest22610-21diskmemload11:37:151
29273992418,1cyclictest14167-21turbostat07:45:001
2927399240,24cyclictest22610-21diskmemload11:54:191
2927399240,23cyclictest0-21swapper/110:36:341
2927399240,0cyclictest22610-21diskmemload11:28:061
2927399240,0cyclictest22610-21diskmemload09:31:061
2927399240,0cyclictest19718-21apt-get09:36:171
2927399240,0cyclictest0-21swapper/112:07:221
29272992423,1cyclictest22610-21diskmemload11:22:450
2927299240,2cyclictest13380-21rm12:21:550
2927299240,24cyclictest0-21swapper/012:08:380
2927299240,23cyclictest28785-21users10:11:260
2927299240,23cyclictest22610-21diskmemload09:41:460
2927299240,23cyclictest22610-21diskmemload09:41:460
2927299240,23cyclictest0-21swapper/011:55:110
2927299240,1cyclictest0-21swapper/012:26:220
2927299240,0cyclictest0-21swapper/010:00:110
29280992323,0cyclictest22610-21diskmemload11:52:313
29280992323,0cyclictest22610-21diskmemload11:38:503
29280992323,0cyclictest22610-21diskmemload10:07:233
29280992321,1cyclictest2566-21apt-get11:16:163
29280992321,1cyclictest0-21swapper/312:07:043
29280992321,1cyclictest0-21swapper/310:35:403
29280992321,1cyclictest0-21swapper/309:36:503
2928099230,2cyclictest733-21grep12:11:213
2928099230,22cyclictest774-21runrttasks09:28:263
2928099230,22cyclictest30371-21sh11:41:063
2928099230,22cyclictest22610-21diskmemload10:31:223
2928099230,22cyclictest20428-21sh12:00:513
2928099230,22cyclictest15278-21diskmemload10:57:463
2928099230,22cyclictest14526-21apt-get07:46:153
2928099230,22cyclictest0-21swapper/312:31:473
2928099230,22cyclictest0-21swapper/309:30:233
2928099230,0cyclictest0-21swapper/311:13:423
2928099230,0cyclictest0-21swapper/309:57:553
29276992323,0cyclictest22610-21diskmemload12:25:502
29276992323,0cyclictest0-21swapper/211:57:212
29276992323,0cyclictest0-21swapper/211:51:332
29276992323,0cyclictest0-21swapper/211:10:462
29276992323,0cyclictest0-21swapper/210:53:562
29276992323,0cyclictest0-21swapper/209:10:082
29276992322,1cyclictest22610-21diskmemload11:17:262
29276992318,2cyclictest0-21swapper/211:06:172
2927699230,23cyclictest0-21swapper/211:36:332
2927699230,22cyclictest556-21ssh09:46:322
2927699230,22cyclictest28577-21ssh12:06:592
2927699230,22cyclictest2536-21rm11:45:002
2927699230,0cyclictest0-21swapper/212:14:142
2927699230,0cyclictest0-21swapper/210:25:512
2927699230,0cyclictest0-21swapper/209:31:372
29273992323,0cyclictest22610-21diskmemload12:31:091
29273992323,0cyclictest0-21swapper/112:26:181
29273992323,0cyclictest0-21swapper/112:00:501
29273992323,0cyclictest0-21swapper/111:20:061
29273992321,1cyclictest1397-21grep12:11:251
29273992320,3cyclictest0-21swapper/109:25:211
29273992319,3cyclictest0-21swapper/107:51:141
29273992319,1cyclictest12314-21ssh10:56:061
2927399230,2cyclictest1120-21rm11:43:061
2927399230,23cyclictest0-21swapper/112:23:331
2927399230,23cyclictest0-21swapper/111:13:341
2927399230,23cyclictest0-21swapper/109:20:551
2927399230,22cyclictest8248-21sendmail-msp08:40:001
2927399230,22cyclictest22610-21diskmemload10:41:111
2927399230,22cyclictest21234-21sh10:06:101
2927399230,22cyclictest0-21swapper/110:50:241
2927399230,20cyclictest23733-21sh09:11:061
2927399230,0cyclictest0-21swapper/110:46:471
2927399230,0cyclictest0-21swapper/110:10:051
29272992323,0cyclictest22610-21diskmemload12:37:140
29272992323,0cyclictest0-21swapper/012:32:400
29272992323,0cyclictest0-21swapper/011:43:440
29272992322,1cyclictest62550irq/127-eno111:17:420
29272992321,1cyclictest0-21swapper/011:26:020
29272992321,1cyclictest0-21swapper/010:37:450
2927299230,23cyclictest0-21swapper/012:12:590
2927299230,23cyclictest0-21swapper/010:46:470
2927299230,23cyclictest0-21swapper/009:59:020
2927299230,23cyclictest0-21swapper/009:50:580
2927299230,22cyclictest22987-21kworker/u8:109:30:460
2927299230,22cyclictest14092-21diskmemload11:54:590
2927299230,22cyclictest0-21swapper/010:05:070
2927299230,1cyclictest22160-21ssh09:37:110
2927299230,1cyclictest0-21swapper/011:36:200
2927299230,0cyclictest22610-21diskmemload12:18:320
2927299230,0cyclictest0-21swapper/011:30:080
2927299230,0cyclictest0-21swapper/010:54:440
2927299230,0cyclictest0-21swapper/009:14:040
29280992222,0cyclictest0-21swapper/311:56:043
29280992221,1cyclictest22610-21diskmemload09:42:573
29280992221,1cyclictest22610-21diskmemload09:42:573
29280992220,1cyclictest32535-21cron10:45:013
29280992220,1cyclictest11300-21/usr/sbin/munin10:26:203
29280992219,2cyclictest3412-21dump-pmu-power08:30:003
2928099220,22cyclictest0-21swapper/310:15:423
2928099220,21cyclictest2425-21apt-get07:21:153
2928099220,21cyclictest17347-21ssh11:00:103
2928099220,21cyclictest13483-21ssh11:26:053
2928099220,21cyclictest0-21swapper/311:45:093
2928099220,21cyclictest0-21swapper/310:00:063
2928099220,21cyclictest0-21swapper/308:11:153
2928099220,21cyclictest0-21swapper/308:06:163
2928099220,1cyclictest0-21swapper/312:25:083
2928099220,1cyclictest0-21swapper/308:41:173
2928099220,0cyclictest0-21swapper/312:22:543
29276992222,0cyclictest22610-21diskmemload09:23:352
29276992222,0cyclictest0-21swapper/212:35:442
29276992222,0cyclictest0-21swapper/212:01:112
29276992222,0cyclictest0-21swapper/210:22:222
29276992222,0cyclictest0-21swapper/210:14:442
29276992222,0cyclictest0-21swapper/209:55:482
29276992222,0cyclictest0-21swapper/209:35:532
29276992222,0cyclictest0-21swapper/207:46:152
29276992221,1cyclictest27239-21sh10:40:102
29276992221,1cyclictest0-21swapper/209:41:242
29276992221,1cyclictest0-21swapper/209:41:242
2927699220,22cyclictest22610-21diskmemload11:00:562
2927699220,21cyclictest0-21swapper/207:56:152
2927699220,1cyclictest3981-21apt-get12:15:022
2927699220,1cyclictest23564-21sh10:06:532
29273992220,1cyclictest19576-21ssh11:01:231
2927399220,21cyclictest3278-21ssh09:50:021
2927399220,1cyclictest26183-21munin-run08:10:011
2927399220,0cyclictest0-21swapper/112:18:041
29272992222,0cyclictest22610-21diskmemload11:05:130
29272992222,0cyclictest22610-21diskmemload09:45:410
29272992222,0cyclictest0-21swapper/011:00:520
29272992222,0cyclictest0-21swapper/007:41:140
29272992220,2cyclictest0-21swapper/009:16:540
29272992219,2cyclictest30953-21sh08:20:010
2927299220,21cyclictest6834-21cron07:30:010
2927299220,21cyclictest0-21swapper/011:10:040
2927299220,21cyclictest0-21swapper/010:25:280
2927299220,21cyclictest0-21swapper/009:01:160
2927299220,21cyclictest0-21swapper/008:26:160
2927299220,1cyclictest19318-21apt-get07:56:150
2927299220,1cyclictest1005-21ls08:25:000
2927299220,1cyclictest0-21swapper/010:30:230
2927299220,1cyclictest0-21swapper/010:21:590
2927299220,1cyclictest0-21swapper/009:27:210
2928099210,20cyclictest0-21swapper/307:26:153
2928099210,0cyclictest0-21swapper/310:50:173
29276992119,1cyclictest0-21swapper/209:10:012
2927699210,21cyclictest0-21swapper/211:45:462
2927699210,21cyclictest0-21swapper/210:45:302
2927699210,21cyclictest0-21swapper/207:26:152
29273992120,1cyclictest0-21swapper/108:35:001
2927399210,21cyclictest17798-21sendmail09:00:011
2927399210,21cyclictest0-21swapper/107:31:141
2927399210,0cyclictest28918-21apt-get08:16:161
29272992118,2cyclictest15435-21wc08:55:010
2927299210,20cyclictest0-21swapper/008:46:140
2927299210,0cyclictest0-21swapper/007:11:160
29280992020,0cyclictest0-21swapper/308:01:133
29280992018,1cyclictest13378-21apt-get08:51:143
29280992018,1cyclictest0-21swapper/307:15:003
2928099200,19cyclictest0-21swapper/307:41:153
29276992020,0cyclictest0-21swapper/207:36:162
2927699200,19cyclictest8236-21perf08:40:002
2927699200,0cyclictest0-21swapper/207:20:022
29273992020,0cyclictest31341-21apt-get08:21:131
29273992020,0cyclictest26523-21apt-get08:11:151
29273992020,0cyclictest1656-21apt-get07:20:021
29273992019,0cyclictest0-21swapper/108:01:141
29272992020,0cyclictest0-21swapper/008:36:150
29272992020,0cyclictest0-21swapper/008:31:160
29276991919,0cyclictest0-21swapper/208:56:162
29276991919,0cyclictest0-21swapper/208:51:142
29276991919,0cyclictest0-21swapper/208:21:132
29276991919,0cyclictest0-21swapper/208:16:152
29276991919,0cyclictest0-21swapper/208:06:152
29276991919,0cyclictest0-21swapper/207:11:162
2927699190,0cyclictest0-21swapper/208:31:162
29273991919,0cyclictest0-21swapper/107:36:161
29273991916,2cyclictest21390-21munin-run08:00:001
29272991919,0cyclictest0-21swapper/008:11:150
29272991919,0cyclictest0-21swapper/008:05:010
29272991919,0cyclictest0-21swapper/007:21:150
29280991818,0cyclictest0-21swapper/309:06:153
29280991818,0cyclictest0-21swapper/308:21:133
29276991818,0cyclictest7216-21apt-get07:31:142
29276991818,0cyclictest18194-21apt-get09:01:162
29273991818,0cyclictest0-21swapper/109:01:161
29273991818,0cyclictest0-21swapper/108:51:141
29273991818,0cyclictest0-21swapper/108:26:161
29272991816,1cyclictest9612-21apt-get07:36:160
2927299181,16cyclictest26148-21cron08:10:010
29276991717,0cyclictest0-21swapper/208:46:142
29273991717,0cyclictest10985-21apt-get08:46:141
29273991716,1cyclictest4819-21apt-get07:26:151
29272991717,0cyclictest0-21swapper/008:56:160
29272991717,0cyclictest0-21swapper/007:16:140
29280991616,0cyclictest16917-21apt-get07:51:143
29280991616,0cyclictest0-21swapper/308:46:143
29280991615,1cyclictest11735-21cron07:40:003
29280991614,1cyclictest0-21swapper/307:35:013
29280991612,3cyclictest20958-21tail07:56:253
29276991615,1cyclictest1370-21apt-get08:26:162
2927399160,7cyclictest0-21swapper/107:16:141
29280991515,0cyclictest0-21swapper/307:16:143
29273991513,1cyclictest10822-21munin-run08:45:011
2927399150,1cyclictest31629-21munin-run07:15:011
2927399150,1cyclictest0-21swapper/109:06:171
29272991515,0cyclictest0-21swapper/008:41:160
29272991515,0cyclictest0-21swapper/007:46:120
29280991412,1cyclictest16685-21memory08:56:213
2928099140,14cyclictest0-21swapper/309:00:123
2928099140,14cyclictest0-21swapper/308:36:583
2928099140,0cyclictest0-21swapper/308:30:083
29276991414,0cyclictest0-21swapper/207:41:282
2927699140,14cyclictest0-21swapper/208:41:352
2927699140,14cyclictest0-21swapper/208:01:262
2927699140,14cyclictest0-21swapper/207:15:472
2927699140,0cyclictest0-21swapper/207:50:422
29273991414,0cyclictest0-21swapper/107:46:051
29272991414,0cyclictest0-21swapper/007:51:580
2927299140,14cyclictest0-21swapper/009:05:450
2927299140,14cyclictest0-21swapper/007:30:070
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional