You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-07 - 04:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Wed Jan 07, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
204362216018,2099sleep08555-1kworker/0:0H21:40:090
1765271217,677sleep31827-21/usr/sbin/munin20:40:273
2744126287,609sleep319609-1kworker/3:1H23:45:323
2744126287,609sleep319609-1kworker/3:1H23:45:323
1246925059,489sleep181-1kworker/1:1H21:15:511
20920248111,460sleep1196-21jbd2/mmcblk2p2-23:25:031
20920248111,460sleep1196-21jbd2/mmcblk2p2-23:25:031
873124036,364sleep0366-21snmpd19:20:350
5199234611,321sleep15244-21ntpq19:10:221
12751227716,246sleep212764-21irqstats21:16:162
22095227517,242sleep00-21swapper/021:45:140
405522666,249sleep00-21swapper/020:49:320
1911022658,245sleep019144-21ps21:35:240
1373226511,238sleep21428-21fschecks_time20:40:182
2562822436,197sleep00-21swapper/023:40:120
74622427,136sleep0815-21latency22:20:200
1889223934,144sleep02031-21grep00:05:290
40622233111,80sleep00-21swapper/000:14:580
4821221828,173sleep00-21swapper/019:10:130
819822078,145sleep08327-21grep21:00:310
819822078,145sleep08327-21grep21:00:310
14376218449,55sleep00-21swapper/021:20:220
10124217923,15sleep00-21swapper/019:25:110
403921778,121sleep00-21swapper/022:30:250
31256217728,111sleep031385-21unixbench_singl23:55:310
1895217516,144sleep12007-21cat20:40:291
26131216738,70sleep00-21swapper/020:15:200
8528216628,102sleep08599-21cat22:45:200
1041215928,78sleep00-21swapper/020:39:590
542321575,140sleep00-21swapper/000:17:570
27593215318,86sleep00-21swapper/020:20:160
668021526,135sleep06700-21dpkg22:40:070
668021526,135sleep06700-21dpkg22:40:070
2305021525,107sleep00-21swapper/023:30:270
31911215145,64sleep00-21swapper/020:34:580
915321437,122sleep00-21swapper/021:05:160
25387214312,116sleep225493-21latency_hist23:39:582
646821427,49sleep06470-21munin-run22:39:580
646821427,49sleep06470-21munin-run22:39:580
8964213743,45sleep00-21swapper/000:30:080
6915213610,112sleep26955-21if_eth022:40:172
6915213610,112sleep26955-21if_eth022:40:172
31855213321,94sleep331811-21latency_hist23:59:583
26366213036,81sleep026368-21cron21:59:560
14246213011,108sleep014317-21latency_hist23:04:580
17188212944,39sleep00-21swapper/021:30:160
31860212625,83sleep231861-21cron20:34:562
27261212629,60sleep027346-21switchtime22:00:290
5973212547,39sleep00-21swapper/000:20:150
5973212547,39sleep00-21swapper/000:20:150
31302212528,75sleep30-21swapper/320:30:293
4846212412,22sleep20-21swapper/220:50:232
15096212445,39sleep00-21swapper/023:05:270
1037021240,8sleep10-21swapper/121:09:561
568321238,103sleep05693-21munin-run20:54:580
16906212323,41sleep00-21swapper/019:45:270
365212227,82sleep0510-21awk00:00:280
6017211822,32sleep20-21swapper/220:55:162
27411211627,48sleep00-21swapper/023:45:130
27411211627,48sleep00-21swapper/023:45:130
6103211514,82sleep36143-21if_eth000:20:183
6103211514,82sleep36143-21if_eth000:20:183
29790211522,64sleep229784-21apt-get22:10:122
2841211535,56sleep32902-21fschecks_time00:10:163
31102211420,76sleep00-21swapper/022:14:580
26444211418,80sleep226507-21sed21:59:582
2486921149,59sleep00-21swapper/021:54:590
3160921127,91sleep031650-21ls22:15:190
19150211031,44sleep019165-21munin-run23:19:570
1060421108,81sleep010605-21/usr/sbin/munin21:10:080
19913210951,42sleep00-21swapper/019:55:240
17987210926,42sleep00-21swapper/019:50:150
21763210812,83sleep021893-21unixbench_singl20:00:290
20066210819,68sleep219926-21latency_hist21:39:592
7821210518,77sleep00-21swapper/019:17:160
16684210521,52sleep00-21swapper/021:28:180
1776210420,65sleep21796-21ls22:24:582
13296210321,16sleep00-21swapper/021:16:260
10225210320,70sleep110238-21fschecks_count19:25:201
2880721020,20sleep14617-21cobalt_printf20:24:571
2045921015,10sleep00-21swapper/023:22:190
2045921015,10sleep00-21swapper/023:22:190
26809210011,76sleep226889-21fschecks_time22:00:182
2514210014,73sleep12629-21latency_hist00:09:581
22915210015,71sleep122957-21fschecks_count20:05:191
22915210015,71sleep122957-21fschecks_count20:05:191
1161421009,78sleep311628-21irqstats22:55:203
2434229911,44sleep00-21swapper/023:35:200
1830329915,67sleep317875-21/usr/sbin/munin19:50:223
1480529915,68sleep314808-21cpu19:40:133
1480529915,68sleep314808-21cpu19:40:133
109182995,29sleep00-21swapper/022:53:030
2529229818,52sleep00-21swapper/020:10:350
2529229818,52sleep00-21swapper/020:10:350
3025429720,59sleep329765-21/usr/sbin/munin22:10:253
716929615,69sleep17190-21awk22:40:241
716929615,69sleep17190-21awk22:40:241
372229621,59sleep13731-21vmstat20:45:301
2481729615,67sleep124834-21idleruntime-cro21:54:561
1419529618,65sleep214230-21awk23:04:562
115392967,28sleep00-21swapper/000:36:390
537329417,62sleep25444-21switchtime19:10:272
400929419,26sleep20-21swapper/200:14:562
2394129415,47sleep00-21swapper/020:08:090
2394129415,47sleep00-21swapper/020:08:090
1693829410,40sleep00-21swapper/023:11:080
753029338,44sleep07596-21fschecks_time00:25:160
753029338,44sleep07596-21fschecks_time00:25:160
2929029323,45sleep00-21swapper/023:50:180
292262936,54sleep00-21swapper/022:06:260
2051329314,62sleep120541-21df21:40:141
1757229314,65sleep117557-21latency_hist23:14:571
872829211,19sleep20-21swapper/222:45:262
3218229219,53sleep332194-21fschecks_count00:00:153
1054229213,62sleep110593-21cpuspeed00:35:141
875129113,25sleep20-21swapper/200:29:582
875129113,25sleep20-21swapper/200:29:582
220362917,74sleep0366-21snmpd23:27:400
220362917,74sleep0366-21snmpd23:27:400
158582905,75sleep00-21swapper/019:42:160
158582905,75sleep00-21swapper/019:42:160
2164228923,51sleep321759-21timerandwakeup20:00:283
183228923,22sleep00-21swapper/020:40:280
454428828,39sleep14563-21munin-run19:09:581
454428828,39sleep14563-21munin-run19:09:581
390928841,25sleep20-21swapper/219:05:272
390928841,25sleep20-21swapper/219:05:272
2242128818,52sleep122573-21latency_hist20:04:581
2104028813,60sleep221113-21fschecks_time23:25:162
2104028813,60sleep221113-21fschecks_time23:25:162
1506928711,59sleep215081-21/usr/sbin/munin19:40:202
1506928711,59sleep215081-21/usr/sbin/munin19:40:202
1389028728,45sleep014006-21timerwakeupswit19:35:290
1389028728,45sleep014006-21timerwakeupswit19:35:290
58722869,58sleep16014-21grep22:35:271
58722869,58sleep16014-21grep22:35:271
2707228629,28sleep20-21swapper/223:44:592
1760228629,27sleep20-21swapper/223:14:582
1380528515,52sleep313884-21sort19:35:283
1380528515,52sleep313884-21sort19:35:283
1756828419,51sleep117620-21processes21:30:251
1579228418,49sleep215872-21gpgv23:09:572
438828315,51sleep24395-21/usr/sbin/munin00:15:162
3227328317,41sleep232354-21fschecks_time20:35:172
2959828326,27sleep20-21swapper/222:10:002
1623728321,43sleep315755-21apt-get23:10:063
60082820,11sleep16005-21df20:55:141
256528219,49sleep12635-21sensors_temp22:25:291
117702827,41sleep011792-21cut19:30:160
590328136,11sleep00-21swapper/020:55:060
183682810,6sleep20-21swapper/219:50:252
490528011,51sleep34908-21/usr/sbin/munin19:10:163
417928036,29sleep0366-21snmpd19:05:570
417928036,29sleep0366-21snmpd19:05:570
416228018,50sleep24303-21latency_hist20:49:592
2295827926,29sleep20-21swapper/220:05:202
2295827926,29sleep20-21swapper/220:05:202
1132927922,16sleep00-21swapper/022:55:090
1908527826,36sleep319109-21munin-run23:19:563
1368627823,24sleep30-21swapper/321:19:573
117327811,48sleep11326-21unixbench_multi22:20:291
3036227721,42sleep130471-21sed20:29:591
2790727718,35sleep227480-21/usr/sbin/munin20:20:222
79692760,6sleep30-21swapper/321:00:253
79692760,6sleep30-21swapper/321:00:253
690027613,41sleep30-21swapper/320:56:143
3260727617,37sleep132616-21modprobe22:19:561
2823927628,28sleep30-21swapper/322:05:123
223527617,11sleep00-21swapper/022:25:200
1456927623,28sleep20-21swapper/219:39:592
1456927623,28sleep20-21swapper/219:39:592
1029227516,41sleep110354-21/usr/sbin/munin22:50:251
639127416,35sleep26397-21modprobe22:39:562
639127416,35sleep26397-21modprobe22:39:562
296072748,17sleep00-21swapper/020:25:290
1519727415,36sleep10-21swapper/123:05:281
760727323,23sleep10-21swapper/100:25:171
760727323,23sleep10-21swapper/100:25:171
2482827311,39sleep20-21swapper/220:10:252
2482827311,39sleep20-21swapper/220:10:252
2688227216,40sleep326883-21cron23:44:563
263627217,24sleep20-21swapper/222:25:302
1643727215,42sleep116451-21fschecks_count19:45:161
1563927217,21sleep10-21swapper/121:25:161
3021827122,32sleep130245-21munin-run23:54:551
2637827124,24sleep30-21swapper/320:15:273
2250427119,27sleep20-21swapper/223:30:132
2215827116,37sleep2378-21sshd20:03:142
1400627114,31sleep30-21swapper/321:20:133
486027013,38sleep34919-21/usr/sbin/munin20:50:243
2650827011,16sleep30-21swapper/321:59:593
254062709,51sleep10-21swapper/120:12:051
254062709,51sleep10-21swapper/120:12:051
2008227014,42sleep320203-21cat23:20:283
2008227014,42sleep320203-21cat23:20:283
1158227010,47sleep311583-21sshd00:37:143
1039127010,45sleep210465-21latency_hist21:09:582
59526928,24sleep30-21swapper/320:35:303
98926818,27sleep20-21swapper/200:04:582
860026814,30sleep10-21swapper/122:45:211
568226818,28sleep30-21swapper/322:35:253
568226818,28sleep30-21swapper/322:35:253
2831326817,32sleep128324-21diskstats22:05:141
2777326820,32sleep327795-21kernelversion20:20:193
1917626814,26sleep20-21swapper/223:19:592
2954026718,36sleep329605-21awk20:25:283
2419726720,28sleep124259-21/usr/sbin/munin23:35:171
499726625,24sleep10-21swapper/120:50:261
3051226615,33sleep20-21swapper/223:55:132
3005126618,25sleep10-21swapper/122:10:191
2356226613,38sleep223607-21cpu21:50:132
801726518,23sleep30-21swapper/322:44:583
801726518,23sleep30-21swapper/322:44:583
734626513,21sleep10-21swapper/119:15:271
67622657,12sleep30-21swapper/319:15:133
405426524,23sleep30-21swapper/319:05:283
405426524,23sleep30-21swapper/319:05:283
2382726516,33sleep323971-21latency_hist23:34:583
2226326517,24sleep30-21swapper/323:29:573
2226326517,24sleep30-21swapper/323:29:573
2054926513,23sleep320553-21diskstats21:40:163
1189126522,22sleep20-21swapper/219:30:182
1130826515,36sleep111433-21sed21:10:301
917126416,23sleep30-21swapper/319:20:583
880726415,27sleep30-21swapper/300:29:593
2738226419,30sleep227384-21/usr/sbin/munin23:45:072
2738226419,30sleep227384-21/usr/sbin/munin23:45:072
212726424,20sleep30-21swapper/322:25:193
1972126420,23sleep30-21swapper/319:55:193
1605126415,30sleep316127-21/usr/sbin/munin21:25:263
1179426417,28sleep211795-21cron00:39:562
1029826414,26sleep20-21swapper/200:34:592
2642926317,31sleep126504-21run-parts20:15:291
2431726319,18sleep30-21swapper/323:35:193
3197726224,16sleep10-21swapper/120:34:591
2652526215,19sleep10-21swapper/123:40:301
2247726219,22sleep10-21swapper/123:30:081
1202526220,22sleep20-21swapper/221:14:592
1010926118,26sleep30-21swapper/319:25:103
751126019,26sleep27504-21apt-get21:00:132
751126019,26sleep27504-21apt-get21:00:132
2979826019,20sleep20-21swapper/220:25:312
1947526016,28sleep119482-21df_inode23:20:141
1947526016,28sleep119482-21df_inode23:20:141
1185026018,19sleep20-21swapper/222:55:272
874725911,26sleep10-21swapper/119:20:481
412425919,25sleep14257-21cut00:14:591
2449025918,23sleep324580-21perl20:10:183
2449025918,23sleep324580-21perl20:10:183
190222595,18sleep10-21swapper/119:53:111
1587525919,17sleep30-21swapper/323:09:583
1450825918,16sleep10-21swapper/121:20:271
1062825916,27sleep310624-21apt-get21:10:113
719225816,25sleep27202-21munin-run00:24:572
719225816,25sleep27202-21munin-run00:24:572
69525814,25sleep3743-21irqstats22:20:183
3184825812,18sleep30-21swapper/322:15:263
282325816,26sleep32816-21apt-get20:45:113
2650525811,32sleep226598-21/usr/sbin/munin20:15:302
68102577,12sleep10-21swapper/100:20:321
68102577,12sleep10-21swapper/100:20:321
2668525717,23sleep126679-21apt-get22:00:131
245092575,13sleep30-21swapper/321:50:583
2067625716,16sleep20-21swapper/221:40:522
1845825718,16sleep30-21swapper/321:34:593
926125618,17sleep30-21swapper/300:30:203
855425614,24sleep28053-21apt-get19:20:082
758525613,18sleep10-21swapper/121:00:161
758525613,18sleep10-21swapper/121:00:161
1932325615,19sleep30-21swapper/321:35:273
1601425617,21sleep20-21swapper/221:25:252
1242525622,17sleep10-21swapper/119:30:291
827325515,22sleep28053-21apt-get19:20:002
8042557,36sleep10-21swapper/100:03:321
3051125514,23sleep130505-21apt-get23:55:121
2872525516,23sleep128701-21latency_hist23:49:591
2872525516,23sleep128701-21latency_hist23:49:591
2200725514,25sleep322003-21apt-get21:45:093
1570725514,26sleep115719-21vmstat19:40:311
1570725514,26sleep115719-21vmstat19:40:311
1455725515,22sleep1286-21dbus-daemon19:39:581
1455725515,22sleep1286-21dbus-daemon19:39:581
564325414,24sleep35653-21munin-run00:19:573
3261925412,28sleep232684-21unin-run22:19:572
2371225413,17sleep30-21swapper/320:05:453
2371225413,17sleep30-21swapper/320:05:453
1766025419,15sleep30-21swapper/319:49:583
1709525420,17sleep20-21swapper/219:45:292
899425312,23sleep19061-21df00:30:131
484125312,23sleep34859-21munin-run22:34:573
415125315,22sleep24222-21wc22:30:272
2936525312,23sleep229413-21ntp_states23:50:202
1948425314,16sleep20-21swapper/223:20:152
1948425314,16sleep20-21swapper/223:20:152
2552825215,22sleep125564-21ntp_states21:55:251
246225213,22sleep32492-21munin-run00:09:563
2240125217,20sleep20-21swapper/221:45:202
1998525210,18sleep20-21swapper/219:55:262
142025212,17sleep20-21swapper/200:05:182
1290425213,15sleep30-21swapper/323:00:093
1269225217,19sleep10-21swapper/122:59:591
1176325212,24sleep311768-21diskstats19:30:153
454525113,24sleep14578-21irqstats00:15:191
378525114,17sleep10-21swapper/122:30:181
2895825112,22sleep328953-21apt-get23:50:093
1945125113,21sleep119521-21df19:55:131
1117225112,18sleep20-21swapper/222:54:592
1008125114,22sleep310174-21memory22:50:193
318242506,35sleep10-21swapper/120:34:271
2275225012,17sleep10-21swapper/121:45:271
1408625015,21sleep214093-21diskstats21:20:162
1864424910,22sleep118641-21apt-get21:35:091
1781624912,21sleep117813-21apt-get23:15:091
103102496,34sleep30-21swapper/321:09:433
83332486,17sleep30-21swapper/322:45:143
251052489,22sleep325101-21apt-get21:55:103
1333524819,13sleep10-21swapper/123:00:231
74402478,20sleep30-21swapper/300:25:113
181372476,29sleep20-21swapper/221:32:072
1024324710,21sleep210346-21perl19:25:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional