You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-19 - 20:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Mar 19, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
263572507415,5046sleep326339-21kworker/3:110:41:293
25696275822,719sleep225704-21/usr/sbin/munin12:23:012
6064252413,496sleep26136-21/usr/sbin/munin07:53:202
31238246667,385sleep131253-21apt-get07:32:521
2328023437,276sleep123316-21ntpq08:48:181
493723407,322sleep113441-21snmpd07:48:551
1584323200,96sleep123139-21cyclictest08:23:221
8374228311,257sleep28422-21unin-run08:02:532
8374228311,257sleep28422-21unin-run08:02:532
18201228189,173sleep118250-21latency11:58:131
26970225993,144sleep127012-21fschecks_count07:18:101
31582225093,53sleep10-21swapper/110:58:101
19572224114,207sleep019644-21fschecks_time12:03:090
2493422319,207sleep125065-21idleruntime07:12:571
16433223122,189sleep216440-21/usr/sbin/munin11:53:092
18230222837,162sleep118281-21cpuspeed08:33:081
24678220111,176sleep024775-21mailstats12:18:180
24678220111,176sleep024775-21mailstats12:18:180
2283621969,93sleep122935-21grep10:28:231
19581219614,128sleep119678-21timerandwakeup10:18:211
19581219614,128sleep119678-21timerandwakeup10:18:211
10924218014,95sleep110972-21latency_hist09:52:541
19995217719,57sleep10-21swapper/108:38:131
7622216931,66sleep17019-21/usr/sbin/munin11:23:211
7622216931,66sleep17019-21/usr/sbin/munin11:23:211
6802216021,35sleep16850-21latency_hist11:22:531
6802216021,35sleep16850-21latency_hist11:22:531
23783216083,62sleep123848-21ls10:33:111
14008215663,78sleep114011-21cron10:02:511
14008215663,78sleep114011-21cron10:02:511
31156215523,106sleep331171-21modprobe10:57:523
20838215414,72sleep120992-21latency_hist12:07:541
14333215330,95sleep00-21swapper/011:45:290
25991214921,109sleep10-21swapper/112:23:121
9933214438,82sleep10-21swapper/108:07:521
10600214441,86sleep110665-21processes11:33:181
10600214441,86sleep110665-21processes11:33:181
6604214213,110sleep1222-21systemd-journal07:55:291
21559214172,19sleep10-21swapper/108:43:131
483221395,12sleep00-21swapper/011:13:310
29089213717,78sleep129145-21systemd-run09:07:521
28817213717,14sleep10-21swapper/112:33:051
472213547,15sleep10-21swapper/109:18:141
22787213524,26sleep10-21swapper/112:13:111
22787213524,26sleep10-21swapper/112:13:111
6893212911,18sleep10-21swapper/109:38:161
1778821299,17sleep10-21swapper/110:13:171
1778821299,17sleep10-21swapper/110:13:171
15281212914,26sleep00-21swapper/011:48:190
30118212737,76sleep130204-21latency10:53:131
22706212418,89sleep00-21swapper/010:28:220
219321237,15sleep10-21swapper/107:42:531
929921229,18sleep10-21swapper/109:47:291
3181721199,90sleep10-21swapper/107:33:411
490921185,10sleep10-21swapper/111:14:291
24542211824,71sleep324598-21tune2fs08:53:123
3165211530,14sleep10-21swapper/109:27:551
5030211420,77sleep35086-21tune2fs09:33:103
5030211420,77sleep35086-21tune2fs09:33:103
23025211431,50sleep023029-21cron07:07:520
1438021149,39sleep30-21swapper/310:03:093
1438021149,39sleep30-21swapper/310:03:093
11164211311,85sleep10-21swapper/109:53:061
11164211311,85sleep10-21swapper/109:53:061
21402211210,18sleep10-21swapper/112:08:141
15930211212,83sleep215990-21fschecks_time10:08:092
9049211113,25sleep10-21swapper/111:28:181
9049211113,25sleep10-21swapper/111:28:181
343521115,12sleep10-21swapper/107:44:311
18776211111,21sleep318834-21users11:58:233
1744421085,92sleep00-21swapper/011:55:380
13268210825,64sleep313290-21ls08:17:523
27445210720,60sleep027651-21latency_hist09:02:540
24982210713,78sleep025138-21cat10:37:540
1735621076,10sleep10-21swapper/111:54:301
29993210673,18sleep130079-21ls09:08:211
2870210613,26sleep22892-21ntp_states07:43:192
10473210616,72sleep310440-21apt-get08:08:273
5015210511,19sleep10-21swapper/109:33:081
5015210511,19sleep10-21swapper/109:33:081
1534121057,76sleep10-21swapper/111:48:201
27949210412,25sleep128098-21grep12:28:211
29823210311,18sleep20-21swapper/210:53:022
25484210313,73sleep225537-21latency_hist12:22:532
25484210313,73sleep225537-21latency_hist12:22:532
2964221025,58sleep10-21swapper/107:27:151
13751210215,68sleep10-21swapper/111:43:201
13560210219,16sleep10-21swapper/108:18:021
7232210110,74sleep07298-21if_eth011:23:120
7232210110,74sleep07298-21if_eth011:23:120
17462210120,13sleep10-21swapper/108:28:241
17462210120,13sleep10-21swapper/108:28:241
9994210020,58sleep210123-21latency_hist08:07:532
27238210025,16sleep10-21swapper/110:43:201
91272990,11sleep30-21swapper/311:28:193
91272990,11sleep30-21swapper/311:28:193
3145329912,67sleep031655-21latency_hist07:32:550
2560529713,41sleep025607-21sshd08:56:280
217242975,11sleep10-21swapper/110:27:051
3175929613,65sleep231253-21apt-get07:33:052
221929611,69sleep02316-21fschecks_time11:08:100
117082968,16sleep10-21swapper/111:38:031
2375329514,64sleep223791-21ntpq07:08:182
114602955,11sleep10-21swapper/108:09:381
2612029423,46sleep226159-21fschecks_count08:58:122
25182947,20sleep123139-21cyclictest11:08:151
38929321,51sleep0469-21tune2fs09:18:130
249562939,72sleep125035-21awk08:53:231
243242915,74sleep10-21swapper/107:09:101
713529016,56sleep37226-21fschecks_time11:23:113
713529016,56sleep37226-21fschecks_time11:23:113
1584328912,20sleep10-21swapper/110:08:041
269742887,15sleep00-21swapper/012:27:430
195602880,6sleep20-21swapper/208:37:522
142942887,13sleep10-21swapper/110:03:031
142942887,13sleep10-21swapper/110:03:031
75932878,67sleep17661-21systemctl07:58:201
75932878,67sleep17661-21systemctl07:58:201
65928717,48sleep0746-21fschecks_time11:03:100
2838528712,61sleep328389-21entropy10:48:113
2244228732,33sleep222590-21grep07:03:222
296102866,68sleep129611-21munin-run10:52:531
2212828624,42sleep30-21swapper/307:03:153
162002867,11sleep00-21swapper/008:25:240
39082857,23sleep30-21swapper/311:13:063
241732838,60sleep10-21swapper/112:18:031
241732838,60sleep10-21swapper/112:18:031
2260828325,32sleep323032-21kworker/u8:108:47:523
2907528212,17sleep00-21swapper/010:48:240
2735328217,46sleep327443-21apt-config09:02:533
1346228218,44sleep213612-21grep09:58:242
1346228218,44sleep213612-21grep09:58:242
367128114,47sleep33377-21/usr/sbin/munin09:28:133
367128114,47sleep33377-21/usr/sbin/munin09:28:133
3101928114,51sleep231026-21/usr/sbin/munin09:13:102
2348428111,57sleep323554-21switchtime08:48:243
90272808,13sleep00-21swapper/008:03:150
46842800,9sleep20-21swapper/209:32:512
46842800,9sleep20-21swapper/209:32:512
271682808,61sleep10-21swapper/109:01:101
261372807,11sleep10-21swapper/110:38:401
465827911,21sleep10-21swapper/109:32:511
465827911,21sleep10-21swapper/109:32:511
223327914,48sleep22386-21cut07:42:542
107622797,61sleep00-21swapper/009:51:230
3131527812,52sleep331448-21latency_hist07:32:543
1948727826,34sleep019512-21awk08:37:510
1453927711,51sleep214566-21irqstats10:03:122
1453927711,51sleep214566-21irqstats10:03:122
1260227725,14sleep10-21swapper/108:13:221
1022027724,32sleep00-21swapper/011:33:090
1022027724,32sleep00-21swapper/011:33:090
320292765,10sleep10-21swapper/109:15:351
241502769,48sleep30-21swapper/312:18:013
241502769,48sleep30-21swapper/312:18:013
842527521,26sleep00-21swapper/008:02:550
842527521,26sleep00-21swapper/008:02:550
555527513,28sleep20-21swapper/209:33:212
555527513,28sleep20-21swapper/209:33:212
15292755,11sleep10-21swapper/111:03:551
561527416,35sleep05661-21fschecks_count07:53:100
455727411,34sleep00-21swapper/007:48:210
2936427427,19sleep20-21swapper/209:08:012
2706627418,39sleep327221-21latency_hist12:27:543
242627412,24sleep00-21swapper/009:23:220
1146427317,43sleep311498-21latency_hist11:37:533
1146427317,43sleep311498-21latency_hist11:37:533
981027221,23sleep30-21swapper/308:06:293
401627213,40sleep24107-21fschecks_time11:13:172
290127212,44sleep02933-21/usr/sbin/munin07:43:210
1396127215,33sleep20-21swapper/208:18:152
617127114,39sleep26302-21/usr/sbin/munin11:18:222
617127114,39sleep26302-21/usr/sbin/munin11:18:222
3039227115,25sleep00-21swapper/010:53:210
1733827112,43sleep017461-21cat08:28:230
1733827112,43sleep017461-21cat08:28:230
1096127115,34sleep210984-21/usr/sbin/munin08:08:542
447027023,21sleep30-21swapper/307:48:203
270427025,25sleep20-21swapper/211:08:202
2099227013,44sleep021049-21processes10:23:210
783226917,24sleep20-21swapper/209:42:542
297662698,52sleep00-21swapper/012:33:530
2746026919,25sleep00-21swapper/007:18:220
2725526914,37sleep226860-21/usr/sbin/munin07:18:152
2598326922,31sleep326130-21unixbench_multi07:13:243
200552696,12sleep00-21swapper/010:20:300
200552696,12sleep00-21swapper/010:20:300
325812685,12sleep00-21swapper/011:00:470
2858126820,25sleep20-21swapper/212:32:542
2287526814,21sleep20-21swapper/208:48:032
1797326814,25sleep20-21swapper/210:13:222
1797326814,25sleep20-21swapper/210:13:222
1637826820,27sleep00-21swapper/010:08:190
1139326819,26sleep20-21swapper/211:37:512
1139326819,26sleep20-21swapper/211:37:512
2969126715,37sleep229706-21idleruntime-cro07:27:522
2865726716,23sleep20-21swapper/209:04:062
279826714,31sleep30-21swapper/311:08:213
2355926720,23sleep00-21swapper/008:48:250
2001926713,25sleep20-21swapper/212:03:192
1457026721,26sleep30-21swapper/311:47:543
340226618,26sleep00-21swapper/009:28:060
340226618,26sleep00-21swapper/009:28:060
3169626615,20sleep30-21swapper/309:13:233
2783226614,37sleep327819-21apt-get09:03:103
1560726614,35sleep215666-21sendmail_mailqu08:23:202
145862669,45sleep00-21swapper/008:19:300
1383126614,34sleep213927-21switchtime11:43:212
123892667,31sleep20-21swapper/211:38:202
989826514,22sleep00-21swapper/011:32:530
989826514,22sleep00-21swapper/011:32:530
834026514,37sleep38395-21latency09:43:153
2303326522,26sleep1291-21systemd-logind07:07:531
1897926513,25sleep30-21swapper/310:18:073
1897926513,25sleep30-21swapper/310:18:073
84226416,24sleep30-21swapper/311:03:133
720526415,23sleep00-21swapper/009:38:230
65126415,30sleep30-21swapper/307:37:553
2923626421,23sleep00-21swapper/009:07:550
2777426422,20sleep30-21swapper/312:28:193
1405826420,20sleep30-21swapper/310:02:533
1405826420,20sleep30-21swapper/310:02:533
407426319,25sleep20-21swapper/207:48:112
2418426310,37sleep324267-21switchtime10:33:213
2884126213,34sleep028889-21ntp_states07:23:180
2655526218,23sleep00-21swapper/010:42:550
243912625,12sleep00-21swapper/007:10:020
2234626217,21sleep00-21swapper/012:12:540
2111326214,26sleep20-21swapper/208:42:542
210552625,18sleep30-21swapper/308:42:473
163302626,13sleep30-21swapper/311:53:023
579726122,21sleep30-21swapper/307:53:133
3273226116,27sleep232246-21apt-get09:18:032
27822615,11sleep20-21swapper/209:23:362
2565726118,20sleep20-21swapper/210:38:172
2246126114,20sleep20-21swapper/210:28:162
1191726115,19sleep00-21swapper/011:38:100
971226018,19sleep20-21swapper/209:48:102
59682605,44sleep00-21swapper/009:34:480
59682605,44sleep00-21swapper/009:34:480
3141826010,36sleep031487-21systemctl09:13:200
31126026,18sleep30-21swapper/311:02:523
2806226019,18sleep20-21swapper/210:47:542
772725913,30sleep37857-21wc07:58:223
772725913,30sleep37857-21wc07:58:223
649825912,32sleep36500-21acpi09:38:023
2556425918,25sleep20-21swapper/207:13:152
2182325910,33sleep321832-21munin-run10:27:523
1639325911,22sleep30-21swapper/308:27:533
1268025913,28sleep20-21swapper/208:13:232
1097525913,25sleep20-21swapper/209:52:552
1097525913,25sleep20-21swapper/209:52:552
82672586,15sleep20-21swapper/211:27:262
82672586,15sleep20-21swapper/211:27:262
2918425820,19sleep20-21swapper/212:33:142
2683725818,21sleep30-21swapper/310:43:103
2613625814,20sleep00-21swapper/007:13:250
2571525814,18sleep20-21swapper/208:57:522
156525814,27sleep31585-21munin-run09:22:523
98925710,18sleep20-21swapper/211:03:162
815825714,18sleep00-21swapper/009:43:120
2394125718,22sleep00-21swapper/012:17:540
2394125718,22sleep00-21swapper/012:17:540
2181825714,26sleep00-21swapper/008:43:200
1489325721,16sleep30-21swapper/311:48:103
1161625714,26sleep011622-21ntp_states09:53:170
1161625714,26sleep011622-21ntp_states09:53:170
278762565,40sleep30-21swapper/307:19:263
2374925616,21sleep30-21swapper/312:16:283
2374925616,21sleep30-21swapper/312:16:283
2103425610,27sleep321030-21apt-get12:08:043
2843625517,20sleep30-21swapper/307:23:093
2308925515,18sleep20-21swapper/212:13:202
2308925515,18sleep20-21swapper/212:13:202
298972547,33sleep20-21swapper/207:27:562
2959925411,28sleep029657-21/usr/sbin/munin09:08:110
2338925415,21sleep30-21swapper/310:32:533
2060925411,23sleep220637-21fschecks_count10:23:112
1978125413,18sleep00-21swapper/008:38:030
1897725412,25sleep20-21swapper/210:18:062
1897725412,25sleep20-21swapper/210:18:062
1748925413,22sleep00-21swapper/010:13:120
1748925413,22sleep00-21swapper/010:13:120
1725325416,15sleep30-21swapper/308:28:223
1725325416,15sleep30-21swapper/308:28:223
167702546,12sleep30-21swapper/310:08:343
599925311,23sleep00-21swapper/011:18:200
599925311,23sleep00-21swapper/011:18:200
547025312,25sleep35488-21apt-get11:18:023
547025312,25sleep35488-21apt-get11:18:023
2044125321,17sleep30-21swapper/312:03:563
1739225312,24sleep317384-21apt-get10:13:073
1739225312,24sleep317384-21apt-get10:13:073
3722526,36sleep00-21swapper/007:35:210
252125215,23sleep32533-21/usr/sbin/munin07:43:123
2488825214,22sleep224916-21munin-run10:37:522
2195325210,18sleep20-21swapper/208:43:222
1611825213,16sleep20-21swapper/211:52:532
1438625213,23sleep014478-21perl10:03:100
1438625213,23sleep014478-21perl10:03:100
1116125211,24sleep311157-21apt-get09:53:043
1116125211,24sleep311157-21apt-get09:53:043
906425113,19sleep20-21swapper/211:28:192
906425113,19sleep20-21swapper/211:28:192
3187225112,25sleep20-21swapper/210:58:152
283792516,34sleep00-21swapper/012:30:540
1793325114,22sleep217927-21apt-get11:58:062
97425011,22sleep31069-21perl07:38:113
278742505,32sleep00-21swapper/010:46:060
3023624910,15sleep30-21swapper/309:08:233
2961224911,19sleep20-21swapper/210:52:542
1171024911,23sleep311777-21df11:38:053
981724814,16sleep30-21swapper/309:48:123
92112486,29sleep20-21swapper/209:46:212
88624812,20sleep0882-21apt-get07:38:040
2888824813,21sleep328895-21df_inode12:33:073
1365824812,21sleep313664-21df_inode08:18:083
1239824815,14sleep00-21swapper/008:13:170
179412477,15sleep20-21swapper/208:32:532
179412477,15sleep20-21swapper/208:32:532
1296024710,21sleep013009-21irqstats09:58:140
1296024710,21sleep013009-21irqstats09:58:140
1049824711,18sleep00-21swapper/008:08:400
2361224614,17sleep30-21swapper/307:08:153
191924611,21sleep31929-21df_inode09:23:083
1959424510,15sleep30-21swapper/308:37:543
181432448,14sleep00-21swapper/011:58:120
210332418,17sleep20-21swapper/212:08:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional