You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 05:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Jun 01, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
48302215210,2126sleep119622-21kworker/u8:223:53:401
48302215210,2126sleep119622-21kworker/u8:223:53:401
4926290510,878sleep010235-1kworker/0:2H20:28:500
12032274920,712sleep212063-21/usr/sbin/munin00:18:482
20481272919,694sleep120548-21/usr/sbin/munin22:58:531
2008024797,460sleep320093-21awk21:13:593
24339241938,306sleep324354-21latency_hist23:13:253
4448239011,362sleep34316-21latency_hist23:53:253
4448239011,362sleep34316-21latency_hist23:53:253
2649023586,317sleep3363-21snmpd21:36:073
496123419,319sleep35044-21tune2fs23:53:543
496123419,319sleep35044-21tune2fs23:53:543
29374234015,313sleep329477-21fschecks_time23:28:433
726123310,11sleep07264-21awk20:34:020
20395229711,89sleep319909-21/usr/sbin/munin22:58:513
1490229722,256sleep01484-21apt-get23:43:370
13143229330,76sleep313201-21awk20:53:443
8751227410,96sleep38772-21dpkg22:23:353
23133227124,155sleep123225-21fschecks_time23:08:421
32079226820,224sleep232093-21modprobe23:38:222
13470225051,183sleep313542-21cat22:38:253
29143223584,72sleep329249-21timerandwakeup21:43:543
2697223122,196sleep32733-21ntpq22:03:493
28112222930,182sleep328136-21kernelversion19:59:263
9429222721,115sleep38582-21/usr/sbin/munin00:08:523
7079222530,71sleep30-21swapper/322:18:423
31807222226,106sleep3225-21systemd-journal23:34:563
666722179,189sleep36686-21apt-get20:33:343
27881221614,190sleep227889-21sshd21:38:572
2501722035,182sleep315998-1kworker/3:1H19:48:563
2501722035,182sleep315998-1kworker/3:1H19:48:563
16744220012,123sleep316793-21fschecks_count19:23:443
26427219412,167sleep126464-21ps19:53:511
29923219120,150sleep230012-21switchtime20:03:572
29923219120,150sleep230012-21switchtime20:03:572
28981218510,159sleep328992-21modprobe23:28:223
15861218268,26sleep30-21swapper/321:03:253
19615218013,153sleep319618-21cron22:58:233
2040217830,112sleep32067-21munin-run22:03:243
17669217664,97sleep317672-21cron00:38:233
1016321748,55sleep30-21swapper/300:13:403
983021738,17sleep00-21swapper/020:43:400
11750217367,87sleep30-21swapper/320:48:483
14856217229,126sleep00-21swapper/022:41:570
11331216213,109sleep311491-21latency_hist19:08:263
18661215678,66sleep318695-21ps19:28:533
4022154122,19sleep3444-21processes23:38:523
27485215212,81sleep227622-21latency_hist23:23:252
486821507,15sleep20-21swapper/220:28:282
13075214948,86sleep313038-21latency_hist00:23:253
6049214813,120sleep36170-21timerwakeupswit22:13:543
26231214810,106sleep326249-21grep19:53:453
13977214311,120sleep314021-21latency22:38:453
8548214128,52sleep30-21swapper/320:38:473
21992214151,26sleep322117-21cat19:38:553
21992214151,26sleep322117-21cat19:38:553
691214065,54sleep30-21swapper/320:13:553
20424213952,70sleep320578-21grep19:33:543
4903213847,70sleep311350-21cyclictest20:28:373
25199213852,15sleep30-21swapper/323:13:533
2829213534,87sleep31-21systemd20:23:243
18775213514,104sleep10-21swapper/119:28:551
9009213228,74sleep09150-21grep20:38:560
3111213048,46sleep33113-21apt-get20:23:363
3111213048,46sleep33113-21apt-get20:23:363
29033213016,90sleep10-21swapper/123:28:241
28506212824,75sleep028525-21apt-get21:43:350
22476212522,20sleep30-21swapper/323:04:573
29346212232,77sleep329358-21grep20:03:423
29346212232,77sleep329358-21grep20:03:423
15558212127,82sleep315566-21sh19:18:523
381212015,90sleep10-21swapper/123:38:501
126521200,9sleep01258-21kworker/u8:020:18:230
5169211822,53sleep35230-21latency_hist22:13:253
17382211811,81sleep30-21swapper/322:48:543
6416211711,88sleep30-21swapper/323:58:383
15230211624,22sleep30-21swapper/300:28:463
6659211211,86sleep26180-21apt-get20:33:292
30473211058,14sleep30-21swapper/321:48:473
2910021100,7sleep20-21swapper/223:28:242
2401210910,23sleep30-21swapper/323:43:573
22841210916,75sleep30-21swapper/319:43:353
5900210815,74sleep15978-21mailstats22:13:521
25672210611,15sleep00-21swapper/023:16:000
17302210618,35sleep20-21swapper/222:48:532
10795210612,27sleep30-21swapper/322:28:523
21472210510,16sleep30-21swapper/321:18:563
1509121058,45sleep30-21swapper/320:58:533
26649210312,74sleep126707-21/usr/sbin/munin23:18:521
31995210212,29sleep332019-21netstat21:53:473
18177210221,22sleep30-21swapper/321:08:543
18177210221,22sleep30-21swapper/321:08:543
9343210012,70sleep19406-21/usr/sbin/munin22:24:051
1644021009,72sleep30-21swapper/321:03:503
15679210021,62sleep215764-21switchtime19:18:552
33952999,18sleep00-21swapper/020:23:500
33952999,18sleep00-21swapper/020:23:500
3137529912,21sleep30-21swapper/320:08:553
458829715,65sleep24725-21unixbench_singl22:08:562
234122976,30sleep30-21swapper/321:26:223
82512966,46sleep30-21swapper/300:07:303
82512966,46sleep30-21swapper/300:07:303
1084629623,15sleep30-21swapper/320:45:213
296892956,20sleep10-21swapper/123:28:491
1393329520,14sleep30-21swapper/319:13:513
138942944,79sleep00-21swapper/020:54:340
1229429418,61sleep112365-21sensors_temp00:18:551
1918129314,27sleep119173-21apt-get21:13:411
133329319,53sleep21454-21latency_hist20:18:272
116562936,12sleep20-21swapper/220:48:472
1379929114,57sleep30-21swapper/300:23:533
80212905,73sleep00-21swapper/000:04:330
80212905,73sleep00-21swapper/000:04:330
492729014,39sleep14899-21apt-get20:28:511
2581929030,29sleep10-21swapper/119:53:261
2549228918,51sleep025556-21sort21:33:420
940728718,50sleep29503-21timerandwakeup22:24:062
2774728717,52sleep127880-21uptime21:38:561
1425328716,55sleep114388-21unixbench_singl19:13:581
1360128712,37sleep00-21swapper/019:13:440
124072876,11sleep00-21swapper/020:50:220
1225228715,51sleep212317-21grep22:33:472
1006628725,48sleep010096-21fschecks_count19:03:430
1007228618,46sleep01-21systemd22:28:240
1448928514,50sleep114608-21/usr/sbin/munin22:38:551
127272854,46sleep30-21swapper/319:12:063
127272854,46sleep30-21swapper/319:12:063
52528413,49sleep1677-21latency_hist21:58:261
1160028324,41sleep211613-21apt-get22:33:232
2394728218,39sleep124028-21fschecks_time21:28:421
2002028214,50sleep220113-21grep22:58:422
1771928111,55sleep117798-21fschecks_time21:08:441
1771928111,55sleep117798-21fschecks_time21:08:441
546628026,22sleep20-21swapper/223:54:052
546628026,22sleep20-21swapper/223:54:052
1018928019,45sleep210248-21awk19:03:452
269822797,34sleep30-21swapper/321:38:383
201827919,39sleep22038-21modprobe22:03:232
362027715,44sleep03703-21wc22:08:260
2161827714,45sleep023758-21sshd21:18:570
1526027714,49sleep015299-21grep00:28:470
2593427621,24sleep20-21swapper/223:18:242
2394427616,29sleep00-21swapper/021:28:410
887927517,41sleep18924-21cat00:08:411
310542755,59sleep00-21swapper/021:49:450
2742327526,23sleep00-21swapper/019:58:260
1582427518,15sleep20-21swapper/221:03:242
1582427518,15sleep20-21swapper/221:03:242
1361527515,36sleep113769-21unixbench_multi20:53:551
1288027519,26sleep20-21swapper/219:13:262
1288027519,26sleep20-21swapper/219:13:262
2322727416,31sleep20-21swapper/223:08:432
2288427417,30sleep00-21swapper/021:23:530
1191127412,32sleep00-21swapper/000:18:460
608727320,34sleep06111-21munin-run23:58:230
608727320,34sleep06111-21munin-run23:58:230
2905027323,27sleep00-21swapper/020:03:230
273642735,12sleep30-21swapper/323:22:293
228227316,23sleep20-21swapper/222:03:352
225527319,28sleep00-21swapper/020:18:570
2108027326,27sleep20-21swapper/219:38:252
1656927314,28sleep20-21swapper/222:48:252
1562627312,47sleep115670-21ntpq22:43:481
1328827314,38sleep013290-21/usr/sbin/munin00:23:340
2439427210,48sleep30-21swapper/321:28:533
2148427227,26sleep00-21swapper/019:38:440
2148427227,26sleep00-21swapper/019:38:440
1641327215,41sleep016489-21latency_hist19:23:250
597927121,33sleep26048-21/usr/sbin/munin22:13:532
33227116,23sleep00-21swapper/023:38:490
57742707,16sleep00-21swapper/022:13:470
385827018,31sleep13852-21apt-get22:08:391
2436027018,26sleep00-21swapper/023:13:260
155262709,27sleep30-21swapper/322:43:453
140692700,8sleep20-21swapper/200:23:552
193962699,50sleep00-21swapper/019:32:380
1799526919,20sleep00-21swapper/019:28:260
2212326816,35sleep122270-21modprobe21:23:261
2129726812,40sleep121368-21/usr/sbin/munin21:18:541
1543726818,27sleep10-21swapper/100:28:521
702726721,26sleep20-21swapper/223:58:582
61726717,31sleep2760-21grep23:38:552
3044526720,32sleep230472-21awk21:48:462
103502675,14sleep10-21swapper/119:03:491
330226610,12sleep00-21swapper/023:48:460
330226610,12sleep00-21swapper/023:48:460
2910126622,18sleep20-21swapper/220:03:252
2412326615,32sleep224149-21munin-run19:48:232
241062667,13sleep00-21swapper/019:48:090
2280026621,22sleep00-21swapper/023:08:250
1368226617,25sleep10-21swapper/100:23:471
860726527,19sleep20-21swapper/200:08:342
1763426524,22sleep20-21swapper/221:08:412
1763426524,22sleep20-21swapper/221:08:412
1223926526,19sleep10-21swapper/119:08:531
1223926526,19sleep10-21swapper/119:08:531
3219926415,25sleep20-21swapper/220:13:262
2073526419,30sleep220753-21apt21:18:342
1927826416,25sleep20-21swapper/221:13:462
1902926417,20sleep20-21swapper/222:53:542
1339126421,28sleep113432-21sendmail22:38:221
681626315,28sleep00-21swapper/023:58:510
3139026321,20sleep10-21swapper/121:53:251
3116626321,25sleep10-21swapper/120:08:491
3013426316,28sleep030248-21/usr/sbin/munin20:03:590
3013426316,28sleep030248-21/usr/sbin/munin20:03:590
2574926321,20sleep00-21swapper/019:53:250
2574926321,20sleep00-21swapper/019:53:250
225182636,48sleep10-21swapper/123:05:271
1834526315,28sleep20-21swapper/219:28:442
1668126315,20sleep20-21swapper/200:33:452
3108926218,28sleep10-21swapper/123:33:451
2775326225,20sleep10-21swapper/119:58:351
2646726212,35sleep226542-21mailstats19:53:522
2588226216,30sleep125939-21/usr/sbin/munin21:33:511
127082627,12sleep00-21swapper/022:33:570
102942628,37sleep010374-21/usr/sbin/munin00:13:450
825326114,30sleep08283-21munin-run22:23:240
68502615,11sleep10-21swapper/123:58:531
340626116,29sleep20-21swapper/223:48:482
340626116,29sleep20-21swapper/223:48:482
2824126124,23sleep028250-21modprobe21:43:230
164142618,15sleep00-21swapper/000:33:390
1596426115,17sleep00-21swapper/022:43:550
837426017,19sleep10-21swapper/100:08:251
837426017,19sleep10-21swapper/100:08:251
387326012,29sleep13094-21/usr/sbin/munin20:23:591
387326012,29sleep13094-21/usr/sbin/munin20:23:591
2442226020,21sleep10-21swapper/119:48:421
2247826014,16sleep20-21swapper/221:23:432
202712600,8sleep10-21swapper/119:33:511
1729026015,25sleep217291-21sshd21:07:562
1722626013,19sleep20-21swapper/219:23:562
156026012,30sleep20-21swapper/223:43:392
1400026018,21sleep20-21swapper/219:13:542
1340126016,19sleep20-21swapper/220:53:522
1168126016,20sleep00-21swapper/022:33:250
1067426017,26sleep110765-21mailstats00:13:541
2942025914,19sleep10-21swapper/120:03:441
2942025914,19sleep10-21swapper/120:03:441
208025918,17sleep20-21swapper/220:18:552
85025818,16sleep10-21swapper/121:58:461
297822585,13sleep20-21swapper/221:48:172
2614725816,25sleep026166-21apt-get23:18:340
1763125816,25sleep017624-21apt-get21:08:390
1763125816,25sleep017624-21apt-get21:08:390
438025716,18sleep20-21swapper/220:28:242
438025716,18sleep20-21swapper/220:28:242
312425710,23sleep10-21swapper/123:48:431
312425710,23sleep10-21swapper/123:48:431
250692575,12sleep10-21swapper/123:13:511
1885725719,20sleep00-21swapper/022:53:520
1617325716,20sleep20-21swapper/200:33:252
1113725714,19sleep10-21swapper/120:48:261
982625614,26sleep29820-21apt-get20:43:382
407825614,24sleep04127-21irqstats22:08:460
320182565,15sleep00-21swapper/020:11:420
3162825615,24sleep231622-21apt-get21:53:382
1615525618,22sleep10-21swapper/121:03:441
1531125612,26sleep115409-21users20:58:551
1531125612,26sleep115409-21users20:58:551
1194525616,26sleep112018-21sed20:48:541
864825511,29sleep10-21swapper/120:38:491
78722556,15sleep10-21swapper/120:37:151
51325511,29sleep1584-21sensors20:13:531
3132525515,24sleep031403-21mailstats23:33:520
2964325516,20sleep00-21swapper/023:28:480
2769825512,18sleep00-21swapper/023:23:350
2294225514,24sleep122949-21df_inode19:43:451
1144025514,26sleep211465-21/usr/sbin/munin00:18:232
1090725515,19sleep10-21swapper/122:28:541
726225416,23sleep20-21swapper/222:18:452
276225413,25sleep12768-21ntp_states22:03:511
168102547,11sleep10-21swapper/122:48:401
1663125415,23sleep116680-21irqstats00:33:441
75425315,22sleep0748-21apt-get21:58:410
2000825314,23sleep020014-21df_inode22:58:400
1672725313,23sleep116736-21diskstats19:23:421
1156425314,23sleep011558-21apt-get19:08:350
1156425314,23sleep011558-21apt-get19:08:350
2927125212,23sleep229265-21apt-get23:28:382
274825214,16sleep10-21swapper/123:48:231
1726325214,23sleep017269-21ntp_states22:48:510
1448225219,17sleep00-21swapper/020:58:340
1448225219,17sleep00-21swapper/020:58:340
835325111,17sleep20-21swapper/220:38:422
699125112,22sleep16986-21apt-get22:18:361
50025111,29sleep0524-21sed21:58:250
2547625012,22sleep225485-21df_inode21:33:402
1973525010,16sleep00-21swapper/019:33:350
1969625013,15sleep10-21swapper/122:58:251
1732625013,21sleep017369-21munin-run21:08:230
227282495,31sleep20-21swapper/223:08:072
2139924912,22sleep221403-21diskstats19:38:422
2139924912,22sleep221403-21diskstats19:38:422
2131524910,23sleep121310-21apt-get19:38:371
2131524910,23sleep121310-21apt-get19:38:371
1918024914,21sleep019173-21apt-get21:13:400
1391924911,22sleep213976-21irqstats22:38:442
2492424811,20sleep225016-21mailstats19:48:552
2492424811,20sleep225016-21mailstats19:48:552
2385624811,22sleep223853-21apt-get21:28:352
911924713,16sleep00-21swapper/000:08:490
2891024713,14sleep10-21swapper/121:43:481
151752419,17sleep115181-21df_inode19:18:421
20412408,15sleep10-21swapper/120:18:541
82892329,12sleep20-21swapper/200:07:582
82892329,12sleep20-21swapper/200:07:582
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional