You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-05 - 11:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Jul 05, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31202170757,1599sleep13175-21sendmail-msp23:59:191
31202170757,1599sleep13175-21sendmail-msp23:59:191
263052103419,1000sleep15780-1kworker/1:2H23:29:311
24600271316,678sleep024650-21/usr/sbin/munin19:59:400
6855251715,426sleep16867-21modprobe22:29:181
1882250111,473sleep01926-21ntpq20:29:490
31786242324,379sleep331804-21/usr/sbin/munin22:04:293
800524168,393sleep18053-21awk20:49:571
27947240511,375sleep128022-21fschecks_time23:34:401
32532235120,220sleep112921-21kworker/u8:323:49:211
8765228615,108sleep10-21swapper/100:14:501
14340228419,248sleep314403-21/usr/sbin/munin00:34:343
14340228419,248sleep314403-21/usr/sbin/munin00:34:343
29069226517,184sleep129206-21latency_hist20:14:231
25731224453,177sleep125775-21ntpq21:44:451
10135224067,123sleep10-21swapper/119:14:341
10135224067,123sleep10-21swapper/119:14:341
12887223138,173sleep10-21swapper/121:04:311
12887223138,173sleep10-21swapper/121:04:311
13612223016,152sleep10-21swapper/122:49:331
1465222289,204sleep114796-21cmp19:29:211
14877222613,197sleep214920-21unin-run22:54:192
32413222116,117sleep10-21swapper/120:24:331
2570922207,198sleep125852-21date20:04:221
8324221516,184sleep38380-21interrupts00:14:393
3229622159,171sleep132334-21processes22:04:561
29964221516,171sleep00-21swapper/023:39:470
12321220316,166sleep312407-21timerandwakeup19:19:493
12321220316,166sleep312407-21timerandwakeup19:19:493
17351219924,113sleep10-21swapper/121:19:211
12428219621,154sleep312532-21latency_hist21:04:213
12428219621,154sleep312532-21latency_hist21:04:213
10278219330,102sleep110383-21date22:39:201
10278219330,102sleep110383-21date22:39:201
30034219210,161sleep10-21swapper/123:39:481
7944218810,163sleep37464-21apt-get20:49:303
2268921859,161sleep10-21swapper/121:34:471
214582185113,53sleep10-21swapper/119:49:401
25918218417,154sleep00-21swapper/020:04:290
5119217418,133sleep00-21swapper/020:39:460
6412216710,114sleep10-21swapper/120:44:401
6910216318,63sleep37051-21unixbench_singl20:44:503
922821626,13sleep10-21swapper/119:11:321
4311216111,138sleep14382-21true20:39:191
6872216014,22sleep20-21swapper/222:29:192
31291216010,92sleep10-21swapper/122:04:191
2333821609,137sleep123383-21awk23:19:421
1004821576,10sleep00-21swapper/022:37:210
1004821576,10sleep00-21swapper/022:37:210
13503215317,122sleep013642-21unixbench_singl00:29:510
20957214716,111sleep00-21swapper/019:49:020
26621214690,43sleep126742-21cat20:04:501
2927621398,94sleep15780-1kworker/1:2H20:14:311
51992136102,19sleep10-21swapper/120:39:471
2540213611,20sleep10-21swapper/122:14:371
2607021350,9sleep00-21swapper/023:29:210
1124221358,18sleep10-21swapper/100:24:311
18590213492,23sleep10-21swapper/121:21:021
31836213020,35sleep10-21swapper/120:20:371
1072221305,13sleep10-21swapper/120:58:331
2922121299,36sleep10-21swapper/121:54:501
1804221276,12sleep10-21swapper/123:04:191
998221268,47sleep30-21swapper/319:14:283
998221268,47sleep30-21swapper/319:14:283
24665212516,17sleep20-21swapper/219:59:422
16600212526,59sleep10-21swapper/119:34:341
31185212411,99sleep330706-21apt-get23:44:273
8940212316,90sleep38988-21awk22:34:333
8940212316,90sleep38988-21awk22:34:333
1912921235,36sleep10-21swapper/119:40:271
14416212210,55sleep17794-21cyclictest00:34:361
14416212210,55sleep17794-21cyclictest00:34:361
11882212280,20sleep10-21swapper/119:19:391
11882212280,20sleep10-21swapper/119:19:391
7211211824,22sleep10-21swapper/100:09:491
5307211826,36sleep15310-21cron22:24:181
3144121170,72sleep331573-21cat22:04:213
22733211715,83sleep022789-21latency_hist23:19:200
8383211616,84sleep08399-21kernelversion00:14:400
2566321165,33sleep10-21swapper/123:27:491
1410221159,91sleep114167-21date00:34:201
794721149,85sleep10-21swapper/122:29:511
15161211353,16sleep10-21swapper/122:54:341
4770211249,51sleep14771-21munin-run00:04:191
2494421126,70sleep10-21swapper/121:42:471
8124211117,76sleep38269-21grep22:29:533
9497211019,72sleep09578-21latency_hist19:14:210
1214821096,16sleep10-21swapper/122:44:371
4048210829,48sleep04042-21apt-get22:19:320
21992210819,72sleep322095-21latency_hist21:34:213
30401210716,74sleep230449-21ntpq21:59:442
631821060,6sleep20-21swapper/219:04:212
4383210614,76sleep24421-21latency_hist20:39:202
123721065,89sleep10-21swapper/123:50:021
10732210530,64sleep1364-21snmpd00:21:281
9499210439,34sleep30-21swapper/320:54:313
7471210414,75sleep27507-21/usr/sbin/munin22:29:402
2103821035,85sleep10-21swapper/123:13:321
2103821035,85sleep10-21swapper/123:13:321
2007021035,64sleep10-21swapper/121:25:101
1564721035,85sleep10-21swapper/121:13:271
582921027,13sleep10-21swapper/100:04:521
1829521027,33sleep10-21swapper/123:04:361
1829521027,33sleep10-21swapper/123:04:361
8001210132,42sleep27963-21apt-get20:49:542
2689321017,37sleep10-21swapper/121:49:291
2240621017,41sleep1364-21snmpd23:16:141
2395521005,54sleep10-21swapper/119:57:471
19738210021,63sleep019819-21timerandwakeup21:24:490
2105129919,61sleep121087-21ntpq21:29:431
183429912,71sleep31836-21/usr/sbin/munin23:54:283
183429912,71sleep31836-21/usr/sbin/munin23:54:283
150512997,35sleep10-21swapper/119:29:331
2450229717,60sleep30-21swapper/321:39:543
200892978,56sleep120125-21ntpq19:44:431
1021829715,68sleep010224-21cat19:14:380
1021829715,68sleep010224-21cat19:14:380
771229616,64sleep07829-21apt-config19:09:200
3212529624,54sleep332153-21basename20:24:193
1133729618,58sleep011427-21fschecks_time00:24:380
238082948,22sleep00-21swapper/021:39:350
1981029413,54sleep019829-21apt-get23:09:270
1981029413,54sleep019829-21apt-get23:09:270
99329325,31sleep10-21swapper/122:09:361
3134029330,50sleep030843-21/usr/sbin/munin20:19:520
1372429313,60sleep313747-21/usr/sbin/munin19:24:463
3068929128,40sleep230710-21munin-run23:44:182
2980629119,57sleep329897-21cut20:15:023
246129114,58sleep00-21swapper/022:14:310
1107229110,36sleep10-21swapper/120:59:301
1107229110,36sleep10-21swapper/120:59:301
151129031,46sleep11501-21apt-get20:29:381
92752899,63sleep00-21swapper/020:54:200
87382899,67sleep38743-21ntp_states19:11:013
29242896,13sleep30-21swapper/322:14:473
217428914,60sleep32178-21kworker/u8:422:14:183
38022880,6sleep00-21swapper/020:34:490
254322874,10sleep30-21swapper/323:24:533
2136328620,40sleep021456-21fschecks_time19:49:390
1659928624,37sleep016593-21apt-get19:34:320
569228411,56sleep05828-21unixbench_multi00:04:510
787428314,46sleep07912-21latency_hist00:14:200
444328326,42sleep24486-21ntpq22:19:422
88828214,53sleep2904-21awk22:09:292
1759028231,26sleep30-21swapper/321:19:353
2677128110,53sleep30-21swapper/323:29:493
2349728121,30sleep30-21swapper/319:54:513
1492128119,46sleep314985-21cut22:54:203
1479728124,27sleep20-21swapper/219:29:222
2978628022,27sleep30-21swapper/321:59:203
1289828014,49sleep212883-21apt-get21:04:362
1289828014,49sleep212883-21apt-get21:04:362
568227915,46sleep05776-21perl22:24:370
1164627921,27sleep10-21swapper/122:42:391
1164627921,27sleep10-21swapper/122:42:391
1096927916,48sleep210975-21ntp_states22:39:452
1096927916,48sleep210975-21ntp_states22:39:452
2641127812,51sleep226445-21ps20:04:472
2175527821,25sleep30-21swapper/323:14:433
318827725,32sleep23204-21munin-run23:59:202
318827725,32sleep23204-21munin-run23:59:202
2903827717,34sleep028768-21apt-get20:14:220
2687427621,27sleep00-21swapper/021:49:280
1235027616,47sleep312372-21cut22:44:553
735427519,37sleep06871-21apt-get22:29:300
2781027516,16sleep30-21swapper/321:49:523
1948227514,48sleep319584-21latency_hist19:44:213
1844527515,43sleep018534-21load19:39:410
1636027520,26sleep00-21swapper/019:34:200
1253627518,40sleep012380-21latency_hist21:04:220
1253627518,40sleep012380-21latency_hist21:04:220
1078127523,33sleep210831-21modprobe20:59:192
320462746,13sleep20-21swapper/222:04:482
2328427413,48sleep023339-21processes19:54:480
232762746,10sleep00-21swapper/021:36:360
229227415,14sleep20-21swapper/222:14:212
1276427424,31sleep012901-21grep22:45:040
2865527328,26sleep20-21swapper/221:54:392
1728627318,39sleep317348-21/usr/sbin/munin22:59:493
1512127321,29sleep00-21swapper/021:09:510
1609127218,34sleep00-21swapper/021:14:390
1146927219,22sleep20-21swapper/219:19:202
1146927219,22sleep20-21swapper/219:19:202
2426227116,28sleep20-21swapper/221:39:492
2990127023,28sleep00-21swapper/020:15:030
2725527022,27sleep30-21swapper/320:09:203
196642709,15sleep30-21swapper/321:24:483
2761626913,37sleep227702-21fw_packets20:09:402
162026919,26sleep30-21swapper/323:54:203
121726917,30sleep21242-21modprobe20:29:182
79726821,25sleep20-21swapper/223:49:472
1950226820,24sleep219533-21nfs_client21:24:432
1879726812,41sleep218877-21tr23:04:492
1879726812,41sleep218877-21tr23:04:492
1855826824,21sleep30-21swapper/323:04:423
1855826824,21sleep30-21swapper/323:04:423
759926720,22sleep20-21swapper/220:49:212
2799326714,36sleep328069-21/usr/sbin/munin20:09:493
977926617,30sleep39672-21/usr/sbin/munin00:19:363
3217726621,27sleep232297-21latency_hist20:24:222
2535326619,29sleep225347-21apt-get21:44:342
2032026623,22sleep20-21swapper/223:09:452
2032026623,22sleep20-21swapper/223:09:452
1805626618,24sleep00-21swapper/021:19:480
1651326621,21sleep30-21swapper/322:59:213
3248326513,38sleep332489-21df_inode20:24:353
2257926513,26sleep20-21swapper/219:54:212
1566726520,20sleep00-21swapper/000:39:200
1566726520,20sleep00-21swapper/000:39:200
1066326516,17sleep00-21swapper/022:39:390
1066326516,17sleep00-21swapper/022:39:390
957526419,27sleep00-21swapper/020:54:360
67362645,15sleep30-21swapper/322:27:473
367526417,20sleep30-21swapper/320:34:493
310812645,11sleep00-21swapper/022:01:390
2919426417,26sleep20-21swapper/223:39:212
263702648,43sleep00-21swapper/021:46:130
216142646,15sleep00-21swapper/021:30:060
2039226416,17sleep30-21swapper/323:09:463
2039226416,17sleep30-21swapper/323:09:463
1648826419,21sleep20-21swapper/222:59:202
148692647,12sleep00-21swapper/022:54:170
1306326419,21sleep30-21swapper/300:29:423
261426322,20sleep00-21swapper/023:54:490
261426322,20sleep00-21swapper/023:54:490
2103026312,38sleep221098-21latency_hist19:49:212
2017226313,28sleep319695-21/usr/sbin/munin19:44:453
1498926311,13sleep00-21swapper/022:54:210
141432639,44sleep30-21swapper/321:09:093
141432639,44sleep30-21swapper/321:09:093
1288426315,28sleep00-21swapper/021:04:300
1288426315,28sleep00-21swapper/021:04:300
107262636,18sleep00-21swapper/000:21:230
762626218,26sleep37710-21apt-config19:09:193
593926220,27sleep25914-21latency_hist20:44:212
55626210,37sleep0630-21find20:24:480
2475926221,22sleep30-21swapper/319:59:433
212162626,12sleep20-21swapper/221:29:482
2154026120,23sleep30-21swapper/319:49:423
626826016,30sleep36289-21/usr/sbin/munin00:09:183
477226014,28sleep20-21swapper/200:04:202
46432606,15sleep00-21swapper/000:03:170
2562526014,18sleep30-21swapper/321:44:433
20822605,12sleep00-21swapper/022:13:060
1051726017,27sleep310510-21apt-get22:39:343
1051726017,27sleep310510-21apt-get22:39:343
893925918,25sleep28933-21apt-get22:34:322
893925918,25sleep28933-21apt-get22:34:322
736025922,17sleep10-21swapper/119:06:031
3182625919,20sleep00-21swapper/022:04:380
2428325916,21sleep00-21swapper/023:24:210
212525922,20sleep30-21swapper/320:29:563
1884625921,21sleep20-21swapper/221:24:182
1331525916,18sleep00-21swapper/019:24:350
92242586,42sleep20-21swapper/219:11:282
2928525813,26sleep20-21swapper/220:14:362
276982588,31sleep30-21swapper/323:34:213
2627925817,18sleep00-21swapper/023:29:300
2415525813,15sleep20-21swapper/219:59:222
1171525815,25sleep211711-21apt-get19:19:322
1171525815,25sleep211711-21apt-get19:19:322
95525714,18sleep00-21swapper/023:49:490
75602578,37sleep20-21swapper/200:11:032
1241025714,26sleep00-21swapper/019:19:500
1241025714,26sleep00-21swapper/019:19:500
412425616,24sleep34130-21diskstats22:19:353
2738425616,25sleep227466-21mailstats21:49:472
2659225615,24sleep226676-21memory23:29:442
1680925615,23sleep316866-21irqstats19:34:403
1255425621,15sleep20-21swapper/200:29:202
2342025517,23sleep323422-21ntp_states23:19:443
2989725415,17sleep30-21swapper/323:39:463
2786125413,24sleep027855-21apt-get23:34:350
2147725413,18sleep20-21swapper/223:14:382
192442545,40sleep00-21swapper/023:05:230
192442545,40sleep00-21swapper/023:05:230
80842537,34sleep20-21swapper/200:14:262
662625310,19sleep30-21swapper/300:09:343
500925313,23sleep25005-21apt-get00:04:332
3160925314,16sleep00-21swapper/023:45:130
2223025313,24sleep322225-21apt-get21:34:333
154062538,33sleep20-21swapper/221:10:212
345325216,20sleep33450-21apt-get23:59:333
1677025215,22sleep216808-21df22:59:362
1554125213,20sleep30-21swapper/319:29:513
993225112,24sleep29966-21irqstats00:19:392
743525113,21sleep07492-21apt-config20:49:190
3176125117,17sleep20-21swapper/223:45:192
167042516,36sleep00-21swapper/022:59:260
1479425113,21sleep214800-21ntp_states00:34:462
1479425113,21sleep214800-21ntp_states00:34:462
1332225113,22sleep213353-21munin-run22:49:182
1744625016,18sleep20-21swapper/219:34:512
1727025012,20sleep30-21swapper/321:19:193
1445625011,23sleep314451-21apt-get21:09:343
1236725011,17sleep20-21swapper/221:04:192
1236725011,17sleep20-21swapper/221:04:192
2502524915,15sleep20-21swapper/223:24:472
2844824811,21sleep028442-21apt-get21:54:330
2356824814,20sleep223652-21latency_hist21:39:212
1855024817,18sleep20-21swapper/219:39:432
1124424716,18sleep30-21swapper/300:24:333
460124611,17sleep30-21swapper/320:39:293
26072465,32sleep20-21swapper/220:32:172
1326424610,18sleep20-21swapper/219:24:332
2318824510,20sleep223237-21irqstats23:19:392
160022438,18sleep20-21swapper/221:14:332
1286624310,16sleep20-21swapper/200:29:372
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional