You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-18 - 01:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Wed Jun 17, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
256242368316,3655sleep224943-21kworker/2:112:07:402
46662273010,2707sleep13149-21kworker/1:107:36:011
126712168210,1660sleep132052-21kworker/1:011:25:401
9046299615,964sleep19075-21users11:12:561
1661924449,418sleep216645-21ntpq11:37:482
2289523299,306sleep122942-21ntpq08:32:491
161052299103,179sleep116109-21diskstats08:12:431
161052299103,179sleep116109-21diskstats08:12:431
9757226919,232sleep09488-21latency_hist11:17:270
6355224819,207sleep3307-21in:imuxsock07:42:253
18008224667,79sleep30-21swapper/311:42:463
21271222613,198sleep11974-21kworker/u8:011:52:491
8144222344,51sleep30-21swapper/311:12:343
25509221917,120sleep327553-21cyclictest10:22:563
25509221917,120sleep327553-21cyclictest10:22:563
25002219121,54sleep30-21swapper/310:54:053
4192220745,110sleep34319-21timerwakeupswit10:57:573
23062220548,74sleep30-21swapper/311:57:563
29512220114,28sleep30-21swapper/307:12:333
12442219415,72sleep30-21swapper/309:42:433
1678021908,170sleep316861-21cat11:37:543
11302219066,109sleep311369-21mailstats09:37:513
11302219066,109sleep311369-21mailstats09:37:513
3250221750,67sleep327553-21cyclictest10:47:263
28457217559,98sleep328542-21latency_hist12:17:263
1533217264,81sleep30-21swapper/309:07:433
1533217264,81sleep30-21swapper/309:07:433
20607216650,89sleep30-21swapper/310:07:503
17978216376,73sleep318043-21grep08:17:503
16488216261,81sleep30-21swapper/308:12:533
16488216261,81sleep30-21swapper/308:12:533
165821607,85sleep30-21swapper/312:32:563
165821607,85sleep30-21swapper/312:32:563
11809215814,129sleep011821-21kernelversion11:22:450
1874621576,131sleep00-21swapper/008:18:030
17534215619,116sleep00-21swapper/011:42:270
5701215524,111sleep15783-21cat07:37:541
27166215016,120sleep227168-21cpu08:47:402
27166215016,120sleep227168-21cpu08:47:402
32388214647,79sleep30-21swapper/312:27:523
4954214510,61sleep30-21swapper/309:17:493
28930214512,114sleep028448-21apt-get08:52:340
28020214419,103sleep30-21swapper/310:32:453
29301214135,89sleep30-21swapper/312:17:543
10570214055,18sleep30-21swapper/309:37:273
3075213925,92sleep03090-21cut09:12:410
26306213111,17sleep00-21swapper/007:02:530
30185212916,83sleep230323-21grep10:37:532
23364212915,88sleep30-21swapper/310:17:433
24476212869,15sleep30-21swapper/312:02:533
19935212543,18sleep320058-21timerandwakeup11:47:553
24152212336,43sleep30-21swapper/308:37:443
29875212132,76sleep3363-21snmpd08:54:313
31055212015,19sleep00-21swapper/010:42:390
20957212045,14sleep30-21swapper/311:52:433
10010211911,95sleep310032-21tr07:52:463
25034211715,81sleep025084-21/usr/sbin/munin10:22:450
25034211715,81sleep025084-21/usr/sbin/munin10:22:450
16704211724,66sleep216853-21unixbench_multi08:12:562
16704211724,66sleep216853-21unixbench_multi08:12:562
11320211713,89sleep011369-21awk07:57:400
11000211716,80sleep111040-21awk07:57:251
2724221168,86sleep00-21swapper/008:47:420
2724221168,86sleep00-21swapper/008:47:420
1488121167,39sleep314883-21ntp_states08:07:523
12592211618,79sleep112634-21true08:02:251
713221148,70sleep37220-21grep11:07:543
31693211216,48sleep30-21swapper/307:17:493
24459211220,74sleep123979-21/usr/sbin/munin12:02:511
1405921117,95sleep00-21swapper/008:06:240
528721107,37sleep30-21swapper/311:02:453
2778321107,25sleep30-21swapper/307:07:403
2778321107,25sleep30-21swapper/307:07:403
26766211017,52sleep30-21swapper/310:27:493
2386321070,6sleep20-21swapper/212:02:262
1412210724,61sleep01441-21grep10:52:240
1412210724,61sleep01441-21grep10:52:240
21441210613,30sleep221450-21munin-run10:12:262
1516721069,83sleep315188-21awk09:52:243
10590210627,17sleep30-21swapper/311:18:073
5163210522,61sleep15218-21fschecks_count11:02:431
30836210512,75sleep30-21swapper/310:42:263
30417210514,75sleep130570-21awk07:12:551
2295221059,73sleep30-21swapper/308:32:513
13817210314,67sleep213959-21sed11:27:552
6619210224,51sleep26638-21apt-get07:42:362
32306210218,22sleep00-21swapper/009:02:360
32306210218,22sleep00-21swapper/009:02:360
30912210217,64sleep230941-21/usr/sbin/munin08:57:492
27529210214,73sleep227634-21latency_hist07:07:272
20704210146,19sleep320706-21munin-run08:27:263
11174210115,69sleep211203-21/usr/sbin/munin09:37:462
11174210115,69sleep211203-21/usr/sbin/munin09:37:462
29205210027,50sleep229210-21grep08:54:182
172621009,19sleep30-21swapper/307:27:263
40532995,12sleep30-21swapper/309:14:463
35129917,63sleep132304-21/usr/sbin/munin09:03:011
35129917,63sleep132304-21/usr/sbin/munin09:03:011
3263229811,70sleep032737-21latency_hist07:22:270
204122987,13sleep10-21swapper/110:07:441
94432965,80sleep30-21swapper/307:52:183
2851929623,50sleep128671-21grep07:07:581
2851929623,50sleep128671-21grep07:07:581
1554429617,52sleep115608-21sort09:52:431
109829613,66sleep11262-21unixbench_multi07:22:561
351329527,21sleep33509-21apt-get07:32:393
3230329532,16sleep30-21swapper/309:02:353
3230329532,16sleep30-21swapper/309:02:353
434529416,61sleep14478-21latency_hist09:17:261
1586929439,21sleep3363-21snmpd11:36:573
983129317,49sleep09871-21sort07:52:430
243029312,64sleep12454-21/usr/sbin/munin07:27:531
81612918,62sleep00-21swapper/009:27:550
1582629114,61sleep215863-21ntp_kernel_err09:52:482
97229016,58sleep2982-21forks12:32:432
97229016,58sleep2982-21forks12:32:432
2874929017,59sleep2225-21systemd-journal07:08:402
2874929017,59sleep2225-21systemd-journal07:08:402
2692429013,62sleep126960-21latency_hist08:47:261
2215028922,22sleep321664-21/usr/sbin/munin10:12:513
153452897,69sleep00-21swapper/008:08:010
996028814,58sleep29462-21apt-get11:17:302
2966128716,56sleep129755-21switchtime08:54:291
76562869,60sleep30-21swapper/309:27:383
193028614,54sleep21425-21apt-get10:52:352
119462869,64sleep311521-21/usr/sbin/munin11:22:483
633428517,47sleep26338-21modprobe11:07:242
589328513,57sleep05982-21ls09:22:260
2675228526,25sleep10-21swapper/107:03:021
215228411,54sleep12215-21fschecks_time10:53:551
1451428416,55sleep114613-21sed09:47:541
1444228419,28sleep00-21swapper/009:47:530
309422838,13sleep30-21swapper/308:57:503
2686328217,44sleep026871-21latency_hist12:12:260
2133428232,33sleep021358-21proc_pri11:52:520
1519628214,51sleep115272-21ls11:32:551
73602815,65sleep30-21swapper/309:27:013
165992805,64sleep30-21swapper/309:55:553
125192805,63sleep30-21swapper/308:01:303
951727920,35sleep00-21swapper/007:52:270
83627921,26sleep10-21swapper/110:47:571
83627921,26sleep10-21swapper/110:47:571
40927914,17sleep00-21swapper/007:22:390
2181427912,48sleep021900-21fw_packets10:12:440
951127819,38sleep29573-21munin-run11:17:252
365627811,53sleep00-21swapper/010:57:450
22252787,33sleep30-21swapper/307:27:473
130942786,11sleep30-21swapper/311:27:363
584227722,34sleep25866-21modprobe09:22:242
2837727718,37sleep128414-21run-parts12:17:241
1370727723,35sleep113815-21grep11:27:541
133682779,56sleep20-21swapper/208:03:032
3110527612,44sleep131127-21df10:42:401
2491127622,31sleep20-21swapper/210:22:432
2491127622,31sleep20-21swapper/210:22:432
119452766,12sleep00-21swapper/009:40:510
119452766,12sleep00-21swapper/009:40:510
311982755,59sleep30-21swapper/312:23:323
2991127511,50sleep229914-21sshd12:21:532
2536027516,26sleep10-21swapper/110:22:541
2536027516,26sleep10-21swapper/110:22:541
2059827517,44sleep220599-21sshd08:26:412
78212747,55sleep3363-21snmpd07:46:203
578627423,25sleep20-21swapper/207:37:552
319927415,42sleep01-21systemd10:57:250
133372748,51sleep30-21swapper/308:03:003
660827329,24sleep10-21swapper/109:22:511
2664927317,39sleep126742-21/usr/sbin/munin10:27:471
256892736,32sleep30-21swapper/312:07:453
1903127320,39sleep11-21systemd11:47:241
1578427321,34sleep115878-21latency_hist08:12:271
1366927317,31sleep213797-21ls09:47:272
99027215,38sleep21056-21users10:47:582
99027215,38sleep21056-21users10:47:582
3155527218,28sleep20-21swapper/210:42:542
258332726,29sleep30-21swapper/308:42:483
2404227220,25sleep00-21swapper/008:37:360
88427113,41sleep2385-21/usr/sbin/munin07:22:532
80182719,17sleep10-21swapper/107:47:271
3225727115,25sleep10-21swapper/112:27:501
306722716,12sleep10-21swapper/112:22:491
1685127114,26sleep20-21swapper/209:57:282
71962705,11sleep00-21swapper/009:24:560
216827011,12sleep20-21swapper/207:27:462
205827016,24sleep20-21swapper/209:07:552
205827016,24sleep20-21swapper/209:07:552
1837927017,24sleep10-21swapper/110:02:271
170312708,19sleep30-21swapper/309:57:373
120527019,20sleep20-21swapper/209:07:262
120527019,20sleep20-21swapper/209:07:262
77662698,19sleep00-21swapper/011:10:540
708426923,27sleep10-21swapper/111:07:521
2840026914,38sleep028517-21cat07:07:570
2840026914,38sleep028517-21cat07:07:570
271752697,45sleep30-21swapper/308:47:413
271752697,45sleep30-21swapper/308:47:413
1986626913,22sleep20-21swapper/211:47:542
190722697,14sleep30-21swapper/310:02:543
306492686,12sleep00-21swapper/007:12:580
234932688,14sleep10-21swapper/110:17:451
1783526816,35sleep217848-21/usr/sbin/munin11:42:412
1527426821,27sleep20-21swapper/211:32:562
325782678,44sleep30-21swapper/310:47:353
325782678,44sleep30-21swapper/310:47:353
221372675,29sleep30-21swapper/308:31:343
221372675,29sleep30-21swapper/308:31:343
196152670,7sleep00-21swapper/008:22:460
93112669,41sleep09317-21diskstats09:32:430
795726613,39sleep08017-21latency_hist07:47:260
2382626514,28sleep223881-21latency_hist08:37:262
931826420,23sleep10-21swapper/109:32:441
3155426414,35sleep231692-21cat12:27:262
3001626414,15sleep00-21swapper/010:37:510
2376726413,22sleep10-21swapper/112:02:261
236762647,50sleep00-21swapper/008:36:270
1668526417,21sleep10-21swapper/111:37:521
138526412,23sleep00-21swapper/012:32:530
138526412,23sleep00-21swapper/012:32:530
3141126321,22sleep20-21swapper/207:17:442
1014026321,22sleep20-21swapper/207:52:492
865826213,18sleep20-21swapper/207:47:532
3180726219,26sleep131853-21apt-config09:02:241
2637526219,22sleep20-21swapper/210:27:402
2538026210,38sleep225486-21latency_hist08:42:272
1077926214,30sleep10-21swapper/109:37:331
1077926214,30sleep10-21swapper/109:37:331
1000226217,25sleep10-21swapper/111:17:411
41372617,43sleep20-21swapper/209:15:502
2687326118,23sleep10-21swapper/112:12:271
2400526121,21sleep20-21swapper/212:02:392
216452616,12sleep10-21swapper/108:27:571
216452616,12sleep10-21swapper/108:27:571
2144526121,23sleep20-21swapper/211:52:542
202892616,11sleep00-21swapper/011:48:400
1841326123,24sleep10-21swapper/108:18:001
1436926115,21sleep20-21swapper/209:47:512
1286226117,19sleep20-21swapper/211:27:272
969626022,20sleep20-21swapper/209:32:542
826826015,28sleep08356-21/usr/sbin/munin11:12:430
2856426018,19sleep10-21swapper/108:52:271
2856426018,19sleep10-21swapper/108:52:271
261526013,31sleep02739-21timerwakeupswit07:27:560
181626015,24sleep00-21swapper/009:07:490
181626015,24sleep00-21swapper/009:07:490
1572726018,21sleep00-21swapper/009:52:460
1309826016,18sleep10-21swapper/108:02:541
3140625914,21sleep10-21swapper/107:17:431
1910725919,24sleep219146-21modprobe08:22:252
483825812,29sleep14979-21latency_hist11:02:271
2793125815,25sleep227925-21apt-get10:32:402
2396325810,33sleep024124-21awk10:17:570
2131725814,24sleep20-21swapper/208:27:502
2131725814,24sleep20-21swapper/208:27:502
1286625813,28sleep112957-21mailstats09:42:541
124502585,40sleep20-21swapper/208:00:412
795625711,23sleep20-21swapper/209:27:492
2834925711,20sleep10-21swapper/110:32:511
258922577,12sleep00-21swapper/008:42:490
2574425717,25sleep225832-21fw_packets08:42:472
1904825715,26sleep119054-21ntp_states10:02:521
1316925712,30sleep013261-21/usr/sbin/munin09:42:570
88425614,26sleep1879-21apt-get12:32:381
88425614,26sleep1879-21apt-get12:32:381
710125615,25sleep17107-21ntp_states07:42:511
530825614,25sleep05070-21/usr/sbin/munin07:37:450
2769325613,25sleep00-21swapper/010:32:260
2707325616,17sleep00-21swapper/012:12:340
2303425615,22sleep20-21swapper/210:17:262
1985225616,23sleep119858-21ntp_states08:22:521
1859425612,24sleep218663-21df10:02:412
3203525515,20sleep00-21swapper/009:02:270
2244425515,24sleep022440-21apt-get11:57:370
2094625514,25sleep020940-21apt-get08:27:400
2094625514,25sleep020940-21apt-get08:27:400
507425417,19sleep00-21swapper/011:02:380
35192548,19sleep20-21swapper/207:32:422
2444225418,18sleep10-21swapper/108:37:501
1736025419,18sleep00-21swapper/008:17:260
1736025419,18sleep00-21swapper/008:17:260
148562545,11sleep00-21swapper/011:32:460
3072725317,16sleep00-21swapper/012:22:520
3057125313,24sleep230637-21users07:12:562
35902529,17sleep00-21swapper/007:32:450
2950325212,21sleep129498-21apt-get10:37:351
2691725214,15sleep30-21swapper/307:03:043
2530625213,26sleep025315-21pluginstate12:07:260
2335925214,22sleep223363-21diskstats10:17:422
775925114,22sleep17767-21df_inode09:27:441
64625119,16sleep20-21swapper/212:32:272
636425115,17sleep20-21swapper/209:22:442
396425110,16sleep20-21swapper/210:57:522
2883525119,17sleep10-21swapper/112:17:441
485425013,25sleep10-21swapper/109:17:471
2878325017,18sleep00-21swapper/012:17:430
279282509,15sleep00-21swapper/010:32:380
2759425016,18sleep20-21swapper/212:12:532
66172499,16sleep20-21swapper/211:07:362
2176024911,16sleep10-21swapper/110:12:411
16582499,14sleep10-21swapper/109:07:451
16582499,14sleep10-21swapper/109:07:451
2013824812,17sleep00-21swapper/010:07:350
1950524814,18sleep10-21swapper/111:47:451
1902424811,22sleep019047-21ntp_states10:02:510
1835524815,17sleep00-21swapper/010:02:260
1442624811,22sleep214422-21apt-get08:07:392
3187524713,19sleep031913-21fschecks_count12:27:410
1777024712,17sleep00-21swapper/011:42:390
1351624710,21sleep013553-21ntp_states11:27:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional