You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 23:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Fri Aug 07, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1442226066,588sleep014461-21ntpq08:42:410
11289235118,317sleep211299-21/usr/sbin/munin08:32:422
11289235118,317sleep211299-21/usr/sbin/munin08:32:422
9013231333,241sleep09021-21awk10:07:460
2702230467,195sleep00-21swapper/008:07:200
184742295141,56sleep00-21swapper/012:22:360
194522283187,43sleep00-21swapper/010:42:180
952622767,230sleep09613-21basename11:52:480
22593225997,24sleep00-21swapper/009:07:490
2161222375,141sleep322107-1kworker/3:1H09:07:193
144862217104,60sleep014489-21cron10:27:160
21915220710,111sleep022043-21timerwakeupswit10:47:470
15005219010,161sleep015150-21latency_hist12:12:180
15005219010,161sleep015150-21latency_hist12:12:180
26283218310,17sleep00-21swapper/009:22:180
20430218212,151sleep020527-21timerandwakeup07:17:530
11364218011,134sleep211402-21idleruntime-cro10:17:162
24469217915,136sleep00-21swapper/010:57:360
24469217915,136sleep00-21swapper/010:57:360
16043217611,152sleep016046-21cron10:32:170
12992217511,150sleep013103-21latency_hist10:22:190
6313216915,97sleep00-21swapper/008:17:420
24735216326,124sleep024786-21cat07:32:430
24735216326,124sleep024786-21cat07:32:430
5821216215,128sleep15865-21ntpq09:57:451
1662121616,107sleep00-21swapper/008:49:070
16563215615,126sleep316628-21mailstats07:07:463
14246215315,126sleep314319-21expr12:07:453
14246215315,126sleep314319-21expr12:07:453
276421518,108sleep02893-21unixbench_multi09:47:510
2623321429,70sleep00-21swapper/007:37:400
1891221420,63sleep218916-21fschecks_time08:57:362
4496214039,63sleep04562-21swap09:53:480
9910213913,48sleep00-21swapper/008:28:360
9910213913,48sleep00-21swapper/008:28:360
16190213918,100sleep316389-21latency_hist10:32:203
18153213711,89sleep018181-21sed07:12:280
18153213711,89sleep018181-21sed07:12:280
2456213630,93sleep00-21swapper/011:31:400
12881213510,81sleep00-21swapper/008:37:410
6128213310,17sleep00-21swapper/009:57:520
572221339,87sleep01-21systemd08:17:170
572221339,87sleep01-21systemd08:17:170
1967221305,30sleep00-21swapper/010:42:260
6458212712,30sleep36494-21ntp_states08:17:453
557121268,70sleep0364-21snmpd11:41:330
28159212521,69sleep20-21swapper/209:27:332
30363212421,56sleep00-21swapper/011:17:180
21532212211,99sleep00-21swapper/009:07:040
19403212138,38sleep219427-21modprobe10:42:162
9872211912,89sleep110015-21cut10:12:191
1525121199,49sleep00-21swapper/012:12:350
30683211811,14sleep00-21swapper/007:52:340
30683211811,14sleep00-21swapper/007:52:340
11477211814,48sleep00-21swapper/008:32:480
11477211814,48sleep00-21swapper/008:32:480
2853821176,13sleep00-21swapper/009:27:440
4578211620,76sleep34626-21grep08:12:403
4578211620,76sleep34626-21grep08:12:403
26799211610,58sleep00-21swapper/009:22:390
26781211520,75sleep126914-21unixbench_singl07:37:531
3038921149,46sleep00-21swapper/007:52:170
2468421148,52sleep01-21systemd09:17:180
16711211418,75sleep116842-21grep07:07:481
27552211311,17sleep00-21swapper/011:07:340
27552211311,17sleep00-21swapper/011:07:340
17832211323,68sleep317862-21modprobe10:37:163
1343211321,72sleep11436-21uniq11:27:371
12306211311,85sleep012371-21ls12:02:370
12306211311,85sleep012371-21ls12:02:370
29833211212,82sleep021555-21kworker/u8:009:32:280
26531211019,75sleep126575-21/usr/sbin/munin09:22:321
16917211024,41sleep00-21swapper/012:17:350
15330211011,21sleep215334-21diskstats12:12:372
23462210933,61sleep00-21swapper/007:27:510
23462210933,61sleep00-21swapper/007:27:510
6087210826,63sleep30-21swapper/311:42:373
27954210714,71sleep327560-21/usr/sbin/munin07:42:443
1715721068,20sleep00-21swapper/008:52:310
11759210615,33sleep311766-21diskstats10:17:343
17158210510,15sleep00-21swapper/010:32:530
3215421047,15sleep00-21swapper/011:22:310
2136821048,60sleep021416-21awk07:22:330
19571210422,61sleep219589-21latency_hist07:17:182
19571210422,61sleep219589-21latency_hist07:17:182
26891210315,60sleep00-21swapper/011:03:300
26891210315,60sleep00-21swapper/011:03:300
1630121039,21sleep016389-21latency07:07:390
3225621027,38sleep00-21swapper/007:57:300
3225621027,38sleep00-21swapper/007:57:300
2147621027,44sleep00-21swapper/012:32:290
2147621027,44sleep00-21swapper/012:32:290
1934121029,37sleep00-21swapper/008:57:470
1058721028,14sleep00-21swapper/011:57:280
1058721028,14sleep00-21swapper/011:57:280
16518210021,55sleep116542-21latency_hist12:17:161
979429835,38sleep39796-21sshd10:12:093
27942989,40sleep00-21swapper/011:32:290
2413129715,69sleep324209-21cat10:57:193
75932968,34sleep00-21swapper/008:22:340
308552969,35sleep00-21swapper/011:17:390
227992969,71sleep00-21swapper/010:52:310
910629512,71sleep29119-21fschecks_count11:52:382
2494429517,65sleep225032-21sed10:57:462
2494429517,65sleep225032-21sed10:57:462
1570529526,45sleep315773-21systemd-run07:07:183
72772947,36sleep00-21swapper/010:02:410
72772947,36sleep00-21swapper/010:02:410
63372947,34sleep00-21swapper/011:42:430
1571229416,64sleep315735-21awk12:12:463
39542936,19sleep20-21swapper/209:53:342
29902926,32sleep00-21swapper/008:07:360
1155329218,58sleep111678-21grep08:32:491
1155329218,58sleep111678-21grep08:32:491
112502925,11sleep00-21swapper/010:15:490
3201729022,47sleep131956-21latency_hist07:57:191
3201729022,47sleep131956-21latency_hist07:57:191
2156129015,56sleep121453-21/usr/sbin/munin12:32:341
2156129015,56sleep121453-21/usr/sbin/munin12:32:341
1337229016,56sleep213439-21/usr/sbin/munin10:22:382
37528920,45sleep1505-21grep11:22:491
2464228911,56sleep30-21swapper/310:57:393
2464228911,56sleep30-21swapper/310:57:393
18372896,32sleep00-21swapper/009:47:180
3048728816,60sleep230571-21expr09:33:032
2164128814,59sleep221724-21cat07:22:442
1242628814,52sleep112570-21unixbench_singl10:17:481
3274928617,25sleep10-21swapper/109:42:181
2631428623,44sleep126455-21latency_hist09:22:191
2239628612,58sleep222463-21awk09:07:472
1568728616,32sleep20-21swapper/208:47:362
567728516,56sleep15800-21latency_hist11:42:191
2387928522,30sleep20-21swapper/209:12:432
1413928519,51sleep314204-21fschecks_time08:42:363
420628413,53sleep24215-21/usr/sbin/munin08:12:192
292532849,56sleep10-21swapper/107:47:381
1969328419,48sleep219785-21latency_hist12:27:192
157752849,57sleep00-21swapper/007:07:190
87642829,56sleep20-21swapper/210:07:392
1385428218,20sleep30-21swapper/308:42:203
872328120,40sleep18765-21true11:52:161
831028116,52sleep28377-21latency_hist10:07:192
831028116,52sleep28377-21latency_hist10:07:192
1454128128,28sleep30-21swapper/310:27:193
20582807,10sleep20-21swapper/209:47:282
168452808,52sleep20-21swapper/207:07:492
451127912,52sleep20-21swapper/208:12:352
451127912,52sleep20-21swapper/208:12:352
297952797,17sleep10-21swapper/109:32:211
285192795,28sleep00-21swapper/007:43:090
2580027926,31sleep125849-21wc07:37:201
2580027926,31sleep125849-21wc07:37:201
2571927922,27sleep20-21swapper/207:37:172
2571927922,27sleep20-21swapper/207:37:172
2020627911,13sleep30-21swapper/312:27:403
2003227931,32sleep00-21swapper/012:27:370
3237827831,37sleep00-21swapper/009:38:100
2466527819,40sleep324679-21modprobe09:17:173
222182788,11sleep20-21swapper/212:32:482
222182788,11sleep20-21swapper/212:32:482
1035227812,50sleep310488-21latency_hist11:57:193
835927712,49sleep28374-21/usr/sbin/munin11:47:492
953127519,39sleep39547-21sort08:28:273
953127519,39sleep39547-21sort08:28:273
935127519,41sleep29364-21acpi08:27:292
935127519,41sleep29364-21acpi08:27:292
859827531,11sleep00-21swapper/011:50:380
61827419,27sleep30-21swapper/309:42:373
43627417,34sleep2558-21timerwakeupswit07:57:572
43627417,34sleep2558-21timerwakeupswit07:57:572
3190727424,26sleep30-21swapper/311:22:183
2170827414,41sleep121740-21/usr/sbin/munin10:47:441
184727423,21sleep10-21swapper/109:47:191
310512737,48sleep30-21swapper/307:52:453
310512737,48sleep30-21swapper/307:52:453
2623927319,35sleep326281-21true09:22:173
174327317,42sleep11787-21ntp_kernel_pll_08:02:491
1432227319,37sleep214387-21latency08:42:392
1257727318,40sleep112586-21/usr/sbin/munin08:37:331
670327214,42sleep26812-21timerwakeupswit08:17:522
616227220,35sleep16213-21fschecks_time08:17:391
484727217,38sleep34906-21/usr/sbin/munin11:37:453
337127221,28sleep20-21swapper/211:32:472
2818527221,31sleep30-21swapper/309:27:343
1784427221,29sleep317302-21apt-get07:12:233
1784427221,29sleep317302-21apt-get07:12:233
1654227218,36sleep216543-21sshd10:32:272
710127120,21sleep10-21swapper/110:02:381
710127120,21sleep10-21swapper/110:02:381
2404927114,36sleep124088-21modprobe10:57:171
238562719,12sleep10-21swapper/109:12:421
2015427118,29sleep10-21swapper/110:42:441
1882127122,29sleep10-21swapper/108:57:361
1314527117,29sleep20-21swapper/208:37:472
434227010,44sleep24363-21apt-get11:37:272
2615727014,26sleep30-21swapper/307:37:393
2288927016,37sleep322907-21apt07:27:293
2288927016,37sleep322907-21apt07:27:293
2202427014,16sleep10-21swapper/109:07:381
1693527022,24sleep20-21swapper/212:17:372
1692427014,24sleep10-21swapper/112:17:361
2209526920,29sleep322236-21unixbench_singl07:22:533
2010126913,37sleep320146-21/usr/sbin/munin10:42:413
1607626919,20sleep30-21swapper/308:47:453
2182126821,29sleep321912-21sed10:47:463
2015426812,20sleep10-21swapper/107:17:461
1491026812,28sleep10-21swapper/110:27:371
2043826718,21sleep10-21swapper/112:27:461
1949426712,22sleep10-21swapper/110:42:191
1349026717,24sleep10-21swapper/112:07:191
1185426722,24sleep30-21swapper/312:02:163
1185426722,24sleep30-21swapper/312:02:163
80726616,33sleep2838-21/usr/sbin/munin09:42:402
456326624,20sleep10-21swapper/109:53:491
1331726619,25sleep10-21swapper/110:22:371
351426523,22sleep10-21swapper/109:52:201
3096026519,27sleep231050-21/usr/sbin/munin07:52:442
3096026519,27sleep231050-21/usr/sbin/munin07:52:442
3028226517,23sleep20-21swapper/211:17:162
2797826512,37sleep328061-21tr11:07:453
2797826512,37sleep328061-21tr11:07:453
2094726514,29sleep20-21swapper/209:02:462
182426523,23sleep20-21swapper/208:02:582
1610526517,32sleep216189-21apt-config10:32:192
148452655,52sleep20-21swapper/212:10:512
148452655,52sleep20-21swapper/212:10:512
1027226511,39sleep110307-21munin-run11:57:171
586726416,20sleep20-21swapper/209:57:462
2128226419,22sleep20-21swapper/212:32:192
908126317,27sleep20-21swapper/208:27:212
39432637,14sleep10-21swapper/108:09:271
3253026319,27sleep232568-21awk11:22:422
2942026318,25sleep30-21swapper/311:12:413
2632726314,24sleep20-21swapper/211:02:432
2632726314,24sleep20-21swapper/211:02:432
208726313,30sleep32082-21apt-get09:47:333
179426316,29sleep31861-21awk11:27:473
627226215,21sleep20-21swapper/211:42:402
351926222,18sleep30-21swapper/308:07:513
3273126219,20sleep30-21swapper/307:57:543
3273126219,20sleep30-21swapper/307:57:543
324822626,11sleep30-21swapper/309:39:303
2902326217,25sleep10-21swapper/111:12:311
2506926215,20sleep20-21swapper/209:17:362
1842526219,27sleep318517-21load10:37:413
1583526217,20sleep10-21swapper/107:07:201
822326113,36sleep30-21swapper/310:06:523
822326113,36sleep30-21swapper/310:06:523
780726122,20sleep30-21swapper/308:22:453
2904926123,22sleep10-21swapper/107:47:221
2717926117,27sleep327208-21munin-run11:07:163
2717926117,27sleep327208-21munin-run11:07:163
258926117,19sleep20-21swapper/211:32:192
2416926123,21sleep20-21swapper/207:32:172
2416926123,21sleep20-21swapper/207:32:172
1855126120,21sleep10-21swapper/112:22:371
1688726113,31sleep316915-21/usr/sbin/munin12:17:343
717326014,29sleep17178-21modprobe11:47:161
520926020,24sleep30-21swapper/309:57:193
2691926016,19sleep20-21swapper/207:37:542
1691826017,29sleep116890-21latency_hist08:52:191
1378126020,21sleep10-21swapper/112:07:351
1378126020,21sleep10-21swapper/112:07:351
1276926015,26sleep20-21swapper/212:02:472
1276926015,26sleep20-21swapper/212:02:472
315992596,12sleep20-21swapper/211:18:542
1725325918,20sleep10-21swapper/108:52:361
550025817,20sleep10-21swapper/108:14:281
550025817,20sleep10-21swapper/108:14:281
2913125814,26sleep229125-21apt-get07:47:312
2318825820,17sleep10-21swapper/107:27:431
2318825820,17sleep10-21swapper/107:27:431
2023925814,28sleep320276-21ntp_states07:17:483
1818225811,19sleep10-21swapper/107:12:291
1818225811,19sleep10-21swapper/107:12:291
1261225817,21sleep30-21swapper/312:02:443
1261225817,21sleep30-21swapper/312:02:443
880325716,25sleep38956-21latency_hist11:52:183
85872577,10sleep10-21swapper/108:23:571
291012577,41sleep30-21swapper/307:47:243
2853125711,32sleep228532-21sshd07:43:192
2567625717,18sleep10-21swapper/111:02:181
2567625717,18sleep10-21swapper/111:02:181
112782579,10sleep30-21swapper/310:16:133
1122825719,16sleep20-21swapper/211:57:472
1122825719,16sleep20-21swapper/211:57:472
3276525613,24sleep20-21swapper/209:42:192
2986325613,25sleep329860-21apt-get09:32:313
2233225611,31sleep322385-21users12:32:493
2233225611,31sleep322385-21users12:32:493
1442325617,23sleep214441-21apt-get07:02:292
1067025614,23sleep10-21swapper/111:57:331
1067025614,23sleep10-21swapper/111:57:331
572125516,25sleep35789-21latency09:57:433
2933525513,27sleep129363-21munin-run09:32:171
1970325513,23sleep219772-21df10:42:322
1841725518,21sleep318441-21/usr/sbin/munin08:57:183
119325513,24sleep10-21swapper/108:02:201
119325513,24sleep10-21swapper/108:02:201
2128225411,22sleep10-21swapper/107:22:221
1838425411,26sleep318379-21apt-get12:22:313
1229925422,16sleep30-21swapper/308:37:193
1229925422,16sleep30-21swapper/308:37:193
2328125314,26sleep21-21systemd10:52:462
2185025313,24sleep321845-21apt-get09:07:323
1715325313,24sleep217149-21apt-get08:52:282
1269825317,21sleep10-21swapper/112:02:461
208502525,36sleep20-21swapper/207:19:242
100625215,16sleep30-21swapper/311:27:193
934825112,23sleep18859-21apt-get08:27:281
934825112,23sleep18859-21apt-get08:27:281
34422518,17sleep30-21swapper/311:32:483
2814925115,20sleep10-21swapper/111:07:471
2814925115,20sleep10-21swapper/111:07:471
2652825114,19sleep30-21swapper/309:22:303
1654125112,22sleep116061-21apt-get10:32:261
2652525012,22sleep226522-21apt-get09:22:292
239425017,18sleep30-21swapper/308:04:283
2166125015,14sleep20-21swapper/210:47:412
2071725014,21sleep120740-21ntp_states09:02:411
189532509,17sleep30-21swapper/308:57:383
129412479,17sleep20-21swapper/210:22:172
329224610,14sleep10-21swapper/111:32:461
2747624617,15sleep20-21swapper/211:07:312
2747624617,15sleep20-21swapper/211:07:312
1020024611,15sleep10-21swapper/110:12:391
267372457,14sleep10-21swapper/111:02:501
267372457,14sleep10-21swapper/111:02:501
144632459,20sleep114505-21ntp_states08:42:421
246792449,18sleep324733-21irqstats07:32:423
246792449,18sleep324733-21irqstats07:32:423
306072428,16sleep10-21swapper/111:17:321
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional