You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-05 - 11:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sat Sep 05, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1880625769,552sleep218844-21ntpq19:14:042
853925507,530sleep230052-1kworker/2:1H23:49:132
662724789,457sleep06663-21ntpq20:18:560
2594124207,402sleep126013-21unin-run19:38:331
28501238318,341sleep028518-21apt-get21:28:440
14600238015,341sleep314619-21apt-get19:03:443
17290237665,295sleep017291-21ntp_states22:34:230
17290237665,295sleep017291-21ntp_states22:34:230
10990233115,301sleep010996-21cat20:33:540
19608232817,291sleep00-21swapper/020:59:050
3271232274,230sleep13361-21sort23:33:341
6397228112,63sleep10-21swapper/123:43:351
1736226382,131sleep11885-21grep21:44:031
7337224810,219sleep07480-21/usr/sbin/munin23:44:020
7337224810,219sleep07480-21/usr/sbin/munin23:44:020
2518122449,194sleep125184-21apt-get21:18:441
16459221911,191sleep110084-1kworker/1:0H22:33:351
23499220322,130sleep123535-21ntpq19:28:561
29110220166,54sleep10-21swapper/119:48:341
29110220166,54sleep10-21swapper/119:48:341
19750219816,129sleep119833-21wc19:18:351
23150219744,93sleep10-21swapper/121:13:351
12564219631,128sleep112644-21tune2fs20:38:521
3397218672,66sleep13466-21sort20:08:561
25042217548,71sleep124536-21apt-get22:58:421
25042217548,71sleep124536-21apt-get22:58:421
29791217421,72sleep129849-21unin-run21:33:331
19670217227,127sleep319691-21cron19:18:333
2305217116,140sleep02361-21sed21:48:340
1066621689,16sleep10-21swapper/120:33:351
881216411,142sleep1945-21ps23:24:011
24653216310,134sleep00-21swapper/019:33:440
11982215813,129sleep311988-21df_inode22:18:523
4850215626,92sleep15006-21cat23:38:341
756215117,53sleep10-21swapper/121:43:341
24858214267,56sleep125066-21latency_hist21:18:351
16072214114,54sleep10-21swapper/120:49:051
20913213924,55sleep10-21swapper/121:04:101
19669213910,117sleep119729-21/usr/sbin/munin00:23:511
31579213819,105sleep331599-21ls21:38:323
31579213819,105sleep331599-21ls21:38:323
28001213761,31sleep128026-21munin-run21:28:321
21736213371,16sleep10-21swapper/122:49:061
20477213054,58sleep320510-21/usr/sbin/munin21:04:003
32613212613,75sleep132643-21latency_hist23:23:331
32613212613,75sleep132643-21latency_hist23:23:331
21693212610,103sleep121763-21cat00:29:001
21693212610,103sleep121763-21cat00:29:001
7685212510,97sleep00-21swapper/022:04:000
27093212310,77sleep127134-21grep23:03:591
27093212310,77sleep127134-21grep23:03:591
17170211921,76sleep317320-21unixbench_singl00:14:053
15019211938,66sleep115032-21fschecks_count00:08:541
15019211938,66sleep115032-21fschecks_count00:08:541
14081211910,54sleep10-21swapper/122:24:031
12035211864,16sleep10-21swapper/123:59:011
2846321170,7sleep00-21swapper/023:08:540
16762211648,15sleep116281-21apt-get22:33:391
16762211648,15sleep116281-21apt-get22:33:391
3214211417,77sleep20-21swapper/220:08:532
11668211329,70sleep111765-21latency_hist22:18:341
6344211116,79sleep26366-21idleruntime-cro23:43:322
934421108,88sleep19444-21cat22:09:011
934421108,88sleep19444-21cat22:09:011
6900210919,74sleep36936-21cat23:43:533
6900210919,74sleep36936-21cat23:43:533
19633210923,68sleep019645-21modprobe22:43:310
19633210923,68sleep019645-21modprobe22:43:310
19731210858,35sleep10-21swapper/121:00:271
6386210623,15sleep10-21swapper/121:59:041
23529210633,57sleep10-21swapper/122:53:541
20696210610,85sleep00-21swapper/000:26:410
19647210651,41sleep10-21swapper/122:43:321
19647210651,41sleep10-21swapper/122:43:321
15893210626,63sleep015913-21cron19:08:330
8414210523,61sleep10-21swapper/123:48:441
31538210510,52sleep130871-21/usr/sbin/munin19:54:031
604921040,5sleep30-21swapper/320:18:343
349621049,78sleep33540-21ntpq20:08:583
15276210319,66sleep314982-21/usr/sbin/munin22:28:573
28443210218,64sleep328155-21/usr/sbin/munin23:08:543
2566210228,55sleep10-21swapper/120:05:051
366229914,20sleep00-21swapper/020:09:040
406729843,35sleep10-21swapper/121:53:431
317532980,11sleep20-21swapper/223:18:572
317532980,11sleep20-21swapper/223:18:572
217092979,16sleep20-21swapper/222:48:462
1995229619,59sleep219979-21/usr/sbin/munin00:23:572
1477029636,43sleep214802-21tune2fs19:04:312
144872966,75sleep10-21swapper/120:44:001
238392959,61sleep00-21swapper/019:29:030
3134329429,16sleep1374-21sshd21:35:311
3134329429,16sleep1374-21sshd21:35:311
3107629415,58sleep331176-21latency_hist23:18:343
3107629415,58sleep331176-21latency_hist23:18:343
2854029417,43sleep20-21swapper/221:28:542
1516929421,19sleep00-21swapper/000:08:570
1516929421,19sleep00-21swapper/000:08:570
2934829321,16sleep10-21swapper/119:48:461
2934829321,16sleep10-21swapper/119:48:461
3238129211,66sleep332382-21sshd23:21:053
3238129211,66sleep332382-21sshd23:21:053
1850029215,21sleep30-21swapper/300:19:013
1804329220,52sleep317103-21/usr/sbin/munin20:54:073
174312926,36sleep1367-21snmpd19:12:581
1682329213,18sleep00-21swapper/000:13:580
167229228,41sleep31705-21munin-plugin-st23:28:313
2848929114,51sleep228647-21sed19:44:112
2848929114,51sleep228647-21sed19:44:112
2649429114,64sleep226500-21/usr/sbin/munin21:23:332
2219929115,60sleep222320-21cat19:24:052
2219929115,60sleep222320-21cat19:24:052
1789929120,16sleep10-21swapper/120:54:051
2336529022,41sleep323520-21grep00:34:033
2336529022,41sleep323520-21grep00:34:033
1908829013,56sleep019214-21/usr/sbin/munin19:14:100
77782895,72sleep10-21swapper/120:23:421
508928921,15sleep10-21swapper/120:13:571
2902328937,36sleep10-21swapper/123:09:041
1999128927,17sleep10-21swapper/122:43:491
2475128817,34sleep20-21swapper/219:33:502
2463728826,38sleep224667-21latency_hist21:18:322
1411028818,49sleep00-21swapper/020:43:510
1296128816,54sleep213042-21grep20:39:012
1008128827,29sleep20-21swapper/222:13:342
1008128827,29sleep20-21swapper/222:13:342
291028715,50sleep10-21swapper/121:48:551
2174528715,52sleep021559-21/usr/sbin/munin19:23:550
2174528715,52sleep021559-21/usr/sbin/munin19:23:550
2110428724,41sleep10-21swapper/119:21:051
1811228731,13sleep10-21swapper/100:18:511
769828629,15sleep10-21swapper/122:04:011
417528613,53sleep00-21swapper/021:53:500
322028615,54sleep33252-21munin-run23:33:323
3099828615,54sleep131047-21sendmail23:18:321
3099828615,54sleep131047-21sendmail23:18:321
2366428611,61sleep223735-21sensors19:29:012
601628514,58sleep26047-21latency_hist20:18:342
2849728517,51sleep328016-21apt-get21:28:433
240072859,15sleep10-21swapper/100:38:331
240072859,15sleep10-21swapper/100:38:331
1199528512,15sleep10-21swapper/122:18:541
1626828415,54sleep216287-21awk22:33:322
1509228417,49sleep015185-21grep22:28:540
190702830,7sleep30-21swapper/322:39:013
190702830,7sleep30-21swapper/322:39:013
1619328315,49sleep230052-1kworker/2:1H19:08:432
10432835,32sleep10-21swapper/120:00:211
133722829,59sleep113397-21cut00:03:531
1131428215,51sleep311444-21cat23:58:353
1131428215,51sleep311444-21cat23:58:353
2989428122,43sleep029967-21grep19:49:010
2989428122,43sleep029967-21grep19:49:010
2789728121,22sleep20-21swapper/223:08:322
2789728121,22sleep20-21swapper/223:08:322
33622800,6sleep20-21swapper/223:33:342
2170128018,43sleep321196-21apt-get22:48:433
1378628012,53sleep313832-21latency_hist20:43:343
134027915,18sleep30-21swapper/320:03:343
1122627912,51sleep111276-40ogrotate23:58:331
1122627912,51sleep111276-40ogrotate23:58:331
566727819,33sleep25664-21apt-get21:58:452
2034627818,43sleep220433-21fschecks_time21:03:512
200912780,6sleep30-21swapper/321:03:353
662427723,13sleep10-21swapper/123:43:431
662427723,13sleep10-21swapper/123:43:431
1666527625,36sleep116014-21cyclictest00:13:541
906627522,36sleep09109-21true20:28:330
256827511,47sleep01935-21/usr/sbin/munin23:29:010
2328027530,25sleep20-21swapper/200:34:022
2328027530,25sleep20-21swapper/200:34:022
203152756,13sleep20-21swapper/219:18:562
197927516,25sleep20-21swapper/220:03:572
1475727518,43sleep014780-21latency_hist22:28:340
1158727528,27sleep311638-21tr22:18:323
1033727517,41sleep010363-21netstat23:53:550
1033727517,41sleep010363-21netstat23:53:550
3069427420,24sleep00-21swapper/021:34:000
3069427420,24sleep00-21swapper/021:34:000
1510927414,24sleep30-21swapper/300:08:563
1510927414,24sleep30-21swapper/300:08:563
144662746,49sleep30-21swapper/320:43:583
98952735,59sleep00-21swapper/022:11:470
98952735,59sleep00-21swapper/022:11:470
3062427311,27sleep00-21swapper/023:14:030
3062427311,27sleep00-21swapper/023:14:030
2013327322,29sleep30-21swapper/319:18:533
1107327312,44sleep210294-21/usr/sbin/munin22:14:032
703427214,27sleep20-21swapper/223:43:562
703427214,27sleep20-21swapper/223:43:562
2817627225,28sleep228173-21apt-get23:08:442
2755327210,25sleep127576-21sed19:43:331
2755327210,25sleep127576-21sed19:43:331
225827212,43sleep02419-21awk20:04:030
2159027220,28sleep30-21swapper/300:28:583
2159027220,28sleep30-21swapper/300:28:583
1846827218,32sleep20-21swapper/220:58:342
1384027215,27sleep00-21swapper/020:43:350
2200227110,45sleep10-21swapper/119:24:001
2200227110,45sleep10-21swapper/119:24:001
173927119,31sleep21888-21latency_hist23:28:342
1679527124,25sleep20-21swapper/222:33:492
1679527124,25sleep20-21swapper/222:33:492
535527026,22sleep00-21swapper/023:38:540
3215427018,37sleep232202-21modprobe19:58:322
263062708,51sleep0231-21systemd-journal23:03:050
263062708,51sleep0231-21systemd-journal23:03:050
2190927017,37sleep221998-21grep21:08:522
1998027010,43sleep320014-21ntp_kernel_pll_00:23:583
923226915,31sleep30-21swapper/322:08:593
923226915,31sleep30-21swapper/322:08:593
5932694,11sleep20-21swapper/221:42:062
3146726910,35sleep00-21swapper/019:54:020
2625526915,39sleep026325-21/usr/sbin/munin19:38:480
2625526915,39sleep026325-21/usr/sbin/munin19:38:480
2474626920,26sleep00-21swapper/021:18:340
2397826916,36sleep324083-21timerandwakeup21:14:053
2362226916,27sleep20-21swapper/222:53:552
1943326921,26sleep219577-21unixbench_singl20:59:032
1122226920,13sleep10-21swapper/120:33:591
1743426817,22sleep30-21swapper/322:34:263
1743426817,22sleep30-21swapper/322:34:263
1694526816,30sleep317097-21sed19:09:043
1426426810,41sleep314265-21sshd00:07:143
135482685,39sleep20-21swapper/222:23:522
1254226812,27sleep00-21swapper/022:19:050
2757726721,24sleep20-21swapper/219:43:342
2757726721,24sleep20-21swapper/219:43:342
2057726717,35sleep320691-21timerwakeupswit22:44:033
66652656,45sleep112-21rcu_sched20:18:571
483726520,26sleep0289-21dbus-daemon23:38:330
2633826520,24sleep30-21swapper/319:38:513
2633826520,24sleep30-21swapper/319:38:513
260492657,12sleep20-21swapper/222:59:482
260492657,12sleep20-21swapper/222:59:482
99726417,19sleep20-21swapper/221:43:472
775926424,23sleep20-21swapper/222:04:022
498926415,34sleep25004-21awk21:54:052
2971726411,26sleep30-21swapper/319:48:563
2971726411,26sleep30-21swapper/319:48:563
224232647,13sleep00-21swapper/022:50:180
2131226417,24sleep00-21swapper/000:28:510
2131226417,24sleep00-21swapper/000:28:510
393026312,36sleep33959-21ntp_states23:33:563
3092026319,19sleep20-21swapper/221:34:232
3092026319,19sleep20-21swapper/221:34:232
2200026312,21sleep30-21swapper/321:08:533
1897826316,19sleep20-21swapper/222:39:012
1897826316,19sleep20-21swapper/222:39:012
1838926317,31sleep018417-21munin-run20:58:320
3255426211,22sleep00-21swapper/019:58:500
2812126217,20sleep30-21swapper/321:28:343
2794126219,27sleep30-21swapper/323:08:333
2794126219,27sleep30-21swapper/323:08:333
2711426215,24sleep00-21swapper/021:23:590
2671526225,23sleep00-21swapper/023:03:500
2671526225,23sleep00-21swapper/023:03:500
255592626,13sleep30-21swapper/322:59:173
255592626,13sleep30-21swapper/322:59:173
2528826220,24sleep30-21swapper/319:34:033
2168126219,27sleep30-21swapper/319:23:543
2168126219,27sleep30-21swapper/319:23:543
1929426221,25sleep2289-21dbus-daemon00:23:322
808826110,33sleep38172-21memory20:23:563
1688826120,18sleep20-21swapper/220:53:342
698626021,24sleep30-21swapper/322:03:343
594226014,31sleep05998-21latency21:58:550
2943426020,19sleep20-21swapper/219:48:512
2943426020,19sleep20-21swapper/219:48:512
2905826016,24sleep329087-21munin-run19:48:323
2905826016,24sleep329087-21munin-run19:48:323
2682826017,26sleep20-21swapper/221:23:532
893425911,22sleep30-21swapper/323:49:343
3137925912,26sleep30-21swapper/319:54:013
3129025915,27sleep031309-21apt-get23:18:440
3129025915,27sleep031309-21apt-get23:18:440
1216125917,25sleep212229-21modprobe20:38:322
935525817,25sleep29347-21apt-get20:28:472
543025815,21sleep30-21swapper/321:58:343
158482585,12sleep30-21swapper/320:48:593
1489625815,25sleep214885-21apt-get00:08:472
1489625815,25sleep214885-21apt-get00:08:472
3250825713,19sleep00-21swapper/021:39:010
300125713,20sleep00-21swapper/021:48:580
2131225714,19sleep20-21swapper/222:48:342
1450625721,18sleep20-21swapper/200:08:352
288062568,16sleep00-21swapper/019:45:180
288062568,16sleep00-21swapper/019:45:180
2339925617,24sleep30-21swapper/319:28:543
2333125615,20sleep00-21swapper/022:53:490
1772025612,19sleep20-21swapper/200:18:322
1173525612,24sleep00-21swapper/023:58:520
840925514,23sleep08411-21apt-get23:48:430
3196625514,22sleep331975-21df_inode21:38:483
2393825513,24sleep023977-21munin-run00:38:320
2393825513,24sleep023977-21munin-run00:38:320
2176425515,16sleep20-21swapper/200:29:012
2176425515,16sleep20-21swapper/200:29:012
1459825513,24sleep114121-21apt-get19:03:421
1402325515,25sleep214020-21apt-get20:43:452
1393525515,18sleep30-21swapper/322:24:013
2516425414,17sleep00-21swapper/021:18:420
2118125414,26sleep021208-21munin-run22:48:320
1436725412,27sleep014394-21munin-run00:08:330
807125313,20sleep20-21swapper/220:23:552
515625314,22sleep35165-21df_inode23:38:493
485625315,24sleep00-21swapper/020:13:520
3246725313,25sleep232464-21apt-get19:58:442
2650125312,28sleep326538-21cat21:23:343
254125313,24sleep32537-21apt-get21:48:453
2312525315,19sleep00-21swapper/021:13:340
99325213,24sleep3989-21apt-get21:43:443
625925213,22sleep36252-21apt-get20:18:473
3018225214,22sleep230188-21ntp_states23:13:572
3018225214,22sleep230188-21ntp_states23:13:572
2025925213,23sleep020254-21apt-get21:03:450
244742515,35sleep30-21swapper/322:58:003
202525118,17sleep30-21swapper/320:03:583
1803625113,23sleep018032-21apt-get00:18:460
1709925114,17sleep00-21swapper/019:09:050
1557925113,19sleep00-21swapper/020:48:520
1217725119,18sleep30-21swapper/323:59:033
749325015,15sleep30-21swapper/322:03:553
368925011,18sleep20-21swapper/223:33:512
1228525014,21sleep20-21swapper/222:18:592
996624912,22sleep29962-21apt-get23:53:452
996624912,22sleep29962-21apt-get23:53:452
260924912,22sleep22618-21df_inode21:48:482
1286924913,19sleep20-21swapper/200:03:342
1087324912,16sleep00-21swapper/022:14:010
935624712,20sleep39423-21df20:28:483
1098224711,20sleep310988-21diskstats20:33:533
105072478,28sleep00-21swapper/020:32:080
74152464,29sleep20-21swapper/220:22:132
200612459,16sleep00-21swapper/019:18:500
7122428,16sleep00-21swapper/023:23:560
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional