You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 19:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Thu Jan 01, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
257322285811,2827sleep125646-21apt-get07:20:061
22764215418,1473sleep218279-21kworker/2:108:49:332
1318628399,816sleep015998-1kworker/0:1H10:04:200
641277149,703sleep0625-21/usr/sbin/munin11:08:530
4573269415,662sleep04634-21/usr/sbin/munin11:19:230
4573269415,662sleep04634-21/usr/sbin/munin11:19:230
25153253613,510sleep125185-21irqstats12:24:141
23589248455,416sleep223641-21if_eth008:54:142
12711239537,226sleep212841-21latency_hist10:03:542
18430238312,354sleep018507-21unin-run08:38:530
8744232415,263sleep28804-21sendmail_mailqu09:49:242
20263232113,209sleep220274-21acpi08:44:042
27513229551,234sleep2369-21snmpd12:31:132
27513229551,234sleep2369-21snmpd12:31:132
1447622559,231sleep214547-21latency_hist10:08:542
25589224588,137sleep20-21swapper/208:59:262
24627222116,185sleep3934-21cron12:23:513
17728218815,159sleep317798-21ls08:34:243
7196218320,49sleep20-21swapper/211:29:132
18837218310,154sleep218914-21fschecks_time08:39:122
19980217856,64sleep220059-21xargs10:24:242
19980217856,64sleep220059-21xargs10:24:242
11879217111,145sleep212036-21apt-key11:43:552
30698217017,134sleep230745-21fschecks_time07:34:482
2566621699,67sleep225646-21apt-get07:19:142
23840216911,146sleep223874-21ps12:19:222
21322216911,137sleep221442-21apt-config12:13:532
31888216810,142sleep231917-21fschecks_count11:04:112
5780216210,137sleep25860-21timerandwakeup09:39:232
5780216210,137sleep25860-21timerandwakeup09:39:232
28752216184,25sleep20-21swapper/207:29:112
3618215017,113sleep03681-21switchtime07:50:210
23138214810,122sleep223259-21latency_hist12:18:542
1693521489,124sleep217053-21date08:33:552
23332214213,58sleep223457-21sed07:13:572
3365214123,96sleep33492-21wc09:33:553
2830621418,114sleep00-21swapper/009:09:090
2830621418,114sleep00-21swapper/009:09:090
14814214161,68sleep214911-21sed08:24:262
32264213716,101sleep332286-21/usr/sbin/munin11:04:203
669321336,10sleep20-21swapper/211:27:162
20758213213,104sleep020813-21processes08:44:220
326821305,10sleep20-21swapper/209:33:182
2263521306,10sleep20-21swapper/207:10:222
1200221306,10sleep20-21swapper/208:15:542
1200221306,10sleep20-21swapper/208:15:542
23665212910,54sleep223725-21munin-node07:14:012
2145212814,96sleep22148-21cron11:13:522
507221275,35sleep20-21swapper/211:21:142
507221275,35sleep20-21swapper/211:21:142
4138212713,17sleep20-21swapper/209:34:232
1095621267,14sleep30-21swapper/308:14:063
1095621267,14sleep30-21swapper/308:14:063
1715821259,104sleep217171-21awk10:14:272
3780212415,92sleep03906-21latency_hist11:18:540
3780212415,92sleep03906-21latency_hist11:18:540
30011212112,93sleep129955-21latency_hist10:58:531
25257212025,17sleep30-21swapper/307:18:543
1031121206,34sleep20-21swapper/208:12:152
2689321196,12sleep10-21swapper/109:05:071
20156211925,71sleep120171-21fschecks_count07:04:111
26051211812,13sleep20-21swapper/210:44:222
26051211812,13sleep20-21swapper/210:44:222
25688211825,75sleep025786-21sed12:24:250
25001211725,80sleep224891-21/usr/sbin/munin12:24:112
2456221155,12sleep00-21swapper/012:23:000
24037211519,76sleep124155-21timerwakeupswit12:19:251
18517211531,72sleep218581-21sed10:19:282
1426221159,94sleep214293-21ps11:49:242
1426221159,94sleep214293-21ps11:49:242
557921146,51sleep20-21swapper/207:56:242
27851211418,68sleep20-21swapper/207:24:272
1410021140,7sleep10-21swapper/111:49:181
1410021140,7sleep10-21swapper/111:49:181
712221125,53sleep20-21swapper/208:00:522
16261211217,78sleep116386-21sort08:29:261
24633211116,81sleep124713-21awk07:15:091
7113210915,74sleep37149-21sed09:44:173
26723210913,78sleep326246-21apt-get09:04:043
26370210917,68sleep326435-21apt-key09:03:553
21852210914,80sleep2372-21sshd10:29:322
7971210822,66sleep20-21swapper/209:48:522
30522210821,67sleep00-21swapper/010:59:170
16591210725,68sleep216661-21systemd-run11:58:522
16591210725,68sleep216661-21systemd-run11:58:522
14175210722,29sleep10-21swapper/108:24:131
1186210726,69sleep21183-21apt-get07:44:052
23848210613,74sleep123861-21/usr/sbin/munin08:54:211
5176210412,76sleep35151-21/usr/sbin/munin09:39:083
5176210412,76sleep35151-21/usr/sbin/munin09:39:083
28612210421,61sleep327980-21/usr/sbin/munin12:34:253
28612210421,61sleep327980-21/usr/sbin/munin12:34:253
17589210314,69sleep017605-21latency_hist10:18:530
31564210222,22sleep10-21swapper/111:03:541
28131210219,16sleep20-21swapper/209:08:552
28131210219,16sleep20-21swapper/209:08:552
32153210140,16sleep20-21swapper/209:19:252
20366210121,58sleep320374-21needreboot12:09:183
16020210123,59sleep20-21swapper/210:13:552
21393210015,73sleep021460-21latency_hist07:08:540
13447210019,68sleep20-21swapper/208:19:252
3132029915,10sleep20-21swapper/211:01:212
955629818,64sleep30-21swapper/309:53:533
576929727,49sleep05798-21modprobe07:58:520
937329643,14sleep20-21swapper/211:35:192
2792429618,13sleep20-21swapper/209:07:502
2386829622,32sleep00-21swapper/007:14:310
3267929522,48sleep132704-21modprobe09:23:521
3104629517,65sleep331092-21processes07:34:573
275032959,21sleep227530-21awk10:49:182
1230529517,12sleep20-21swapper/211:44:522
1230529517,12sleep20-21swapper/211:44:522
307602945,78sleep20-21swapper/209:14:282
1088929419,11sleep20-21swapper/209:56:392
1088929419,11sleep20-21swapper/209:56:392
2894129319,12sleep20-21swapper/212:34:442
2894129319,12sleep20-21swapper/212:34:442
2510029320,12sleep20-21swapper/210:41:152
201882937,73sleep00-21swapper/012:09:150
1727229317,62sleep117373-21fschecks_time08:34:141
97929215,62sleep3984-21entropy11:09:133
747729217,21sleep30-21swapper/311:29:183
291012929,64sleep20-21swapper/210:54:182
2869029219,50sleep028853-21grep12:34:260
2869029219,50sleep028853-21grep12:34:260
173752928,15sleep20-21swapper/208:34:152
123962926,12sleep10-21swapper/111:44:591
123962926,12sleep10-21swapper/111:44:591
181772917,16sleep20-21swapper/212:03:522
181772917,16sleep20-21swapper/212:03:522
8152905,75sleep20-21swapper/207:42:482
301629018,11sleep20-21swapper/207:49:382
162829018,11sleep20-21swapper/209:27:002
2374528859,15sleep20-21swapper/210:38:532
1882628859,15sleep20-21swapper/212:04:172
1882628859,15sleep20-21swapper/212:04:172
1107228817,47sleep010441-21/usr/sbin/munin11:39:250
343828717,11sleep20-21swapper/211:15:332
343828717,11sleep20-21swapper/211:15:332
2856628717,57sleep328581-21kernelversion09:09:163
2856628717,57sleep328581-21kernelversion09:09:163
2727628611,59sleep027295-21ntp_states09:05:300
1666228623,34sleep30-21swapper/311:58:533
1666228623,34sleep30-21swapper/311:58:533
1554728618,47sleep115311-21/usr/sbin/munin11:54:151
1554728618,47sleep115311-21/usr/sbin/munin11:54:151
1099728612,58sleep00-21swapper/008:14:310
1099728612,58sleep00-21swapper/008:14:310
789828520,16sleep20-21swapper/208:04:152
353528528,29sleep30-21swapper/307:50:203
2314728518,50sleep123331-21grep07:13:561
2314728518,50sleep123331-21grep07:13:561
2261528420,27sleep30-21swapper/308:49:263
2609728315,56sleep026186-21mailstats10:44:240
2609728315,56sleep026186-21mailstats10:44:240
2037728312,24sleep320466-21grep08:44:133
3004328221,41sleep130129-21gpgv07:33:541
2472128212,50sleep324844-21latency_hist08:58:553
2005128222,40sleep120060-21grep08:43:551
1822028225,36sleep30-21swapper/312:03:533
1822028225,36sleep30-21swapper/312:03:533
1320028216,43sleep10-21swapper/110:04:291
77572800,7sleep00-21swapper/008:04:120
750228013,52sleep07538-21ntpq11:29:190
3034827912,54sleep330424-21run-parts09:14:213
2827627914,52sleep128283-21acpi09:09:021
2827627914,52sleep128283-21acpi09:09:021
2725027919,33sleep00-21swapper/012:29:260
2725027919,33sleep00-21swapper/012:29:260
214727914,32sleep10-21swapper/109:29:131
98727817,44sleep01064-21grep11:09:140
772527817,21sleep30-21swapper/308:04:123
3270527815,27sleep00-21swapper/007:39:220
3186027813,49sleep031913-21latency_hist07:38:540
264027819,36sleep02779-21gpgv07:48:550
2451427813,28sleep10-21swapper/110:39:241
3084927718,18sleep10-21swapper/107:34:511
1105827713,47sleep111086-21idleruntime-cro09:58:521
1105827713,47sleep111086-21idleruntime-cro09:58:521
428627613,46sleep04439-21unixbench_multi09:34:250
2695827619,42sleep327010-21ls07:23:553
649327515,46sleep16646-21latency_hist09:43:541
649327515,46sleep16646-21latency_hist09:43:541
1947327510,15sleep30-21swapper/310:24:133
1947327510,15sleep30-21swapper/310:24:133
1092727514,45sleep110940-21/usr/sbin/munin11:39:221
1056927521,28sleep10-21swapper/108:13:551
655627412,44sleep16588-21/usr/sbin/munin07:59:221
1570627413,44sleep015736-21awk11:54:180
1570627413,44sleep015736-21awk11:54:180
1534227425,30sleep00-21swapper/008:28:520
1479927420,23sleep30-21swapper/310:09:113
2373127319,35sleep123743-21modprobe10:38:521
2050227318,28sleep10-21swapper/108:44:151
1406727320,30sleep00-21swapper/011:49:180
1406727320,30sleep00-21swapper/011:49:180
1313727313,44sleep312652-21apt-get10:04:033
954427221,28sleep30-21swapper/308:09:173
2777127222,27sleep10-21swapper/110:49:251
1976327215,41sleep319918-21sed12:08:543
1976327215,41sleep319918-21sed12:08:543
1165427213,26sleep30-21swapper/309:59:183
390827124,20sleep10-21swapper/111:18:551
2132627126,27sleep31-21systemd07:08:533
1113027116,26sleep30-21swapper/309:58:543
1113027116,26sleep30-21swapper/309:58:543
1891527013,43sleep318970-21cat08:39:133
1476427012,43sleep014786-21df10:09:080
1176027011,46sleep111801-21ntpq09:59:211
894426918,23sleep10-21swapper/108:08:551
8462690,8sleep327599-21kworker/3:209:24:173
2535126811,16sleep30-21swapper/310:43:543
2132126820,28sleep221325-21cron07:08:522
2125926815,33sleep121320-21apt-config12:13:521
1923126823,23sleep00-21swapper/008:39:220
1263726815,36sleep012652-21apt10:03:520
835226714,23sleep00-21swapper/009:49:140
3218826715,23sleep10-21swapper/107:39:111
1281326716,25sleep30-21swapper/308:19:143
863026611,43sleep18638-21needreboot09:49:191
493726613,15sleep10-21swapper/109:38:541
3273026617,29sleep332743-21latency_hist09:23:543
2211326620,23sleep10-21swapper/108:49:161
2183626616,34sleep321889-21/usr/sbin/munin07:09:143
2183626616,34sleep321889-21/usr/sbin/munin07:09:143
988626519,27sleep38863-21/usr/sbin/munin11:35:283
666726517,29sleep30-21swapper/307:59:243
450426517,26sleep30-21swapper/311:19:223
450426517,26sleep30-21swapper/311:19:223
429226515,23sleep10-21swapper/111:19:161
429226515,23sleep10-21swapper/111:19:161
2540126521,17sleep00-21swapper/007:18:560
1246026515,25sleep30-21swapper/311:45:013
1246026515,25sleep30-21swapper/311:45:013
802426417,22sleep00-21swapper/009:48:540
73626418,17sleep10-21swapper/109:24:161
485726417,32sleep34858-21cron09:38:523
478426414,28sleep10-21swapper/107:54:131
2904426415,26sleep00-21swapper/010:54:160
2769026418,16sleep00-21swapper/010:49:240
2247526415,31sleep322470-21apt-get10:34:053
1168326419,25sleep10-21swapper/108:15:211
1168326419,25sleep10-21swapper/108:15:211
65326320,22sleep10-21swapper/111:08:541
617626313,20sleep10-21swapper/111:24:241
2505126322,19sleep10-21swapper/108:59:131
2305726314,22sleep00-21swapper/012:18:520
101082635,11sleep00-21swapper/008:09:340
9942627,12sleep00-21swapper/009:24:230
244226225,23sleep1228-21systemd-journal07:48:521
1797526217,24sleep30-21swapper/310:19:163
3014426115,27sleep130211-21memory09:14:151
2993026117,27sleep329936-21modprobe10:58:513
1816926114,29sleep118176-21modprobe12:03:511
1816926114,29sleep118176-21modprobe12:03:511
471926021,23sleep30-21swapper/307:54:113
2808326013,28sleep10-21swapper/112:34:111
2808326013,28sleep10-21swapper/112:34:111
2189226014,22sleep00-21swapper/007:09:150
2189226014,22sleep00-21swapper/007:09:150
1945926013,27sleep119465-21diskstats10:24:111
1945926013,27sleep119465-21diskstats10:24:111
1914326019,20sleep10-21swapper/110:23:541
1263626015,24sleep012711-21tune2fs08:19:110
836225915,27sleep18388-21munin-run11:33:521
71772597,43sleep00-21swapper/008:01:350
316825916,24sleep10-21swapper/107:50:091
2586125912,34sleep125898-21grep10:44:161
2586125912,34sleep125898-21grep10:44:161
1429725918,18sleep30-21swapper/308:24:153
1280025921,21sleep00-21swapper/011:45:100
1280025921,21sleep00-21swapper/011:45:100
2666525814,31sleep31-21systemd10:46:323
2666525814,31sleep31-21systemd10:46:323
2336125816,21sleep00-21swapper/008:54:030
886525715,22sleep10-21swapper/111:34:051
252125718,19sleep10-21swapper/111:14:111
2178825716,23sleep121781-21apt-get12:14:041
2157925716,18sleep00-21swapper/010:29:260
2984325613,26sleep029836-21apt-get09:14:060
2780625614,23sleep10-21swapper/107:24:261
2728125616,24sleep327287-21diskstats07:24:113
2527225615,25sleep325284-21memory12:24:163
1652925617,23sleep316537-21sshd08:29:283
1631125614,23sleep116333-21df10:14:101
805225515,24sleep18203-21cat09:48:551
3046925513,23sleep330520-21irqstats10:59:163
243325514,24sleep32428-21apt-get11:14:053
243325514,24sleep32428-21apt-get11:14:053
1792025512,27sleep017926-21diskstats10:19:130
602425414,23sleep06077-21users09:39:250
602425414,23sleep06077-21users09:39:250
2948825418,18sleep30-21swapper/307:29:283
2204925415,22sleep30-21swapper/312:14:513
1571625415,23sleep015718-21diskstats08:29:120
1508525417,22sleep315095-21munin-run11:53:533
1508525417,22sleep315095-21munin-run11:53:533
119225414,24sleep31193-21http07:44:103
1007725415,22sleep00-21swapper/009:54:150
1007725415,22sleep00-21swapper/009:54:150
3260825311,21sleep30-21swapper/307:39:213
2701325314,17sleep00-21swapper/007:23:560
2119625314,23sleep30-21swapper/310:29:173
19852536,34sleep10-21swapper/111:11:451
1843325312,25sleep118452-21apt-get12:04:041
1843325312,25sleep118452-21apt-get12:04:041
1705825314,24sleep017128-21basename11:59:140
1705825314,24sleep017128-21basename11:59:140
1087925314,23sleep310918-21ntp_states11:39:193
894525213,24sleep08882-21apt-get11:34:480
689425117,16sleep30-21swapper/311:28:543
66962515,34sleep00-21swapper/011:27:180
3150925111,24sleep031513-21diskstats09:19:100
2803425113,20sleep30-21swapper/310:49:273
2247725110,19sleep00-21swapper/010:34:060
1260925115,21sleep112620-21df_inode08:19:081
2958025011,19sleep00-21swapper/007:29:290
2224225019,16sleep00-21swapper/008:49:190
1854025013,22sleep018544-21diskstats12:04:110
1854025013,22sleep018544-21diskstats12:04:110
2653624912,16sleep10-21swapper/112:29:101
2653624912,16sleep10-21swapper/112:29:101
1763624915,15sleep30-21swapper/310:18:563
2643524811,18sleep30-21swapper/312:29:043
2643524811,18sleep30-21swapper/312:29:043
228724811,20sleep02331-21irqstats09:29:160
2544924613,14sleep00-21swapper/008:59:240
2432524610,18sleep00-21swapper/010:39:190
1955824611,20sleep019592-21fw_packets10:24:140
1955824611,20sleep019592-21fw_packets10:24:140
217062448,14sleep10-21swapper/110:29:271
147112449,17sleep10-21swapper/110:09:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional