You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-03 - 18:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Sat Jan 03, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7600268116,648sleep17602-21sshd08:34:011
2085025918,571sleep320910-21unin-run10:58:463
11425229942,202sleep311464-21fschecks_count10:29:063
8929228925,248sleep39042-21latency_hist08:38:473
2485822799,257sleep324938-21sed07:44:163
302822670,12sleep312346-21cyclictest10:03:263
1752522540,46sleep312346-21cyclictest07:23:483
17318223215,194sleep00-21swapper/009:04:080
3152922040,17sleep312-21rcu_sched09:49:133
3152922040,17sleep312-21rcu_sched09:49:133
1105219751,130sleep31244-21latency_hist11:38:473
26235219012,161sleep026206-21latency_hist09:33:480
26235219012,161sleep026206-21latency_hist09:33:480
14402218977,46sleep30-21swapper/312:19:093
16724218261,108sleep316784-21sendmail_mailqu10:44:213
27277218113,112sleep327302-21modprobe11:18:443
27277218113,112sleep327302-21modprobe11:18:443
7156217721,135sleep07272-21latency_hist08:33:480
7156217721,135sleep07272-21latency_hist08:33:480
435621745,40sleep30-21swapper/310:05:373
435621745,40sleep30-21swapper/310:05:373
9504216441,75sleep39546-21true10:23:453
15463216123,121sleep115528-21uniq08:59:051
1351621597,138sleep313551-21idleruntime-cro08:53:453
24398215816,120sleep224415-21latency_hist09:28:452
32556215281,13sleep30-21swapper/308:09:163
32556215281,13sleep30-21swapper/308:09:163
18881215211,79sleep318884-21cron07:28:463
7359214916,110sleep20-21swapper/211:58:492
2455521468,85sleep324572-21uniq11:09:043
678214517,101sleep30-21swapper/309:54:143
14457214547,41sleep30-21swapper/310:39:003
14321214517,54sleep30-21swapper/307:13:523
807421418,84sleep38204-21sed10:18:493
29172213814,111sleep229207-21if_err_eth007:59:092
14221213829,51sleep314289-21latency_hist10:38:473
14221213829,51sleep314289-21latency_hist10:38:473
1885213714,55sleep30-21swapper/311:39:163
6436213641,37sleep30-21swapper/311:54:143
22093213625,89sleep022188-21timerwakeupswit11:00:000
31046213316,55sleep30-21swapper/311:29:103
2400221338,113sleep3369-21snmpd11:06:253
29310213111,57sleep30-21swapper/311:24:053
27482213012,105sleep027547-21uniq07:54:060
28643212911,58sleep328646-21cron07:58:463
19532212821,87sleep119049-21apt-get10:53:551
21942212512,55sleep322022-21switchtime10:59:273
2646621247,75sleep30-21swapper/309:33:593
32149212225,75sleep232244-21fschecks_time08:09:072
32149212225,75sleep232244-21fschecks_time08:09:072
17469212125,45sleep30-21swapper/312:29:103
32380212023,77sleep232388-21modprobe09:53:452
32380212023,77sleep232388-21modprobe09:53:452
2079421199,13sleep00-21swapper/009:14:180
2079421199,13sleep00-21swapper/009:14:180
24417211811,52sleep324466-21systemd-run09:28:463
14278211815,85sleep00-21swapper/008:54:150
15857211715,82sleep215915-21/usr/sbin/munin08:59:142
1908621140,8sleep20-21swapper/207:28:482
14030211420,72sleep113893-21munin-node07:13:501
4020211319,79sleep14146-21cat10:04:181
4020211319,79sleep14146-21cat10:04:181
555121128,50sleep35658-21timerwakeupswit10:09:163
555121128,50sleep35658-21timerwakeupswit10:09:163
20818211213,78sleep220837-21apt-get10:58:452
7793211115,78sleep17921-21cat10:18:471
2597211113,71sleep00-21swapper/011:43:450
29585211028,69sleep329589-21grep09:44:013
29585211028,69sleep329589-21grep09:44:013
537121097,53sleep3369-21snmpd11:49:413
4940210745,46sleep35056-21sort08:24:183
2781421067,15sleep00-21swapper/011:19:060
2781421067,15sleep00-21swapper/011:19:060
17787210617,74sleep117832-21/usr/sbin/munin10:49:031
29537210315,70sleep30-21swapper/307:59:183
5570210212,77sleep05603-21latency_hist08:28:470
3710210114,65sleep23319-21/usr/sbin/munin10:04:102
3710210114,65sleep23319-21/usr/sbin/munin10:04:102
25769210112,72sleep2228-21systemd-journal11:13:472
25400210021,58sleep30-21swapper/309:30:043
25400210021,58sleep30-21swapper/309:30:043
21282210043,17sleep30-21swapper/309:18:453
21282210043,17sleep30-21swapper/309:18:453
1232229913,70sleep012442-21latency_hist07:08:480
96229831,52sleep31107-21modprobe08:13:483
1122129811,70sleep211316-21mailstats08:44:162
597129713,64sleep16016-21/usr/sbin/munin11:53:591
573429715,69sleep15792-21latency_hist11:53:471
93862968,75sleep00-21swapper/010:22:130
163672965,12sleep30-21swapper/309:00:123
1938729510,60sleep30-21swapper/307:28:583
1427029510,58sleep30-21swapper/308:54:143
822529427,48sleep30-21swapper/311:59:183
3091429439,43sleep30-21swapper/308:04:113
3091429439,43sleep30-21swapper/308:04:113
1771629421,43sleep00-21swapper/007:24:060
122492949,21sleep00-21swapper/012:13:490
122492949,21sleep00-21swapper/012:13:490
1040129432,15sleep30-21swapper/312:08:473
269022935,12sleep30-21swapper/307:51:113
3027329221,47sleep030246-21latency_hist08:03:470
2511729215,52sleep125147-21cut09:29:581
2511729215,52sleep125147-21cut09:29:581
1815729215,59sleep2197-21jbd2/mmcblk2p2-10:49:122
1305429230,21sleep313118-21sendmail_mailqu07:09:153
2285429115,63sleep322857-21cron09:23:463
1824229136,14sleep30-21swapper/309:08:483
1259829118,53sleep112610-21modprobe10:33:451
101352915,11sleep30-21swapper/308:40:013
2631129014,57sleep026324-21/usr/sbin/munin07:49:140
2172629030,16sleep30-21swapper/307:35:163
44828923,36sleep2591-21grep11:34:182
194942895,12sleep30-21swapper/309:10:373
1377228912,58sleep113840-21cat12:18:461
1377228912,58sleep113840-21cat12:18:461
1195028922,47sleep212032-21apt-config12:13:472
339528756,16sleep30-21swapper/308:19:173
1820328722,47sleep30-21swapper/310:49:133
621628633,25sleep26370-21latency_hist10:13:472
621628633,25sleep26370-21latency_hist10:13:472
1243128630,14sleep30-21swapper/312:14:003
1243128630,14sleep30-21swapper/312:14:003
1204128610,50sleep30-21swapper/312:13:483
2276228521,48sleep30-21swapper/307:38:593
1106928514,53sleep10-21swapper/107:04:051
3176428423,38sleep131789-21modprobe08:08:451
3176428423,38sleep131789-21modprobe08:08:451
1531828423,33sleep10-21swapper/112:23:471
100632849,63sleep310094-21latency10:24:103
70972827,60sleep37154-21sort08:33:473
70972827,60sleep37154-21sort08:33:473
2872428114,54sleep128788-21cat07:58:481
134628111,56sleep21390-21if_err_eth008:14:072
1130428113,53sleep011433-21sed12:09:170
459428026,30sleep00-21swapper/010:08:450
459428026,30sleep00-21swapper/010:08:450
1601728023,16sleep30-21swapper/312:24:143
1524228012,15sleep00-21swapper/010:39:170
2457827913,48sleep024667-21load07:44:090
2219127923,34sleep122301-21sed11:00:011
2644827811,33sleep20-21swapper/209:33:582
2632527821,30sleep10-21swapper/107:49:151
1266827819,39sleep212738-21fschecks_time07:09:062
1263927813,51sleep212713-21fschecks_time12:15:042
1263927813,51sleep212713-21fschecks_time12:15:042
3157727712,14sleep00-21swapper/009:49:150
3157727712,14sleep00-21swapper/009:49:150
2759027720,41sleep30-21swapper/311:18:573
2759027720,41sleep30-21swapper/311:18:573
82527618,18sleep20-21swapper/209:54:172
507527617,42sleep15133-21if_eth010:09:061
507527617,42sleep15133-21if_eth010:09:061
2924827620,11sleep30-21swapper/309:43:083
28872768,28sleep30-21swapper/311:44:003
2820527617,45sleep128313-21timerandwakeup11:19:151
2820527617,45sleep128313-21timerandwakeup11:19:151
75327513,45sleep1824-21/usr/sbin/munin09:54:161
665527522,30sleep20-21swapper/208:29:172
665527522,30sleep20-21swapper/208:29:172
41142755,37sleep10-21swapper/111:48:261
292127514,46sleep12981-21if_eth008:19:071
2908227519,36sleep129169-21tune2fs07:59:081
1982527518,11sleep30-21swapper/312:37:503
1163327518,10sleep30-21swapper/308:44:253
1623427418,40sleep116298-21idleruntime07:19:071
138872745,11sleep00-21swapper/010:35:250
138872745,11sleep00-21swapper/010:35:250
423227312,44sleep24382-21latency_hist11:48:472
2507127314,44sleep125204-21awk07:44:181
2110527311,12sleep20-21swapper/209:16:262
2110527311,12sleep20-21swapper/209:16:262
2016627324,22sleep00-21swapper/007:30:280
1916427327,25sleep10-21swapper/109:09:221
1505927213,38sleep215120-21sendmail-msp08:58:452
650127115,19sleep10-21swapper/108:29:161
650127115,19sleep10-21swapper/108:29:161
306802717,51sleep00-21swapper/011:28:590
2913727114,36sleep229112-21/usr/sbin/munin11:24:002
2772727121,34sleep027731-21cron09:38:450
238812716,10sleep20-21swapper/207:42:012
2066327012,44sleep220670-21modprobe07:33:452
1977627017,23sleep20-21swapper/209:13:472
1719527012,18sleep20-21swapper/209:04:062
1686527017,37sleep117019-21latency_hist12:28:481
1571827021,28sleep00-21swapper/012:24:060
1080127017,24sleep00-21swapper/008:44:050
977826914,29sleep10-21swapper/108:39:181
1319026922,32sleep113296-21ls07:09:171
804926817,22sleep10-21swapper/111:59:161
773626817,30sleep00-21swapper/010:18:460
3205726816,31sleep10-21swapper/108:09:011
3205726816,31sleep10-21swapper/108:09:011
29852688,46sleep03087-21cut11:44:050
2447926817,29sleep024415-21latency_hist09:28:470
2300126832,21sleep00-21swapper/011:04:040
485126719,35sleep24939-21cat08:24:172
160426715,34sleep01610-21ntp_states08:14:130
1437326718,29sleep00-21swapper/007:13:530
8432665,13sleep00-21swapper/011:36:260
2640826616,23sleep00-21swapper/011:14:090
2640826616,23sleep00-21swapper/011:14:090
2173426616,29sleep00-21swapper/007:35:170
2091926625,20sleep00-21swapper/010:58:470
1840226613,32sleep118530-21/usr/sbin/munin07:24:191
1389726616,28sleep00-21swapper/007:13:490
1230226619,27sleep312319-21munin-run07:08:473
970126515,22sleep10-21swapper/112:04:171
432626514,36sleep14351-21sh08:24:031
3098126519,25sleep20-21swapper/208:04:142
3098126519,25sleep20-21swapper/208:04:142
282626515,25sleep00-21swapper/008:19:060
2461426510,39sleep124814-21latency_hist09:28:481
2166326516,22sleep10-21swapper/107:35:141
2120326519,21sleep20-21swapper/207:34:102
17962658,13sleep00-21swapper/009:59:040
1654826517,28sleep00-21swapper/007:19:140
1584026510,17sleep20-21swapper/207:18:472
963426416,28sleep20-21swapper/208:39:152
219972649,12sleep10-21swapper/109:19:111
1771426419,19sleep20-21swapper/212:29:172
1503026414,29sleep20-21swapper/210:39:152
1356726415,31sleep10-21swapper/108:53:481
3267826311,29sleep10-21swapper/111:34:131
1631826319,21sleep20-21swapper/210:44:032
134726318,20sleep10-21swapper/111:39:011
107142639,12sleep10-21swapper/108:44:021
918326218,26sleep29263-21fschecks_time12:04:062
642926221,20sleep20-21swapper/210:14:002
48862629,41sleep04935-21awk10:09:010
48862629,41sleep04935-21awk10:09:010
3132126216,28sleep231446-21sed11:29:172
2831426218,23sleep20-21swapper/211:19:162
2831426218,23sleep20-21swapper/211:19:162
2708226216,24sleep10-21swapper/109:34:181
2667526218,21sleep20-21swapper/211:14:152
2667526218,21sleep20-21swapper/211:14:152
1989526215,30sleep119917-21apt12:38:451
3047126119,19sleep10-21swapper/111:28:481
2975526127,18sleep00-21swapper/009:44:060
2975526127,18sleep00-21swapper/009:44:060
1885226116,30sleep018892-21/usr/sbin/munin12:34:080
186526120,19sleep10-21swapper/109:59:051
1737026115,20sleep10-21swapper/112:29:081
1109926117,26sleep00-21swapper/010:28:470
880726018,19sleep20-21swapper/212:03:452
572426018,20sleep00-21swapper/011:53:460
2515026016,25sleep225207-21/usr/sbin/munin09:29:592
2515026016,25sleep225207-21/usr/sbin/munin09:29:592
2087126013,26sleep120990-21ls07:33:481
1872926014,24sleep20-21swapper/212:34:062
1848626015,29sleep018480-21apt-get09:09:050
139522608,15sleep20-21swapper/210:36:162
139522608,15sleep20-21swapper/210:36:162
138226011,35sleep21419-21df11:39:022
1100626015,29sleep111032-21netstat12:09:101
3205625916,26sleep032051-21apt-get08:09:000
3205625916,26sleep032051-21apt-get08:09:000
301512596,43sleep00-21swapper/011:25:360
292272597,16sleep20-21swapper/209:42:512
2746825913,19sleep20-21swapper/207:54:042
2136325915,25sleep10-21swapper/109:18:471
2136325915,25sleep10-21swapper/109:18:471
187442596,40sleep00-21swapper/010:49:550
1575225916,25sleep115837-21apt-config10:43:461
1686025817,19sleep10-21swapper/110:44:231
155425825,17sleep00-21swapper/009:58:470
802625718,19sleep00-21swapper/008:36:100
55662576,16sleep20-21swapper/211:52:152
330725722,18sleep20-21swapper/208:19:162
2333925714,24sleep00-21swapper/007:39:160
225752575,10sleep20-21swapper/209:20:052
3123625616,24sleep131320-21cat11:29:161
2468325615,17sleep20-21swapper/207:44:112
2401825613,21sleep10-21swapper/107:43:451
2159925614,25sleep021595-21apt-get09:18:580
124425618,24sleep11345-21grep08:14:061
1099725612,27sleep210993-21apt-get07:04:022
3003825515,23sleep230045-21ntp_states09:44:122
3003825515,23sleep230045-21ntp_states09:44:122
2445225514,23sleep024464-21apt-get11:08:560
239452555,11sleep10-21swapper/111:05:381
2320225516,22sleep223296-21memory11:04:102
1662625512,26sleep216693-21/usr/sbin/munin07:19:162
1598825518,20sleep00-21swapper/008:59:160
1458725515,26sleep00-21swapper/012:19:140
1243025515,20sleep00-21swapper/008:49:180
1017125517,21sleep00-21swapper/012:06:260
2601125412,24sleep126006-21apt-get11:13:581
2601125412,24sleep126006-21apt-get11:13:581
1840525416,18sleep00-21swapper/012:33:470
1555625412,24sleep215550-21apt-get12:24:002
123032547,11sleep00-21swapper/010:29:560
1042025414,25sleep010429-21modprobe08:43:450
958125316,19sleep10-21swapper/110:23:481
85892535,39sleep00-21swapper/012:00:530
3113225314,23sleep131125-21apt-get09:49:031
3113225314,23sleep131125-21apt-get09:49:031
3051425313,24sleep030508-21apt-get08:04:000
3051425313,24sleep030508-21apt-get08:04:000
1749525319,20sleep00-21swapper/010:48:460
1401325314,22sleep214081-21df12:19:002
1131225312,25sleep20-21swapper/210:28:572
887825213,17sleep20-21swapper/210:19:572
2454025214,23sleep124547-21df_inode11:09:011
2187225219,20sleep221941-21systemctl10:59:262
1818025215,20sleep10-21swapper/109:08:461
1273625214,23sleep212742-21ntp_states08:49:242
986825114,20sleep29874-21df_inode10:24:052
269582515,13sleep20-21swapper/207:51:542
756625010,23sleep27087-21apt-get08:33:542
187125012,21sleep21878-21diskstats09:59:062
3000124913,21sleep130037-21ntp_states09:44:111
3000124913,21sleep130037-21ntp_states09:44:111
2854724914,20sleep01-21systemd09:39:130
1288324911,21sleep112878-21apt-get10:33:571
1288324911,21sleep112878-21apt-get10:33:571
2812224811,21sleep128127-21diskstats09:39:021
1432024714,17sleep10-21swapper/108:54:161
647024613,15sleep00-21swapper/011:54:150
993624511,13sleep10-21swapper/110:24:081
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional