You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-19 - 08:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Mar 19, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19961922490,4sleep10-21swapper/119:05:171
20083199191187,3cyclictest259448-21kworker/u8:021:59:583
24647421230,2sleep30-21swapper/321:15:113
21660021150,2sleep20-21swapper/219:50:182
20842121090,4sleep10-21swapper/119:30:001
20060228239,14sleep20-21swapper/219:09:252
139099780,67rtkit-daemon0-21swapper/019:08:340
20000427746,22sleep30-21swapper/319:05:243
2524582490,3sleep10-21swapper/121:30:211
2689772460,4sleep126-21ksoftirqd/122:20:131
2390412360,3sleep1251rcuc/120:55:131
139099330,2rtkit-daemon0-21swapper/000:34:010
139099320,2rtkit-daemon0-21swapper/020:47:330
139099320,2rtkit-daemon0-21swapper/000:12:030
139099280,4rtkit-daemon1389-21rtkit-daemon23:16:090
139099270,4rtkit-daemon1389-21rtkit-daemon00:24:161
2017882250,3sleep30-21swapper/319:10:173
20083199258,15cyclictest217526-21wc19:55:013
200831992515,6cyclictest231541-21turbostat20:35:013
20083199251,11cyclictest250514-21python321:25:203
2284102230,4sleep33056-21PK-Backend20:25:113
20083199232,15cyclictest279136-21chrt22:50:113
20083199232,15cyclictest279136-21chrt22:50:113
200828992319,3cyclictest34-21ksoftirqd/200:00:002
200823992320,2cyclictest3056-21PK-Backend21:57:491
139099230,4rtkit-daemon1389-21rtkit-daemon21:25:530
139099230,4rtkit-daemon1389-21rtkit-daemon20:34:320
139099230,17rtkit-daemon953-21gmain19:42:530
20083199229,11cyclictest282621-21unin-run23:00:003
20083199224,16cyclictest263255-21cron22:04:593
20083199224,15cyclictest289512-21cron23:20:003
20083199222,17cyclictest226174-21lsb_release20:18:193
20083199221,17cyclictest232666-21python320:35:213
200828992213,8cyclictest257770-21NetworkManager21:46:372
200823992220,1cyclictest3056-21PK-Backend21:29:441
200823992220,1cyclictest0-21swapper/120:54:561
200823992220,1cyclictest0-21swapper/120:49:151
200823992218,2cyclictest3056-21PK-Backend21:21:531
20082399220,20cyclictest313463-21python300:25:191
20081999220,20cyclictest0-21swapper/021:42:230
139099220,3rtkit-daemon1389-21rtkit-daemon23:26:380
139099220,3rtkit-daemon1389-21rtkit-daemon22:00:330
2461412210,4sleep00-21swapper/021:15:000
20083199215,14cyclictest224262-21perf20:15:003
20083199213,13cyclictest277401-21apt22:45:113
20083199211,15cyclictest305609-21chrt00:05:013
20083199210,8cyclictest2524592who21:30:213
20083199210,19cyclictest3056-21PK-Backend00:25:473
200828992119,1cyclictest0-21swapper/221:54:512
200828992119,1cyclictest0-21swapper/221:40:522
200828992113,6cyclictest436-20systemd-journal23:45:002
200828992113,6cyclictest232512-21ntp_kernel_pll_20:35:182
20082899210,20cyclictest285582-21timerandwakeup23:05:212
20082899210,20cyclictest0-21swapper/222:14:452
20082899210,20cyclictest0-21swapper/220:51:242
20082899210,20cyclictest0-21swapper/219:19:442
20082899210,1cyclictest0-21swapper/219:48:172
200823992119,1cyclictest3056-21PK-Backend23:07:321
200823992119,1cyclictest3056-21PK-Backend22:17:161
200823992119,1cyclictest3056-21PK-Backend22:03:371
200823992119,1cyclictest3056-21PK-Backend21:47:421
200823992119,1cyclictest0-21swapper/123:34:011
200823992119,1cyclictest0-21swapper/122:35:291
200823992119,1cyclictest0-21swapper/120:39:421
200823992119,1cyclictest0-21swapper/120:23:061
200823992119,1cyclictest0-21swapper/119:35:201
200823992118,2cyclictest3056-21PK-Backend22:08:371
200823992118,2cyclictest3056-21PK-Backend21:09:361
200823992113,6cyclictest317398-21sshd00:38:271
20082399210,19cyclictest3056-21PK-Backend20:03:121
200819992118,2cyclictest220785-21sh20:04:590
200819992117,3cyclictest0-21swapper/023:40:110
20081999210,19cyclictest436-20systemd-journal00:20:000
139099210,3rtkit-daemon1389-21rtkit-daemon21:08:300
139099210,3rtkit-daemon1389-21rtkit-daemon19:54:170
139099210,3rtkit-daemon1389-21rtkit-daemon19:31:261
2563692200,5sleep10-21swapper/121:45:011
20083199203,15cyclictest220804-21munin-run20:04:593
200831992018,1cyclictest0-21swapper/322:09:343
200831992017,2cyclictest3056-21PK-Backend20:42:003
200831992017,2cyclictest0-21swapper/300:00:193
200831992016,3cyclictest886-21NetworkManager21:03:583
200831992012,5cyclictest271946-21idleruntime-cro22:30:003
200831992010,9cyclictest42-21ksoftirqd/319:20:003
20083199200,7cyclictest267938-21taskset22:15:193
20083199200,1cyclictest3056-21PK-Backend23:41:413
20083199200,1cyclictest3056-21PK-Backend21:35:403
20083199200,1cyclictest3056-21PK-Backend20:22:063
20083199200,1cyclictest3056-21PK-Backend00:11:013
20083199200,1cyclictest0-21swapper/323:56:143
20083199200,1cyclictest0-21swapper/322:32:113
20083199200,19cyclictest3056-21PK-Backend22:38:403
20083199200,19cyclictest3056-21PK-Backend20:54:103
20083199200,19cyclictest0-21swapper/323:48:503
20083199200,19cyclictest0-21swapper/323:10:243
20083199200,19cyclictest0-21swapper/323:03:283
20083199200,19cyclictest0-21swapper/322:23:533
20083199200,19cyclictest0-21swapper/322:12:103
20083199200,19cyclictest0-21swapper/321:06:003
200828992018,1cyclictest0-21swapper/223:28:132
200828992018,1cyclictest0-21swapper/219:32:082
200828992018,1cyclictest0-21swapper/200:08:142
200828992012,6cyclictest436-20systemd-journal21:30:002
200828992012,6cyclictest235972-21awk20:45:162
20082899200,1cyclictest0-21swapper/223:36:372
20082899200,1cyclictest0-21swapper/222:01:062
20082899200,1cyclictest0-21swapper/221:57:412
20082899200,1cyclictest0-21swapper/200:36:262
20082899200,19cyclictest0-21swapper/222:45:482
20082899200,19cyclictest0-21swapper/222:44:462
20082899200,19cyclictest0-21swapper/221:23:412
20082899200,19cyclictest0-21swapper/221:05:162
20082899200,19cyclictest0-21swapper/220:12:442
20082899200,19cyclictest0-21swapper/219:35:412
20082899200,19cyclictest0-21swapper/200:27:482
20082899200,19cyclictest0-21swapper/200:04:562
20082899200,18cyclictest0-21swapper/200:31:062
200823992018,1cyclictest3056-21PK-Backend21:11:391
200823992018,1cyclictest3056-21PK-Backend20:33:131
200823992018,1cyclictest3056-21PK-Backend20:17:051
200823992018,1cyclictest3056-21PK-Backend20:07:451
200823992018,1cyclictest0-21swapper/121:38:361
200823992018,1cyclictest0-21swapper/100:10:321
200823992018,1cyclictest0-21swapper/100:06:121
200823992017,2cyclictest3056-21PK-Backend22:55:081
200823992017,2cyclictest3056-21PK-Backend22:48:251
200823992017,2cyclictest3056-21PK-Backend20:28:061
200823992017,2cyclictest3056-21PK-Backend20:13:331
200823992017,2cyclictest3056-21PK-Backend19:17:221
20082399200,1cyclictest3056-21PK-Backend23:19:551
20082399200,1cyclictest3056-21PK-Backend19:55:341
20082399200,19cyclictest3056-21PK-Backend23:03:401
20082399200,19cyclictest3056-21PK-Backend19:41:561
20082399200,18cyclictest3056-21PK-Backend22:28:591
200819992018,1cyclictest0-21swapper/020:22:330
200819992018,1cyclictest0-21swapper/020:06:240
200819992018,1cyclictest0-21swapper/019:31:230
20081999200,1cyclictest0-21swapper/023:09:030
20081999200,19cyclictest0-21swapper/022:41:310
20081999200,19cyclictest0-21swapper/021:55:550
20081999200,19cyclictest0-21swapper/021:10:260
20081999200,19cyclictest0-21swapper/020:59:510
20081999200,19cyclictest0-21swapper/020:25:040
20081999200,19cyclictest0-21swapper/000:25:540
20081999200,19cyclictest0-21swapper/000:05:330
20083199192,5cyclictest213428-21cat19:40:203
200831991917,1cyclictest3056-21PK-Backend00:16:213
200831991917,1cyclictest0-21swapper/323:22:133
200831991917,1cyclictest0-21swapper/321:47:233
200831991917,1cyclictest0-21swapper/319:24:333
200831991912,5cyclictest292683-21sshd23:26:163
20083199190,6cyclictest222527-21dump-pmu-power20:09:593
20083199190,1cyclictest3056-21PK-Backend21:44:333
20083199190,1cyclictest3056-21PK-Backend20:58:023
20083199190,1cyclictest3056-21PK-Backend19:39:003
20083199190,1cyclictest0-21swapper/323:05:303
20083199190,16cyclictest244625-21grep21:10:123
20082899199,9cyclictest269444-21ntpq22:20:182
200828991917,1cyclictest0-21swapper/223:17:112
200828991917,1cyclictest0-21swapper/222:31:352
200828991917,1cyclictest0-21swapper/221:16:322
200828991917,1cyclictest0-21swapper/220:25:142
200828991917,1cyclictest0-21swapper/219:22:012
200828991914,4cyclictest301640-21cron23:54:592
200828991913,5cyclictest283302-21cat23:00:152
200828991913,4cyclictest270714-21cat22:25:142
200828991910,7cyclictest287396-21users23:10:212
20082899190,1cyclictest0-21swapper/223:49:082
20082899190,1cyclictest0-21swapper/223:22:042
20082899190,1cyclictest0-21swapper/222:08:432
20082899190,1cyclictest0-21swapper/221:39:512
20082899190,18cyclictest0-21swapper/220:04:262
20082899190,18cyclictest0-21swapper/219:42:292
200823991917,1cyclictest3056-21PK-Backend19:24:111
200823991917,1cyclictest0-21swapper/100:17:551
200823991916,1cyclictest3056-21PK-Backend23:25:511
200823991911,6cyclictest295470-21awk23:35:171
20082399190,1cyclictest3056-21PK-Backend23:20:571
20082399190,1cyclictest3056-21PK-Backend23:13:131
20082399190,1cyclictest3056-21PK-Backend22:53:051
20082399190,1cyclictest3056-21PK-Backend22:53:051
20082399190,1cyclictest3056-21PK-Backend19:49:481
20082399190,1cyclictest3056-21PK-Backend00:01:531
20082399190,18cyclictest3056-21PK-Backend21:53:091
20082399190,18cyclictest3056-21PK-Backend19:10:261
20082399190,18cyclictest0-21swapper/100:30:391
200819991917,1cyclictest0-21swapper/022:48:470
200819991917,1cyclictest0-21swapper/022:20:560
200819991917,1cyclictest0-21swapper/021:36:080
200819991917,1cyclictest0-21swapper/020:51:190
200819991917,1cyclictest0-21swapper/020:41:460
200819991917,1cyclictest0-21swapper/019:29:060
200819991917,1cyclictest0-21swapper/019:18:250
200819991913,4cyclictest299924-21awk23:50:000
20081999190,1cyclictest0-21swapper/023:50:250
20081999190,1cyclictest0-21swapper/019:35:560
20081999190,18cyclictest0-21swapper/023:57:220
20081999190,18cyclictest0-21swapper/023:31:470
20081999190,18cyclictest0-21swapper/020:11:590
20081999190,18cyclictest0-21swapper/019:21:440
200831991816,1cyclictest3056-21PK-Backend23:36:333
200831991816,1cyclictest3056-21PK-Backend19:32:243
200831991816,1cyclictest0-21swapper/321:50:143
200831991816,1cyclictest0-21swapper/319:59:433
200831991816,1cyclictest0-21swapper/300:33:113
200831991813,3cyclictest300667-21/usr/sbin/munin23:50:143
200831991811,4cyclictest1095-21runrttasks19:45:113
200828991816,1cyclictest0-21swapper/221:10:232
200828991816,1cyclictest0-21swapper/220:42:382
200828991816,1cyclictest0-21swapper/220:16:292
200828991816,1cyclictest0-21swapper/200:10:312
200828991814,3cyclictest978-21snmpd20:59:162
200828991812,4cyclictest281276-21expr22:55:122
200828991811,6cyclictest274513-21memory22:35:152
200828991810,6cyclictest436-20systemd-journal19:55:142
20082899180,16cyclictest207992-21sed19:25:212
200823991816,1cyclictest3056-21PK-Backend23:55:451
200823991816,1cyclictest3056-21PK-Backend23:50:381
200823991816,1cyclictest3056-21PK-Backend22:11:141
200823991816,1cyclictest3056-21PK-Backend21:00:511
200823991816,1cyclictest0-21swapper/123:45:311
200823991816,1cyclictest0-21swapper/122:31:491
200823991816,1cyclictest0-21swapper/120:43:341
200823991813,3cyclictest297394-21strings23:40:171
200819991816,1cyclictest0-21swapper/022:27:590
200819991816,1cyclictest0-21swapper/000:36:370
200819991812,4cyclictest295329-21expr23:35:150
200819991811,5cyclictest281567-21awk22:55:150
20081999180,1cyclictest0-21swapper/021:32:290
20081999180,1cyclictest0-21swapper/021:02:420
20081999180,17cyclictest0-21swapper/023:10:030
200831991715,1cyclictest3056-21PK-Backend23:30:253
200831991715,1cyclictest3056-21PK-Backend22:43:203
20083199171,2cyclictest207678-21ntp_states19:25:193
200831991712,4cyclictest248957-21ls21:20:223
20082899179,6cyclictest279522-21fw_packets22:50:142
20082899179,6cyclictest279522-21fw_packets22:50:142
20082899179,6cyclictest251773-21fw_packets21:30:142
20082899172,13cyclictest267542-21wc22:15:142
200828991715,1cyclictest0-21swapper/221:03:132
200828991715,1cyclictest0-21swapper/219:12:482
200828991715,1cyclictest0-21swapper/200:23:302
200828991712,3cyclictest230172-21cpuspeed_turbos20:30:102
200823991714,2cyclictest3056-21PK-Backend21:15:461
20082399171,11cyclictest275669-21apt22:40:101
20081999179,6cyclictest275403-21/usr/sbin/munin22:40:000
200819991715,1cyclictest0-21swapper/022:32:250
200819991715,1cyclictest0-21swapper/022:15:430
200819991715,1cyclictest0-21swapper/021:47:500
200819991715,1cyclictest0-21swapper/021:20:260
200819991715,1cyclictest0-21swapper/020:15:300
200819991715,1cyclictest0-21swapper/000:20:340
200819991713,3cyclictest283067-21cstates23:00:120
200819991712,3cyclictest264440-21switchtime22:05:200
20081999170,1cyclictest0-21swapper/019:14:260
20081999170,16cyclictest0-21swapper/020:37:060
20082899169,6cyclictest221810-21ntpq20:05:192
200828991614,1cyclictest0-21swapper/220:20:142
200828991611,3cyclictest310684-21cat00:20:002
200819991614,1cyclictest0-21swapper/019:45:490
200819991610,4cyclictest265773-21cut22:10:150
200819991610,4cyclictest2156-21gnome-shell22:54:090
200819991610,4cyclictest2156-21gnome-shell22:54:090
200831991512,2cyclictest3056-21PK-Backend20:45:523
200831991512,2cyclictest3056-21PK-Backend00:21:423
200819991513,1cyclictest0-21swapper/021:54:070
200819991513,1cyclictest0-21swapper/000:01:270
200819991512,2cyclictest289764-21/usr/sbin/munin23:20:210
200828991410,3cyclictest1095-21runrttasks23:32:352
200823991410,3cyclictest3056-21PK-Backend19:51:571
20081999140,5cyclictest141rcu_preempt19:55:110
200831991310,2cyclictest3056-21PK-Backend00:39:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional