You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 22:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Mon Jun 01, 2026 12:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
50594821010,2sleep20-21swapper/210:50:212
4254102810,4sleep30-21swapper/307:05:223
139099750,60rtkit-daemon0-21swapper/107:05:481
42473127242,21sleep00-21swapper/007:05:070
42533926743,15sleep20-21swapper/207:05:212
4845062500,4sleep23056-21PK-Backend09:50:182
5425222470,2sleep00-21swapper/012:35:130
139099410,4rtkit-daemon1389-21rtkit-daemon08:55:022
5038012330,4chrt0-21swapper/310:45:173
139099280,4rtkit-daemon1389-21rtkit-daemon07:37:093
4876042240,4sleep23056-21PK-Backend10:00:132
426350992418,4cyclictest461067-21cron08:45:011
426352992316,5cyclictest1-21systemd10:50:002
42635299230,22cyclictest1389-21rtkit-daemon10:42:522
42635299230,21cyclictest527164-21users11:50:222
42635099239,12cyclictest487091-21basename10:00:001
426350992318,3cyclictest487722-21df_inode10:00:131
426347992318,3cyclictest515406-21munin-run11:20:000
139099230,3rtkit-daemon1389-21rtkit-daemon11:32:023
426355992218,3cyclictest439507-21idleruntime-cro07:45:013
426355992214,6cyclictest532748-21apt-get12:10:003
42635599220,20cyclictest3056-21PK-Backend11:39:143
426352992217,4cyclictest34-21ksoftirqd/211:43:592
42635299220,20cyclictest467485-21threads09:00:222
426350992220,1cyclictest0-21swapper/112:35:191
426350992220,1cyclictest0-21swapper/111:44:501
426350992220,1cyclictest0-21swapper/109:10:591
426350992219,2cyclictest3056-21PK-Backend12:11:331
426350992219,2cyclictest3056-21PK-Backend11:56:131
426350992219,2cyclictest3056-21PK-Backend10:46:371
426350992219,2cyclictest3056-21PK-Backend07:37:371
426350992211,9cyclictest2636-21xdg-desktop-por07:23:561
42635099220,21cyclictest3056-21PK-Backend09:49:251
426347992215,6cyclictest428970-21users07:15:030
426347992215,6cyclictest428970-21users07:15:030
426347992214,6cyclictest439497-21apt-get07:45:000
426347992214,6cyclictest436-20systemd-journal08:10:000
139099220,15rtkit-daemon0-21swapper/107:42:221
5258452210,4sleep33056-21PK-Backend11:50:003
42635599213,2cyclictest141rcu_preempt09:30:213
42635599210,20cyclictest3056-21PK-Backend09:08:193
42635599210,1cyclictest3056-21PK-Backend09:10:563
42635599210,19cyclictest3056-21PK-Backend09:43:343
426352992117,3cyclictest541948-21cron12:35:002
426352992113,6cyclictest883-21dbus-daemon12:10:002
42635299210,20cyclictest0-21swapper/212:01:102
42635299210,20cyclictest0-21swapper/210:14:022
42635099214,15cyclictest525808-21cron11:50:001
426350992119,1cyclictest0-21swapper/112:24:311
426350992119,1cyclictest0-21swapper/111:11:381
426350992119,1cyclictest0-21swapper/111:11:381
426350992119,1cyclictest0-21swapper/110:32:181
426350992119,1cyclictest0-21swapper/110:26:431
426350992119,1cyclictest0-21swapper/110:18:451
426350992119,1cyclictest0-21swapper/109:31:061
426350992119,1cyclictest0-21swapper/109:07:201
426350992119,1cyclictest0-21swapper/108:06:431
426350992119,1cyclictest0-21swapper/107:16:551
426350992119,1cyclictest0-21swapper/107:16:551
426350992118,2cyclictest3056-21PK-Backend10:38:391
426350992118,2cyclictest3056-21PK-Backend08:51:521
426350992118,2cyclictest0-21swapper/108:19:421
426350992118,2cyclictest0-21swapper/108:19:421
426350992115,4cyclictest511081-21strings11:05:171
426350992114,5cyclictest883-21dbus-daemon12:30:011
426350992113,6cyclictest507002-21awk10:55:121
426350992113,6cyclictest480128-21cron09:40:011
426350992113,6cyclictest450409-21cron08:14:591
426350992113,6cyclictest450409-21cron08:14:591
42635099210,20cyclictest3056-21PK-Backend10:42:381
42635099210,20cyclictest3056-21PK-Backend09:02:541
42635099210,20cyclictest0-21swapper/110:10:131
42635099210,20cyclictest0-21swapper/107:27:101
42635099210,19cyclictest3056-21PK-Backend08:56:051
426347992114,5cyclictest436-20systemd-journal12:04:550
426347992112,7cyclictest462110-21awk08:45:210
42634799210,20cyclictest0-21swapper/009:10:560
42634799210,20cyclictest0-21swapper/009:02:520
42634799210,20cyclictest0-21swapper/008:21:290
139099210,3rtkit-daemon0-21swapper/107:10:121
139099210,15rtkit-daemon3056-21PK-Backend12:06:061
426355992018,1cyclictest3056-21PK-Backend11:11:093
426355992018,1cyclictest3056-21PK-Backend11:11:083
426355992018,1cyclictest3056-21PK-Backend08:05:203
426355992018,1cyclictest0-21swapper/308:47:383
426355992017,2cyclictest3056-21PK-Backend10:21:413
426355992017,2cyclictest3056-21PK-Backend10:21:403
426355992015,3cyclictest535017-21apt12:15:103
426355992014,4cyclictest436-20systemd-journal08:50:173
426355992011,7cyclictest537826-21/usr/sbin/munin12:20:233
42635599200,1cyclictest3056-21PK-Backend08:15:203
42635599200,1cyclictest3056-21PK-Backend08:15:203
42635599200,19cyclictest3056-21PK-Backend12:28:553
42635599200,19cyclictest3056-21PK-Backend11:02:503
42635599200,19cyclictest3056-21PK-Backend10:13:223
42635599200,19cyclictest3056-21PK-Backend07:57:223
42635599200,19cyclictest0-21swapper/311:20:143
42635599200,19cyclictest0-21swapper/309:00:293
42635599200,19cyclictest0-21swapper/307:50:263
42635599200,18cyclictest3056-21PK-Backend08:12:173
42635599200,18cyclictest3056-21PK-Backend08:12:163
426352992018,1cyclictest0-21swapper/210:39:582
426352992018,1cyclictest0-21swapper/209:07:392
426352992018,1cyclictest0-21swapper/208:27:362
426352992017,2cyclictest3056-21PK-Backend11:15:282
426352992015,4cyclictest34-21ksoftirqd/207:50:172
426352992011,7cyclictest458959-21users08:35:242
42635299200,1cyclictest0-21swapper/211:07:152
42635299200,1cyclictest0-21swapper/209:31:522
42635299200,19cyclictest0-21swapper/211:55:222
42635299200,19cyclictest0-21swapper/211:24:122
42635299200,19cyclictest0-21swapper/209:16:572
42635299200,19cyclictest0-21swapper/209:16:572
42635299200,19cyclictest0-21swapper/208:41:162
42635299200,19cyclictest0-21swapper/207:44:522
426350992018,1cyclictest3056-21PK-Backend08:48:291
426350992018,1cyclictest0-21swapper/111:50:381
426350992018,1cyclictest0-21swapper/111:29:431
426350992018,1cyclictest0-21swapper/111:20:241
426350992018,1cyclictest0-21swapper/111:16:241
426350992018,1cyclictest0-21swapper/109:40:401
426350992018,1cyclictest0-21swapper/109:28:431
426350992015,3cyclictest494695-21tr10:20:121
426350992015,3cyclictest494695-21tr10:20:121
426350992013,5cyclictest541427-21threads12:30:211
42635099200,1cyclictest0-21swapper/109:22:341
42635099200,19cyclictest3056-21PK-Backend07:56:171
42635099200,19cyclictest0-21swapper/109:16:271
42635099200,19cyclictest0-21swapper/109:16:261
426347992018,1cyclictest0-21swapper/012:09:020
426347992018,1cyclictest0-21swapper/011:59:140
426347992018,1cyclictest0-21swapper/011:36:300
426347992018,1cyclictest0-21swapper/011:29:200
426347992018,1cyclictest0-21swapper/010:14:570
426347992018,1cyclictest0-21swapper/009:46:120
426347992015,4cyclictest13-21ksoftirqd/010:33:570
426347992013,6cyclictest486492-21wc09:55:220
42634799200,19cyclictest0-21swapper/011:49:270
42634799200,19cyclictest0-21swapper/011:31:440
42634799200,19cyclictest0-21swapper/010:16:340
42634799200,19cyclictest0-21swapper/010:00:100
42634799200,19cyclictest0-21swapper/008:19:120
42634799200,19cyclictest0-21swapper/008:19:120
42634799200,19cyclictest0-21swapper/007:14:230
426355991917,1cyclictest3056-21PK-Backend09:27:123
426355991917,1cyclictest3056-21PK-Backend07:19:243
426355991917,1cyclictest3056-21PK-Backend07:19:233
426355991917,1cyclictest0-21swapper/311:53:543
426355991917,1cyclictest0-21swapper/310:32:563
426355991917,1cyclictest0-21swapper/309:58:283
426355991917,1cyclictest0-21swapper/309:45:583
426355991917,1cyclictest0-21swapper/308:57:533
426355991917,1cyclictest0-21swapper/308:25:213
426355991917,1cyclictest0-21swapper/307:14:443
426355991916,2cyclictest0-21swapper/310:50:133
426355991914,3cyclictest441498-21dump-pmu-power07:50:003
42635599191,16cyclictest533258-21/usr/sbin/munin12:10:223
42635599190,1cyclictest0-21swapper/311:56:173
42635599190,18cyclictest3056-21PK-Backend12:01:033
42635599190,18cyclictest0-21swapper/310:00:043
426352991917,1cyclictest0-21swapper/212:15:232
426352991917,1cyclictest0-21swapper/209:20:082
426352991917,1cyclictest0-21swapper/207:47:082
426352991912,5cyclictest517690-21expr11:25:132
42635299190,1cyclictest0-21swapper/209:27:532
42635299190,1cyclictest0-21swapper/208:22:562
42635299190,1cyclictest0-21swapper/208:01:542
42635299190,18cyclictest0-21swapper/212:25:232
42635299190,18cyclictest0-21swapper/212:11:582
42635299190,18cyclictest0-21swapper/210:06:392
42635299190,18cyclictest0-21swapper/208:50:412
42635299190,18cyclictest0-21swapper/208:18:232
42635299190,18cyclictest0-21swapper/208:18:232
42635299190,18cyclictest0-21swapper/208:12:502
42635299190,18cyclictest0-21swapper/208:12:502
426350991912,5cyclictest521320-21cat11:35:131
426350991912,5cyclictest446252-21awk08:00:181
426350991911,6cyclictest913-21gdbus07:35:001
426350991911,6cyclictest436-20systemd-journal08:34:341
42635099190,1cyclictest3056-21PK-Backend11:31:181
42635099190,1cyclictest0-21swapper/111:03:411
42635099190,18cyclictest3056-21PK-Backend07:54:141
42635099190,18cyclictest0-21swapper/108:35:021
42634799197,4cyclictest141rcu_preempt08:00:010
426347991917,1cyclictest3056-21PK-Backend07:35:050
426347991917,1cyclictest0-21swapper/011:20:480
426347991917,1cyclictest0-21swapper/009:09:070
426347991917,1cyclictest0-21swapper/008:52:250
426347991917,1cyclictest0-21swapper/008:30:340
426347991917,1cyclictest0-21swapper/008:00:060
426347991916,2cyclictest3056-21PK-Backend12:21:240
426347991916,2cyclictest0-21swapper/011:11:160
426347991916,2cyclictest0-21swapper/011:11:150
426347991912,6cyclictest455600-21seq08:26:360
426347991911,2cyclictest13-21ksoftirqd/011:10:010
426347991910,7cyclictest539606-21python312:25:210
42634799190,1cyclictest3056-21PK-Backend10:40:270
42634799190,1cyclictest0-21swapper/011:00:540
42634799190,1cyclictest0-21swapper/010:57:430
42634799190,1cyclictest0-21swapper/010:54:320
42634799190,1cyclictest0-21swapper/007:24:230
42634799190,18cyclictest0-21swapper/009:22:320
42634799190,18cyclictest0-21swapper/008:55:020
42634799190,18cyclictest0-21swapper/007:46:330
42635599189,7cyclictest543689-21cron12:40:003
426355991816,1cyclictest3056-21PK-Backend12:30:313
426355991816,1cyclictest3056-21PK-Backend11:07:233
426355991816,1cyclictest3056-21PK-Backend09:17:473
426355991816,1cyclictest3056-21PK-Backend09:17:463
426355991816,1cyclictest0-21swapper/308:37:513
426355991815,2cyclictest3056-21PK-Backend10:25:253
426355991813,3cyclictest513932-21apt11:15:113
426355991813,3cyclictest1095-21runrttasks10:40:193
426355991811,5cyclictest484829-21timerandwakeup09:50:233
426352991816,1cyclictest0-21swapper/210:28:372
426352991816,1cyclictest0-21swapper/209:44:492
426352991816,1cyclictest0-21swapper/207:24:292
426352991812,4cyclictest524516-21tr11:45:122
426352991811,6cyclictest444426-21grep07:55:172
426352991811,5cyclictest498455-21awk10:30:162
426352991811,5cyclictest470732-21ntpq09:10:172
42635299180,1cyclictest0-21swapper/212:21:112
426350991816,1cyclictest0-21swapper/110:53:331
426347991816,1cyclictest0-21swapper/009:16:250
426347991816,1cyclictest0-21swapper/009:16:250
426347991814,3cyclictest433243-21if_err_enp2s007:25:170
426347991813,3cyclictest475938-21ntp_offset09:25:190
426347991813,3cyclictest450458-21apt-key08:15:000
426347991813,3cyclictest450458-21apt-key08:15:000
426347991812,5cyclictest477408-21interrupts09:30:150
426347991811,5cyclictest481122-21ntp_kernel_pll_09:40:170
42634799180,1cyclictest0-21swapper/010:46:480
42634799180,1cyclictest0-21swapper/008:36:220
426355991715,1cyclictest3056-21PK-Backend10:56:073
426355991713,2cyclictest432262-21turbostat07:24:593
42635599170,4cyclictest494029-21awk10:19:593
42635599170,16cyclictest3056-21PK-Backend11:42:253
42635599170,15cyclictest3056-21PK-Backend08:01:373
42635299179,6cyclictest883-21dbus-daemon09:58:572
426352991715,1cyclictest0-21swapper/211:00:062
426352991715,1cyclictest0-21swapper/208:48:042
426352991715,1cyclictest0-21swapper/208:09:312
42635099178,6cyclictest485330-21cron09:55:001
42635099172,9cyclictest439497-21apt-get07:45:031
42635099172,12cyclictest489086-21apt10:05:121
426350991711,5cyclictest453574-21ls08:20:211
42634799179,6cyclictest883-21dbus-daemon11:45:000
426347991715,1cyclictest0-21swapper/010:20:330
426347991715,1cyclictest0-21swapper/010:20:320
426347991712,4cyclictest13-21ksoftirqd/008:45:000
426347991712,3cyclictest434667-21grep07:30:130
426347991710,6cyclictest484722-21sshd09:50:210
42635599168,6cyclictest42-21ksoftirqd/310:35:123
426355991614,1cyclictest3056-21PK-Backend11:26:223
426355991614,1cyclictest3056-21PK-Backend07:25:183
426355991613,2cyclictest3056-21PK-Backend09:40:003
426355991613,2cyclictest3056-21PK-Backend09:22:063
42635599160,4cyclictest436085-21latency_hist07:35:013
42635299169,5cyclictest512852-21ntpq11:10:182
42635299169,5cyclictest512852-21ntpq11:10:182
42635299168,6cyclictest480132-21cron09:40:002
42635299168,6cyclictest436713-21cat07:35:152
426352991614,1cyclictest0-21swapper/212:37:122
426352991614,1cyclictest0-21swapper/207:34:092
426352991614,1cyclictest0-21swapper/207:29:502
426352991611,4cyclictest427898-21vmstat07:10:222
42635099168,5cyclictest454547-21cpuspeed_turbos08:25:121
426350991612,3cyclictest1095-21runrttasks12:02:121
426347991614,1cyclictest0-21swapper/010:07:480
426347991612,3cyclictest533450-21tr12:10:110
426347991612,3cyclictest478728-21cpu09:35:120
426347991611,3cyclictest540978-21kernelversion12:30:150
426347991611,3cyclictest442314-21grep07:50:160
426347991610,5cyclictest496884-21cat10:25:170
42635599159,4cyclictest490548-21apt10:10:003
426355991513,1cyclictest3056-21PK-Backend08:32:173
426355991513,1cyclictest3056-21PK-Backend08:23:323
42635299158,5cyclictest456924-21open_inodes08:30:202
42635299158,5cyclictest436-20systemd-journal11:35:002
42635299154,2cyclictest141rcu_preempt07:15:202
42635299154,2cyclictest141rcu_preempt07:15:202
426352991513,1cyclictest0-21swapper/210:55:202
426352991511,3cyclictest494756-21cstates10:20:142
426352991511,3cyclictest494756-21cstates10:20:132
42635299150,2cyclictest141rcu_preempt11:35:092
426350991510,4cyclictest3056-21PK-Backend12:20:001
42634799159,5cyclictest526626-21cat11:50:150
42634799149,4cyclictest535196-21tr12:15:110
426347991412,1cyclictest3056-21PK-Backend10:35:390
42635599138,4cyclictest3056-21PK-Backend08:41:213
42635299139,3cyclictest978-21snmpd10:15:362
42635299139,3cyclictest1095-21runrttasks09:45:262
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional