You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-22 - 06:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Mon Jun 22, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
370729721300,2sleep33056-21PK-Backend22:00:223
371888621140,4sleep13056-21PK-Backend22:35:171
371534021020,3sleep00-21swapper/022:25:130
371534021020,3sleep00-21swapper/022:25:130
139099890,58rtkit-daemon0-21swapper/319:07:453
364456727347,17sleep10-21swapper/119:05:131
364453827240,23sleep00-21swapper/019:05:120
364571026443,14sleep20-21swapper/219:06:092
37400642540,15sleep30-21swapper/323:35:183
37413952340,2sleep13741396-21cpuspeed_turbos23:40:121
37050162280,12sleep126-21ksoftirqd/121:55:151
3646194992517,6cyclictest436-20systemd-journal20:40:003
3646188992516,6cyclictest436-20systemd-journal21:44:591
37116212240,2chrt0-21swapper/322:15:133
364619099240,22cyclictest978-21snmpd23:31:282
3646188992415,6cyclictest3725083-21cron22:55:001
3646188992410,12cyclictest3723398-21unin-run22:50:001
139099240,17rtkit-daemon3056-21PK-Backend19:38:283
139099240,16rtkit-daemon3056-21PK-Backend21:20:193
364619099230,3cyclictest3687349-21df_inode21:05:152
3646188992316,4cyclictest436-20systemd-journal23:09:561
3646184992315,6cyclictest3682713-21fwupdmgr20:51:440
3646194992214,6cyclictest3713611-21diskstats22:20:143
3646194992211,9cyclictest3734054-21unin-run23:20:003
364619499220,20cyclictest3056-21PK-Backend19:46:313
3646190992220,1cyclictest0-21swapper/221:47:122
3646190992220,1cyclictest0-21swapper/221:32:312
3646190992220,1cyclictest0-21swapper/220:09:182
3646190992220,1cyclictest0-21swapper/219:29:162
3646190992220,1cyclictest0-21swapper/219:29:162
3646190992220,1cyclictest0-21swapper/219:16:112
3646190992220,1cyclictest0-21swapper/200:29:482
3646190992220,1cyclictest0-21swapper/200:12:122
3646190992214,6cyclictest3729458-21df_inode23:05:132
364619099221,2cyclictest0-21swapper/200:00:012
364619099220,20cyclictest0-21swapper/220:47:172
3646188992220,1cyclictest3056-21PK-Backend23:53:031
3646188992212,7cyclictest3709512-21cron22:10:001
3646188992211,9cyclictest3672613-21unin-run20:25:001
3646184992220,1cyclictest0-21swapper/022:57:560
3646184992220,1cyclictest0-21swapper/020:32:440
3646184992220,1cyclictest0-21swapper/020:24:310
3646184992220,1cyclictest0-21swapper/019:34:030
3646184992220,1cyclictest0-21swapper/019:13:080
3646184992220,1cyclictest0-21swapper/000:28:280
364618499220,21cyclictest0-21swapper/022:14:360
3646194992118,2cyclictest3056-21PK-Backend00:36:443
364619499210,20cyclictest3736234-21ls23:25:113
364619499210,20cyclictest3736234-21ls23:25:113
364619499210,20cyclictest0-21swapper/322:12:113
364619499210,19cyclictest3056-21PK-Backend20:52:493
364619499210,19cyclictest3056-21PK-Backend20:18:013
364619499210,19cyclictest3056-21PK-Backend19:58:553
3646190992119,1cyclictest0-21swapper/223:35:202
3646190992119,1cyclictest0-21swapper/223:14:522
3646190992119,1cyclictest0-21swapper/223:04:462
3646190992119,1cyclictest0-21swapper/222:32:282
3646190992119,1cyclictest0-21swapper/221:36:512
3646190992119,1cyclictest0-21swapper/221:26:092
3646190992119,1cyclictest0-21swapper/221:17:112
3646190992119,1cyclictest0-21swapper/220:37:442
3646190992119,1cyclictest0-21swapper/220:34:532
3646190992119,1cyclictest0-21swapper/220:25:322
3646190992119,1cyclictest0-21swapper/220:25:322
3646190992119,1cyclictest0-21swapper/219:45:052
3646190992113,6cyclictest3712274-21awk22:15:212
3646190992112,7cyclictest3748550-21df_inode00:00:122
364619099210,20cyclictest0-21swapper/223:46:092
364619099210,20cyclictest0-21swapper/221:41:112
364619099210,20cyclictest0-21swapper/219:41:332
364619099210,20cyclictest0-21swapper/200:33:402
364618899219,9cyclictest1-21systemd23:49:481
3646188992119,1cyclictest3056-21PK-Backend23:20:031
3646188992119,1cyclictest3056-21PK-Backend00:29:531
3646188992119,1cyclictest0-21swapper/123:14:091
3646188992119,1cyclictest0-21swapper/123:00:031
3646188992119,1cyclictest0-21swapper/122:01:081
3646188992119,1cyclictest0-21swapper/121:20:201
3646188992119,1cyclictest0-21swapper/119:54:551
3646188992119,1cyclictest0-21swapper/119:45:291
3646188992119,1cyclictest0-21swapper/119:44:401
3646188992118,2cyclictest3056-21PK-Backend23:58:371
3646188992118,2cyclictest3056-21PK-Backend20:36:591
3646188992111,7cyclictest883-21dbus-daemon20:34:591
364618899210,20cyclictest3056-21PK-Backend19:22:571
364618899210,19cyclictest3056-21PK-Backend23:36:051
364618899210,19cyclictest3056-21PK-Backend19:14:541
364618899210,19cyclictest3056-21PK-Backend00:18:161
3646184992119,1cyclictest0-21swapper/023:58:060
3646184992119,1cyclictest0-21swapper/023:39:130
3646184992119,1cyclictest0-21swapper/023:22:090
3646184992119,1cyclictest0-21swapper/022:37:480
3646184992119,1cyclictest0-21swapper/022:30:040
3646184992119,1cyclictest0-21swapper/021:56:320
3646184992119,1cyclictest0-21swapper/021:54:500
3646184992119,1cyclictest0-21swapper/021:32:260
3646184992119,1cyclictest0-21swapper/019:58:230
3646184992119,1cyclictest0-21swapper/019:40:180
3646184992119,1cyclictest0-21swapper/019:28:010
3646184992119,1cyclictest0-21swapper/019:28:000
3646184992119,1cyclictest0-21swapper/019:16:190
3646184992119,1cyclictest0-21swapper/000:37:320
3646184992114,5cyclictest13-21ksoftirqd/020:10:160
3646184992113,6cyclictest3686974-21awk21:05:110
364618499210,20cyclictest37215892sleep022:44:380
364618499210,20cyclictest0-21swapper/021:15:350
364618499210,20cyclictest0-21swapper/021:02:450
364618499210,20cyclictest0-21swapper/020:47:230
364618499210,20cyclictest0-21swapper/020:43:310
364618499210,20cyclictest0-21swapper/020:19:240
139099210,14rtkit-daemon0-21swapper/223:50:232
36838582200,11sleep20-21swapper/220:55:122
364619499202,16cyclictest3720818-21grep22:40:173
3646194992018,1cyclictest3056-21PK-Backend21:14:323
3646194992017,2cyclictest3056-21PK-Backend22:47:203
3646194992017,2cyclictest3056-21PK-Backend20:12:343
3646194992012,7cyclictest3691926-21basename21:19:593
3646194992012,6cyclictest3739225-21cron23:35:003
364619499200,19cyclictest3056-21PK-Backend22:54:443
364619499200,19cyclictest0-21swapper/323:58:373
364619499200,19cyclictest0-21swapper/322:32:323
364619499200,19cyclictest0-21swapper/300:04:123
364619499200,18cyclictest3056-21PK-Backend23:45:323
364619499200,18cyclictest3056-21PK-Backend23:02:543
364619499200,18cyclictest3056-21PK-Backend21:59:343
364619499200,18cyclictest3056-21PK-Backend21:41:303
364619499200,18cyclictest3056-21PK-Backend21:30:143
364619499200,18cyclictest3056-21PK-Backend20:08:153
364619499200,18cyclictest3056-21PK-Backend19:29:203
364619499200,18cyclictest3056-21PK-Backend19:29:203
3646190992018,1cyclictest3056-21PK-Backend20:10:262
3646190992018,1cyclictest0-21swapper/222:48:562
3646190992018,1cyclictest0-21swapper/222:12:072
3646190992018,1cyclictest0-21swapper/222:07:132
3646190992018,1cyclictest0-21swapper/221:55:292
3646190992018,1cyclictest0-21swapper/221:53:482
3646190992018,1cyclictest0-21swapper/221:20:212
3646190992018,1cyclictest0-21swapper/220:40:072
3646190992018,1cyclictest0-21swapper/220:17:022
3646190992018,1cyclictest0-21swapper/219:52:552
3646190992018,1cyclictest0-21swapper/200:38:552
3646190992013,5cyclictest3647820-21cron19:15:002
364619099200,1cyclictest0-21swapper/219:55:522
364619099200,19cyclictest0-21swapper/220:02:212
364619099200,19cyclictest0-21swapper/200:20:432
3646188992018,1cyclictest3056-21PK-Backend23:33:361
3646188992018,1cyclictest3056-21PK-Backend22:26:091
3646188992018,1cyclictest3056-21PK-Backend22:26:091
3646188992018,1cyclictest3056-21PK-Backend19:18:381
3646188992018,1cyclictest0-21swapper/121:32:291
3646188992018,1cyclictest0-21swapper/121:08:221
3646188992018,1cyclictest0-21swapper/120:49:511
3646188992018,1cyclictest0-21swapper/120:14:351
3646188992018,1cyclictest0-21swapper/119:25:491
3646188992018,1cyclictest0-21swapper/119:25:491
3646188992018,1cyclictest0-21swapper/100:35:061
3646188992018,1cyclictest0-21swapper/100:07:151
3646188992017,2cyclictest3056-21PK-Backend22:12:041
3646188992017,2cyclictest3056-21PK-Backend00:01:061
3646188992017,2cyclictest0-21swapper/100:10:331
364618899200,1cyclictest3056-21PK-Backend22:15:491
364618899200,19cyclictest3056-21PK-Backend22:31:031
364618899200,19cyclictest0-21swapper/121:47:441
364618899200,19cyclictest0-21swapper/121:25:331
3646184992018,1cyclictest3056-21PK-Backend00:10:490
3646184992018,1cyclictest0-21swapper/023:41:090
3646184992018,1cyclictest0-21swapper/023:32:170
3646184992018,1cyclictest0-21swapper/022:21:470
3646184992018,1cyclictest0-21swapper/021:36:450
3646184992018,1cyclictest0-21swapper/000:05:020
364618499200,19cyclictest0-21swapper/023:50:350
364618499200,19cyclictest0-21swapper/021:47:050
364618499200,19cyclictest0-21swapper/021:26:040
364618499200,19cyclictest0-21swapper/020:37:510
364618499200,19cyclictest0-21swapper/019:52:280
364618499200,19cyclictest0-21swapper/019:47:200
364618499200,19cyclictest0-21swapper/019:38:090
364618499200,19cyclictest0-21swapper/000:21:170
364618499200,19cyclictest0-21swapper/000:16:030
364618499200,18cyclictest3700844-21munin-run21:44:590
139099200,14rtkit-daemon3056-21PK-Backend19:53:243
139099200,13rtkit-daemon3056-21PK-Backend23:09:423
3646194991917,1cyclictest3056-21PK-Backend20:48:563
3646194991917,1cyclictest0-21swapper/320:29:243
3646194991917,1cyclictest0-21swapper/320:29:243
3646194991917,1cyclictest0-21swapper/300:34:483
3646194991917,1cyclictest0-21swapper/300:16:353
3646194991917,1cyclictest0-21swapper/300:06:083
3646194991912,5cyclictest3686304-21sed21:00:213
364619499190,1cyclictest3056-21PK-Backend21:45:483
364619499190,1cyclictest3056-21PK-Backend20:04:303
364619499190,1cyclictest3056-21PK-Backend19:34:013
364619499190,18cyclictest0-21swapper/320:42:413
364619499190,18cyclictest0-21swapper/319:13:403
364619499190,17cyclictest3719297-21/usr/sbin/munin22:35:203
3646190991917,1cyclictest0-21swapper/223:41:432
3646190991917,1cyclictest0-21swapper/223:19:382
3646190991917,1cyclictest0-21swapper/222:59:252
3646190991917,1cyclictest0-21swapper/221:14:342
3646190991912,5cyclictest3685981-21ntpq21:00:182
3646190991911,6cyclictest3754710-21awk00:15:212
364619099190,18cyclictest0-21swapper/219:30:372
3646188991917,1cyclictest3056-21PK-Backend23:29:581
3646188991917,1cyclictest3056-21PK-Backend23:29:581
3646188991917,1cyclictest3056-21PK-Backend20:43:361
3646188991917,1cyclictest0-21swapper/121:11:001
3646188991917,1cyclictest0-21swapper/120:06:301
3646188991917,1cyclictest0-21swapper/119:58:411
3646188991916,2cyclictest3056-21PK-Backend21:04:161
3646188991916,2cyclictest0-21swapper/120:52:481
3646188991912,5cyclictest26-21ksoftirqd/120:55:011
364618899190,1cyclictest3056-21PK-Backend22:20:431
364618899190,1cyclictest0-21swapper/120:15:101
364618899190,1cyclictest0-21swapper/120:02:451
3646184991917,1cyclictest0-21swapper/023:45:340
3646184991917,1cyclictest0-21swapper/023:15:070
3646184991917,1cyclictest0-21swapper/021:20:500
3646184991917,1cyclictest0-21swapper/019:20:380
3646184991917,1cyclictest0-21swapper/000:00:020
3646184991912,5cyclictest3736325-21cat23:25:110
3646184991912,5cyclictest3736325-21cat23:25:110
364618499190,18cyclictest0-21swapper/022:06:330
139099190,13rtkit-daemon0-21swapper/222:01:222
3646194991816,1cyclictest3056-21PK-Backend21:29:403
3646194991816,1cyclictest3056-21PK-Backend19:41:383
3646194991814,3cyclictest1095-21runrttasks22:25:583
3646194991814,3cyclictest1095-21runrttasks22:25:583
3646194991811,5cyclictest42-21ksoftirqd/320:30:243
3646194991811,5cyclictest3758556-21cron00:30:003
3646194991811,5cyclictest3756094-21ntp_kernel_pll_00:20:173
3646194991811,5cyclictest3685108-21cut21:00:013
3646194991810,6cyclictest3651777-21/usr/sbin/munin19:25:003
364619499180,1cyclictest3056-21PK-Backend23:10:323
364619099188,8cyclictest3736483-21cat23:25:152
364619099188,8cyclictest3736483-21cat23:25:142
3646190991816,1cyclictest0-21swapper/220:50:132
3646190991816,1cyclictest0-21swapper/219:21:052
3646190991816,1cyclictest0-21swapper/200:08:532
3646190991813,4cyclictest3714729-21systemctl22:24:592
3646188991813,2cyclictest3725422-21cpuspeed22:55:111
3646188991812,3cyclictest26-21ksoftirqd/121:15:171
3646188991811,5cyclictest3673559-21ntpq20:25:201
3646188991811,5cyclictest3673559-21ntpq20:25:201
3646188991810,6cyclictest3703703-21python321:50:231
364618899180,1cyclictest3056-21PK-Backend00:24:051
3646184991816,1cyclictest0-21swapper/023:03:500
3646184991816,1cyclictest0-21swapper/022:16:180
3646184991816,1cyclictest0-21swapper/020:59:130
3646184991816,1cyclictest0-21swapper/000:31:460
3646184991813,3cyclictest3673312-21if_err_enp2s020:25:180
3646184991813,3cyclictest3673312-21if_err_enp2s020:25:180
3646184991810,6cyclictest3725100-21cron22:55:010
364618499180,1cyclictest0-21swapper/020:03:370
364619499179,6cyclictest3744703-21apt23:50:113
3646194991715,1cyclictest3056-21PK-Backend21:08:243
3646194991715,1cyclictest3056-21PK-Backend19:17:253
3646194991715,1cyclictest0-21swapper/320:20:173
364619499170,15cyclictest3734685-21hddtemp_smartct23:20:163
3646190991715,1cyclictest0-21swapper/222:25:182
3646190991715,1cyclictest0-21swapper/222:25:172
3646190991710,6cyclictest3734667-21seq23:20:152
3646190991710,5cyclictest3724614-21grep22:50:232
3646190991710,5cyclictest3720335-21grep22:40:122
3646188991710,5cyclictest3656505-21cat19:35:221
3646184991715,1cyclictest0-21swapper/023:09:450
3646184991715,1cyclictest0-21swapper/021:11:290
364619499169,5cyclictest42-21ksoftirqd/322:55:173
364619499169,5cyclictest3703402-21cat21:50:183
3646194991614,1cyclictest0-21swapper/322:08:273
364619499161,13cyclictest3751884-21/usr/sbin/munin00:10:213
3646190991610,5cyclictest3656181-21grep19:35:192
364618899169,6cyclictest3654777-21sed19:30:211
364618899169,5cyclictest26-21ksoftirqd/123:19:591
364618899162,11cyclictest3697588-21cnf-update-db21:35:031
3646188991614,1cyclictest3056-21PK-Backend00:31:141
364618499169,5cyclictest436-20systemd-journal22:05:010
364619099157,6cyclictest3718927-21cut22:35:182
364618499159,5cyclictest13-21ksoftirqd/022:45:180
364618499159,4cyclictest3731886-21sed23:10:220
3646184991513,1cyclictest0-21swapper/020:07:560
364619099149,4cyclictest3671415-21expr20:20:132
3646188991410,3cyclictest3056-21PK-Backend22:40:141
364619499138,4cyclictest3699121-21turbostat21:40:003
3646194991311,1cyclictest3056-21PK-Backend23:42:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional