You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 21:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Jun 14, 2026 12:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
369736121260,4sleep13697363-21latency_hist11:35:011
368893221260,3sleep20-21swapper/211:10:132
364833621030,2sleep33056-21PK-Backend09:15:123
366192021010,4sleep10-21swapper/109:55:011
360298028039,13sleep10-21swapper/107:07:511
360291627255,12sleep00-21swapper/007:07:020
36557912680,4sleep33056-21PK-Backend09:35:173
360243326845,15sleep20-21swapper/207:05:242
360300326344,13sleep30-21swapper/307:08:093
37114412560,9sleep126-21ksoftirqd/112:15:111
36330952550,3sleep33633094-21wc08:30:223
36782722480,2sleep20-21swapper/210:40:132
36887312380,3sleep13056-21PK-Backend11:10:111
36947992340,3sleep20-21swapper/211:25:192
3603291993012,5cyclictest3626996-21sed08:15:012
3603291993010,5cyclictest3638047-21ls08:45:172
3603291993010,5cyclictest3638047-21ls08:45:162
3603291992910,5cyclictest3677687-21dpkg10:40:012
36826912280,4chrt0-21swapper/010:50:210
360329199289,7cyclictest3652085-21cat09:25:162
360329199282,5cyclictest0-21swapper/212:25:102
3603291992810,4cyclictest3627694-21sed08:15:162
139099280,4rtkit-daemon1389-21rtkit-daemon12:30:283
360329199272,7cyclictest3609443-21apt07:25:112
360329199270,12cyclictest3635901-21grep08:40:132
360329099271,7cyclictest3670924-21apt-get10:20:041
3603290992713,3cyclictest3636466-21netstat08:40:171
139099270,4rtkit-daemon1389-21rtkit-daemon12:19:183
139099270,4rtkit-daemon1389-21rtkit-daemon10:48:313
3603295992613,11cyclictest3660286-21awk09:50:113
360329199268,3cyclictest3687861-21timerwakeupswit11:05:222
360329199267,5cyclictest3620898-21cat07:55:222
360329199261,6cyclictest3641189-21apt-get08:55:032
3603291992611,3cyclictest1095-21runrttasks08:22:162
3603291992610,4cyclictest3671566-21sed10:20:162
360329099269,4cyclictest1095-21runrttasks09:20:211
360329099261,8cyclictest1-21systemd10:37:411
3603290992610,4cyclictest3609259-21wc07:25:011
360329199258,3cyclictest3714053-21timerandwakeup12:20:212
360329199252,7cyclictest3630568-21apt08:25:122
360329199250,9cyclictest3659418-21taskset09:45:202
360329199250,11cyclictest3613580-21cut07:35:172
360329099256,3cyclictest0-21swapper/111:55:201
360329099253,7cyclictest3668962-21apt10:15:111
3603290992516,6cyclictest3704139-21cron11:55:011
3603290992513,2cyclictest3613767-21sensors_temp07:35:191
3603290992512,3cyclictest3678050-21apt-config10:40:011
360329099251,10cyclictest3623638-21apt08:05:141
3603290992510,4cyclictest3639653-21basename08:50:151
3603290992510,4cyclictest3630470-21sed08:25:021
139099250,3rtkit-daemon1389-21rtkit-daemon07:10:491
360329599249,13cyclictest3662151-21grep09:55:133
360329199248,3cyclictest3683188-21latency_hist10:55:002
360329199248,3cyclictest0-21swapper/209:50:142
360329199246,5cyclictest3634758-21awk08:35:182
360329199243,3cyclictest3698465-21timerandwakeup11:35:212
360329199240,9cyclictest3701695-21ntp_states11:45:192
360329199240,3cyclictest3607178-21latency_hist07:20:002
360329199240,18cyclictest0-21swapper/210:46:082
360329099248,14cyclictest0-21swapper/111:15:171
360329099241,9cyclictest3700685-21idleruntime-cro11:44:591
360329099240,19cyclictest3056-21PK-Backend12:12:161
360329199237,4cyclictest3604501-21expr07:10:202
360329199231,7cyclictest3662696-21needreboot09:55:182
360329199231,6cyclictest3683365-21apt10:55:102
3603291992310,3cyclictest1095-21runrttasks07:20:022
360329199230,1cyclictest0-21swapper/212:37:412
360329099239,3cyclictest0-21swapper/109:40:171
360329099231,8cyclictest3655952-21python309:35:201
360329099231,6cyclictest3708748-21sshd12:05:201
3603290992316,6cyclictest3056-21PK-Backend08:19:501
360328599230,22cyclictest0-21swapper/009:15:120
36098542220,4sleep10-21swapper/107:25:151
3603295992220,1cyclictest3056-21PK-Backend09:20:323
3603295992217,3cyclictest3686561-21sh11:05:003
360329199228,3cyclictest978-21snmpd11:04:552
360329199228,3cyclictest3657764-21xargs09:40:212
3603291992220,1cyclictest0-21swapper/210:30:192
3603291992217,4cyclictest0-21swapper/209:07:402
3603291992210,3cyclictest886-21NetworkManager07:47:402
3603290992218,3cyclictest3056-21PK-Backend08:56:131
3603290992213,7cyclictest436-20systemd-journal08:05:001
3603285992220,1cyclictest0-21swapper/011:47:270
3603285992220,1cyclictest0-21swapper/011:44:020
3603285992220,1cyclictest0-21swapper/010:48:050
3603285992215,6cyclictest3708747-21python312:05:200
360328599220,20cyclictest0-21swapper/009:44:580
139099220,4rtkit-daemon1389-21rtkit-daemon07:49:030
3603295992118,2cyclictest3056-21PK-Backend07:42:573
3603295992113,6cyclictest3693750-21cron11:25:003
360329599210,5cyclictest141rcu_preempt10:09:273
360329599210,1cyclictest3056-21PK-Backend07:28:183
360329599210,19cyclictest3056-21PK-Backend11:45:033
360329599210,19cyclictest3056-21PK-Backend10:17:573
360329599210,19cyclictest3056-21PK-Backend10:13:453
360329599210,19cyclictest3056-21PK-Backend08:43:073
360329599210,19cyclictest3056-21PK-Backend07:36:083
3603291992119,1cyclictest3056-21PK-Backend07:40:202
3603291992119,1cyclictest0-21swapper/212:16:322
3603291992119,1cyclictest0-21swapper/211:57:392
3603291992119,1cyclictest0-21swapper/211:17:512
3603291992119,1cyclictest0-21swapper/210:17:152
3603291992119,1cyclictest0-21swapper/208:50:422
3603291992119,1cyclictest0-21swapper/207:34:042
3603291992114,6cyclictest3625129-21systemd-run08:10:012
3603291992114,5cyclictest3633555-21sshd08:32:332
360329199210,20cyclictest0-21swapper/212:10:312
360329199210,20cyclictest0-21swapper/211:52:252
360329199210,20cyclictest0-21swapper/210:05:252
360329099215,5cyclictest3675282-21/usr/sbin/munin10:30:181
3603290992119,1cyclictest3056-21PK-Backend08:11:111
3603290992119,1cyclictest0-21swapper/112:00:401
3603290992119,1cyclictest0-21swapper/108:34:501
3603290992119,1cyclictest0-21swapper/107:53:141
360329099210,20cyclictest0-21swapper/111:47:011
3603285992119,1cyclictest3056-21PK-Backend12:35:260
3603285992119,1cyclictest0-21swapper/012:28:230
3603285992119,1cyclictest0-21swapper/012:01:540
3603285992119,1cyclictest0-21swapper/011:31:250
3603285992119,1cyclictest0-21swapper/010:56:300
3603285992119,1cyclictest0-21swapper/008:34:290
3603285992119,1cyclictest0-21swapper/008:26:310
3603285992119,1cyclictest0-21swapper/008:12:180
3603285992119,1cyclictest0-21swapper/007:51:160
3603285992119,1cyclictest0-21swapper/007:25:550
3603285992119,1cyclictest0-21swapper/007:11:000
360328599210,20cyclictest0-21swapper/012:11:060
360328599210,20cyclictest0-21swapper/008:56:190
360328599210,20cyclictest0-21swapper/007:35:200
139099210,18rtkit-daemon945-21thermald07:17:080
3603295992018,1cyclictest3056-21PK-Backend11:35:513
3603295992018,1cyclictest0-21swapper/312:25:463
3603295992018,1cyclictest0-21swapper/311:30:523
3603295992018,1cyclictest0-21swapper/308:51:113
3603295992018,1cyclictest0-21swapper/308:14:073
3603295992012,6cyclictest3684227-21python310:55:183
360329599200,1cyclictest3056-21PK-Backend11:58:293
360329599200,19cyclictest0-21swapper/312:21:203
360329599200,19cyclictest0-21swapper/311:09:363
360329599200,19cyclictest0-21swapper/310:54:203
360329599200,19cyclictest0-21swapper/307:24:533
360329599200,18cyclictest3056-21PK-Backend10:00:483
360329599200,18cyclictest3056-21PK-Backend07:30:553
3603291992018,1cyclictest0-21swapper/212:32:272
3603291992018,1cyclictest0-21swapper/211:32:312
3603291992018,1cyclictest0-21swapper/210:25:532
3603291992018,1cyclictest0-21swapper/210:00:242
3603291992018,1cyclictest0-21swapper/209:13:282
360329199200,1cyclictest0-21swapper/209:23:412
360329199200,19cyclictest3056-21PK-Backend12:00:162
360329199200,19cyclictest0-21swapper/212:07:532
360329199200,19cyclictest0-21swapper/211:23:052
360329199200,19cyclictest0-21swapper/208:00:332
360329199200,19cyclictest0-21swapper/207:52:152
360329099207,9cyclictest436-20systemd-journal10:00:001
360329099202,7cyclictest3692256-21apt-get11:20:051
3603290992018,1cyclictest3056-21PK-Backend12:30:271
3603290992018,1cyclictest0-21swapper/111:25:111
3603290992018,1cyclictest0-21swapper/107:30:151
3603290992014,5cyclictest0-21swapper/109:19:051
3603290992014,4cyclictest3720024-21idleruntime-cro12:39:591
3603290992011,7cyclictest1-21systemd07:43:331
360329099200,1cyclictest3056-21PK-Backend11:06:321
360329099200,1cyclictest3056-21PK-Backend10:50:241
360329099200,1cyclictest3056-21PK-Backend09:05:191
360329099200,1cyclictest0-21swapper/110:55:111
360329099200,19cyclictest3056-21PK-Backend09:04:311
360329099200,19cyclictest0-21swapper/108:25:171
360329099200,19cyclictest0-21swapper/107:55:021
360329099200,10cyclictest36731672awk10:25:191
3603285992018,1cyclictest0-21swapper/011:50:040
3603285992018,1cyclictest0-21swapper/011:26:250
3603285992018,1cyclictest0-21swapper/011:23:000
3603285992018,1cyclictest0-21swapper/011:13:330
3603285992018,1cyclictest0-21swapper/011:06:440
3603285992018,1cyclictest0-21swapper/011:04:080
3603285992018,1cyclictest0-21swapper/009:53:160
3603285992012,2cyclictest13-21ksoftirqd/009:20:120
3603285992011,2cyclictest13-21ksoftirqd/009:00:200
360328599200,1cyclictest0-21swapper/009:57:430
360328599200,1cyclictest0-21swapper/009:34:170
360328599200,19cyclictest0-21swapper/012:18:430
360328599200,19cyclictest0-21swapper/010:40:420
360328599200,19cyclictest0-21swapper/010:15:400
360328599200,19cyclictest0-21swapper/010:03:180
360328599200,19cyclictest0-21swapper/009:36:060
360328599200,19cyclictest0-21swapper/009:28:160
360328599200,19cyclictest0-21swapper/008:53:080
360328599200,19cyclictest0-21swapper/008:24:280
360328599200,19cyclictest0-21swapper/007:42:090
360328599200,19cyclictest0-21swapper/007:30:060
139099200,14rtkit-daemon0-21swapper/309:31:093
36379642190,4chrt3056-21PK-Backend08:45:161
36379642190,4chrt3056-21PK-Backend08:45:161
3603295991917,1cyclictest3056-21PK-Backend12:04:173
3603295991917,1cyclictest3056-21PK-Backend11:25:503
3603295991917,1cyclictest3056-21PK-Backend11:19:013
3603295991917,1cyclictest3056-21PK-Backend10:21:223
3603295991917,1cyclictest0-21swapper/309:47:283
3603295991917,1cyclictest0-21swapper/309:25:333
3603295991917,1cyclictest0-21swapper/309:07:203
3603295991917,1cyclictest0-21swapper/308:25:443
3603295991917,1cyclictest0-21swapper/308:16:573
3603295991917,1cyclictest0-21swapper/308:09:273
3603295991916,2cyclictest3056-21PK-Backend10:42:443
3603295991916,2cyclictest3056-21PK-Backend10:25:343
3603295991916,2cyclictest3056-21PK-Backend07:53:523
3603295991916,1cyclictest3056-21PK-Backend11:40:033
3603295991914,3cyclictest3641339-21wc08:55:113
3603295991914,3cyclictest3620773-21python307:55:223
3603295991911,6cyclictest3718559-21apt12:35:123
3603295991910,7cyclictest3704149-21cron11:55:003
360329599190,5cyclictest141rcu_preempt07:47:243
360329599190,1cyclictest0-21swapper/308:48:483
360329599190,1cyclictest0-21swapper/308:48:483
3603291991917,1cyclictest0-21swapper/210:14:382
3603291991917,1cyclictest0-21swapper/209:34:092
360329199190,1cyclictest0-21swapper/209:38:352
360329199190,1cyclictest0-21swapper/209:00:152
360329099192,12cyclictest3667231-21apt10:10:111
3603290991917,1cyclictest3056-21PK-Backend12:27:031
3603290991914,3cyclictest3665216-21cron10:05:001
360329099190,1cyclictest3056-21PK-Backend12:21:161
3603285991917,1cyclictest0-21swapper/012:30:130
3603285991917,1cyclictest0-21swapper/012:23:090
3603285991917,1cyclictest0-21swapper/009:46:000
3603285991917,1cyclictest0-21swapper/008:18:460
3603285991912,5cyclictest3636022-21expr08:40:120
3603285991912,2cyclictest13-21ksoftirqd/008:39:590
3603285991911,6cyclictest3672702-21grep10:25:130
3603285991911,2cyclictest13-21ksoftirqd/011:55:120
3603285991911,2cyclictest13-21ksoftirqd/011:35:120
3603285991910,3cyclictest13-21ksoftirqd/008:00:240
360328599190,1cyclictest0-21swapper/010:30:080
139099190,13rtkit-daemon3056-21PK-Backend08:39:023
3603295991816,1cyclictest0-21swapper/309:42:013
3603295991816,1cyclictest0-21swapper/308:20:493
360329599181,15cyclictest3710695-21users12:10:213
3603295991811,5cyclictest3689006-21cat11:10:143
360329599181,15cyclictest3606437-21ntpq07:15:183
3603295991810,6cyclictest3622380-21ntp_states08:00:213
360329599180,16cyclictest3677671-21munin-run10:40:023
360329199180,17cyclictest0-21swapper/211:44:252
3603290991812,4cyclictest3647890-21idleruntime-cro09:15:011
3603290991811,5cyclictest3617462-21cat07:45:231
3603285991816,1cyclictest0-21swapper/007:57:320
3603285991811,6cyclictest3691356-21sed11:15:210
360328599181,13cyclictest3670436-21apt-get10:20:030
360328599180,16cyclictest3646168-21basename09:10:000
3603295991715,1cyclictest3056-21PK-Backend12:08:303
360329599171,4cyclictest3643233-21cstates09:00:133
360329599171,4cyclictest141rcu_preempt10:35:013
360329099179,6cyclictest3651632-21grep09:25:111
360329099173,12cyclictest3635599-21latency_hist08:40:001
3603290991715,1cyclictest3056-21PK-Backend09:47:101
3603290991715,1cyclictest0-21swapper/110:09:471
3603290991712,3cyclictest3679884-21acpi10:45:111
3603290991710,5cyclictest3699153-21ls11:40:001
3603285991715,1cyclictest0-21swapper/008:07:380
3603285991712,3cyclictest3646870-21if_enp2s009:10:170
3603285991711,5cyclictest3607756-21ls07:20:140
3603285991710,5cyclictest883-21dbus-daemon08:48:030
3603285991710,5cyclictest883-21dbus-daemon08:48:030
3603285991710,5cyclictest3668680-21cron10:15:000
3603285991710,5cyclictest3666234-21ntpq10:05:220
139099170,13rtkit-daemon1389-21rtkit-daemon07:15:091
3603295991613,2cyclictest3056-21PK-Backend07:13:163
3603291991614,1cyclictest0-21swapper/209:16:032
3603285991614,1cyclictest0-21swapper/010:35:080
3603295991510,4cyclictest3056-21PK-Backend09:14:083
3603290991513,1cyclictest3056-21PK-Backend09:33:171
3603290991412,1cyclictest3056-21PK-Backend11:04:561
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional