You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-08 - 05:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri May 08, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
344309521240,3sleep00-21swapper/000:10:150
333560727439,9sleep00-21swapper/019:08:370
333462827444,21sleep30-21swapper/319:05:153
333559426343,14sleep20-21swapper/219:08:272
333545526344,13sleep10-21swapper/119:06:411
34284912470,3sleep13056-21PK-Backend23:30:141
34040492450,2sleep10-21swapper/122:20:171
33601652440,4sleep13056-21PK-Backend20:15:181
34058762410,3sleep33056-21PK-Backend22:25:163
34022332310,1sleep13402225-21latency22:15:191
34022332310,1sleep13402225-21latency22:15:191
3335852993121,8cyclictest1-21systemd19:32:261
333585499309,6cyclictest3377539-21wc21:05:172
333585499299,6cyclictest3428475-21expr23:30:142
333585499299,5cyclictest3364016-21ls20:25:212
33940622280,5sleep20-21swapper/221:50:222
3335854992710,4cyclictest3357424-21dpkg20:10:012
139099270,4rtkit-daemon1389-21rtkit-daemon21:54:410
3335857992617,6cyclictest3369727-21perf20:45:003
3335854992613,3cyclictest3438694-21latency_hist00:00:012
139099260,4rtkit-daemon1389-21rtkit-daemon21:14:430
3335854992511,12cyclictest3385877-21unin-run21:30:002
333585499250,1cyclictest0-21swapper/223:04:552
333584899251,5cyclictest3346115-21needreboot19:35:160
333585499240,23cyclictest0-21swapper/222:37:312
33779002230,4sleep13056-21PK-Backend21:05:221
3335857992321,1cyclictest3056-21PK-Backend20:48:473
3335857992320,2cyclictest3056-21PK-Backend22:44:053
3335857992315,5cyclictest3377792-21python321:05:213
3335854992318,4cyclictest0-21swapper/200:02:142
3335854992317,5cyclictest0-21swapper/223:06:582
3335852992314,7cyclictest436-20systemd-journal20:50:001
34066592220,4sleep23406647-21munin-run22:30:002
333585799228,3cyclictest3367999-21ls20:40:013
3335857992220,1cyclictest0-21swapper/323:38:393
3335857992220,1cyclictest0-21swapper/321:29:293
3335857992219,2cyclictest3056-21PK-Backend21:49:443
3335857992212,7cyclictest3410118-21cron22:40:003
333585799220,21cyclictest0-21swapper/321:23:003
333585799220,20cyclictest3056-21PK-Backend21:01:383
3335854992220,1cyclictest0-21swapper/223:48:142
3335854992220,1cyclictest0-21swapper/222:49:412
3335854992220,1cyclictest0-21swapper/220:50:442
3335854992220,1cyclictest0-21swapper/219:20:002
3335854992220,1cyclictest0-21swapper/200:12:072
3335854992215,1cyclictest0-21swapper/220:58:352
333585499221,10cyclictest3436347-21python323:50:192
3335852992220,1cyclictest0-21swapper/119:58:501
3335848992220,1cyclictest0-21swapper/023:32:080
3335848992220,1cyclictest0-21swapper/020:49:320
3335848992220,1cyclictest0-21swapper/020:41:070
3335848992220,1cyclictest0-21swapper/020:32:500
3335848992220,1cyclictest0-21swapper/019:40:030
139099220,14rtkit-daemon3056-21PK-Backend23:50:351
33420942210,3sleep10-21swapper/119:25:131
33364532210,2sleep30-21swapper/319:10:163
3335857992119,1cyclictest3056-21PK-Backend21:51:333
3335857992119,1cyclictest3056-21PK-Backend21:13:073
3335857992119,1cyclictest0-21swapper/323:59:223
3335857992119,1cyclictest0-21swapper/323:20:283
3335857992119,1cyclictest0-21swapper/322:50:343
3335857992119,1cyclictest0-21swapper/320:22:513
3335857992119,1cyclictest0-21swapper/320:14:063
3335857992119,1cyclictest0-21swapper/300:01:383
3335857992118,2cyclictest3056-21PK-Backend19:37:153
3335857992118,2cyclictest3056-21PK-Backend19:27:083
3335857992112,5cyclictest1095-21runrttasks20:30:203
333585799211,19cyclictest3423142-21/usr/sbin/munin23:15:133
333585799211,19cyclictest3423142-21/usr/sbin/munin23:15:133
333585799211,17cyclictest3399109-21python322:05:203
333585799210,20cyclictest3056-21PK-Backend19:59:393
333585799210,20cyclictest0-21swapper/320:08:253
333585799210,20cyclictest0-21swapper/319:21:333
333585799210,20cyclictest0-21swapper/300:05:583
333585799210,19cyclictest3056-21PK-Backend20:52:113
333585799210,19cyclictest3056-21PK-Backend00:15:513
333585799210,19cyclictest3056-21PK-Backend00:15:513
3335854992119,1cyclictest0-21swapper/223:35:102
3335854992119,1cyclictest0-21swapper/222:58:122
3335854992119,1cyclictest0-21swapper/222:50:422
3335854992119,1cyclictest0-21swapper/222:34:402
3335854992119,1cyclictest0-21swapper/222:05:062
3335854992119,1cyclictest0-21swapper/221:39:312
3335854992119,1cyclictest0-21swapper/221:14:302
3335854992119,1cyclictest0-21swapper/220:41:192
3335854992119,1cyclictest0-21swapper/220:37:052
3335854992119,1cyclictest0-21swapper/220:31:582
3335854992119,1cyclictest0-21swapper/220:19:552
3335854992119,1cyclictest0-21swapper/220:12:112
3335854992119,1cyclictest0-21swapper/219:25:332
3335854992119,1cyclictest0-21swapper/200:16:402
3335854992119,1cyclictest0-21swapper/200:16:402
3335854992116,4cyclictest0-21swapper/200:30:042
3335854992115,4cyclictest978-21snmpd22:12:172
3335854992113,6cyclictest3386823-21ntpq21:30:192
333585499210,20cyclictest0-21swapper/222:18:522
333585499210,20cyclictest0-21swapper/222:18:522
333585499210,20cyclictest0-21swapper/222:04:052
333585499210,20cyclictest0-21swapper/200:25:322
333585499210,20cyclictest0-21swapper/200:23:162
3335852992119,1cyclictest3056-21PK-Backend22:10:101
3335852992119,1cyclictest3056-21PK-Backend19:11:591
3335852992119,1cyclictest0-21swapper/121:17:111
3335852992119,1cyclictest0-21swapper/100:32:261
3335852992118,2cyclictest3056-21PK-Backend20:11:141
3335852992118,2cyclictest0-21swapper/119:50:181
3335852992114,5cyclictest3373107-21sshd20:54:061
333585299210,20cyclictest3056-21PK-Backend22:03:211
333585299210,20cyclictest0-21swapper/121:50:161
333585299210,20cyclictest0-21swapper/120:57:171
333585299210,1cyclictest0-21swapper/100:13:401
3335848992119,1cyclictest0-21swapper/023:03:360
3335848992119,1cyclictest0-21swapper/022:53:150
3335848992119,1cyclictest0-21swapper/022:10:090
3335848992119,1cyclictest0-21swapper/021:42:310
3335848992119,1cyclictest0-21swapper/020:36:550
3335848992119,1cyclictest0-21swapper/020:29:450
3335848992119,1cyclictest0-21swapper/020:22:480
3335848992119,1cyclictest0-21swapper/000:09:130
333584899210,20cyclictest0-21swapper/023:47:030
139099210,15rtkit-daemon3056-21PK-Backend23:11:311
139099210,15rtkit-daemon3056-21PK-Backend22:55:281
139099210,14rtkit-daemon3056-21PK-Backend23:35:111
139099210,14rtkit-daemon3056-21PK-Backend00:06:011
3335857992018,1cyclictest3056-21PK-Backend21:55:593
3335857992018,1cyclictest3056-21PK-Backend19:32:433
3335857992018,1cyclictest0-21swapper/323:49:423
3335857992018,1cyclictest0-21swapper/323:10:423
3335857992018,1cyclictest0-21swapper/323:05:483
3335857992018,1cyclictest0-21swapper/322:56:153
3335857992018,1cyclictest0-21swapper/319:17:143
3335857992017,2cyclictest3056-21PK-Backend22:11:203
3335857992016,2cyclictest3056-21PK-Backend00:30:043
3335857992014,3cyclictest3364526-21turbostat20:30:003
3335857992012,5cyclictest3429112-21python323:30:223
3335857992012,5cyclictest3417499-21grep23:00:123
333585799200,19cyclictest0-21swapper/323:41:443
333585799200,19cyclictest0-21swapper/322:03:023
3335854992018,1cyclictest0-21swapper/221:03:482
3335854992018,1cyclictest0-21swapper/219:51:152
3335854992018,1cyclictest0-21swapper/219:46:292
3335854992018,1cyclictest0-21swapper/219:41:432
3335854992018,1cyclictest0-21swapper/219:23:172
3335854992018,1cyclictest0-21swapper/200:37:422
3335854992015,3cyclictest978-21snmpd23:29:312
3335854992013,5cyclictest3361504-21expr20:20:112
3335854992012,6cyclictest3425588-21awk23:20:222
333585499200,19cyclictest0-21swapper/223:13:402
333585499200,19cyclictest0-21swapper/219:56:022
333585499200,19cyclictest0-21swapper/219:36:562
3335852992018,1cyclictest3056-21PK-Backend23:29:331
3335852992018,1cyclictest0-21swapper/122:42:551
3335852992018,1cyclictest0-21swapper/120:02:211
3335852992018,1cyclictest0-21swapper/119:45:331
3335852992018,1cyclictest0-21swapper/100:19:141
3335852992018,1cyclictest0-21swapper/100:19:141
3335852992017,1cyclictest3056-21PK-Backend21:27:391
3335852992012,6cyclictest3412860-21ntpq22:45:191
333585299200,1cyclictest0-21swapper/122:38:161
333585299200,19cyclictest3056-21PK-Backend23:47:101
333585299200,19cyclictest3056-21PK-Backend23:24:521
333585299200,19cyclictest3056-21PK-Backend20:31:011
333585299200,19cyclictest0-21swapper/122:32:001
333585299200,19cyclictest0-21swapper/100:35:441
3335848992018,1cyclictest0-21swapper/022:26:320
3335848992018,1cyclictest0-21swapper/022:23:410
3335848992018,1cyclictest0-21swapper/022:01:040
3335848992018,1cyclictest0-21swapper/021:38:190
3335848992018,1cyclictest0-21swapper/021:09:400
3335848992018,1cyclictest0-21swapper/019:49:360
3335848992018,1cyclictest0-21swapper/019:27:260
3335848992018,1cyclictest0-21swapper/000:33:330
3335848992018,1cyclictest0-21swapper/000:26:570
3335848992018,1cyclictest0-21swapper/000:18:050
3335848992018,1cyclictest0-21swapper/000:18:050
3335848992018,1cyclictest0-21swapper/000:02:510
3335848992015,3cyclictest3446931-21ntp_states00:20:220
3335848992014,5cyclictest3438725-21systemd-run00:00:010
3335848992014,4cyclictest3408364-21cron22:35:000
333584899200,1cyclictest0-21swapper/023:09:170
333584899200,1cyclictest0-21swapper/020:19:440
333584899200,19cyclictest3056-21PK-Backend20:13:020
333584899200,19cyclictest0-21swapper/023:41:080
333584899200,19cyclictest0-21swapper/023:17:010
333584899200,19cyclictest0-21swapper/023:17:010
333584899200,19cyclictest0-21swapper/023:13:090
333584899200,19cyclictest0-21swapper/022:09:080
333584899200,19cyclictest0-21swapper/021:49:210
333584899200,19cyclictest0-21swapper/020:07:210
333584899200,19cyclictest0-21swapper/020:04:300
333584899200,19cyclictest0-21swapper/019:58:080
333584899200,19cyclictest0-21swapper/019:15:290
139099200,2rtkit-daemon1389-21rtkit-daemon20:56:220
3335857991917,1cyclictest0-21swapper/323:25:353
3335857991917,1cyclictest0-21swapper/320:15:073
333585799190,1cyclictest3056-21PK-Backend22:32:093
333585799190,1cyclictest3056-21PK-Backend21:41:193
333585799190,1cyclictest3056-21PK-Backend21:35:313
333585799190,1cyclictest3056-21PK-Backend19:50:063
333585799190,1cyclictest3056-21PK-Backend00:36:263
333585799190,1cyclictest0-21swapper/323:50:433
333585799190,17cyclictest3387058-21find21:30:223
3335854991917,1cyclictest0-21swapper/223:15:432
3335854991917,1cyclictest0-21swapper/223:15:432
3335854991917,1cyclictest0-21swapper/221:45:322
3335854991917,1cyclictest0-21swapper/200:05:322
3335854991916,2cyclictest0-21swapper/221:57:492
3335854991913,4cyclictest978-21snmpd19:10:302
333585499190,1cyclictest0-21swapper/223:44:092
333585499190,1cyclictest0-21swapper/221:21:462
333585499190,18cyclictest0-21swapper/221:43:432
3335852991917,1cyclictest3056-21PK-Backend22:06:461
3335852991917,1cyclictest3056-21PK-Backend21:20:491
3335852991917,1cyclictest3056-21PK-Backend20:23:451
3335852991917,1cyclictest0-21swapper/121:40:021
3335852991917,1cyclictest0-21swapper/119:44:161
3335852991916,2cyclictest3056-21PK-Backend19:16:521
3335852991913,5cyclictest26-21ksoftirqd/100:20:141
333585299190,1cyclictest3056-21PK-Backend21:35:021
333585299190,1cyclictest3056-21PK-Backend20:06:471
333585299190,1cyclictest3056-21PK-Backend19:20:451
333585299190,1cyclictest3056-21PK-Backend00:01:371
333585299190,1cyclictest0-21swapper/123:44:201
333585299190,18cyclictest0-21swapper/121:57:191
333585299190,18cyclictest0-21swapper/119:39:301
333585299190,17cyclictest3056-21PK-Backend20:37:301
3335848991917,1cyclictest0-21swapper/023:21:070
3335848991917,1cyclictest0-21swapper/021:57:250
3335848991917,1cyclictest0-21swapper/021:34:060
3335848991917,1cyclictest0-21swapper/021:28:530
3335848991917,1cyclictest0-21swapper/000:35:500
3335848991914,4cyclictest3413337-21sshd22:47:100
3335848991912,6cyclictest3426681-21cpuspeed_turbos23:25:130
3335848991912,5cyclictest3436485-21cat23:50:200
3335848991912,5cyclictest3351114-21cat19:50:150
333584899190,1cyclictest0-21swapper/022:43:560
333584899190,1cyclictest0-21swapper/021:03:240
333584899190,1cyclictest0-21swapper/019:20:500
333584899190,18cyclictest0-21swapper/022:59:440
333584899190,18cyclictest0-21swapper/020:52:090
139099190,2rtkit-daemon1389-21rtkit-daemon21:46:291
34483282180,3sleep10-21swapper/100:25:151
3335857991816,1cyclictest3056-21PK-Backend22:24:523
3335857991811,5cyclictest3446521-21cat00:20:143
3335854991816,1cyclictest0-21swapper/219:30:062
333585499180,1cyclictest0-21swapper/221:16:192
3335852991816,1cyclictest3056-21PK-Backend20:27:221
3335852991811,5cyclictest3405260-21sshd22:25:101
333585299180,7cyclictest436-20systemd-journal21:15:001
3335848991816,1cyclictest0-21swapper/023:35:130
3335848991812,4cyclictest3402681-21sed22:15:240
3335848991812,4cyclictest3402681-21sed22:15:240
333584899180,1cyclictest0-21swapper/021:19:330
139099180,15rtkit-daemon0-21swapper/121:00:401
3335857991715,1cyclictest3056-21PK-Backend22:46:563
3335857991715,1cyclictest3056-21PK-Backend21:19:223
3335857991715,1cyclictest3056-21PK-Backend20:58:133
3335857991715,1cyclictest3056-21PK-Backend19:46:343
3335857991715,1cyclictest3056-21PK-Backend19:43:033
3335854991715,1cyclictest0-21swapper/222:43:122
3335854991715,1cyclictest0-21swapper/220:48:072
3335854991715,1cyclictest0-21swapper/220:02:512
3335852991715,1cyclictest3056-21PK-Backend23:57:451
3335852991715,1cyclictest3056-21PK-Backend23:09:321
3335852991715,1cyclictest3056-21PK-Backend20:40:411
3335852991714,2cyclictest3056-21PK-Backend22:54:381
333585299170,1cyclictest3056-21PK-Backend23:15:471
333585299170,1cyclictest3056-21PK-Backend23:15:471
333584899179,6cyclictest3343684-21cat19:30:120
3335848991710,5cyclictest3409659-21grep22:35:220
3335857991614,1cyclictest3056-21PK-Backend00:26:463
3335857991614,1cyclictest3056-21PK-Backend00:11:323
3335852991614,1cyclictest3056-21PK-Backend21:33:271
333585299160,15cyclictest3056-21PK-Backend23:02:291
3335857991513,1cyclictest3056-21PK-Backend20:02:573
333585799150,13cyclictest3056-21PK-Backend22:15:163
333585799150,13cyclictest3056-21PK-Backend22:15:163
333584899158,5cyclictest13-21ksoftirqd/021:20:150
333584899148,4cyclictest3336183-21tr19:10:130
333585499121,10cyclictest3403595-21tr22:20:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional