You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 17:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun May 17, 2026 12:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
33783222260,8sleep0922-21systemd-logind07:10:020
33718221210,9sleep30-21swapper/307:05:263
39624821070,3sleep10-21swapper/109:50:221
33769326838,23sleep20-21swapper/207:09:112
33758226545,14sleep10-21swapper/107:07:471
4111972510,2sleep00-21swapper/010:35:120
3422452480,4sleep03056-21PK-Backend07:20:110
3422452480,4sleep03056-21PK-Backend07:20:110
3744472460,2sleep20-21swapper/208:50:162
3666392430,2sleep10-21swapper/108:30:101
3934462420,3sleep00-21swapper/009:45:040
337907992822,4cyclictest452793-21unin-run12:35:002
33790999260,4cyclictest0-21swapper/307:55:003
337907992620,5cyclictest34-21ksoftirqd/207:29:182
33790999253,5cyclictest345911-21diskstats07:30:133
33790999251,8cyclictest444473-21chrt12:10:153
33790999251,8cyclictest444473-21chrt12:10:143
337909992516,5cyclictest390606-21python309:35:213
139099250,4rtkit-daemon1389-21rtkit-daemon11:59:502
139099250,4rtkit-daemon1389-21rtkit-daemon11:44:282
33790999246,16cyclictest3704752sleep308:40:123
33790999243,6cyclictest345259-21cron07:30:003
337898992413,9cyclictest436-20systemd-journal07:45:000
337907992314,7cyclictest409467-21df_inode10:30:132
337898992321,1cyclictest0-21swapper/011:40:190
337898992315,6cyclictest408842-21/usr/sbin/munin10:30:010
33789899230,3cyclictest420247-21irqstats11:00:160
139099230,4rtkit-daemon1389-21rtkit-daemon09:51:172
3877462220,2sleep00-21swapper/009:30:000
337909992220,1cyclictest0-21swapper/308:13:263
337903992220,1cyclictest0-21swapper/111:36:561
337903992220,1cyclictest0-21swapper/111:33:521
337903992220,1cyclictest0-21swapper/109:09:051
337903992219,2cyclictest414966-21strings10:45:181
33790399220,3cyclictest0-21swapper/109:30:141
33790399220,21cyclictest0-21swapper/110:59:581
33790399220,20cyclictest3056-21PK-Backend12:15:211
337898992220,1cyclictest0-21swapper/012:18:240
337898992220,1cyclictest0-21swapper/012:13:310
337898992220,1cyclictest0-21swapper/012:13:310
337898992220,1cyclictest0-21swapper/010:48:270
337898992220,1cyclictest0-21swapper/007:36:030
337898992213,7cyclictest417467-21sshd10:54:540
4494292210,4chrt0-21swapper/212:25:022
4441882210,2sleep10-21swapper/112:10:131
4441882210,2sleep10-21swapper/112:10:121
3922152210,4sleep13056-21PK-Backend09:40:201
33790999214,6cyclictest42-21ksoftirqd/312:25:103
33790999213,6cyclictest42-21ksoftirqd/310:40:483
33790999213,3cyclictest42-21ksoftirqd/310:40:003
33790999212,7cyclictest42-21ksoftirqd/311:17:003
337909992119,1cyclictest3056-21PK-Backend12:09:303
337909992119,1cyclictest3056-21PK-Backend10:55:163
337909992119,1cyclictest3056-21PK-Backend10:49:213
337909992119,1cyclictest3056-21PK-Backend09:19:163
337909992119,1cyclictest3056-21PK-Backend08:38:003
337909992119,1cyclictest0-21swapper/312:32:213
337909992119,1cyclictest0-21swapper/308:01:303
337909992119,1cyclictest0-21swapper/307:48:313
337909992119,1cyclictest0-21swapper/307:11:143
33790999211,6cyclictest433655-21taskset11:40:063
33790999211,17cyclictest356002-21munin-run08:00:003
33790999210,7cyclictest389264-21taskset09:32:353
33790999210,20cyclictest3056-21PK-Backend10:03:113
33790999210,20cyclictest3056-21PK-Backend09:06:463
33790999210,19cyclictest385142-21python309:20:203
337907992119,1cyclictest0-21swapper/212:21:102
337907992119,1cyclictest0-21swapper/212:09:062
337907992119,1cyclictest0-21swapper/211:39:192
337907992119,1cyclictest0-21swapper/211:31:082
337907992119,1cyclictest0-21swapper/210:24:362
337907992119,1cyclictest0-21swapper/210:03:412
337907992119,1cyclictest0-21swapper/208:01:192
337907992117,3cyclictest453543-21latency12:35:152
33790799210,20cyclictest0-21swapper/210:42:132
33790799210,20cyclictest0-21swapper/210:16:112
33790799210,1cyclictest0-21swapper/207:50:112
337903992119,1cyclictest3056-21PK-Backend07:55:391
337903992119,1cyclictest3056-21PK-Backend07:43:071
337903992119,1cyclictest0-21swapper/112:07:381
337903992119,1cyclictest0-21swapper/109:57:391
337903992119,1cyclictest0-21swapper/108:36:001
337903992118,2cyclictest3056-21PK-Backend07:12:111
337903992113,6cyclictest436-20systemd-journal08:25:001
337903992113,6cyclictest436-20systemd-journal08:25:001
33790399210,20cyclictest3056-21PK-Backend12:00:561
33790399210,20cyclictest3056-21PK-Backend07:21:381
33790399210,20cyclictest3056-21PK-Backend07:21:381
33790399210,20cyclictest0-21swapper/110:50:111
33790399210,20cyclictest0-21swapper/110:17:541
33790399210,20cyclictest0-21swapper/108:57:371
33790399210,19cyclictest3056-21PK-Backend07:49:101
337898992119,1cyclictest0-21swapper/012:32:160
337898992119,1cyclictest0-21swapper/012:05:460
337898992119,1cyclictest0-21swapper/011:52:080
337898992119,1cyclictest0-21swapper/011:17:140
337898992119,1cyclictest0-21swapper/011:09:150
337898992119,1cyclictest0-21swapper/010:17:110
337898992119,1cyclictest0-21swapper/010:13:050
337898992119,1cyclictest0-21swapper/010:13:050
337898992119,1cyclictest0-21swapper/010:08:590
337898992119,1cyclictest0-21swapper/009:55:550
337898992119,1cyclictest0-21swapper/009:52:510
337898992119,1cyclictest0-21swapper/009:39:320
337898992119,1cyclictest0-21swapper/007:58:000
337898992119,1cyclictest0-21swapper/007:25:350
337898992119,1cyclictest0-21swapper/007:11:020
33789899210,20cyclictest0-21swapper/011:59:510
33789899210,20cyclictest0-21swapper/011:34:090
33789899210,20cyclictest0-21swapper/011:26:120
33789899210,20cyclictest0-21swapper/010:23:200
33789899210,20cyclictest0-21swapper/008:46:320
33789899210,19cyclictest371859-21munin-run08:45:000
3743412200,3sleep10-21swapper/108:50:161
33790999203,6cyclictest42-21ksoftirqd/308:46:173
33790999203,5cyclictest411rcuc/311:10:283
33790999203,5cyclictest411rcuc/311:10:273
33790999202,5cyclictest365930-21cat08:25:223
337909992018,1cyclictest3056-21PK-Backend11:01:093
337909992018,1cyclictest3056-21PK-Backend09:55:133
337909992018,1cyclictest3056-21PK-Backend08:32:403
337909992018,1cyclictest0-21swapper/311:47:553
337909992018,1cyclictest0-21swapper/308:18:203
337909992018,1cyclictest0-21swapper/307:21:413
337909992018,1cyclictest0-21swapper/307:21:413
33790999201,6cyclictest42-21ksoftirqd/310:06:043
337909992012,6cyclictest382270-21cron09:14:593
33790999200,1cyclictest3056-21PK-Backend11:21:453
33790999200,19cyclictest3056-21PK-Backend11:06:033
33790999200,19cyclictest3056-21PK-Backend10:23:253
33790999200,19cyclictest3056-21PK-Backend09:52:093
33790999200,19cyclictest0-21swapper/312:15:263
33790999200,19cyclictest0-21swapper/311:57:403
33790999200,19cyclictest0-21swapper/311:39:423
33790999200,19cyclictest0-21swapper/308:05:083
33790999200,19cyclictest0-21swapper/307:18:583
33790999200,19cyclictest0-21swapper/307:18:583
33790799204,3cyclictest141rcu_preempt08:29:592
337907992018,1cyclictest0-21swapper/212:12:102
337907992018,1cyclictest0-21swapper/212:12:102
337907992018,1cyclictest0-21swapper/211:05:332
337907992018,1cyclictest0-21swapper/210:58:232
337907992018,1cyclictest0-21swapper/208:43:042
337907992018,1cyclictest0-21swapper/208:16:212
337907992015,4cyclictest34-21ksoftirqd/211:54:292
337907992015,3cyclictest34-21ksoftirqd/208:15:012
33790799200,1cyclictest0-21swapper/208:47:372
33790799200,19cyclictest3056-21PK-Backend10:35:182
33790799200,19cyclictest0-21swapper/212:18:052
33790799200,19cyclictest0-21swapper/212:02:112
33790799200,19cyclictest0-21swapper/211:00:262
33790799200,19cyclictest0-21swapper/210:54:172
33790799200,19cyclictest0-21swapper/209:40:572
33790799200,19cyclictest0-21swapper/209:26:292
33790799200,19cyclictest0-21swapper/209:00:342
33790799200,19cyclictest0-21swapper/208:36:152
33790799200,19cyclictest0-21swapper/208:33:582
33790799200,19cyclictest0-21swapper/207:42:552
33790799200,19cyclictest0-21swapper/207:10:502
33790399207,11cyclictest403647-21unin-run10:15:001
33790399207,11cyclictest403647-21unin-run10:15:001
337903992018,1cyclictest3056-21PK-Backend12:32:391
337903992018,1cyclictest3056-21PK-Backend11:58:041
337903992018,1cyclictest3056-21PK-Backend11:43:521
337903992018,1cyclictest3056-21PK-Backend07:30:031
337903992018,1cyclictest0-21swapper/112:37:331
337903992018,1cyclictest0-21swapper/111:16:541
337903992018,1cyclictest0-21swapper/111:08:581
337903992018,1cyclictest0-21swapper/111:02:001
337903992018,1cyclictest0-21swapper/109:18:321
337903992013,5cyclictest26-21ksoftirqd/110:35:121
337903992010,7cyclictest883-21dbus-daemon09:30:001
33790399200,1cyclictest3056-21PK-Backend07:35:571
33790399200,1cyclictest0-21swapper/111:22:491
33790399200,19cyclictest3056-21PK-Backend11:52:231
33790399200,19cyclictest3056-21PK-Backend11:49:321
33790399200,19cyclictest0-21swapper/110:24:021
33790399200,19cyclictest0-21swapper/110:09:421
33790399200,19cyclictest0-21swapper/110:00:431
33790399200,19cyclictest0-21swapper/108:43:381
337898992018,1cyclictest0-21swapper/012:38:120
337898992018,1cyclictest0-21swapper/012:24:180
337898992018,1cyclictest0-21swapper/012:04:450
337898992018,1cyclictest0-21swapper/010:56:250
337898992018,1cyclictest0-21swapper/009:20:130
337898992018,1cyclictest0-21swapper/008:35:380
337898992018,1cyclictest0-21swapper/008:13:210
337898992018,1cyclictest0-21swapper/008:08:410
337898992018,1cyclictest0-21swapper/007:16:160
337898992018,1cyclictest0-21swapper/007:16:150
337898992013,6cyclictest414040-21latency_hist10:45:010
33789899200,1cyclictest0-21swapper/008:00:370
33789899200,19cyclictest0-21swapper/011:14:090
33789899200,19cyclictest0-21swapper/011:14:090
33789899200,19cyclictest0-21swapper/010:03:520
33789899200,19cyclictest0-21swapper/009:30:540
33789899200,19cyclictest0-21swapper/009:12:480
33789899200,19cyclictest0-21swapper/007:51:300
139099200,13rtkit-daemon0-21swapper/008:16:070
33790999198,3cyclictest42-21ksoftirqd/309:03:253
33790999193,5cyclictest42-21ksoftirqd/310:30:203
33790999193,14cyclictest392997-21apt-config09:45:003
33790999192,6cyclictest3056-21PK-Backend10:16:303
337909991917,1cyclictest3056-21PK-Backend11:25:503
337909991917,1cyclictest3056-21PK-Backend10:54:143
337909991917,1cyclictest0-21swapper/309:29:583
337909991917,1cyclictest0-21swapper/307:44:063
33790999191,6cyclictest42-21ksoftirqd/308:56:283
337909991914,3cyclictest393855-21diskstats09:45:133
337909991911,6cyclictest437156-21apt11:50:123
33790999190,1cyclictest3056-21PK-Backend10:25:283
33790999190,1cyclictest3056-21PK-Backend08:52:123
33790799194,3cyclictest141rcu_preempt10:30:002
337907991917,1cyclictest0-21swapper/211:26:012
337907991917,1cyclictest0-21swapper/209:23:112
337907991917,1cyclictest0-21swapper/209:19:552
337907991917,1cyclictest0-21swapper/207:56:532
337907991914,4cyclictest34-21ksoftirqd/207:40:012
33790799190,1cyclictest0-21swapper/211:45:272
33790799190,1cyclictest0-21swapper/208:23:172
33790799190,1cyclictest0-21swapper/208:23:162
33790799190,1cyclictest0-21swapper/208:06:472
33790799190,1cyclictest0-21swapper/207:49:102
33790799190,18cyclictest0-21swapper/209:35:222
33790399195,3cyclictest141rcu_preempt10:43:091
337903991917,1cyclictest3056-21PK-Backend12:22:041
337903991912,5cyclictest429331-21cat11:25:221
33790399190,1cyclictest3056-21PK-Backend08:07:411
33790399190,1cyclictest3056-21PK-Backend07:17:471
33790399190,1cyclictest3056-21PK-Backend07:17:461
33790399190,18cyclictest3056-21PK-Backend09:45:501
337898991917,1cyclictest0-21swapper/011:38:160
337898991917,1cyclictest0-21swapper/009:41:480
337898991917,1cyclictest0-21swapper/008:25:110
337898991917,1cyclictest0-21swapper/008:22:060
337898991917,1cyclictest0-21swapper/008:22:050
337898991911,6cyclictest426703-21apt11:20:120
33789899190,1cyclictest0-21swapper/008:31:190
33789899190,1cyclictest0-21swapper/007:46:020
33789899190,18cyclictest0-21swapper/012:25:200
337909991816,1cyclictest3056-21PK-Backend12:35:273
337909991816,1cyclictest0-21swapper/312:23:233
337909991816,1cyclictest0-21swapper/308:20:233
337909991816,1cyclictest0-21swapper/308:20:223
337909991814,3cyclictest411rcuc/311:32:263
337909991810,6cyclictest347322-21awk07:35:113
33790999180,6cyclictest441858-21runrttasks12:02:113
33790799186,4cyclictest141rcu_preempt11:14:032
33790799186,4cyclictest141rcu_preempt11:14:032
33790799184,2cyclictest141rcu_preempt10:15:002
33790799184,2cyclictest141rcu_preempt10:14:592
337907991816,1cyclictest3056-21PK-Backend09:55:292
337907991816,1cyclictest0-21swapper/210:46:192
337907991816,1cyclictest0-21swapper/207:31:312
337907991813,3cyclictest340280-21kernelversion07:15:042
337907991813,3cyclictest340280-21kernelversion07:15:042
337907991812,5cyclictest376169-21sed08:55:172
33790799180,17cyclictest0-21swapper/209:45:492
33790799180,17cyclictest0-21swapper/209:07:032
337903991816,1cyclictest3056-21PK-Backend08:02:281
337903991816,1cyclictest3056-21PK-Backend07:25:441
337903991812,4cyclictest373589-21cron08:50:011
33790399180,1cyclictest3056-21PK-Backend09:36:361
33789899180,16cyclictest380570-21systemd-run09:10:000
3883922170,4sleep20-21swapper/209:30:142
3755862170,2sleep00-21swapper/008:55:120
33790999170,6cyclictest3056-21PK-Backend10:11:203
33790999170,6cyclictest3056-21PK-Backend10:11:203
337907991712,3cyclictest342897-21processes07:20:202
337907991712,3cyclictest342897-21processes07:20:202
33790799170,1cyclictest0-21swapper/211:22:502
337903991715,1cyclictest3056-21PK-Backend08:18:371
337903991715,1cyclictest0-21swapper/110:30:241
337903991711,5cyclictest378360-21ls09:00:231
337903991711,4cyclictest384925-21grep09:20:161
33789899179,6cyclictest374174-21cat08:50:120
337898991715,1cyclictest0-21swapper/007:34:270
33790799166,2cyclictest141rcu_preempt09:10:192
337907991614,1cyclictest0-21swapper/211:15:462
33790399169,5cyclictest381555-21ntpq09:10:181
337903991614,1cyclictest3056-21PK-Backend08:11:191
337903991614,1cyclictest0-21swapper/108:25:331
337903991613,2cyclictest3056-21PK-Backend07:50:111
337903991610,5cyclictest423336-21sort11:10:151
337903991610,5cyclictest423336-21sort11:10:151
337903991610,4cyclictest407938-21load10:25:171
337898991614,1cyclictest0-21swapper/011:48:160
337898991614,1cyclictest0-21swapper/010:30:420
337898991610,5cyclictest377511-21grep09:00:150
33790799156,8cyclictest401080-21needreboot10:05:172
337903991513,1cyclictest3056-21PK-Backend12:26:581
33789899157,5cyclictest1095-21runrttasks09:15:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional