You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-24 - 17:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Mar 24, 2026 12:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
332987421280,21sleep00-21swapper/007:05:150
344288121220,3sleep3411rcuc/312:20:203
139099890,76rtkit-daemon0-21swapper/107:05:551
333077627050,14sleep30-21swapper/307:07:433
333091426444,14sleep20-21swapper/207:09:282
33774542490,8sleep234-21ksoftirqd/209:15:212
333109799330,31cyclictest0-21swapper/211:25:192
333109799319,7cyclictest436-20systemd-journal09:35:002
333109799310,2cyclictest0-21swapper/210:56:402
3331097993010,5cyclictest3402448-21munin-plugin-st10:30:002
333109799300,1cyclictest0-21swapper/210:40:162
3331097992911,6cyclictest3425896-21pluginstate11:35:012
333109799289,5cyclictest3448225-21cat12:35:152
333109799289,5cyclictest3392623-21expr10:00:132
333109799281,10cyclictest82332-21update-manager11:00:132
34453032250,4sleep13056-21PK-Backend12:30:001
3331101992516,7cyclictest3388539-21cron09:50:003
333109799253,7cyclictest3356882-40unattended-upgr08:18:342
333109799251,22cyclictest3381781-21cat09:30:012
333109799250,19cyclictest0-21swapper/210:48:282
34424252240,4sleep20-21swapper/212:20:142
3331101992419,3cyclictest3402452-21cut10:30:003
333109799243,6cyclictest3335256-21apt07:20:112
333109799241,6cyclictest3426137-21apt11:35:142
139099240,15rtkit-daemon3056-21PK-Backend11:48:591
34449552230,5sleep342-21ksoftirqd/312:26:263
3331097992316,1cyclictest0-21swapper/212:29:332
3331091992318,4cyclictest13-21ksoftirqd/011:36:340
3331101992217,3cyclictest3353872-21df_inode08:10:153
333110199220,21cyclictest0-21swapper/310:34:053
333110199220,20cyclictest3056-21PK-Backend09:19:503
3331097992218,3cyclictest0-21swapper/207:46:042
333109799221,7cyclictest82332-21update-manager12:11:462
333109799221,6cyclictest3338746-21apt07:30:122
3331097992216,1cyclictest0-21swapper/208:02:002
3331097992215,1cyclictest0-21swapper/208:42:482
3331094992220,1cyclictest3056-21PK-Backend09:11:301
3331094992220,1cyclictest0-21swapper/109:21:161
3331094992220,1cyclictest0-21swapper/108:10:331
3331094992219,2cyclictest3056-21PK-Backend09:29:141
3331094992219,2cyclictest3056-21PK-Backend07:40:381
3331094992211,9cyclictest3400722-21unin-run10:24:591
333109499220,20cyclictest3056-21PK-Backend11:52:231
333109499220,20cyclictest0-21swapper/112:35:021
333109199228,12cyclictest1-21systemd11:45:010
3331091992220,1cyclictest0-21swapper/009:46:480
3331091992220,1cyclictest0-21swapper/008:36:450
3331091992220,1cyclictest0-21swapper/007:36:500
3331091992217,4cyclictest13-21ksoftirqd/012:21:490
3331091992217,4cyclictest13-21ksoftirqd/011:10:530
333109199220,3cyclictest3448141-21df12:35:120
139099220,15rtkit-daemon3056-21PK-Backend10:32:551
139099220,15rtkit-daemon3056-21PK-Backend09:50:111
139099220,14rtkit-daemon0-21swapper/108:01:071
3331101992119,1cyclictest3056-21PK-Backend07:58:113
3331101992119,1cyclictest3056-21PK-Backend07:14:383
3331101992119,1cyclictest0-21swapper/310:00:593
3331101992119,1cyclictest0-21swapper/309:02:253
3331101992119,1cyclictest0-21swapper/308:51:113
3331101992119,1cyclictest0-21swapper/308:49:223
3331101992118,2cyclictest3056-21PK-Backend10:06:213
3331101992114,6cyclictest42-21ksoftirqd/311:20:113
333110199211,18cyclictest3414416-21irqstats11:00:163
333110199210,20cyclictest3347486-21turbostat07:55:003
333110199210,20cyclictest3056-21PK-Backend10:10:403
333110199210,20cyclictest0-21swapper/311:31:593
333110199210,20cyclictest0-21swapper/309:53:093
333110199210,20cyclictest0-21swapper/307:38:443
333110199210,1cyclictest0-21swapper/311:50:103
333110199210,19cyclictest3056-21PK-Backend08:25:433
333110199210,19cyclictest3056-21PK-Backend08:05:073
333110199210,19cyclictest3056-21PK-Backend08:00:343
3331097992115,4cyclictest34-21ksoftirqd/211:05:342
3331097992115,1cyclictest0-21swapper/208:20:452
3331097992114,1cyclictest0-21swapper/211:41:412
3331097992113,7cyclictest3436483-21sshd12:04:252
333109799210,20cyclictest0-21swapper/208:14:362
333109799210,19cyclictest0-21swapper/211:57:372
3331094992119,1cyclictest3056-21PK-Backend10:37:421
3331094992119,1cyclictest3056-21PK-Backend09:31:171
3331094992119,1cyclictest3056-21PK-Backend07:21:101
3331094992119,1cyclictest0-21swapper/111:35:411
3331094992119,1cyclictest0-21swapper/111:13:381
3331094992119,1cyclictest0-21swapper/110:10:241
3331094992119,1cyclictest0-21swapper/110:04:501
3331094992119,1cyclictest0-21swapper/109:17:111
3331094992119,1cyclictest0-21swapper/108:25:461
3331094992119,1cyclictest0-21swapper/107:55:181
3331094992118,2cyclictest3056-21PK-Backend11:26:411
3331094992118,2cyclictest3056-21PK-Backend10:47:421
333109499210,19cyclictest3056-21PK-Backend11:59:061
3331091992119,1cyclictest0-21swapper/012:29:320
3331091992119,1cyclictest0-21swapper/011:49:170
3331091992119,1cyclictest0-21swapper/011:28:140
3331091992119,1cyclictest0-21swapper/011:15:100
3331091992119,1cyclictest0-21swapper/010:46:100
3331091992119,1cyclictest0-21swapper/010:11:420
3331091992119,1cyclictest0-21swapper/009:27:000
3331091992119,1cyclictest0-21swapper/009:12:070
3331091992119,1cyclictest0-21swapper/008:43:350
3331091992119,1cyclictest0-21swapper/007:17:370
3331091992118,2cyclictest0-21swapper/011:00:150
333109199210,20cyclictest0-21swapper/012:00:050
333109199210,20cyclictest0-21swapper/010:57:120
333109199210,20cyclictest0-21swapper/010:05:070
333109199210,20cyclictest0-21swapper/007:29:190
333109199210,20cyclictest0-21swapper/007:29:180
139099210,15rtkit-daemon3056-21PK-Backend08:39:361
333110199202,16cyclictest3428736-21python311:40:173
3331101992018,1cyclictest0-21swapper/312:17:273
3331101992018,1cyclictest0-21swapper/312:07:553
3331101992018,1cyclictest0-21swapper/312:02:003
3331101992018,1cyclictest0-21swapper/310:20:343
3331101992018,1cyclictest0-21swapper/309:41:543
3331101992018,1cyclictest0-21swapper/309:12:543
3331101992018,1cyclictest0-21swapper/308:44:083
3331101992018,1cyclictest0-21swapper/307:46:293
333110199201,7cyclictest3434213-21python311:55:203
3331101992015,3cyclictest978-21snmpd10:55:403
3331101992012,6cyclictest3384691-21awk09:35:213
333110199201,17cyclictest3342280-21apt07:40:113
3331101992010,8cyclictest1346-21snapd08:18:353
333110199200,1cyclictest3056-21PK-Backend11:16:373
333110199200,1cyclictest3056-21PK-Backend11:10:433
333110199200,19cyclictest3056-21PK-Backend08:34:413
333110199200,19cyclictest0-21swapper/312:34:313
333110199200,19cyclictest0-21swapper/311:47:063
333110199200,19cyclictest0-21swapper/309:07:393
333110199200,19cyclictest0-21swapper/308:57:133
333110199200,19cyclictest0-21swapper/307:20:403
3331097992018,1cyclictest0-21swapper/209:20:482
3331097992018,1cyclictest0-21swapper/208:25:462
3331097992013,6cyclictest3347405-21sshd07:54:142
3331097992011,7cyclictest3405568-21python310:35:212
333109799200,19cyclictest0-21swapper/210:14:342
333109799200,19cyclictest0-21swapper/207:37:332
3331094992018,1cyclictest3056-21PK-Backend12:06:501
3331094992018,1cyclictest3056-21PK-Backend11:42:371
3331094992018,1cyclictest3056-21PK-Backend10:28:291
3331094992018,1cyclictest3056-21PK-Backend09:47:461
3331094992018,1cyclictest3056-21PK-Backend09:41:041
3331094992018,1cyclictest3056-21PK-Backend08:58:391
3331094992018,1cyclictest3056-21PK-Backend08:48:251
3331094992018,1cyclictest3056-21PK-Backend08:41:351
3331094992018,1cyclictest3056-21PK-Backend07:28:011
3331094992018,1cyclictest3056-21PK-Backend07:28:011
3331094992018,1cyclictest0-21swapper/108:33:101
3331094992018,1cyclictest0-21swapper/108:05:261
3331094992018,1cyclictest0-21swapper/107:30:581
3331094992017,2cyclictest3056-21PK-Backend12:20:021
3331094992017,2cyclictest3056-21PK-Backend07:52:541
333109499200,19cyclictest3056-21PK-Backend09:08:531
333109499200,18cyclictest3056-21PK-Backend12:00:071
333109199203,2cyclictest141rcu_preempt08:24:200
3331091992018,1cyclictest3056-21PK-Backend12:30:330
3331091992018,1cyclictest3056-21PK-Backend09:05:180
3331091992018,1cyclictest0-21swapper/012:16:200
3331091992018,1cyclictest0-21swapper/012:13:290
3331091992018,1cyclictest0-21swapper/011:59:030
3331091992018,1cyclictest0-21swapper/010:24:400
3331091992018,1cyclictest0-21swapper/009:56:270
3331091992018,1cyclictest0-21swapper/009:36:000
3331091992018,1cyclictest0-21swapper/007:42:100
3331091992012,6cyclictest883-21dbus-daemon12:06:420
333109199200,1cyclictest0-21swapper/010:42:040
333109199200,19cyclictest3056-21PK-Backend08:25:090
333109199200,19cyclictest0-21swapper/011:07:120
333109199200,19cyclictest0-21swapper/010:27:440
333109199200,19cyclictest0-21swapper/009:40:530
333109199200,19cyclictest0-21swapper/009:32:550
333109199200,19cyclictest0-21swapper/008:57:260
333109199200,19cyclictest0-21swapper/008:46:120
333109199200,19cyclictest0-21swapper/008:06:100
333109199200,18cyclictest3056-21PK-Backend08:15:020
333109199200,18cyclictest3056-21PK-Backend07:30:060
139099200,13rtkit-daemon3056-21PK-Backend11:24:061
139099200,13rtkit-daemon0-21swapper/008:03:500
3331101991911,6cyclictest3426671-21cat11:35:183
333110199190,1cyclictest3056-21PK-Backend09:21:533
333110199190,1cyclictest0-21swapper/310:39:123
333110199190,1cyclictest0-21swapper/307:31:133
333110199190,18cyclictest0-21swapper/310:53:323
333110199190,18cyclictest0-21swapper/309:31:533
333110199190,18cyclictest0-21swapper/309:29:513
333110199190,17cyclictest3439171-21sed12:10:173
333110199190,17cyclictest3423102-21if_enp2s011:25:183
333110199190,17cyclictest3408169-21apt-get10:45:043
333109799199,8cyclictest3432535-21nm-dispatcher11:51:382
333109799199,8cyclictest2254-21gdbus07:28:072
333109799199,8cyclictest2254-21gdbus07:28:072
3331097991917,1cyclictest0-21swapper/212:30:362
3331097991917,1cyclictest0-21swapper/212:17:092
3331097991917,1cyclictest0-21swapper/210:15:502
3331097991917,1cyclictest0-21swapper/210:05:422
3331097991917,1cyclictest0-21swapper/209:54:342
3331097991917,1cyclictest0-21swapper/209:35:082
3331097991917,1cyclictest0-21swapper/209:07:232
3331097991917,1cyclictest0-21swapper/208:52:292
3331097991917,1cyclictest0-21swapper/208:38:232
3331097991917,1cyclictest0-21swapper/208:08:422
3331097991914,4cyclictest978-21snmpd11:46:282
333109799191,13cyclictest3371177-21apt09:00:132
3331097991910,7cyclictest883-21dbus-daemon11:19:592
333109799190,6cyclictest34006972latency_hist10:25:002
333109799190,1cyclictest0-21swapper/212:05:342
333109799190,1cyclictest0-21swapper/207:43:402
333109799190,1cyclictest0-21swapper/207:18:062
333109799190,18cyclictest0-21swapper/209:47:582
333109799190,18cyclictest0-21swapper/208:34:582
3331094991917,1cyclictest3056-21PK-Backend10:57:411
3331094991917,1cyclictest3056-21PK-Backend10:50:451
3331094991917,1cyclictest3056-21PK-Backend10:40:461
3331094991917,1cyclictest3056-21PK-Backend07:45:591
3331094991917,1cyclictest0-21swapper/111:05:401
3331094991917,1cyclictest0-21swapper/110:08:081
3331094991917,1cyclictest0-21swapper/108:18:371
3331094991913,4cyclictest26-21ksoftirqd/111:30:181
3331094991912,5cyclictest3333667-21tr07:15:131
333109499190,1cyclictest3056-21PK-Backend11:02:361
333109499190,1cyclictest0-21swapper/111:15:401
3331091991917,1cyclictest0-21swapper/010:50:150
3331091991917,1cyclictest0-21swapper/008:30:560
3331091991917,1cyclictest0-21swapper/007:20:200
3331091991912,5cyclictest3377437-21cat09:15:220
3331091991912,5cyclictest3347511-21http07:55:010
3331091991911,6cyclictest3348375-21df07:55:140
333109199190,8cyclictest3425859-21sh11:34:590
333109199190,1cyclictest0-21swapper/010:16:020
333109199190,1cyclictest0-21swapper/007:12:570
333109199190,18cyclictest0-21swapper/009:03:280
139099190,13rtkit-daemon0-21swapper/010:37:530
139099190,13rtkit-daemon0-21swapper/009:22:420
139099190,12rtkit-daemon3056-21PK-Backend10:15:041
3331101991816,1cyclictest3056-21PK-Backend12:36:343
3331101991816,1cyclictest0-21swapper/310:16:143
3331101991816,1cyclictest0-21swapper/308:38:533
3331101991813,3cyclictest3390650-21sort09:55:133
3331101991812,4cyclictest3408175-21latency_hist10:45:013
3331101991812,4cyclictest3358802-21systemd-run08:25:003
333110199180,1cyclictest0-21swapper/307:26:073
333110199180,1cyclictest0-21swapper/307:26:073
3331097991816,1cyclictest0-21swapper/207:11:292
3331094991816,1cyclictest3056-21PK-Backend12:19:011
3331094991816,1cyclictest3056-21PK-Backend09:59:291
3331094991816,1cyclictest3056-21PK-Backend07:39:501
3331094991816,1cyclictest0-21swapper/107:11:181
3331091991816,1cyclictest0-21swapper/008:12:050
3331101991715,1cyclictest3056-21PK-Backend07:16:003
3331101991712,3cyclictest886-21NetworkManager11:05:123
333109799172,14cyclictest1095-21runrttasks10:30:422
3331097991712,3cyclictest3374629-21acpi09:10:112
3331094991714,2cyclictest3056-21PK-Backend08:51:011
3331094991711,4cyclictest3372693-21sed09:05:011
3331091991715,1cyclictest0-21swapper/011:52:210
3331091991715,1cyclictest0-21swapper/007:48:190
333109799168,6cyclictest3410846-21which10:50:162
333109799168,6cyclictest3386126-21pluginstate09:40:192
3331097991614,1cyclictest0-21swapper/211:20:122
3331097991614,1cyclictest0-21swapper/208:56:552
333109799161,12cyclictest2156-21gnome-shell08:45:092
3331097991610,4cyclictest3418033-21grep11:10:182
3331094991610,4cyclictest3446223-21if_enp2s012:30:151
3331091991614,1cyclictest0-21swapper/009:53:100
333109799158,5cyclictest3390873-21expr09:55:132
333109199159,4cyclictest3392724-21awk10:00:140
333109199157,2cyclictest3368355-21needreboot08:50:190
3331091991512,2cyclictest3404171-21sh10:34:590
139099150,3rtkit-daemon0-21swapper/011:22:060
3331094991411,2cyclictest3056-21PK-Backend12:12:321
3331094991411,2cyclictest3056-21PK-Backend08:24:581
3331094991410,3cyclictest3056-21PK-Backend09:39:591
333109799137,4cyclictest0-21swapper/207:55:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional