You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-11 - 14:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Sun Jan 11, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1180822610,3sleep20-21swapper/207:05:172
129292212163,37sleep00-21swapper/007:07:470
131032203167,24sleep10-21swapper/107:10:001
127562203165,26sleep30-21swapper/307:05:323
1811521630,3sleep01319099cyclictest07:18:260
89052670,3sleep00-21swapper/009:25:110
245332530,1sleep30-21swapper/308:45:213
34952490,1sleep20-21swapper/208:00:002
13200993527,7cyclictest0-21swapper/210:48:182
1320799341,29cyclictest309162sleep311:27:543
1319599341,17cyclictest0-21swapper/109:17:531
1319599341,17cyclictest0-21swapper/109:17:531
1319099347,13cyclictest16674-21munin-run08:30:010
13190993410,11cyclictest20125-21basename11:05:010
1319099333,14cyclictest17905-21munin-run11:00:010
1320799320,11cyclictest0-21swapper/311:47:463
1319099327,12cyclictest1260-21munin-run07:55:010
1320799311,23cyclictest0-21swapper/307:36:263
1320799310,24cyclictest0-21swapper/309:57:453
13195993112,3cyclictest0-21swapper/112:30:161
1319599311,14cyclictest0-21swapper/111:45:161
1319099314,12cyclictest0-21swapper/008:05:010
1319099314,12cyclictest0-21swapper/008:05:010
1320799301,26cyclictest0-21swapper/311:11:343
1320799301,12cyclictest0-21swapper/311:30:583
1320799301,11cyclictest0-21swapper/309:06:583
1319599301,10cyclictest2630-21fschecks_time09:10:131
1320799291,13cyclictest0-21swapper/309:24:423
1320799291,10cyclictest0-21swapper/309:10:463
1320799290,23cyclictest0-21swapper/310:30:263
1320799290,22cyclictest0-21swapper/310:08:453
1319599291,12cyclictest0-21swapper/112:10:201
1319099296,11cyclictest23536-21munin-run12:30:000
1319099291,13cyclictest908-21munin-run11:35:010
1320799281,13cyclictest0-21swapper/311:06:023
1320799281,13cyclictest0-21swapper/309:19:223
1320799281,13cyclictest0-21swapper/309:19:223
1320799281,12cyclictest0-21swapper/311:03:143
1320799281,11cyclictest0-21swapper/307:16:143
1320799280,3cyclictest0-21swapper/310:26:063
1320799280,3cyclictest0-21swapper/310:26:053
1320799280,22cyclictest0-21swapper/309:25:383
1319599281,22cyclictest0-21swapper/112:15:401
1320799271,9cyclictest0-21swapper/307:52:583
1320799271,9cyclictest0-21swapper/307:31:383
1320799271,12cyclictest0-21swapper/312:37:543
1320799271,12cyclictest0-21swapper/312:05:373
1320799271,12cyclictest0-21swapper/310:35:423
1320799271,12cyclictest0-21swapper/310:14:453
1320799271,11cyclictest0-21swapper/312:10:573
1320799271,11cyclictest0-21swapper/309:49:093
1320799271,11cyclictest0-21swapper/308:11:013
1320799271,11cyclictest0-21swapper/307:28:143
1320799271,11cyclictest0-21swapper/307:23:103
1320799271,10cyclictest0-21swapper/312:18:333
1320799271,10cyclictest0-21swapper/310:50:343
1320799271,10cyclictest0-21swapper/307:58:093
1320799270,17cyclictest0-21swapper/311:50:343
1320799270,12cyclictest0-21swapper/312:28:053
1320799270,12cyclictest0-21swapper/311:36:063
1320799270,12cyclictest0-21swapper/310:01:533
1320799270,10cyclictest0-21swapper/310:21:333
1320099272,10cyclictest25194-21kworker/2:109:05:062
13195992713,2cyclictest0-21swapper/111:39:591
1319599271,11cyclictest0-21swapper/111:40:471
1319599271,11cyclictest0-21swapper/111:40:471
1320799261,12cyclictest0-21swapper/311:17:023
1320799261,12cyclictest0-21swapper/308:44:263
1320799261,11cyclictest0-21swapper/308:31:013
1320799261,10cyclictest0-21swapper/307:11:093
1320799260,3cyclictest0-21swapper/310:56:023
1320799260,16cyclictest0-21swapper/309:51:333
1320799260,12cyclictest0-21swapper/311:44:463
1320799260,12cyclictest0-21swapper/311:44:463
1320799260,12cyclictest0-21swapper/311:23:463
1320799260,11cyclictest0-21swapper/309:37:223
13200992611,3cyclictest25194-21kworker/2:107:13:302
1319599264,3cyclictest0-21swapper/111:05:271
1319599261,9cyclictest0-21swapper/107:40:541
1319599261,23cyclictest0-21swapper/112:05:221
1319599261,16cyclictest0-21swapper/109:35:121
1319599261,13cyclictest0-21swapper/108:05:261
1319599261,12cyclictest0-21swapper/110:12:441
1319599261,10cyclictest26032-21diskstats08:50:141
1319599261,10cyclictest0-21swapper/109:53:121
1319599260,8cyclictest0-21swapper/109:00:251
1320799251,12cyclictest0-21swapper/308:01:573
1320799251,12cyclictest0-21swapper/308:01:573
1320799251,11cyclictest13563-21ntp_states08:20:193
1320799251,11cyclictest0-21swapper/311:57:103
1320799251,10cyclictest0-21swapper/307:42:143
1320799250,11cyclictest0-21swapper/312:01:053
1320799250,10cyclictest0-21swapper/308:52:343
1319599251,8cyclictest4745-21ntpq08:00:221
1319599251,8cyclictest4745-21ntpq08:00:221
1320799241,9cyclictest0-21swapper/312:30:463
1320799241,9cyclictest0-21swapper/307:46:023
1320799240,9cyclictest0-21swapper/308:37:093
1320799240,11cyclictest0-21swapper/310:16:013
1320099242,8cyclictest25194-21kworker/2:107:36:332
1320099241,5cyclictest0-21swapper/209:38:282
1319599243,6cyclictest0-21swapper/110:22:441
1319599241,9cyclictest0-21swapper/108:38:141
1319599241,6cyclictest0-21swapper/109:40:541
1319599241,11cyclictest0-21swapper/110:35:121
1319599241,11cyclictest0-21swapper/110:02:001
1319599241,10cyclictest0-21swapper/112:27:321
1320799231,11cyclictest0-21swapper/308:19:513
1320799231,10cyclictest0-21swapper/309:41:543
1320799230,9cyclictest0-21swapper/312:23:253
1320799230,9cyclictest0-21swapper/309:02:543
1320799230,10cyclictest0-21swapper/308:25:413
1320099237,3cyclictest500-21kworker/2:009:40:372
1320099236,5cyclictest13706-21unixbench_singl12:00:262
1320099231,6cyclictest0-21swapper/212:29:522
1319599237,2cyclictest0-21swapper/107:50:241
1319599231,9cyclictest0-21swapper/110:45:071
1319599231,9cyclictest0-21swapper/110:29:001
1319599231,9cyclictest0-21swapper/110:29:001
1319599231,8cyclictest0-21swapper/112:03:351
1319599231,17cyclictest0-21swapper/107:33:491
1319099235,15cyclictest15367-21munin-run07:15:020
1319099231,6cyclictest0-21swapper/011:08:100
13190992316,4cyclictest12189-21cut10:45:160
1320799228,3cyclictest0-21swapper/308:56:323
1320799227,3cyclictest0-21swapper/308:05:313
1320799220,11cyclictest0-21swapper/310:42:023
1320799220,10cyclictest0-21swapper/310:45:583
1320099221,6cyclictest0-21swapper/212:38:142
1320099221,6cyclictest0-21swapper/211:48:232
1320099221,6cyclictest0-21swapper/210:34:452
1320099221,5cyclictest0-21swapper/211:28:542
1320099220,12cyclictest311ktimersoftd/210:00:122
1319599225,3cyclictest0-21swapper/108:48:541
1319599221,9cyclictest27136-21kernelversion10:05:181
1319599221,3cyclictest9470-21/usr/sbin/munin08:10:251
1319599221,16cyclictest0-21swapper/107:37:151
1319599221,13cyclictest0-21swapper/107:10:041
13195992210,3cyclictest14064-21df10:50:131
1319599220,9cyclictest11364-21timerwakeupswit11:55:261
1319099226,14cyclictest7-21ksoftirqd/009:25:010
1319099221,19cyclictest0-21swapper/011:37:210
1319099221,12cyclictest5906-21/usr/sbin/munin08:05:170
1320799215,3cyclictest0-21swapper/309:30:583
1320099211,4cyclictest13338-21kworker/u8:011:55:262
1319599217,3cyclictest0-21swapper/108:42:181
1319599215,4cyclictest0-21swapper/111:15:231
1319599211,9cyclictest11583-21hddtemp_smartct09:30:151
1319599211,13cyclictest0-21swapper/110:31:001
1319599211,12cyclictest0-21swapper/107:55:561
1319099219,3cyclictest21907-21/usr/sbin/munin09:55:270
13190992111,7cyclictest13338-21kworker/u8:011:10:230
1319099211,17cyclictest0-21swapper/012:18:550
1319099211,10cyclictest5823-21cat10:30:160
91242200,4sleep230-21rcuc/210:40:022
305732200,4sleep230-21rcuc/210:15:012
269302201,2sleep00-21swapper/008:50:240
1320099207,4cyclictest27197-21expr11:20:122
1320099201,6cyclictest1128-21sshd11:35:072
13200992013,4cyclictest30682-21grep09:00:172
1319599206,4cyclictest0-21swapper/109:22:331
1319599205,8cyclictest1903-21diskmemload09:50:021
1319599201,9cyclictest0-21swapper/107:25:131
1319599201,7cyclictest21969-21kworker/u8:211:20:531
1319599201,4cyclictest16125-21users07:15:041
13195992013,3cyclictest0-21swapper/107:49:171
1319599201,2cyclictest87450irq/129-enp1s0-10:43:381
1319599201,18cyclictest0-21swapper/110:17:141
1319599201,18cyclictest0-21swapper/108:57:381
1319599201,17cyclictest0-21swapper/108:31:331
13195992011,2cyclictest0-21swapper/111:25:191
1319599201,11cyclictest23593-21ntp_states11:10:211
1319599201,11cyclictest0-21swapper/109:05:191
1319599201,10cyclictest18814-21df07:20:111
1319599201,10cyclictest14377-21seq08:23:591
1319599201,10cyclictest10682-21latency_hist09:30:011
1319599201,10cyclictest10646-21diskstats08:15:151
1319099204,7cyclictest0-21swapper/012:11:240
1319099201,4cyclictest0-21swapper/012:04:300
1319099201,11cyclictest0-21swapper/010:42:020
1319099201,11cyclictest0-21swapper/010:17:020
1319099201,10cyclictest19093-21/usr/sbin/munin08:35:140
1319099201,10cyclictest17074-21sshd09:42:280
1320099197,3cyclictest87150irq/126-enp1s0-09:33:002
1320099196,2cyclictest9331-21/usr/sbin/munin10:40:162
1320099196,2cyclictest12050irq/123-ahci[0008:45:022
1320099193,6cyclictest6112-40dnf09:17:552
1320099193,6cyclictest6112-40dnf09:17:552
1320099191,7cyclictest0-21swapper/212:15:282
1320099191,10cyclictest20058-21sed12:20:132
1320099191,10cyclictest0-21swapper/209:25:102
1319599196,11cyclictest221ktimersoftd/111:30:101
1319599191,17cyclictest0-21swapper/108:27:261
1319599191,10cyclictest19193-21ntp_states11:00:211
1319599191,10cyclictest16607-21hddtemp_smartct10:55:151
1319599190,10cyclictest0-21swapper/111:50:221
1319099198,6cyclictest0-21swapper/012:24:570
1319099198,3cyclictest0-21swapper/011:53:400
1319099198,3cyclictest0-21swapper/011:46:150
1319099197,3cyclictest0-21swapper/009:48:100
1319099195,2cyclictest87150irq/126-enp1s0-11:19:510
1319099192,8cyclictest27840-21memory11:20:180
1319099191,10cyclictest23204-21sshd08:44:270
1319099191,10cyclictest1596-21kworker/u8:109:10:120
1319099191,10cyclictest0-21swapper/011:25:110
1319099191,10cyclictest0-21swapper/010:00:280
1319099190,9cyclictest0-21swapper/012:06:260
1319099190,11cyclictest0-21swapper/009:30:530
291972181,2sleep00-21swapper/007:44:350
156792180,3sleep00-21swapper/010:54:520
1320099188,2cyclictest87450irq/129-enp1s0-10:15:172
1320099187,2cyclictest87150irq/126-enp1s0-10:25:142
1320099187,2cyclictest87150irq/126-enp1s0-10:25:142
1320099183,5cyclictest848-21diskstats10:20:142
1320099183,3cyclictest12050irq/123-ahci[0008:20:022
1320099182,7cyclictest21907-21/usr/sbin/munin09:55:232
1320099182,6cyclictest21682-21munin-run09:55:012
1320099182,6cyclictest0-21swapper/209:11:082
1320099182,2cyclictest87350irq/128-enp1s0-08:50:232
1320099182,14cyclictest0-21swapper/211:40:092
1320099182,14cyclictest0-21swapper/211:40:092
1320099181,7cyclictest0-21swapper/211:50:132
1320099181,7cyclictest0-21swapper/211:06:322
1320099181,7cyclictest0-21swapper/207:25:202
1320099181,5cyclictest1903-21diskmemload11:10:452
1320099181,12cyclictest0-21swapper/210:07:152
1320099181,10cyclictest0-21swapper/212:10:072
1320099181,10cyclictest0-21swapper/208:10:152
1320099180,8cyclictest311ktimersoftd/208:30:132
1320099180,10cyclictest311ktimersoftd/208:00:132
1320099180,10cyclictest311ktimersoftd/208:00:122
13195991815,2cyclictest0-21swapper/112:39:111
1319599181,16cyclictest0-21swapper/112:21:411
1319599181,16cyclictest0-21swapper/109:56:141
1319099187,3cyclictest0-21swapper/010:39:090
1319099186,5cyclictest0-21swapper/009:50:470
1319099186,3cyclictest0-21swapper/011:58:140
1319099185,5cyclictest0-21swapper/012:30:010
1319099182,4cyclictest20453-21munin-run07:25:010
1319099181,9cyclictest1903-21diskmemload10:21:410
1319099181,9cyclictest0-21swapper/009:17:570
1319099181,9cyclictest0-21swapper/009:17:570
1319099181,8cyclictest252202cpuspeed_turbos07:35:130
1319099181,7cyclictest29856-21munin-run09:00:000
1319099181,3cyclictest26133-21cat12:35:120
1319099181,14cyclictest12493-21/usr/sbin/munin08:20:180
13190991811,4cyclictest0-21swapper/010:10:150
1319099181,11cyclictest0-21swapper/011:42:330
1319099181,11cyclictest0-21swapper/011:42:330
1319099180,5cyclictest0-21swapper/009:38:110
1320099177,2cyclictest28952-21ntp_states08:55:222
1320099175,4cyclictest19307-21/usr/sbin/munin11:00:242
1320099174,6cyclictest87150irq/126-enp1s0-11:33:492
1320099174,3cyclictest87450irq/129-enp1s0-07:45:162
1320099174,3cyclictest30-21rcuc/212:30:232
1320099173,7cyclictest0-21swapper/209:20:362
1320099171,7cyclictest14526-21cat08:25:022
1320099171,6cyclictest0-21swapper/212:05:142
1320099171,6cyclictest0-21swapper/210:59:162
13200991712,3cyclictest16441-21cut07:15:132
13200991711,4cyclictest14199-21fschecks_time10:50:162
13200991711,3cyclictest19093-21head07:20:152
1320099171,10cyclictest0-21swapper/211:15:252
1320099171,10cyclictest0-21swapper/209:46:412
1319099175,4cyclictest4006-21smart_sda10:25:250
1319099175,4cyclictest4006-21smart_sda10:25:250
1319099171,7cyclictest0-21swapper/008:15:160
13190991712,3cyclictest32619-21df09:05:120
13190991712,3cyclictest29469-21/usr/sbin/munin07:45:260
13190991712,3cyclictest26992-21hwlatdetect10:05:160
13190991712,3cyclictest22873-21/usr/sbin/munin07:30:120
1319099171,13cyclictest20812-21cut07:25:130
1320099161,6cyclictest21059-21latency_hist08:40:022
1320099161,4cyclictest30-21rcuc/208:05:192
13200991611,3cyclictest24885-21cat08:45:262
1320099160,14cyclictest0-21swapper/207:40:072
1319099168,2cyclictest0-21swapper/008:30:150
1319099166,7cyclictest23859-21fschecks_count08:45:140
1319099161,6cyclictest0-21swapper/007:55:170
1319099161,14cyclictest0-21swapper/008:11:410
13190991610,3cyclictest31166-21open_files09:00:220
11362161,2sleep20-21swapper/207:53:392
1320099151,8cyclictest0-21swapper/208:25:212
13200991510,3cyclictest24226-21cat07:30:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional