You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-17 - 18:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Jul 17, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
118802239164,24sleep10-21swapper/107:05:471
121482234163,59sleep00-21swapper/007:09:170
118962230158,24sleep20-21swapper/207:06:022
118622204167,25sleep30-21swapper/307:05:343
1229399107103,3cyclictest0-21swapper/207:26:212
12293997772,2cyclictest975-21diskmemload11:40:442
102422560,3sleep29-21rcu_preempt08:15:192
217092540,1sleep30-21swapper/309:55:173
322742520,2sleep121-21rcuc/107:50:261
150902460,1sleep30-21swapper/308:25:253
12293994643,2cyclictest0-21swapper/210:03:552
12279993813,12cyclictest22375-21awk08:45:010
179492370,1sleep20-21swapper/209:45:252
1227999369,13cyclictest31776-21munin-run10:20:010
1229399331,25cyclictest0-21swapper/208:26:042
1227999339,11cyclictest11-21rcuc/011:35:020
1229399321,13cyclictest0-21swapper/207:32:532
1229399311,17cyclictest0-21swapper/209:23:082
1229399311,12cyclictest0-21swapper/208:30:482
1229399310,23cyclictest0-21swapper/209:33:402
1229399310,22cyclictest0-21swapper/210:14:572
1229399310,13cyclictest0-21swapper/211:35:032
1227999316,12cyclictest28957-21munin-run09:00:010
12299993018,6cyclictest7117-21unixbench_multi09:20:253
1229399301,10cyclictest0-21swapper/207:36:132
1229399300,23cyclictest0-21swapper/210:26:572
1229399300,22cyclictest0-21swapper/212:24:472
1229399300,22cyclictest0-21swapper/212:24:472
74772290,5sleep230-21rcuc/211:50:152
1229999294,14cyclictest18242-21grep07:20:173
1229399291,14cyclictest0-21swapper/208:39:202
1229399291,12cyclictest0-21swapper/208:13:362
1229399291,11cyclictest0-21swapper/211:29:382
1229399290,17cyclictest0-21swapper/208:53:202
1229399290,17cyclictest0-21swapper/208:53:202
1229399290,14cyclictest0-21swapper/208:09:202
24312280,4sleep011-21rcuc/010:25:160
1229999285,11cyclictest27647-21switchtime07:40:263
1229999285,11cyclictest27647-21switchtime07:40:263
1229399281,13cyclictest0-21swapper/211:22:342
1229399281,11cyclictest0-21swapper/207:51:132
1229399281,10cyclictest0-21swapper/209:39:202
1229399281,10cyclictest0-21swapper/208:57:482
1229399280,19cyclictest0-21swapper/207:50:012
1229399280,13cyclictest0-21swapper/209:03:482
1229399280,12cyclictest0-21swapper/210:50:252
1229999276,9cyclictest7775-21tune2fs08:10:143
1229999276,12cyclictest7821-21/usr/sbin/munin09:25:253
1229999274,12cyclictest30253-21tune2fs10:15:183
12293992711,4cyclictest16508-21irqstats12:10:192
1229399271,12cyclictest0-21swapper/211:48:112
1229399271,12cyclictest0-21swapper/211:31:272
1229399271,10cyclictest0-21swapper/209:18:242
1229399271,10cyclictest0-21swapper/207:56:042
1229399270,9cyclictest0-21swapper/212:08:192
1229399270,18cyclictest0-21swapper/208:46:122
1229399270,13cyclictest0-21swapper/207:44:172
1229399270,13cyclictest0-21swapper/207:44:172
1229399270,12cyclictest0-21swapper/210:30:292
1229999262,9cyclictest1847-21/usr/sbin/munin10:25:243
1229399261,11cyclictest0-21swapper/210:55:342
1229399261,11cyclictest0-21swapper/209:08:062
1229399261,11cyclictest0-21swapper/207:24:472
1229399260,18cyclictest0-21swapper/209:56:332
1229399260,16cyclictest0-21swapper/209:42:402
1229399260,10cyclictest0-21swapper/211:12:342
1228599264,14cyclictest0-21swapper/110:37:061
242522250,2sleep081ktimersoftd/010:00:210
1229999252,16cyclictest19857-21idleruntime-cro12:20:013
1229999251,15cyclictest11778-21wc08:20:123
1229999251,12cyclictest0-21swapper/308:30:323
1229399251,9cyclictest975-21diskmemload12:35:512
1229399251,9cyclictest0-21swapper/209:11:042
1229399251,10cyclictest0-21swapper/211:16:222
1229399251,10cyclictest0-21swapper/208:21:542
1229399250,11cyclictest0-21swapper/212:32:192
1229399250,10cyclictest0-21swapper/210:49:092
1229399250,10cyclictest0-21swapper/210:37:132
12285992514,4cyclictest0-21swapper/109:25:021
1228599251,16cyclictest29139-21memory11:25:191
1229999245,6cyclictest26473-21awk12:35:013
1229999241,14cyclictest11368-21munin-run08:20:023
1229999241,10cyclictest5010-21/usr/sbin/munin08:05:283
1229399241,9cyclictest0-21swapper/212:15:432
1229399241,9cyclictest0-21swapper/211:02:102
1229399241,7cyclictest0-21swapper/208:42:162
1229399241,11cyclictest0-21swapper/210:08:092
1229399240,9cyclictest0-21swapper/212:04:232
1229399240,9cyclictest0-21swapper/211:56:032
1229399240,12cyclictest0-21swapper/209:25:282
1229399240,11cyclictest0-21swapper/209:51:052
1229399240,11cyclictest0-21swapper/208:04:282
1229399240,10cyclictest0-21swapper/211:08:102
1229399240,10cyclictest0-21swapper/210:44:492
1229399240,10cyclictest0-21swapper/210:22:532
1228599241,16cyclictest0-21swapper/111:06:201
1228599241,16cyclictest0-21swapper/110:54:071
1228599240,9cyclictest0-21swapper/112:34:281
1227999248,8cyclictest827-21runrttasks11:21:170
1227999241,12cyclictest26475-21idleruntime-cro12:35:000
1227999241,12cyclictest21763-21awk07:30:000
1229999238,7cyclictest28104-21ls11:25:013
1229999238,6cyclictest19159-21diskstats09:50:123
1229999236,2cyclictest13501-21/usr/sbin/munin12:05:223
1229999234,3cyclictest0-21swapper/311:45:263
1229999232,6cyclictest8539-21cut10:40:123
1229999232,12cyclictest28574-21diskstats11:25:133
1229999231,8cyclictest827-21runrttasks11:02:053
1229999231,7cyclictest0-21swapper/309:39:053
1229999231,6cyclictest0-21swapper/309:31:093
1229999231,14cyclictest23360-21seq11:12:043
1229999231,14cyclictest20091-21/usr/sbin/munin12:20:183
1229999231,14cyclictest20091-21/usr/sbin/munin12:20:183
1229999230,14cyclictest26378-21missed_timers10:05:203
1229399231,11cyclictest0-21swapper/212:27:392
1229399230,9cyclictest0-21swapper/207:13:122
1228599231,5cyclictest0-21swapper/110:32:331
1228599231,16cyclictest0-21swapper/110:08:371
1228599231,15cyclictest975-21diskmemload12:10:001
1228599230,18cyclictest0-21swapper/109:56:031
1229999224,15cyclictest3426-21/usr/sbin/munin09:15:183
1229999223,10cyclictest8862-21latency_hist11:55:013
1229999221,6cyclictest0-21swapper/312:14:473
12299992213,5cyclictest17304-40dnf07:19:313
1229999221,18cyclictest808-21snmpd10:53:433
1229399226,3cyclictest0-21swapper/207:15:502
1229399220,10cyclictest0-21swapper/210:17:052
1228599223,15cyclictest21779-40dnf12:21:341
1228599223,15cyclictest21779-40dnf12:21:341
1228599221,6cyclictest0-21swapper/112:38:371
1228599221,16cyclictest0-21swapper/110:19:071
1228599221,15cyclictest0-21swapper/111:03:231
1228599221,14cyclictest975-21diskmemload11:39:461
1228599221,13cyclictest87650irq/129-enp1s0-12:26:141
1228599220,9cyclictest0-21swapper/109:19:591
1228599220,16cyclictest0-21swapper/111:56:541
1227999221,14cyclictest0-21swapper/011:35:250
1229999217,5cyclictest4388-21diskstats10:30:153
1229999213,6cyclictest28603-21sshd12:39:323
1229999213,6cyclictest22934-21diskstats08:45:143
1229999213,15cyclictest28567-21/usr/sbin/munin07:45:133
1229999212,11cyclictest23830-21/usr/sbin/munin11:15:203
1229999211,5cyclictest18245-1kworker/u9:211:44:283
1229999211,11cyclictest0-21swapper/310:57:003
1229999210,8cyclictest0-21swapper/310:10:193
1229999210,13cyclictest975-21diskmemload10:45:383
1228599216,12cyclictest0-21swapper/109:10:151
1228599215,2cyclictest0-21swapper/111:12:271
1228599213,10cyclictest5847-21wc11:45:231
1228599211,3cyclictest16993-21taskset07:15:321
1228599211,18cyclictest0-21swapper/107:49:111
1228599211,17cyclictest7808-21memory11:50:181
1228599211,15cyclictest0-21swapper/112:01:381
1228599211,15cyclictest0-21swapper/109:37:111
1228599211,11cyclictest579-21systemd-journal09:05:011
1228599211,11cyclictest23694-21grep10:00:151
12285992110,3cyclictest1611-21latency_hist10:25:021
1228599210,16cyclictest0-21swapper/111:16:301
1227999218,2cyclictest2246-21kworker/0:207:57:110
1227999211,9cyclictest12750-21awk08:20:220
1227999211,8cyclictest0-21swapper/009:36:100
12279992115,3cyclictest20918-21unixbench-2d11:05:260
12279992110,3cyclictest25854-21ntp_states08:50:240
12279992110,3cyclictest25854-21ntp_states08:50:240
12279992110,2cyclictest20076-21smart_sda09:50:260
1229999208,3cyclictest31694-21swap11:30:263
1229999205,4cyclictest87350irq/126-enp1s0-12:00:203
1229999204,7cyclictest0-21swapper/307:59:373
1229999204,5cyclictest87350irq/126-enp1s0-12:25:233
1229999204,4cyclictest87650irq/129-enp1s0-08:40:153
1229999203,4cyclictest30243-21ntp_states09:00:203
1229999203,4cyclictest0-21swapper/311:35:193
1229999203,2cyclictest21977-21/usr/sbin/munin07:30:153
1229999201,7cyclictest0-21swapper/310:36:253
1229999201,6cyclictest1611-21latency_hist10:25:023
12299992011,5cyclictest23554-21diskstats10:00:133
1229999201,13cyclictest0-21swapper/311:05:043
1229999201,12cyclictest0-21swapper/309:47:323
1229999201,11cyclictest18194-21/usr/sbin/munin08:35:263
1229999200,8cyclictest9279-21cstates11:55:123
1228599209,3cyclictest11932-21df_inode08:20:121
1228599201,8cyclictest0-21swapper/110:11:591
12285992017,2cyclictest0-21swapper/110:46:341
12285992017,2cyclictest0-21swapper/108:11:551
12285992016,3cyclictest0-21swapper/107:41:011
12285992016,3cyclictest0-21swapper/107:41:011
12285992015,3cyclictest3370-21memory11:40:211
1228599201,3cyclictest0-21swapper/112:11:421
12285992013,2cyclictest0-21swapper/109:29:001
1228599201,18cyclictest0-21swapper/107:31:071
1228599201,17cyclictest0-21swapper/108:53:471
1228599201,17cyclictest0-21swapper/108:53:471
1228599201,17cyclictest0-21swapper/108:31:361
1228599201,17cyclictest0-21swapper/107:56:531
1228599201,17cyclictest0-21swapper/107:14:171
1228599201,15cyclictest0-21swapper/109:35:001
1228599201,14cyclictest87650irq/129-enp1s0-10:28:311
1228599201,14cyclictest0-21swapper/111:24:551
1228599201,14cyclictest0-21swapper/110:42:191
1228599201,11cyclictest5029-21grep08:05:151
1228599201,11cyclictest19843-21cpuspeed_turbos07:25:131
1228599201,11cyclictest0-21swapper/108:55:191
1228599200,15cyclictest0-21swapper/109:51:341
1227999209,3cyclictest7542-21sshd09:24:320
1227999205,3cyclictest0-21swapper/011:52:090
1227999201,9cyclictest0-21swapper/010:35:140
1227999201,11cyclictest5010-21/usr/sbin/munin08:05:210
1227999201,11cyclictest22293-21/usr/sbin/munin12:25:160
1227999201,11cyclictest0-21swapper/010:31:470
1227999200,7cyclictest0-21swapper/010:55:340
1229999193,8cyclictest15117-21idleruntime09:40:173
1229999192,8cyclictest0-21swapper/308:00:263
1229999192,5cyclictest26985-21/usr/sbin/munin08:55:193
1229999192,4cyclictest0-21swapper/308:54:253
1229999192,4cyclictest0-21swapper/308:54:253
1228599199,3cyclictest24259-21cpuspeed_turbos07:35:121
12285991915,3cyclictest0-21swapper/108:01:271
1228599191,17cyclictest0-21swapper/108:45:191
1228599191,16cyclictest0-21swapper/109:08:121
1228599191,15cyclictest21924-21tail08:40:261
1228599191,14cyclictest0-21swapper/109:47:121
1228599191,12cyclictest0-21swapper/109:42:321
1228599191,10cyclictest16208-21ps10:55:241
1228599190,17cyclictest0-21swapper/107:20:181
1227999199,3cyclictest8435-21/usr/sbin/munin10:40:110
1227999198,3cyclictest2801-21/usr/sbin/munin08:00:180
1227999197,4cyclictest0-21swapper/011:42:590
1227999197,4cyclictest0-21swapper/009:19:450
1227999195,6cyclictest0-21swapper/009:31:310
1227999195,3cyclictest28649-21sh12:40:010
1227999195,11cyclictest16416-21basename12:10:170
1227999191,7cyclictest0-21swapper/011:26:400
1227999191,6cyclictest18245-1kworker/u9:211:46:400
12279991913,3cyclictest17596-21grep11:00:160
1227999191,15cyclictest31990-21grep09:05:170
1227999191,12cyclictest0-21swapper/007:40:240
1227999191,12cyclictest0-21swapper/007:40:240
1227999191,10cyclictest0-21swapper/010:22:430
1227999191,10cyclictest0-21swapper/009:29:470
1229999188,3cyclictest12490-21/usr/sbin/munin07:10:183
1229999181,9cyclictest19778-21/usr/sbin/munin07:25:273
1229999181,7cyclictest0-21swapper/307:35:593
12299991810,5cyclictest0-21swapper/309:10:163
12285991815,2cyclictest0-21swapper/108:35:281
12285991812,1cyclictest0-21swapper/111:31:211
1228599181,16cyclictest0-21swapper/108:29:291
1228599180,17cyclictest0-21swapper/112:15:051
1227999189,3cyclictest14412-21sshd07:14:310
1227999188,3cyclictest975-21diskmemload09:40:220
1227999187,3cyclictest9400-21/usr/sbin/munin08:15:180
1227999185,5cyclictest0-21swapper/009:56:410
1227999185,3cyclictest0-21swapper/012:22:130
1227999185,3cyclictest0-21swapper/012:22:130
1227999181,9cyclictest2815-21users09:10:260
1227999181,9cyclictest0-21swapper/011:10:020
1227999181,5cyclictest0-21swapper/011:56:040
1227999181,5cyclictest0-21swapper/010:54:200
12279991813,3cyclictest29165-21unixbench_singl10:10:260
12279991813,3cyclictest10632-21/usr/sbin/munin10:45:210
1227999181,11cyclictest0-21swapper/012:05:130
1227999181,10cyclictest7200-21/usr/sbin/munin08:10:150
1227999181,10cyclictest11285-21/usr/sbin/munin12:00:190
1229999177,7cyclictest31238-21fschecks_time07:50:173
1229999176,3cyclictest87350irq/126-enp1s0-09:05:133
12285991713,3cyclictest0-21swapper/108:15:121
1227999175,6cyclictest755-21rngd10:05:210
1227999172,10cyclictest0-21swapper/011:15:130
1227999171,6cyclictest32322-21grep07:50:280
1227999171,6cyclictest0-21swapper/012:15:180
12279991712,3cyclictest25685-21grep07:35:270
12279991711,4cyclictest30248-21ntp_states09:00:220
1227999171,13cyclictest16143-21if_enp2s007:15:170
1227999170,5cyclictest0-21swapper/009:47:060
1227999170,16cyclictest0-21swapper/008:47:000
1227999161,7cyclictest28567-21/usr/sbin/munin07:45:170
1227999161,6cyclictest18022-21fschecks_count07:20:160
12279991612,3cyclictest0-21swapper/007:33:290
12279991611,3cyclictest18194-21/usr/sbin/munin08:35:160
12279991611,3cyclictest13799-21/usr/sbin/munin08:25:210
12279991510,3cyclictest16736-21awk08:30:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional