You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 20:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Jun 12, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
239942236162,23sleep20-21swapper/207:08:362
238142228145,70sleep00-21swapper/007:06:160
240352217147,57sleep10-21swapper/107:09:071
238652202165,24sleep30-21swapper/307:06:563
80972590,4sleep232-21ksoftirqd/211:25:112
10502590,3sleep21054-21unixbench-2d12:20:252
193582560,3sleep319360-21gltestperf11:50:163
245202540,4sleep02418499cyclictest12:00:250
24197995249,2cyclictest0-21swapper/210:58:592
317202480,1sleep10-21swapper/112:19:561
3492460,1sleep20-21swapper/209:55:032
158432440,1sleep20-21swapper/210:30:012
24184994115,12cyclictest7-21ksoftirqd/011:05:020
182042370,4sleep081ktimersoftd/009:20:160
182042370,4sleep081ktimersoftd/009:20:160
2419799361,22cyclictest0-21swapper/210:16:512
2418499369,12cyclictest0-21swapper/008:05:010
2419799351,14cyclictest0-21swapper/212:06:072
2419799351,13cyclictest0-21swapper/211:30:112
24197993420,9cyclictest0-21swapper/208:40:552
2419799341,26cyclictest0-21swapper/209:14:312
2419799341,20cyclictest0-21swapper/211:03:552
2419799341,14cyclictest0-21swapper/212:12:592
2418499346,13cyclictest18531-21idleruntime-cro11:50:010
2418499346,13cyclictest18531-21idleruntime-cro11:50:010
2419799331,22cyclictest0-21swapper/207:39:512
2419799331,21cyclictest0-21swapper/208:13:592
2419799331,21cyclictest0-21swapper/208:13:592
2419799331,20cyclictest0-21swapper/210:09:392
2419799331,15cyclictest0-21swapper/211:11:512
24197993216,8cyclictest0-21swapper/209:17:082
2419799321,21cyclictest0-21swapper/208:56:432
2419799321,16cyclictest0-21swapper/211:45:032
2419799321,16cyclictest0-21swapper/211:45:032
2419799321,15cyclictest0-21swapper/211:36:512
2419799321,15cyclictest0-21swapper/210:02:312
2419799321,14cyclictest0-21swapper/210:41:112
2419799320,13cyclictest0-21swapper/211:22:272
24197993116,7cyclictest4444-21ntp_states11:15:192
2419799311,23cyclictest0-21swapper/208:07:272
2419799311,21cyclictest0-21swapper/209:06:152
2419799311,19cyclictest0-21swapper/212:27:232
2419799311,13cyclictest0-21swapper/212:00:512
2419799310,14cyclictest0-21swapper/211:44:352
2419799309,9cyclictest0-21swapper/208:02:512
24197993018,3cyclictest0-21swapper/208:23:472
24197993016,7cyclictest0-21swapper/209:41:482
2419799301,20cyclictest0-21swapper/211:07:552
2419799301,20cyclictest0-21swapper/208:27:152
2419799301,15cyclictest0-21swapper/210:13:112
2419799301,14cyclictest0-21swapper/209:46:202
2419799300,14cyclictest0-21swapper/211:58:272
2419799300,14cyclictest0-21swapper/207:50:272
288692290,4sleep12419099cyclictest09:45:131
2419799291,9cyclictest12910-21diskmemload12:35:112
24197992918,3cyclictest0-21swapper/212:34:192
24197992918,3cyclictest0-21swapper/209:56:122
2419799291,19cyclictest0-21swapper/212:16:562
2419799291,18cyclictest0-21swapper/210:52:112
2419799291,15cyclictest0-21swapper/210:20:352
2419799291,13cyclictest0-21swapper/207:28:232
2419799290,12cyclictest0-21swapper/209:36:082
24197992819,3cyclictest0-21swapper/207:48:392
2419799281,13cyclictest0-21swapper/209:33:202
2419799281,13cyclictest0-21swapper/208:49:032
2419799281,13cyclictest0-21swapper/207:11:472
2419799281,11cyclictest0-21swapper/210:49:112
2419799281,11cyclictest0-21swapper/207:18:072
2419799281,10cyclictest0-21swapper/209:26:362
2419799280,14cyclictest0-21swapper/211:52:072
2419799280,12cyclictest0-21swapper/207:56:312
2418499280,13cyclictest30498-21munin-run09:50:010
247102270,4sleep121-21rcuc/112:00:261
2419799271,20cyclictest0-21swapper/209:01:152
24197992711,3cyclictest6983-21ntp_states07:40:232
2419799271,11cyclictest0-21swapper/210:39:112
2419799270,20cyclictest0-21swapper/208:32:312
2419799270,12cyclictest0-21swapper/209:24:032
2419799270,12cyclictest0-21swapper/209:24:032
2419099271,10cyclictest0-21swapper/111:15:531
2419799269,8cyclictest0-21swapper/208:53:032
2419799269,11cyclictest0-21swapper/207:20:472
2419799261,12cyclictest0-21swapper/210:34:192
2419799261,12cyclictest0-21swapper/210:34:192
2419799260,13cyclictest0-21swapper/208:37:072
2419099267,8cyclictest0-21swapper/111:45:131
2419099267,8cyclictest0-21swapper/111:45:131
2419099261,17cyclictest30551-21ntp_states07:20:221
2419099261,17cyclictest14988-21expr08:00:121
2418499261,12cyclictest941-21munin-run11:10:020
2418499261,12cyclictest10724-21awk09:05:000
2419799251,9cyclictest0-21swapper/207:30:552
24190992514,3cyclictest19110-21/usr/sbin/munin08:10:191
24190992514,3cyclictest19110-21/usr/sbin/munin08:10:191
2418499252,10cyclictest0-21swapper/007:55:000
2419799240,18cyclictest0-21swapper/208:17:032
2419099242,12cyclictest26508-21munin-node07:15:011
2419099241,19cyclictest26106-21turbostat09:40:011
2419099241,18cyclictest0-21swapper/110:40:341
2418499237,14cyclictest0-21swapper/007:45:010
2420299225,8cyclictest87350irq/126-enp1s0-12:15:013
2420299222,11cyclictest457-21/usr/sbin/munin09:55:163
2420299222,11cyclictest1165-21/usr/sbin/munin11:10:203
24202992217,3cyclictest13027-21grep10:20:263
2420299221,12cyclictest16566-21/usr/sbin/munin11:45:163
2420299221,12cyclictest16566-21/usr/sbin/munin11:45:163
2420299221,10cyclictest0-21swapper/312:20:273
2419099228,3cyclictest16249-21missed_timers09:15:191
2419099221,3cyclictest5778-21irqstats10:05:171
2419099221,18cyclictest32312-21users09:50:281
2419099221,18cyclictest10224-21unixbench_multi09:00:261
2419099221,16cyclictest0-21swapper/111:14:291
2419099221,15cyclictest0-21swapper/111:21:061
24184992210,4cyclictest27396-21ntp_states09:40:210
236172220,4sleep023621-21cpuspeed_turbos08:20:120
2420299217,11cyclictest39-21rcuc/309:15:253
24202992113,5cyclictest4885-21grep11:15:233
24202992111,3cyclictest87650irq/129-enp1s0-09:41:283
2420299211,11cyclictest7160-21/usr/sbin/munin10:10:143
2420299211,10cyclictest0-21swapper/312:35:243
2419099211,4cyclictest0-21swapper/111:54:151
2419099211,3cyclictest31793-21seq08:36:381
2419099211,3cyclictest10530-21perl11:30:141
2419099211,18cyclictest0-21swapper/109:05:271
2419099211,17cyclictest32314-21diskstats12:20:141
2419099211,17cyclictest17504-21grep10:30:241
2419099211,17cyclictest17504-21grep10:30:241
2419099211,17cyclictest13415-21/usr/sbin/munin11:35:231
2419099211,15cyclictest0-21swapper/107:32:371
2419099211,14cyclictest0-21swapper/107:47:421
2419099211,10cyclictest5684-21turbostat.cron07:40:011
2418499216,6cyclictest0-21swapper/010:47:110
2418499216,5cyclictest0-21swapper/009:11:290
2418499213,16cyclictest14143-21munin-run11:40:010
2418499211,7cyclictest0-21swapper/010:38:510
2418499211,12cyclictest13812-21kworker/0:011:43:590
2418499211,11cyclictest25810-21ntp_states10:50:210
2420299208,3cyclictest6988-21ntp_states07:40:243
2420299206,5cyclictest15593-21cat08:00:193
2420299203,6cyclictest15040-40dnf09:14:253
2420299201,6cyclictest0-21swapper/310:48:363
2420299201,3cyclictest0-21swapper/312:16:343
2420299201,11cyclictest4965-21/usr/sbin/munin10:05:193
2420299201,11cyclictest12170-21/usr/sbin/munin11:35:223
2420299201,10cyclictest9975-21/usr/sbin/munin11:30:133
2420299201,10cyclictest8363-21sshd08:58:263
2420299201,10cyclictest0-21swapper/308:11:253
2420299201,10cyclictest0-21swapper/308:11:253
2419099209,3cyclictest18964-21sed09:20:261
2419099209,3cyclictest18964-21sed09:20:261
2419099209,3cyclictest18492-21df_inode10:35:131
2419099201,8cyclictest0-21swapper/110:54:071
24190992013,2cyclictest0-21swapper/107:54:011
24190992013,2cyclictest0-21swapper/107:16:491
2419099201,2cyclictest12910-21diskmemload09:30:121
2419099201,2cyclictest0-21swapper/110:10:141
2419099201,2cyclictest0-21swapper/107:58:381
24190992012,3cyclictest0-21swapper/107:28:081
2419099201,17cyclictest0-21swapper/109:40:111
2419099201,17cyclictest0-21swapper/108:25:271
2419099201,14cyclictest0-21swapper/111:44:151
2419099201,13cyclictest0-21swapper/110:47:241
2419099201,11cyclictest6998-21interrupts12:35:161
2419099201,11cyclictest28459-21unixbench_multi10:55:261
2419099201,10cyclictest31534-21expr11:05:101
24190992010,3cyclictest3402-21smart_sda08:45:281
2419099200,4cyclictest0-21swapper/110:26:261
2419099200,18cyclictest0-21swapper/112:10:201
2419099200,18cyclictest0-21swapper/107:40:151
2418499209,3cyclictest20918-21fschecks_time10:40:150
2418499208,4cyclictest15667-21df09:15:140
2418499206,6cyclictest0-21swapper/011:31:030
2418499204,6cyclictest0-21swapper/010:55:560
2418499204,6cyclictest0-21swapper/010:09:280
2418499201,8cyclictest31911-21kworker/0:108:47:230
2418499201,6cyclictest0-21swapper/010:34:020
2418499201,6cyclictest0-21swapper/010:34:020
2418499201,5cyclictest0-21swapper/010:16:380
2418499201,10cyclictest25373-21/usr/sbin/munin12:05:150
117422201,3sleep30-21swapper/307:50:283
311212191,2sleep30-21swapper/307:20:533
2420299198,3cyclictest24122-21/usr/sbin/munin09:35:163
2420299198,3cyclictest0-21swapper/308:23:183
2420299197,3cyclictest23512-21users09:30:263
2420299192,10cyclictest10025-21sshd07:49:263
2420299191,5cyclictest0-21swapper/310:34:483
2420299191,5cyclictest0-21swapper/310:34:483
2420299191,4cyclictest0-21swapper/310:55:093
24202991914,3cyclictest22432-21grep11:55:243
24202991912,4cyclictest647-21cat07:25:293
2420299191,16cyclictest0-21swapper/308:06:313
2420299191,15cyclictest30660-21sed11:00:273
2420299191,11cyclictest0-21swapper/309:24:553
2420299191,11cyclictest0-21swapper/309:24:553
2420299191,10cyclictest0-21swapper/310:35:233
2420299191,10cyclictest0-21swapper/307:55:183
2420299190,8cyclictest0-21swapper/312:06:313
2419099192,13cyclictest28975-21ntp_states08:30:231
2419099191,8cyclictest0-21swapper/112:34:131
2419099191,8cyclictest0-21swapper/110:20:511
24190991915,3cyclictest0-21swapper/109:58:231
2419099191,17cyclictest0-21swapper/110:16:171
2419099191,14cyclictest0-21swapper/111:27:481
2419099191,10cyclictest7964-21timerwakeupswit08:55:251
2419099191,10cyclictest32439-21cpuspeed_turbos08:40:131
2419099191,10cyclictest21988-21/usr/sbin/munin11:55:191
2418499192,4cyclictest858-21df_inode09:55:130
2418499191,8cyclictest5201-21kworker/0:111:20:440
24184991914,3cyclictest5188-21grep07:35:280
24184991912,4cyclictest12209-21tail10:20:150
2418499190,8cyclictest0-21swapper/011:11:120
213422190,3sleep10-21swapper/109:25:361
35852181,2sleep00-21swapper/012:27:430
305802181,2sleep00-21swapper/012:15:190
303692180,4sleep339-21rcuc/308:35:123
2420299186,3cyclictest87350irq/126-enp1s0-11:40:223
2420299182,15cyclictest0-21swapper/309:05:593
2420299181,9cyclictest0-21swapper/311:20:193
2420299181,14cyclictest2135-21/usr/sbin/munin08:45:173
2420299181,10cyclictest0-21swapper/311:29:093
2420299181,10cyclictest0-21swapper/310:18:273
24190991815,2cyclictest0-21swapper/112:07:051
2419099181,16cyclictest0-21swapper/111:01:171
24190991811,4cyclictest23513-21bc08:20:121
24190991811,2cyclictest0-21swapper/108:07:211
2419099181,11cyclictest0-21swapper/108:15:171
2419099180,16cyclictest0-21swapper/112:25:401
2419099180,16cyclictest0-21swapper/110:00:191
2419099180,16cyclictest0-21swapper/108:53:191
2418499187,4cyclictest0-21swapper/010:00:140
2418499187,3cyclictest12910-21diskmemload12:10:050
2418499186,3cyclictest0-21swapper/011:27:440
2418499185,4cyclictest6659-21kworker/u8:010:25:100
2418499184,9cyclictest0-21swapper/008:35:210
2418499183,3cyclictest29483-21/usr/sbin/munin07:20:200
2418499182,2cyclictest87350irq/126-enp1s0-09:29:340
2418499181,8cyclictest20673-21kworker/0:311:56:070
2418499181,14cyclictest12400-21grep09:05:260
2418499181,12cyclictest0-21swapper/011:19:190
2418499181,11cyclictest0-21swapper/009:50:110
2418499180,5cyclictest0-21swapper/009:34:590
103592181,3sleep30-21swapper/309:00:503
243842171,2sleep00-21swapper/007:10:080
2420299173,11cyclictest27905-21/usr/sbin/munin08:30:273
2420299171,7cyclictest0-21swapper/310:00:213
2420299171,6cyclictest27634-21diskstats07:15:143
2420299171,6cyclictest0-21swapper/312:30:203
2420299171,6cyclictest0-21swapper/310:26:583
2420299171,6cyclictest0-21swapper/309:53:163
2420299171,4cyclictest0-21swapper/311:05:153
24202991713,3cyclictest0-21swapper/312:02:523
24202991713,3cyclictest0-21swapper/308:18:313
24202991713,3cyclictest0-21swapper/307:39:293
24202991712,3cyclictest5598-21smart_sda08:50:253
24202991712,3cyclictest21698-21dump-pmu-power09:30:013
24202991712,3cyclictest20342-21/usr/sbin/munin10:40:113
2420299171,10cyclictest0-21swapper/312:25:503
24190991714,2cyclictest0-21swapper/109:11:181
24184991712,3cyclictest3001-21grep07:30:280
2418499171,13cyclictest32297-21/usr/sbin/munin08:40:260
24184991711,3cyclictest13913-21expr07:55:230
2418499171,10cyclictest0-21swapper/012:36:130
2418499171,10cyclictest0-21swapper/012:31:190
2418499171,10cyclictest0-21swapper/012:21:360
24184991710,4cyclictest25694-21grep09:35:240
2418499170,5cyclictest0-21swapper/011:50:210
257002160,2sleep00-21swapper/008:25:080
2420299166,3cyclictest87650irq/129-enp1s0-08:25:193
2420299161,6cyclictest3410-21sshd07:34:093
24202991612,2cyclictest0-21swapper/310:50:123
24202991611,3cyclictest28700-21cat09:45:123
2418499166,7cyclictest19235-21grep08:10:140
2418499166,7cyclictest19235-21grep08:10:140
2418499163,5cyclictest0-21swapper/010:10:200
2418499161,6cyclictest0-21swapper/008:50:160
2418499161,2cyclictest87550irq/128-enp1s0-07:15:170
24184991612,3cyclictest0-21swapper/007:29:290
24184991611,3cyclictest29165-21sensors_temp08:30:260
185372161,2sleep00-21swapper/008:05:450
24202991510,3cyclictest827-21runrttasks07:11:213
24202991510,3cyclictest418-21gltestperf08:40:163
2418499151,12cyclictest0-21swapper/008:55:110
24184991510,3cyclictest827-21runrttasks08:15:190
24184991510,3cyclictest8112-21/usr/sbin/munin07:45:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional