You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-09 - 01:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sat Aug 08, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
217942201164,15sleep20-21swapper/207:07:262
219142190168,15sleep30-21swapper/307:09:003
218102178105,29sleep10-21swapper/107:07:371
219412169101,61sleep00-21swapper/007:09:220
2199527758,11sleep30-21swapper/307:10:003
23892690,7sleep228-21ksoftirqd/209:52:062
23892690,7sleep228-21ksoftirqd/209:52:052
297232620,2sleep30-21swapper/309:48:413
5952610,2sleep00-21swapper/011:03:420
15332590,2sleep10-21swapper/110:15:281
53062570,1sleep30-21swapper/310:19:033
219632560,3sleep021925-21munin-run11:20:000
21422540,1sleep30-21swapper/310:40:233
48562520,1sleep14857-21diskmemload10:42:371
153492520,1sleep00-21swapper/012:26:340
7412490,2sleep10-21swapper/111:51:391
72192490,1sleep00-21swapper/009:55:250
284982470,1sleep00-21swapper/011:48:420
226522470,2sleep012-21ksoftirqd/010:07:210
276042460,2sleep20-21swapper/209:46:312
283152450,2sleep10-21swapper/111:48:301
248272420,1sleep00-21swapper/010:33:330
2226999422,20cyclictest23604-21ssh11:45:122
2226999380,8cyclictest27360-21grep10:10:252
255402360,2chrt12-21ksoftirqd/009:45:170
22269993611,22cyclictest8905-21sed07:50:242
22269993511,21cyclictest26547-21seq09:21:212
2226999349,22cyclictest21866-21grep10:55:202
22269993412,19cyclictest2851-21seq12:17:222
22269993410,21cyclictest13345-21sed08:00:252
2226999340,7cyclictest11326-21sh10:47:262
81562330,2chrt0-21swapper/307:50:153
2226999339,21cyclictest18377-21cat09:40:152
2226999339,20cyclictest10746-21kernelversion11:35:192
300222320,11sleep012-21ksoftirqd/010:13:020
2226999321,28cyclictest18384-21grep11:40:252
2226999319,19cyclictest2875-21gltestperf11:05:182
2226999319,19cyclictest20323-21seq10:54:472
2226999318,20cyclictest20547-21fschecks_time10:30:162
2226999313,3cyclictest0-21swapper/209:05:142
2226999312,6cyclictest10065-21diskmemload11:25:112
2226999311,25cyclictest0-21swapper/209:35:122
2226799311,6cyclictest0-21swapper/012:15:000
221642310,2sleep10-21swapper/111:44:011
55062300,21sleep122-21ksoftirqd/112:20:011
2226999308,19cyclictest31336-21cat12:15:012
2226999306,4cyclictest21220-21expr08:20:142
22269993022,6cyclictest0-21swapper/208:14:172
22269993011,16cyclictest29574-21sed11:00:252
2226999300,8cyclictest1431-21ssh11:52:222
2226999300,7cyclictest2772-21ssh10:40:292
2226999300,7cyclictest14896-21ssh12:26:052
2226999300,6cyclictest21226-21ssh12:30:452
22267993026,3cyclictest131rcuc/008:17:550
2226799302,12cyclictest5449-21rm11:06:510
22270992922,5cyclictest30824-21python308:40:233
2226999297,4cyclictest4632-1kworker/u9:210:26:102
2226999297,19cyclictest0-21swapper/211:33:372
22269992922,5cyclictest0-21swapper/208:55:512
22269992922,5cyclictest0-21swapper/208:17:522
2226999292,11cyclictest10505-21apt-get09:10:142
2226999291,6cyclictest9011-21ssh11:10:122
22269992915,12cyclictest0-21swapper/207:45:112
2226999291,23cyclictest0-21swapper/208:54:032
2226899291,4cyclictest21880-21missed_timers08:20:191
2226799293,24cyclictest29542-21cut08:40:000
2226799292,11cyclictest27266-21rm10:59:520
22270992823,3cyclictest13659-21taskset08:01:093
2226999288,17cyclictest23396-21cstates08:25:132
2226999288,17cyclictest18661-21gltestperf12:05:172
2226999288,17cyclictest18661-21gltestperf12:05:162
2226999287,4cyclictest1517-21ls08:50:002
2226999287,3cyclictest6900-21netstat12:20:222
2226999286,5cyclictest26526-21ls07:20:012
2226999286,4cyclictest30091-21fschecks_time08:40:152
2226999286,4cyclictest24223-21rm11:20:342
2226999286,4cyclictest24223-21rm11:20:332
2226999286,19cyclictest0-21swapper/210:16:442
2226999282,23cyclictest30538-21apt-get09:25:102
2226999281,9cyclictest27473-21/usr/sbin/munin10:35:202
2226999281,8cyclictest11431-21/usr/sbin/munin07:55:262
2226999281,4cyclictest0-21swapper/209:15:132
2226999281,3cyclictest3853-21memory07:40:212
2226999281,25cyclictest0-21swapper/208:31:132
22269992812,13cyclictest32617-21vmstat07:30:272
22269992811,14cyclictest10065-21diskmemload12:35:392
2226999280,26cyclictest0-21swapper/208:39:032
2226999280,26cyclictest0-21swapper/208:08:312
2226899281,25cyclictest0-21swapper/108:52:351
2226899281,25cyclictest0-21swapper/107:55:361
2226899281,25cyclictest0-21swapper/107:49:001
2226799284,6cyclictest14674-21ssh09:36:550
2226799284,6cyclictest11667-21sh10:23:420
2226799281,3cyclictest16239-21taskset08:09:110
2226799281,25cyclictest0-21swapper/007:56:490
2226799281,10cyclictest3239-21ssh09:28:500
2226799281,10cyclictest18922-21/usr/sbin/munin09:40:200
2227099271,24cyclictest0-21swapper/309:01:163
2227099271,24cyclictest0-21swapper/309:01:153
2227099271,24cyclictest0-21swapper/307:58:283
2226999278,6cyclictest19619-21gltestperf10:05:172
2226999272,7cyclictest636-21runrttasks07:10:122
2226999271,9cyclictest4941-21fschecks_count11:55:172
2226999271,9cyclictest11766-21gltestperf12:00:172
2226999271,8cyclictest11916-21cron10:00:012
2226999271,8cyclictest11886-21ssh10:23:562
2226999271,23cyclictest7690-21ssh09:31:352
2226999270,25cyclictest0-21swapper/209:01:402
2226999270,25cyclictest0-21swapper/209:01:392
2226899271,25cyclictest0-21swapper/109:01:231
2226899271,25cyclictest0-21swapper/109:01:231
2226899271,25cyclictest0-21swapper/108:35:021
2226899271,24cyclictest0-21swapper/108:30:451
2226899271,24cyclictest0-21swapper/108:25:021
2226899271,24cyclictest0-21swapper/108:17:051
2226899271,24cyclictest0-21swapper/108:11:071
2226899270,25cyclictest0-21swapper/107:52:391
2226799274,10cyclictest16129-21ssh10:02:300
2226799273,8cyclictest17435-21perl11:40:150
2226799272,12cyclictest48572diskmemload10:42:360
2226799271,25cyclictest0-21swapper/008:48:480
2226799271,25cyclictest0-21swapper/008:29:010
2226799271,24cyclictest0-21swapper/009:02:500
2226799271,24cyclictest0-21swapper/009:02:490
2226799271,24cyclictest0-21swapper/008:10:290
2226799271,24cyclictest0-21swapper/008:01:130
2226799271,24cyclictest0-21swapper/007:50:280
2226799271,24cyclictest0-21swapper/007:40:260
2226799271,24cyclictest0-21swapper/007:38:300
2226799271,24cyclictest0-21swapper/007:25:150
2226799271,24cyclictest0-21swapper/007:25:140
201482270,1sleep00-21swapper/012:05:470
201482270,1sleep00-21swapper/012:05:460
22270992624,1cyclictest0-21swapper/308:54:093
22270992622,2cyclictest0-21swapper/308:06:463
22270992622,2cyclictest0-21swapper/307:43:363
2227099261,24cyclictest0-21swapper/308:56:033
2227099261,23cyclictest0-21swapper/308:31:223
2227099261,23cyclictest0-21swapper/308:22:593
2227099261,23cyclictest0-21swapper/308:16:413
2227099261,23cyclictest0-21swapper/308:12:053
2227099261,23cyclictest0-21swapper/307:45:213
2226999261,8cyclictest13023-21/usr/sbin/munin10:00:152
2226999261,7cyclictest2251-21grep07:35:252
2226999260,6cyclictest17143-21awk11:15:262
22268992624,1cyclictest0-21swapper/108:56:301
22268992624,1cyclictest0-21swapper/107:35:271
2226899261,2cyclictest0-21swapper/108:00:571
2226899261,24cyclictest0-21swapper/108:48:411
2226899260,24cyclictest0-21swapper/109:08:201
2226899260,24cyclictest0-21swapper/108:40:061
2226899260,24cyclictest0-21swapper/108:05:441
2226899260,24cyclictest0-21swapper/107:40:191
2226899260,24cyclictest0-21swapper/107:30:171
2226799264,6cyclictest8949-21ssh12:21:480
2226799263,9cyclictest31861-21ssh12:39:480
2226799263,8cyclictest1499-21timerandwakeup10:15:290
2226799263,8cyclictest10093-21sh10:46:090
2226799262,8cyclictest28486-21seq07:22:160
2226799262,8cyclictest22670-21seq08:21:180
22267992624,1cyclictest12-21ksoftirqd/007:45:160
22267992622,3cyclictest131rcuc/011:38:150
2226799261,24cyclictest0-21swapper/008:53:380
2226799261,24cyclictest0-21swapper/008:40:060
2226799261,24cyclictest0-21swapper/007:30:250
2226799261,1cyclictest0-21swapper/008:31:390
2226799261,10cyclictest4989-21rm09:30:150
2226799260,24cyclictest0-21swapper/008:55:430
146462260,2sleep10-21swapper/110:01:001
22270992523,1cyclictest0-21swapper/307:30:193
22270992518,5cyclictest34-21ksoftirqd/311:20:003
2227099251,1cyclictest0-21swapper/308:47:263
2227099251,1cyclictest0-21swapper/308:25:413
2227099250,23cyclictest0-21swapper/309:05:093
2227099250,23cyclictest0-21swapper/308:35:463
2227099250,23cyclictest0-21swapper/307:35:043
2226999251,7cyclictest29763-21grep07:25:182
2226999251,7cyclictest29763-21grep07:25:182
2226999251,6cyclictest27516-21cat07:20:182
2226799254,6cyclictest2776-21ssh11:53:420
2226799254,5cyclictest12-21ksoftirqd/009:13:060
2226799252,8cyclictest29143-21runrttasks10:36:090
2226799252,7cyclictest12-21ksoftirqd/011:58:210
22267992521,3cyclictest131rcuc/012:01:460
2226799251,8cyclictest4002-21sh12:18:330
2226799251,8cyclictest3977-21sh09:53:440
2226799251,8cyclictest3977-21sh09:53:430
2226799251,6cyclictest24958-21ssh12:34:320
2226799251,1cyclictest0-21swapper/009:06:010
2226799251,10cyclictest20016-21rm10:54:290
185392250,1sleep10-21swapper/112:29:471
82882240,2sleep10-21swapper/111:33:461
46172240,10sleep122-21ksoftirqd/109:30:011
305382240,1sleep10-21swapper/112:38:261
22270992414,7cyclictest25321-21df12:10:143
2226799242,8cyclictest19049-21ssh09:15:330
2226799242,6cyclictest16357-21seq10:26:430
22267992421,2cyclictest131rcuc/011:30:080
22267992421,2cyclictest131rcuc/011:24:240
22267992421,2cyclictest131rcuc/011:24:230
2226799240,5cyclictest0-21swapper/011:10:200
199022240,9sleep334-21ksoftirqd/310:54:233
186802240,2sleep122-21ksoftirqd/109:40:161
22270992313,7cyclictest26080-21df10:10:133
2226799234,5cyclictest2091-21ssh11:29:140
2226799232,8cyclictest26023-21seq07:16:070
2226799231,6cyclictest25822-21runrttasks09:20:380
120702230,3sleep310065-21diskmemload12:00:203
22268992213,6cyclictest6041-21sed11:55:261
9012210,1sleep10-21swapper/111:04:001
2227099213,9cyclictest22397-21ssh11:20:133
2227099213,9cyclictest22397-21ssh11:20:123
22270992112,4cyclictest8994-21sh10:45:253
2226899213,10cyclictest26740-21ssh10:35:141
2226899212,17cyclictest20597-21head10:30:171
2226899211,8cyclictest211782sh12:30:431
22268992114,4cyclictest6663-21/usr/sbin/munin10:20:181
22268992111,7cyclictest7356-21unixbench_singl09:55:251
22268992110,9cyclictest20088-21ntp_offset10:05:211
22267992117,3cyclictest131rcuc/007:14:440
265522200,2sleep10-21swapper/110:10:191
2227099201,5cyclictest0-21swapper/311:49:153
22270992014,4cyclictest34-21ksoftirqd/309:20:003
22270992014,4cyclictest13872-21switchtime12:25:253
22270992013,5cyclictest30247-21seq12:38:093
22270992013,5cyclictest19271-21pluginstate09:40:233
22270992011,7cyclictest4235-21cat11:30:203
22268992012,5cyclictest636-21runrttasks11:18:051
22268992011,6cyclictest560-21dbus-daemon11:15:001
22268992010,6cyclictest52950irq/141-enp3s0-09:45:121
2227099199,6cyclictest16128-21ssh11:39:423
2227099192,13cyclictest17904-21diskmemload12:05:043
2227099192,13cyclictest17904-21diskmemload12:05:043
2227099192,12cyclictest29036-21apt-get11:25:143
2227099191,7cyclictest11857-21diskstats09:35:143
22270991914,3cyclictest25893-21sh10:58:273
22270991914,3cyclictest20083-21cpu10:30:143
22270991913,4cyclictest34-21ksoftirqd/310:40:003
22270991913,4cyclictest3067-21sh12:17:373
22270991913,4cyclictest2574-21sh11:53:303
22270991912,5cyclictest19479-21expr12:30:143
22270991912,5cyclictest13680-21cat10:25:143
22270991912,5cyclictest10331-21sensors11:10:253
22270991912,5cyclictest10300-21seq09:10:023
22270991912,4cyclictest26380-21sh09:21:123
22270991912,4cyclictest10065-21diskmemload09:26:323
22270991911,6cyclictest11108-21sed09:35:003
22270991911,5cyclictest6578-21cat12:20:193
22270991910,6cyclictest19888-21cat10:05:203
2226899199,7cyclictest25056-21cat07:15:171
2226899198,9cyclictest26229-21sh09:21:021
22268991912,5cyclictest2210-21seq09:27:481
22268991912,5cyclictest14836-21ssh09:37:061
22268991911,6cyclictest11885-21sed09:10:241
2226899191,13cyclictest24783-21ssh12:10:031
22268991910,5cyclictest22440-21gzip10:55:241
2227099189,7cyclictest4899-21diskstats11:55:173
2227099189,6cyclictest12346-21apt-get10:00:103
2227099188,6cyclictest53050irq/142-enp3s0-11:00:133
22270991813,3cyclictest31868-21fschecks_count09:50:163
22270991813,3cyclictest31868-21fschecks_count09:50:153
22270991811,5cyclictest9574-21kworker/u8:211:05:273
22270991811,5cyclictest6709-21ntp_kernel_pll_09:55:203
22270991811,5cyclictest29113-21grep07:25:133
22270991811,5cyclictest29113-21grep07:25:133
22270991811,5cyclictest24711-21expr07:15:143
22270991811,4cyclictest9688-21sh10:21:423
22270991811,4cyclictest19716-21sh11:41:313
2226899185,11cyclictest11171-21cat12:00:121
2226899183,13cyclictest19229-21head12:05:231
2226899183,13cyclictest19229-21head12:05:221
22268991812,4cyclictest7819-21sh11:09:161
22268991812,4cyclictest17284-21sh10:27:391
22268991811,5cyclictest11157-21proc_pri11:35:231
2226899181,12cyclictest75-21kswapd009:50:121
2226899181,12cyclictest75-21kswapd009:50:111
22268991810,6cyclictest24270-21sed07:15:011
22268991810,6cyclictest17389-21cat09:15:131
2227099179,5cyclictest26835-21cpu07:20:133
2226899178,7cyclictest27304-21grep07:20:171
22268991712,3cyclictest25049-21sh11:21:241
22268991712,3cyclictest25049-21sh11:21:241
22268991711,4cyclictest8592-21ls10:45:211
22268991711,4cyclictest16541-21sh10:50:581
22268991710,5cyclictest6125-21expr12:20:151
2226899167,6cyclictest29948-21sed07:25:221
2226899167,6cyclictest29948-21sed07:25:211
22268991610,4cyclictest29303-21expr11:25:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional