You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-28 - 17:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Thu May 28, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17072237163,24sleep20-21swapper/207:08:072
17112232162,57sleep10-21swapper/107:08:101
16782221151,57sleep30-21swapper/307:07:433
14992200161,26sleep00-21swapper/007:05:380
2462421640,5sleep3195899cyclictest09:10:223
135521410,3sleep2195499cyclictest10:45:212
1954991132,109cyclictest0-21swapper/207:28:122
1954991132,109cyclictest0-21swapper/207:28:122
1954991070,105cyclictest0-21swapper/210:31:572
1954997269,2cyclictest0-21swapper/211:34:542
5172490,1sleep3519-21grep08:15:253
194299477,12cyclictest19834-21munin-run11:30:020
32092430,1sleep00-21swapper/007:10:200
195899391,28cyclictest0-21swapper/309:00:183
195899360,16cyclictest0-21swapper/308:20:153
194299361,13cyclictest3552-21kworker/0:111:07:020
1954993432,1cyclictest0-21swapper/208:12:242
194799344,14cyclictest2310-21kworker/u8:008:29:301
1942993410,11cyclictest9824-21awk08:40:010
1942993410,11cyclictest9824-21awk08:40:010
195899331,14cyclictest0-21swapper/312:24:523
195899331,12cyclictest0-21swapper/312:10:223
195499335,6cyclictest0-21swapper/210:07:532
1954993221,2cyclictest25087-21kworker/u8:107:10:352
195499320,10cyclictest0-21swapper/210:55:122
194299322,11cyclictest7118-21kworker/0:112:16:570
1958993119,2cyclictest0-21swapper/307:20:223
195899311,6cyclictest0-21swapper/308:39:233
195899311,6cyclictest0-21swapper/308:39:233
195899311,19cyclictest0-21swapper/307:10:153
195499311,11cyclictest0-21swapper/209:17:492
194299312,12cyclictest12520-21kworker/0:110:07:120
194299312,10cyclictest13192-21kworker/0:211:55:120
194299311,11cyclictest3552-21kworker/0:111:03:020
195899301,12cyclictest0-21swapper/309:08:123
195499305,5cyclictest0-21swapper/209:20:452
194799301,9cyclictest0-21swapper/111:38:571
194799301,8cyclictest0-21swapper/108:36:531
194799301,8cyclictest0-21swapper/108:36:531
194799301,13cyclictest0-21swapper/107:41:171
194799300,13cyclictest0-21swapper/110:07:051
194299307,11cyclictest11674-21perf12:25:010
1958992915,3cyclictest0-21swapper/308:58:043
195899291,20cyclictest0-21swapper/310:06:053
1958992911,4cyclictest0-21swapper/311:50:233
195899291,13cyclictest0-21swapper/310:22:393
195899290,24cyclictest0-21swapper/308:14:053
1954992911,4cyclictest15496-21ntp_states08:50:212
194799291,13cyclictest0-21swapper/110:24:361
194799291,12cyclictest0-21swapper/110:55:481
194799290,21cyclictest0-21swapper/108:10:161
194299292,11cyclictest6298-21kworker/0:010:58:570
195899281,1cyclictest0-21swapper/308:31:393
195899281,13cyclictest0-21swapper/309:58:153
195899281,12cyclictest0-21swapper/311:09:323
195899280,19cyclictest0-21swapper/308:00:333
195899280,11cyclictest0-21swapper/308:27:333
195499285,5cyclictest0-21swapper/210:04:012
194799281,9cyclictest0-21swapper/107:54:171
194799281,11cyclictest0-21swapper/109:59:371
194799281,11cyclictest0-21swapper/107:36:291
194799280,9cyclictest0-21swapper/111:53:181
194799280,10cyclictest0-21swapper/109:05:291
194299289,3cyclictest13192-21kworker/0:212:06:290
194299289,3cyclictest13192-21kworker/0:212:06:290
194299286,3cyclictest32314-21kworker/0:210:44:170
194299282,13cyclictest5423-21munin-run08:30:010
194299282,11cyclictest3552-21kworker/0:111:15:100
194299281,11cyclictest7494-21kworker/0:208:49:530
194299281,11cyclictest7225-21kworker/0:107:40:360
194299281,11cyclictest23473-21kworker/0:010:27:010
1958992711,4cyclictest0-21swapper/312:35:153
195899271,13cyclictest0-21swapper/312:28:243
195899271,12cyclictest0-21swapper/311:18:053
195899271,12cyclictest0-21swapper/310:45:113
195899270,5cyclictest0-21swapper/308:45:193
195899270,12cyclictest0-21swapper/308:05:213
194799271,12cyclictest0-21swapper/109:26:071
194799271,11cyclictest0-21swapper/108:51:491
194799271,10cyclictest0-21swapper/111:21:131
194799271,10cyclictest0-21swapper/107:48:091
194799270,2cyclictest0-21swapper/110:38:401
194299273,2cyclictest5805-21kworker/0:009:57:280
194299272,10cyclictest930-21kworker/0:108:20:170
194299272,10cyclictest26321-21kworker/0:011:47:260
194299272,10cyclictest13192-21kworker/0:211:38:170
194299271,13cyclictest1283-21kworker/0:109:45:040
194299271,11cyclictest13192-21kworker/0:211:24:570
195899266,3cyclictest0-21swapper/310:35:013
195899261,9cyclictest0-21swapper/307:27:443
195899261,9cyclictest0-21swapper/307:27:443
195899261,18cyclictest0-21swapper/310:28:213
195899261,12cyclictest0-21swapper/310:52:363
195899261,11cyclictest17429-21cstates10:10:123
195899261,10cyclictest0-21swapper/310:35:333
195899260,2cyclictest0-21swapper/310:00:233
195899260,17cyclictest0-21swapper/311:00:173
195899260,16cyclictest0-21swapper/311:36:163
195899260,12cyclictest0-21swapper/311:27:333
195899260,12cyclictest0-21swapper/311:20:083
195899260,12cyclictest0-21swapper/307:47:093
195899260,11cyclictest0-21swapper/308:41:173
195899260,10cyclictest0-21swapper/308:50:143
195499261,5cyclictest0-21swapper/209:38:582
1954992611,3cyclictest15787-21munin-run07:40:012
1954992611,3cyclictest10497-21irqstats12:20:172
194799261,8cyclictest0-21swapper/112:15:411
194799261,8cyclictest0-21swapper/107:22:011
194799261,18cyclictest0-21swapper/109:35:531
194799261,12cyclictest0-21swapper/109:17:511
194799261,10cyclictest0-21swapper/112:31:201
194799261,10cyclictest0-21swapper/112:05:361
194799261,10cyclictest0-21swapper/112:05:361
194799260,9cyclictest0-21swapper/109:14:051
194799260,7cyclictest0-21swapper/107:55:201
194799260,12cyclictest0-21swapper/111:41:341
194799260,12cyclictest0-21swapper/111:34:541
194799260,12cyclictest0-21swapper/110:41:131
194799260,10cyclictest0-21swapper/109:31:171
194299269,3cyclictest7225-21kworker/0:107:25:150
194299269,3cyclictest7225-21kworker/0:107:25:140
194299262,10cyclictest11581-21kworker/0:212:36:540
194299261,10cyclictest7225-21kworker/0:108:02:090
194299261,10cyclictest1283-21kworker/0:109:32:360
195899252,13cyclictest87650irq/129-enp1s0-09:41:173
195899252,13cyclictest87650irq/129-enp1s0-09:41:173
195899251,9cyclictest0-21swapper/310:17:253
195899251,10cyclictest23963-21kworker/u8:011:45:253
195899250,9cyclictest0-21swapper/310:41:093
195899250,9cyclictest0-21swapper/307:39:053
195499257,3cyclictest0-21swapper/211:46:222
195499257,3cyclictest0-21swapper/209:57:572
195499256,3cyclictest0-21swapper/210:41:002
1954992510,4cyclictest15071-21strings12:30:172
194799251,8cyclictest22999-21diskmemload10:19:401
194799251,16cyclictest0-21swapper/110:11:251
194799251,11cyclictest0-21swapper/108:05:481
194799251,10cyclictest0-21swapper/108:02:101
194799251,10cyclictest0-21swapper/107:13:121
194799250,11cyclictest0-21swapper/112:01:181
194799250,10cyclictest0-21swapper/112:36:081
194799250,10cyclictest0-21swapper/110:47:171
194799250,10cyclictest0-21swapper/110:32:241
194299251,9cyclictest30081-21kworker/0:010:45:090
194299251,13cyclictest12520-21kworker/0:110:02:200
1942992511,2cyclictest5805-21kworker/0:009:52:240
194299251,10cyclictest4901-21kworker/0:012:10:400
195899246,11cyclictest0-21swapper/309:49:573
195899244,5cyclictest0-21swapper/307:53:053
195899243,10cyclictest0-21swapper/310:56:063
195899241,9cyclictest0-21swapper/307:58:453
1958992411,3cyclictest0-21swapper/312:05:443
1958992411,3cyclictest0-21swapper/312:05:443
1958992411,3cyclictest0-21swapper/307:42:053
195899240,7cyclictest0-21swapper/312:19:053
195899240,11cyclictest0-21swapper/311:10:193
195899240,11cyclictest0-21swapper/309:35:173
195899240,11cyclictest0-21swapper/309:25:533
195499249,3cyclictest0-21swapper/209:49:372
195499249,2cyclictest0-21swapper/209:29:092
195499247,3cyclictest0-21swapper/211:36:412
195499247,2cyclictest0-21swapper/208:35:242
195499247,2cyclictest0-21swapper/208:35:242
195499241,10cyclictest0-21swapper/212:18:302
194799241,9cyclictest0-21swapper/107:34:371
194799241,16cyclictest0-21swapper/111:45:051
194799241,11cyclictest0-21swapper/112:20:351
194799241,11cyclictest0-21swapper/108:30:371
194799241,10cyclictest0-21swapper/109:02:511
194799240,8cyclictest0-21swapper/108:41:531
194799240,11cyclictest0-21swapper/109:49:431
194799240,10cyclictest0-21swapper/111:09:431
194799240,10cyclictest0-21swapper/109:21:211
194299247,3cyclictest21103-21kworker/0:210:31:050
1942992410,3cyclictest13192-21kworker/0:211:44:410
15252240,4sleep3195899cyclictest12:00:153
195899238,3cyclictest0-21swapper/311:30:293
195899238,2cyclictest0-21swapper/307:30:253
195899232,11cyclictest26010-21perl09:15:153
195899231,10cyclictest0-21swapper/309:50:073
195899230,11cyclictest0-21swapper/311:41:413
195899230,10cyclictest0-21swapper/309:21:523
195499238,3cyclictest0-21swapper/207:49:332
195499236,5cyclictest0-21swapper/211:07:012
195499231,6cyclictest21823-21cat10:20:142
194799232,11cyclictest7208-21df_inode11:00:121
194799231,9cyclictest0-21swapper/107:15:311
194799231,8cyclictest0-21swapper/109:42:191
194799231,8cyclictest0-21swapper/109:42:191
194799231,11cyclictest0-21swapper/112:26:121
194799231,11cyclictest0-21swapper/111:10:541
194799231,11cyclictest0-21swapper/108:19:121
194799230,9cyclictest0-21swapper/111:15:581
194799230,10cyclictest0-21swapper/111:57:021
194799230,10cyclictest0-21swapper/108:46:531
194299237,4cyclictest27112-21kworker/0:209:40:200
194299237,4cyclictest27112-21kworker/0:209:40:200
194299235,5cyclictest755-21rngd12:00:240
194299232,11cyclictest1798-21kworker/0:207:24:410
194299232,11cyclictest1283-21kworker/0:109:38:280
194299231,3cyclictest0-21swapper/011:52:360
194299231,13cyclictest27821-21kworker/0:110:38:050
194299231,11cyclictest7225-21kworker/0:107:46:480
194299231,10cyclictest21027-21kworker/0:109:14:290
194299231,10cyclictest13843-21kworker/0:012:33:450
1942992310,3cyclictest21103-21kworker/0:210:18:000
1942992310,2cyclictest31079-21kworker/0:308:15:010
195899226,3cyclictest0-21swapper/311:55:123
195899220,10cyclictest0-21swapper/307:16:533
195499226,3cyclictest0-21swapper/210:14:492
195499226,3cyclictest0-21swapper/209:14:292
195499224,5cyclictest19448-21users11:25:252
195499224,10cyclictest12017-21awk08:45:002
195499222,7cyclictest16013-21/usr/sbin/munin07:40:182
195499221,6cyclictest0-21swapper/210:52:342
195499220,16cyclictest0-21swapper/211:10:262
194799221,8cyclictest0-21swapper/108:55:571
194799220,9cyclictest0-21swapper/109:50:291
194299229,2cyclictest28947-21kworker/0:208:10:130
194299228,5cyclictest13468-21uptime12:25:270
194299223,2cyclictest87650irq/129-enp1s0-10:12:190
194299221,10cyclictest2476-21/usr/sbin/munin10:50:190
194299220,11cyclictest0-21swapper/010:20:130
195899214,13cyclictest13861-40dnf12:30:043
195899211,13cyclictest0-21swapper/309:32:113
195499219,3cyclictest0-21swapper/211:00:242
195499219,3cyclictest0-21swapper/208:02:092
195499215,8cyclictest5750irq/9-acpi09:30:142
195499212,3cyclictest87350irq/126-enp1s0-11:55:242
195499211,9cyclictest5200-40dnf08:27:222
195499211,6cyclictest6996-21sh07:20:002
195499211,4cyclictest0-21swapper/210:37:542
1954992111,3cyclictest0-21swapper/212:36:542
1954992111,3cyclictest0-21swapper/208:20:172
195499211,11cyclictest0-21swapper/211:15:102
194799211,15cyclictest0-21swapper/111:25:571
194799211,11cyclictest24905-21ntp_states10:25:181
194799211,11cyclictest14874-21cat10:05:021
194799211,11cyclictest0-21swapper/108:20:201
194799210,9cyclictest0-21swapper/110:51:081
194799210,10cyclictest0-21swapper/112:10:471
194299219,3cyclictest19785-21kworker/0:011:31:130
194299218,3cyclictest87650irq/129-enp1s0-07:15:070
194299211,8cyclictest9739-21kworker/0:008:40:170
194299211,3cyclictest7225-21kworker/0:107:57:140
194299211,12cyclictest21027-21kworker/0:109:20:410
1942992110,2cyclictest21027-21kworker/0:109:16:170
195499209,3cyclictest0-21swapper/208:49:522
195499208,3cyclictest0-21swapper/209:00:052
195499207,3cyclictest0-21swapper/212:10:412
195499206,3cyclictest0-21swapper/210:27:012
195499204,3cyclictest0-21swapper/209:41:212
195499204,3cyclictest0-21swapper/209:41:212
195499201,7cyclictest1550-21which12:00:162
195499201,6cyclictest0-21swapper/212:08:372
195499201,6cyclictest0-21swapper/212:08:372
195499201,10cyclictest18597-21munin-run09:00:022
194799201,11cyclictest0-21swapper/107:25:131
194799201,11cyclictest0-21swapper/107:25:131
194299202,11cyclictest13843-21kworker/0:108:50:130
194299201,7cyclictest20409-21/usr/sbin/munin07:50:160
194299200,9cyclictest29553-21kworker/0:009:25:040
195499197,2cyclictest0-21swapper/207:50:132
195499196,2cyclictest87550irq/128-enp1s0-07:55:222
195499195,3cyclictest87550irq/128-enp1s0-11:20:182
195499195,3cyclictest87350irq/126-enp1s0-10:15:192
195499195,3cyclictest5192grep08:15:252
195499191,7cyclictest0-21swapper/212:26:422
195499191,7cyclictest0-21swapper/207:20:252
195499191,6cyclictest8457-21/usr/sbin/munin09:50:152
195499191,3cyclictest24986-21fschecks_time11:40:152
195499190,6cyclictest13588-21crond07:35:012
195499190,10cyclictest0-21swapper/211:54:542
194299196,2cyclictest7225-21kworker/0:107:35:120
194299191,6cyclictest27012-21/usr/sbin/munin08:05:130
194299191,5cyclictest22618-21users09:05:250
194299191,5cyclictest19737-21memory09:00:180
194299191,15cyclictest5645-21/usr/sbin/munin08:30:210
194299190,9cyclictest0-21swapper/007:30:160
195499184,3cyclictest87350irq/126-enp1s0-09:05:172
195499181,9cyclictest27012-21/usr/sbin/munin08:05:242
194299183,3cyclictest18088-21/usr/sbin/munin08:55:240
1954991712,3cyclictest5645-21/usr/sbin/munin08:30:242
194299171,6cyclictest0-21swapper/011:10:050
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional