You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-18 - 12:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Sat Apr 18, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450600,0irq/25-eth10-21swapper/318:55:463
2328323627,0sleep10-21swapper/118:57:061
2335523526,0sleep20-21swapper/218:57:552
2303623423,0sleep00-21swapper/018:54:130
234449988,0cyclictest3514-21snmpd19:17:150
339960,0migration/35046-21sh22:08:053
339960,0migration/34002-21sh22:46:373
339960,0migration/32655-21sh23:44:533
339960,0migration/322874-21hwlatdetect21:38:243
339960,0migration/31380-21taskset20:40:243
29715060,0irq/26-eth0-rx-3419-21mailstats21:43:293
29715060,0irq/26-eth0-rx-22316-21sh21:18:203
234469960,0cyclictest0-21swapper/223:56:002
234459966,0cyclictest21685-21diskmemload23:39:521
234449960,0cyclictest0-21swapper/023:48:080
149960,0migration/031820-21sh23:03:250
149960,0migration/02540-21sh23:23:550
149960,0migration/019155-21sh22:53:530
149960,0migration/013749-21sh21:53:180
4150,0ktimersoftd/00-21swapper/022:43:390
339950,0migration/37263-21fschecks_time19:58:233
339950,0migration/37012-21kthreadcore23:28:263
339950,0migration/36907-21irqcore21:48:223
339950,0migration/35387-21diskmemload21:07:493
339950,0migration/35221-21latency_hist22:08:093
339950,0migration/34993-21df_abs19:08:213
339950,0migration/330812-21taskset19:24:543
339950,0migration/329448-21munin-node22:23:193
339950,0migration/327812-21irqrtprio20:13:243
339950,0migration/325292-21taskset19:46:363
339950,0migration/324209-21fschecks_time18:58:233
339950,0migration/322724-21munin-node22:18:253
339950,0migration/321858-21irqrtprio23:18:253
339950,0migration/321250-21munin-node20:08:253
339950,0migration/321250-21munin-node20:08:253
339950,0migration/320823-21latency_hist22:58:093
339950,0migration/319621-21runrttasks22:34:363
339950,0migration/318783-21taskset23:13:333
339950,0migration/316307-21kthreadcore20:28:263
339950,0migration/315622-21kthreadcore19:38:263
339950,0migration/315124-21kthreadcore22:13:263
339950,0migration/314810-21kthreadcore22:53:263
339950,0migration/312858-21munin-node23:33:213
339950,0migration/310721-21taskset00:10:203
339950,0migration/310247-21kthreadcore19:33:263
29725050,0irq/27-eth0-tx-8073-21tr21:48:232
29725050,0irq/27-eth0-tx-3909-21munin-node00:08:292
29725050,0irq/27-eth0-tx-32241-21ssh22:23:262
29725050,0irq/27-eth0-tx-3122-21wc22:03:312
29725050,0irq/27-eth0-tx-29462-21sed23:23:222
29725050,0irq/27-eth0-tx-275102sleep223:20:312
29725050,0irq/27-eth0-tx-22220-21munin-node21:18:272
29725050,0irq/27-eth0-tx-21685-21diskmemload22:13:192
29725050,0irq/27-eth0-tx-21131-21munin-node22:38:262
29725050,0irq/27-eth0-tx-20343-21ssh21:34:022
29725050,0irq/27-eth0-tx-10383-21ssh23:48:462
29725050,0irq/27-eth0-tx-0-21swapper/223:45:432
29725050,0irq/27-eth0-tx-0-21swapper/223:09:172
29725050,0irq/27-eth0-tx-0-21swapper/222:36:182
29725050,0irq/27-eth0-tx-0-21swapper/222:28:262
29725050,0irq/27-eth0-tx-0-21swapper/200:21:042
29725050,0irq/27-eth0-tx-0-21swapper/200:15:322
29725050,0irq/27-eth0-tx-0-21swapper/020:58:240
29725050,0irq/27-eth0-tx-0-21swapper/020:42:300
29725050,0irq/27-eth0-tx-0-21swapper/019:32:180
29725050,0irq/27-eth0-tx-0-21swapper/019:02:130
29715050,0irq/26-eth0-rx-8300-21kthreadcore22:28:263
29715050,0irq/26-eth0-rx-2065-21ssh00:24:353
29715050,0irq/26-eth0-rx-20020-21sh00:18:203
29715050,0irq/26-eth0-rx-19509-21sensors21:53:313
29715050,0irq/26-eth0-rx-14411-21cut23:53:253
29715050,0irq/26-eth0-rx-0-21swapper/323:38:253
29715050,0irq/26-eth0-rx-0-21swapper/323:25:043
29715050,0irq/26-eth0-rx-0-21swapper/322:48:263
29715050,0irq/26-eth0-rx-0-21swapper/322:02:493
29715050,0irq/26-eth0-rx-0-21swapper/321:30:513
29715050,0irq/26-eth0-rx-0-21swapper/321:13:573
29715050,0irq/26-eth0-rx-0-21swapper/300:15:123
29715050,0irq/26-eth0-rx-0-21swapper/300:03:263
259950,0migration/228367-21gltestperf20:38:202
259950,0migration/221465-21sh21:16:572
259950,0migration/221275-21sh21:57:362
259950,0migration/212705-21aten_r1power_en23:13:192
249950,0watchdog/227839-21sh21:38:322
249950,0watchdog/21154-21kthreadcore22:43:272
249950,0watchdog/211238-21missed_timers21:08:282
234469955,0cyclictest0-21swapper/223:04:362
234469955,0cyclictest0-21swapper/222:56:432
234469955,0cyclictest0-21swapper/221:01:002
234469950,0cyclictest0-21swapper/219:28:272
234459955,0cyclictest21685-21diskmemload22:21:591
234459955,0cyclictest0-21swapper/123:14:581
234459955,0cyclictest0-21swapper/123:04:241
234459955,0cyclictest0-21swapper/121:41:001
234459954,0cyclictest3556-21lldpd21:37:541
234459950,0cyclictest0-21swapper/123:36:051
234459950,0cyclictest0-21swapper/122:31:461
234459950,0cyclictest0-21swapper/120:58:541
234449955,0cyclictest0-21swapper/023:52:420
234449954,0cyclictest22091-21kthreadcore19:43:270
234449954,0cyclictest21855-21awk22:58:240
234449950,0cyclictest0-21swapper/023:32:310
234449950,0cyclictest0-21swapper/000:13:210
179950,0migration/126218-21kthreadcore21:58:271
179950,0migration/123211-21kthreadcore23:18:261
149950,0migration/08297-21kthreadcore00:08:260
149950,0migration/06198-21runrttasks19:55:400
149950,0migration/04302-21taskset19:07:070
149950,0migration/03243-21kthreadcore20:18:260
149950,0migration/032299-21kthreadcore22:23:260
149950,0migration/030213-21latency_hist21:23:090
149950,0migration/028551-21sh22:21:240
149950,0migration/028540-21runrttasks21:39:330
149950,0migration/028386-21aten_r1power_po00:03:210
149950,0migration/027912-21taskset00:02:550
149950,0migration/027448-21sh22:38:580
149950,0migration/026678-21aten_r1power_en20:13:190
149950,0migration/024118-21kthreadcore23:38:250
149950,0migration/021606-21sh21:37:390
149950,0migration/020803-21aten_r1power_vo23:18:210
149950,0migration/018957-21munin-node19:18:240
149950,0migration/018855-21runrttasks20:03:290
149950,0migration/018121-21rt-features23:33:290
149950,0migration/016312-21kthreadcore23:13:260
149950,0migration/012909-21sh21:30:410
149950,0migration/012382-21turbostat23:13:090
8215040,0irq/24-0000:00:0-21swapper/219:53:102
339940,0migration/38892-21kthreadcore23:48:273
339940,0migration/317635-21taskset19:16:423
339940,0migration/310940-21kthreadcore20:48:263
29725040,0irq/27-eth0-tx-4363-21munin-node23:28:252
29725040,0irq/27-eth0-tx-21926-21kthreadcore23:38:242
29725040,0irq/27-eth0-tx-11567-21munin-node21:28:272
29725040,0irq/27-eth0-tx-0-21swapper/222:18:162
29725040,0irq/27-eth0-tx-0-21swapper/221:03:212
29725040,0irq/27-eth0-tx-0-21swapper/200:03:242
29725040,0irq/27-eth0-tx-0-21swapper/020:33:320
29725040,0irq/27-eth0-tx-0-21swapper/020:23:270
29725040,0irq/27-eth0-tx-0-21swapper/019:48:200
29715040,0irq/26-eth0-rx-5375-21ssh23:08:223
29715040,0irq/26-eth0-rx-0-21swapper/323:58:313
29715040,0irq/26-eth0-rx-0-21swapper/323:03:213
29715040,0irq/26-eth0-rx-0-21swapper/322:40:113
29715040,0irq/26-eth0-rx-0-21swapper/321:33:233
29715040,0irq/26-eth0-rx-0-21swapper/321:23:153
29715040,0irq/26-eth0-rx-0-21swapper/321:08:243
29715040,0irq/26-eth0-rx-0-21swapper/320:58:243
29715040,0irq/26-eth0-rx-0-21swapper/320:53:213
29715040,0irq/26-eth0-rx-0-21swapper/320:33:263
29715040,0irq/26-eth0-rx-0-21swapper/319:53:293
259940,0migration/28152-21kthreadcore19:33:262
259940,0migration/27271-21missed_timers20:43:272
259940,0migration/24850-21kthreadcore20:18:272
259940,0migration/2418-21fschecks_time19:53:222
259940,0migration/225987-21kthreadcore18:58:262
259940,0migration/224197-21kthreadcore20:33:262
259940,0migration/223352-21kthreadcore22:58:262
259940,0migration/221891-21aten_r1power_vo21:58:202
259940,0migration/219902-21kthreadcore19:18:262
259940,0migration/21880-21kthreadcore21:23:272
259940,0migration/218169-21timerwakeupswit19:38:322
259940,0migration/215908-21kthreadcore23:33:252
259940,0migration/213262-21kthreadcore19:13:272
249940,0watchdog/2946-21grep00:23:272
249940,0watchdog/25302-21ssh22:08:112
234469943,0cyclictest7609-21awk19:58:252
234469943,0cyclictest3556-21lldpd23:58:252
234469943,0cyclictest3556-21lldpd22:48:242
234469943,0cyclictest3556-21lldpd21:43:242
234469943,0cyclictest3514-21snmpd20:55:492
234469943,0cyclictest3514-21snmpd20:30:492
234469943,0cyclictest3514-21snmpd20:16:492
234469943,0cyclictest3514-21snmpd20:08:492
234469943,0cyclictest3514-21snmpd20:08:492
234469943,0cyclictest3514-21snmpd19:11:492
234469943,0cyclictest25665-21awk19:23:252
234459944,0cyclictest0-21swapper/122:48:381
234459944,0cyclictest0-21swapper/122:34:451
234459944,0cyclictest0-21swapper/100:02:151
234459943,0cyclictest6570-21awk19:33:241
234459943,0cyclictest6235-21awk22:08:231
234459943,0cyclictest5572-21awk19:08:231
234459943,0cyclictest3556-21lldpd23:43:251
234459943,0cyclictest3556-21lldpd23:28:241
234459943,0cyclictest3556-21lldpd22:59:251
234459943,0cyclictest3556-21lldpd21:45:241
234459943,0cyclictest3556-21lldpd21:17:241
234459943,0cyclictest3556-21lldpd20:53:241
234459943,0cyclictest3556-21lldpd19:30:541
234459943,0cyclictest3556-21lldpd18:58:231
234459943,0cyclictest3556-21lldpd00:11:251
234459943,0cyclictest3241-21irqbalance21:19:081
234459943,0cyclictest30670-21awk22:03:241
234459943,0cyclictest20834-21awk20:08:241
234459943,0cyclictest20834-21awk20:08:241
234459943,0cyclictest14166-21awk20:03:231
234449944,0cyclictest19337-21kworker/0:220:43:250
234449943,0cyclictest6220-21awk22:08:230
234449943,0cyclictest5676-21awk22:48:230
234449943,0cyclictest3556-21lldpd22:03:250
234449943,0cyclictest3556-21lldpd21:50:250
234449943,0cyclictest3556-21lldpd19:58:540
234449943,0cyclictest3556-21lldpd19:39:540
234449943,0cyclictest3556-21lldpd00:24:560
234449943,0cyclictest3241-21irqbalance23:53:170
234449943,0cyclictest3241-21irqbalance21:18:180
234449943,0cyclictest20943-21awk00:18:250
234449943,0cyclictest14853-21awk21:13:240
179940,0migration/17994-21kthreadcore21:08:261
179940,0migration/16409-21df_inode21:28:211
179940,0migration/131486-21kthreadcore23:23:261
179940,0migration/130901-21taskset19:25:571
179940,0migration/130493-21kthreadcore00:03:271
179940,0migration/129331-21cpu22:43:221
179940,0migration/128966-21kthreadcore19:48:261
179940,0migration/127675-21irqcore20:13:241
179940,0migration/12763-21kthreadcore21:03:281
179940,0migration/122869-21kthreadcore00:18:261
179940,0migration/122724-21kthreadcore19:43:271
179940,0migration/11835-21latency_hist20:43:101
179940,0migration/111757-21kthreadcore20:23:271
179940,0migration/111423-21kthreadcore20:48:261
179940,0migration/110478-21sh23:49:021
169940,0watchdog/121156-21aten2.4-expect22:38:191
169940,0watchdog/116210-21kthreadcore00:13:271
169940,0watchdog/112124-21taskset21:49:161
169940,0watchdog/111684-21ssh23:11:111
169940,0watchdog/10-21swapper/123:53:311
169940,0watchdog/10-21swapper/122:53:271
169940,0watchdog/10-21swapper/122:15:431
169940,0watchdog/10-21swapper/120:38:281
169940,0watchdog/10-21swapper/119:13:321
169940,0watchdog/10-21swapper/100:23:271
149940,0migration/06090-21cpuspeed_turbos21:08:220
149940,0migration/05782-21irqcore19:08:240
149940,0migration/032165-21kthreadcore21:43:250
149940,0migration/026231-21kthreadcore21:58:270
149940,0migration/019453-21runrttasks22:34:070
149940,0migration/017110-21kthreadcore20:53:260
339930,0migration/34045-21taskset19:04:063
29715030,0irq/26-eth0-rx-0-21swapper/320:18:203
29715030,0irq/26-eth0-rx-0-21swapper/320:05:203
29715030,0irq/26-eth0-rx-0-21swapper/319:50:283
29715030,0irq/26-eth0-rx-0-21swapper/319:20:273
234469933,0cyclictest0-21swapper/220:23:382
234469932,0cyclictest3514-21snmpd20:49:042
234469932,0cyclictest3514-21snmpd20:04:492
234469932,0cyclictest3514-21snmpd19:05:492
234469932,0cyclictest24665-21sensors19:43:312
234459933,0cyclictest0-21swapper/119:18:471
234459932,0cyclictest797-21awk19:53:241
234459932,0cyclictest3556-21lldpd22:23:541
234459932,0cyclictest21960-21awk20:33:241
234459932,0cyclictest17911-21kthreadcore21:53:271
234459932,0cyclictest1762-21awk20:18:251
234459930,0cyclictest0-21swapper/119:03:301
234449933,0cyclictest19337-21kworker/0:222:13:210
234449933,0cyclictest19337-21kworker/0:221:03:270
234449933,0cyclictest19337-21kworker/0:220:48:170
234449933,0cyclictest19337-21kworker/0:220:28:130
234449933,0cyclictest19337-21kworker/0:220:08:170
234449933,0cyclictest19337-21kworker/0:220:08:170
234449933,0cyclictest19337-21kworker/0:219:33:150
234449933,0cyclictest19337-21kworker/0:219:23:150
234449932,0cyclictest3241-21irqbalance22:28:180
179930,0migration/114335-21aten_r1power_cu20:28:211
169930,0watchdog/10-21swapper/121:23:281
329920,0watchdog/38004-40yum-updatesd-he20:43:573
29715020,0irq/26-eth0-rx-0-21swapper/320:23:103
29715020,0irq/26-eth0-rx-0-21swapper/319:28:253
169920,0watchdog/10-21swapper/119:58:121
169920,0watchdog/10-21swapper/119:38:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional