You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-12 - 21:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Fri Dec 12, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31979210298,0sleep30-21swapper/318:37:053
2577728887,0sleep20-21swapper/218:33:562
3204623525,0sleep10-21swapper/118:37:511
3176723423,0sleep00-21swapper/018:34:380
322089970,0cyclictest14178-21munin-node21:09:081
322079970,0cyclictest0-21swapper/020:45:260
28415070,0irq/27-eth0-tx-0-21swapper/221:08:092
249970,0watchdog/20-21swapper/219:20:522
339960,0migration/35079-21sh21:01:323
322099960,0cyclictest0-21swapper/221:59:082
322079966,0cyclictest0-21swapper/023:34:460
28415060,0irq/27-eth0-tx-0-21swapper/223:53:392
28415060,0irq/27-eth0-tx-0-21swapper/223:39:442
28415060,0irq/27-eth0-tx-0-21swapper/223:34:242
28415060,0irq/27-eth0-tx-0-21swapper/223:29:252
28415060,0irq/27-eth0-tx-0-21swapper/223:26:312
28415060,0irq/27-eth0-tx-0-21swapper/223:21:342
28415060,0irq/27-eth0-tx-0-21swapper/223:10:542
28415060,0irq/27-eth0-tx-0-21swapper/222:59:532
28415060,0irq/27-eth0-tx-0-21swapper/222:58:182
28415060,0irq/27-eth0-tx-0-21swapper/222:49:272
28415060,0irq/27-eth0-tx-0-21swapper/222:43:002
28415060,0irq/27-eth0-tx-0-21swapper/222:33:572
28415060,0irq/27-eth0-tx-0-21swapper/222:33:112
28415060,0irq/27-eth0-tx-0-21swapper/222:19:582
28415060,0irq/27-eth0-tx-0-21swapper/222:18:042
28415060,0irq/27-eth0-tx-0-21swapper/221:52:282
28415060,0irq/27-eth0-tx-0-21swapper/221:13:232
28415060,0irq/27-eth0-tx-0-21swapper/220:43:452
28415060,0irq/27-eth0-tx-0-21swapper/200:00:082
28415060,0irq/27-eth0-tx-0-21swapper/018:54:140
28415060,0irq/27-eth0-tx-0-21swapper/018:54:140
28415060,0irq/27-eth0-tx-0-21swapper/018:49:080
28405060,0irq/26-eth0-rx-31313-21kthreadcore20:14:133
28405060,0irq/26-eth0-rx-25862-21df_abs23:19:073
28405060,0irq/26-eth0-rx-20086-21cut23:34:123
28405060,0irq/26-eth0-rx-11040-21munin-node21:24:163
28405060,0irq/26-eth0-rx-0-21swapper/323:24:143
28405060,0irq/26-eth0-rx-0-21swapper/322:14:083
179960,0migration/15605-21munin-node18:39:151
169960,0watchdog/10-21swapper/120:58:521
169960,0watchdog/10-21swapper/100:02:431
149960,0migration/030006-21sh21:18:550
4150,0ktimersoftd/00-21swapper/021:37:190
4150,0ktimersoftd/00-21swapper/021:02:560
35150,0ktimersoftd/30-21swapper/323:00:473
339950,0migration/38989-21sh20:44:133
339950,0migration/37523-21rt-features23:04:163
339950,0migration/37014-21unixbench_singl19:04:163
339950,0migration/35444-21kthreadcore22:04:113
339950,0migration/34953-21aten_r1power_fr21:44:073
339950,0migration/329862-21kthreadcore22:39:133
339950,0migration/327895-21sh21:57:493
339950,0migration/327679-21munin-node18:59:093
339950,0migration/32762-21latency_hist19:53:533
339950,0migration/327050-21kthreadcore23:39:153
339950,0migration/322665-21taskset20:08:283
339950,0migration/31964-21kthreadcore00:04:143
339950,0migration/318008-21sh22:31:093
339950,0migration/316887-21taskset20:27:373
339950,0migration/316672-21kthreadcore19:14:093
339950,0migration/314995-21munin-node21:09:103
339950,0migration/314460-21munin-node18:49:113
339950,0migration/313352-21fschecks_time21:49:073
339950,0migration/31107-21munin-node22:19:183
339950,0migration/310297-21diskmemload23:49:123
322099955,0cyclictest0-21swapper/221:53:562
322089955,0cyclictest23301-21kthreadcore19:44:091
322089955,0cyclictest0-21swapper/123:19:411
322089955,0cyclictest0-21swapper/122:12:121
322089955,0cyclictest0-21swapper/121:35:371
322089955,0cyclictest0-21swapper/120:19:091
322089954,0cyclictest21181-21ssh20:51:101
322089950,0cyclictest0-21swapper/123:48:411
322089950,0cyclictest0-21swapper/121:58:011
322089950,0cyclictest0-21swapper/121:30:441
322089950,0cyclictest0-21swapper/100:07:461
322079954,0cyclictest23046-21awk21:14:120
322079954,0cyclictest12466-21ps22:29:130
322079950,0cyclictest0-21swapper/000:04:050
28415050,0irq/27-eth0-tx-5607-21munin-node21:24:192
28415050,0irq/27-eth0-tx-3633-21ssh20:39:140
28415050,0irq/27-eth0-tx-22485-21munin-node20:54:182
28415050,0irq/27-eth0-tx-21037-21munin-node19:19:160
28415050,0irq/27-eth0-tx-15324-21munin-node19:39:100
28415050,0irq/27-eth0-tx-0-21swapper/223:55:272
28415050,0irq/27-eth0-tx-0-21swapper/222:06:242
28415050,0irq/27-eth0-tx-0-21swapper/221:40:322
28415050,0irq/27-eth0-tx-0-21swapper/221:20:332
28415050,0irq/27-eth0-tx-0-21swapper/220:59:102
28415050,0irq/27-eth0-tx-0-21swapper/220:50:262
28415050,0irq/27-eth0-tx-0-21swapper/220:45:132
28415050,0irq/27-eth0-tx-0-21swapper/200:06:532
28415050,0irq/27-eth0-tx-0-21swapper/020:04:100
28415050,0irq/27-eth0-tx-0-21swapper/019:44:040
28415050,0irq/27-eth0-tx-0-21swapper/019:09:130
28415050,0irq/27-eth0-tx-0-21swapper/019:04:050
28415050,0irq/27-eth0-tx-0-21swapper/018:39:000
28405050,0irq/26-eth0-rx-8924-21cat23:28:533
28405050,0irq/26-eth0-rx-8205-21port_sshpriv22:44:143
28405050,0irq/26-eth0-rx-29815-21kthreadcore19:24:103
28405050,0irq/26-eth0-rx-27941-21sed20:54:193
28405050,0irq/26-eth0-rx-24656-21ntpdc22:34:143
28405050,0irq/26-eth0-rx-2447-21kthreadcore18:39:113
28405050,0irq/26-eth0-rx-19736-21grep22:54:123
28405050,0irq/26-eth0-rx-0-21swapper/323:55:023
28405050,0irq/26-eth0-rx-0-21swapper/323:14:113
28405050,0irq/26-eth0-rx-0-21swapper/322:49:163
28405050,0irq/26-eth0-rx-0-21swapper/321:39:113
28405050,0irq/26-eth0-rx-0-21swapper/321:38:293
28405050,0irq/26-eth0-rx-0-21swapper/321:31:003
28405050,0irq/26-eth0-rx-0-21swapper/320:39:203
259950,0migration/225395-21missed_timers18:54:112
259950,0migration/225395-21missed_timers18:54:112
259950,0migration/222031-21latency_hist21:13:532
249950,0watchdog/28343-21python22:44:152
249950,0watchdog/26058-21python23:44:192
249950,0watchdog/25163-21irqrtprio20:19:122
249950,0watchdog/22879-21if_eth1.219:29:082
249950,0watchdog/220745-21kthreadcore23:14:112
179950,0migration/17093-21munin-node20:44:111
179950,0migration/15548-21kthreadcore22:24:121
179950,0migration/14013-21sh20:39:181
179950,0migration/131324-21kthreadcore22:59:121
179950,0migration/126326-21missed_timers19:19:131
179950,0migration/116227-21rt-features22:49:191
179950,0migration/112268-21kthreadcore19:34:121
169950,0watchdog/16547-21runrttasks21:20:201
169950,0watchdog/15389-21kthreadcore22:04:121
169950,0watchdog/10-21swapper/123:41:271
169950,0watchdog/10-21swapper/123:05:151
169950,0watchdog/10-21swapper/122:00:521
169950,0watchdog/10-21swapper/121:04:121
169950,0watchdog/10-21swapper/120:59:121
149950,0migration/08223-21taskset19:31:190
149950,0migration/07065-21cpuspeed_turbos18:44:060
149950,0migration/03827-21fschecks_count22:24:080
149950,0migration/03498-21taskset21:40:310
149950,0migration/031958-21latency18:59:110
149950,0migration/029939-21kthreadcore21:59:140
149950,0migration/027974-21kthreadcore22:59:110
149950,0migration/027021-21taskset22:17:230
149950,0migration/025233-21kthreadcore20:54:180
149950,0migration/024808-21sh23:17:380
149950,0migration/022295-21kthreadcore21:54:110
149950,0migration/020397-21unixbench_multi20:49:210
149950,0migration/018048-21latency21:49:120
149950,0migration/01638-21needreboot19:49:130
149950,0migration/014135-21fschecks_time21:29:070
149950,0migration/013510-21kthreadcore20:24:110
149950,0migration/013507-21sh21:07:500
149950,0migration/012455-21fschecks_count22:09:070
149950,0migration/01118-21timerwakeupswit19:24:160
339940,0migration/38930-21irqrtprio19:09:113
339940,0migration/36470-21sh23:44:333
339940,0migration/330026-21kthreadcore21:59:143
339940,0migration/327245-21kthreadcore20:09:113
339940,0migration/312844-21runrttasks18:44:483
322099943,0cyclictest9526-21awk19:34:092
322099943,0cyclictest5791-21awk21:44:112
322099943,0cyclictest3844-21awk19:54:122
322099943,0cyclictest3425-21lldpd20:31:222
322099943,0cyclictest3425-21lldpd19:47:222
322099943,0cyclictest3425-21lldpd18:51:522
322099943,0cyclictest30590-21awk20:14:112
322099943,0cyclictest2601-21awk23:04:092
322099943,0cyclictest13172-21awk19:09:142
322089943,0cyclictest6445-21snmpd23:14:011
322089943,0cyclictest6445-21snmpd21:26:001
322089943,0cyclictest6445-21snmpd21:14:301
322089943,0cyclictest6445-21snmpd19:30:581
322089943,0cyclictest6445-21snmpd19:10:271
322089943,0cyclictest3425-21lldpd22:56:531
322089943,0cyclictest3425-21lldpd20:24:221
322089943,0cyclictest3425-21lldpd19:39:221
322089943,0cyclictest31362-21kworker/1:221:49:131
322089943,0cyclictest24761-21awk20:34:131
322089943,0cyclictest23758-21kworker/1:119:59:311
322089943,0cyclictest18081-21awk20:29:101
322089943,0cyclictest17255-21awk20:04:121
322079944,0cyclictest0-21swapper/023:59:410
322079943,0cyclictest9935-21awk23:29:100
322079943,0cyclictest8767-21sadc23:28:530
322079943,0cyclictest6528-21awk21:24:120
322079943,0cyclictest4964-21awk22:04:090
322079943,0cyclictest4832-21awk20:19:110
322079943,0cyclictest3425-21lldpd23:51:290
322079943,0cyclictest3425-21lldpd20:35:520
322079943,0cyclictest3425-21lldpd20:29:220
322079943,0cyclictest3110-21irqbalance23:54:070
322079943,0cyclictest3110-21irqbalance22:54:170
322079943,0cyclictest3110-21irqbalance22:49:170
322079943,0cyclictest3110-21irqbalance22:44:070
322079943,0cyclictest3110-21irqbalance22:39:270
322079943,0cyclictest3110-21irqbalance22:34:270
322079943,0cyclictest3110-21irqbalance21:09:070
322079943,0cyclictest3110-21irqbalance19:53:580
322079943,0cyclictest3110-21irqbalance19:34:280
322079943,0cyclictest3110-21irqbalance19:14:080
322079943,0cyclictest30517-21awk20:14:110
322079943,0cyclictest25787-21awk23:39:130
322079943,0cyclictest10521-21awk19:59:080
28415040,0irq/27-eth0-tx-0-21swapper/222:23:562
28405040,0irq/26-eth0-rx-4007-21perl22:24:083
28405040,0irq/26-eth0-rx-2994-21aten_r1power_en19:54:073
28405040,0irq/26-eth0-rx-0-21swapper/323:12:583
28405040,0irq/26-eth0-rx-0-21swapper/322:10:123
28405040,0irq/26-eth0-rx-0-21swapper/321:23:003
28405040,0irq/26-eth0-rx-0-21swapper/321:14:143
28405040,0irq/26-eth0-rx-0-21swapper/321:05:553
28405040,0irq/26-eth0-rx-0-21swapper/320:49:323
28405040,0irq/26-eth0-rx-0-21swapper/319:39:193
28405040,0irq/26-eth0-rx-0-21swapper/319:33:533
28405040,0irq/26-eth0-rx-0-21swapper/300:00:083
259940,0migration/26327-21munin-node19:04:112
259940,0migration/23835-21kthreadcore18:39:122
259940,0migration/229244-21kthreadcore18:59:112
259940,0migration/228566-21processes20:09:142
259940,0migration/223380-21kthreadcore21:34:162
259940,0migration/217160-21kthreadcore19:39:112
259940,0migration/215143-21kthreadcore20:24:112
259940,0migration/214373-21runrttasks19:13:562
259940,0migration/214314-21munin-node21:29:082
249940,0watchdog/211982-21ssh22:09:042
249940,0watchdog/20-21swapper/220:04:082
249940,0watchdog/20-21swapper/218:44:202
179940,0migration/19867-21kthreadcore18:44:111
179940,0migration/1608-21kthreadcore20:14:131
179940,0migration/15009-21kthreadcore22:44:111
179940,0migration/13547-21kthreadcore23:24:131
179940,0migration/132548-21timerwakeupswit18:59:161
179940,0migration/132502-21kthreadcore21:39:111
179940,0migration/130606-21kthreadcore19:49:111
179940,0migration/124497-21kthreadcore20:09:101
179940,0migration/120326-21fschecks_count22:14:051
179940,0migration/118478-21kthreadcore18:49:131
179940,0migration/117776-21idleruntime23:34:091
179940,0migration/114906-21fschecks_count19:14:061
179940,0migration/114328-21kthreadcore23:29:131
179940,0migration/113719-21kthreadcore23:09:111
179940,0migration/110302-21kthreadcore23:49:121
169940,0watchdog/16950-21unixbench_multi19:04:161
169940,0watchdog/121681-21smartctl23:54:191
169940,0watchdog/12072-21ssh22:21:041
169940,0watchdog/10-21swapper/122:41:191
169940,0watchdog/10-21swapper/122:34:121
169940,0watchdog/10-21swapper/122:31:281
169940,0watchdog/10-21swapper/121:46:441
169940,0watchdog/10-21swapper/119:54:041
169940,0watchdog/10-21swapper/119:24:121
149940,0migration/029352-21kthreadcore22:19:130
149940,0migration/027312-21kthreadcore23:19:110
149940,0migration/012922-21kthreadcore23:09:110
322099933,0cyclictest3425-21lldpd20:01:222
322099932,0cyclictest3425-21lldpd20:38:522
322099931,0cyclictest29182-21kthreadcore19:24:102
322089933,0cyclictest23758-21kworker/1:118:53:551
322089933,0cyclictest23758-21kworker/1:118:53:551
322079932,0cyclictest5659-21hwlatdetect21:44:100
322079932,0cyclictest3425-21lldpd23:06:540
322079932,0cyclictest1189-21awk23:44:120
28415030,0irq/27-eth0-tx-0-21swapper/020:09:040
28405030,0irq/26-eth0-rx-0-21swapper/320:38:173
28405030,0irq/26-eth0-rx-0-21swapper/320:30:343
28405030,0irq/26-eth0-rx-0-21swapper/320:18:553
28405030,0irq/26-eth0-rx-0-21swapper/319:58:573
28405030,0irq/26-eth0-rx-0-21swapper/319:48:303
28405030,0irq/26-eth0-rx-0-21swapper/319:34:053
28405030,0irq/26-eth0-rx-0-21swapper/319:23:033
28405030,0irq/26-eth0-rx-0-21swapper/318:54:163
28405030,0irq/26-eth0-rx-0-21swapper/318:54:163
249930,0watchdog/20-21swapper/219:49:082
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional