You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-02 - 23:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Fri May 02, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
291350440,0irq/25-eth10-21swapper/306:46:283
124923624,0sleep20-21swapper/206:44:002
133823423,0sleep00-21swapper/006:45:020
119823430,0sleep10-21swapper/106:43:271
4160,0ktimersoftd/00-21swapper/008:59:420
339960,0migration/319907-21sh10:01:053
28415060,0irq/27-eth0-tx-5763-21munin-node08:27:280
28415060,0irq/27-eth0-tx-0-21swapper/212:07:482
28415060,0irq/27-eth0-tx-0-21swapper/212:02:112
28415060,0irq/27-eth0-tx-0-21swapper/211:54:272
28415060,0irq/27-eth0-tx-0-21swapper/211:51:332
28415060,0irq/27-eth0-tx-0-21swapper/211:42:572
28415060,0irq/27-eth0-tx-0-21swapper/211:38:292
28415060,0irq/27-eth0-tx-0-21swapper/211:28:222
28415060,0irq/27-eth0-tx-0-21swapper/210:57:542
28415060,0irq/27-eth0-tx-0-21swapper/210:52:222
28415060,0irq/27-eth0-tx-0-21swapper/210:50:492
28415060,0irq/27-eth0-tx-0-21swapper/210:19:232
28415060,0irq/27-eth0-tx-0-21swapper/210:15:012
28415060,0irq/27-eth0-tx-0-21swapper/210:11:182
28415060,0irq/27-eth0-tx-0-21swapper/210:02:532
28415060,0irq/27-eth0-tx-0-21swapper/209:54:032
28415060,0irq/27-eth0-tx-0-21swapper/209:43:452
28415060,0irq/27-eth0-tx-0-21swapper/209:14:122
28415060,0irq/27-eth0-tx-0-21swapper/209:09:262
28415060,0irq/27-eth0-tx-0-21swapper/209:00:482
28415060,0irq/27-eth0-tx-0-21swapper/208:54:192
28415060,0irq/27-eth0-tx-0-21swapper/008:17:250
28415060,0irq/27-eth0-tx-0-21swapper/008:02:290
249960,0watchdog/26372-21iceccd10:32:212
249960,0watchdog/20-21swapper/211:59:452
249960,0watchdog/20-21swapper/211:22:292
249960,0watchdog/20-21swapper/210:28:412
249960,0watchdog/20-21swapper/207:37:252
249960,0watchdog/20-21swapper/207:10:532
15919960,0cyclictest0-21swapper/209:37:182
15899965,0cyclictest1475-21tail07:07:310
149960,0migration/025910-21df_abs11:07:250
149960,0migration/023099-21aten_r1power_vo09:22:270
149960,0migration/010670-21munin-node10:57:240
8225050,0irq/24-0000:00:22664-21idleruntime-cro07:27:102
339950,0migration/39602-21munin-node11:57:223
339950,0migration/332706-21kthreadcore07:07:303
339950,0migration/332312-21taskset11:09:223
339950,0migration/33024-21kthreadcore12:12:303
339950,0migration/329402-21kthreadcore10:27:293
339950,0migration/328128-21kthreadcore08:17:313
339950,0migration/327338-21kthreadcore07:52:293
339950,0migration/32564-21missed_timers09:47:303
339950,0migration/32546-21munin-node07:32:333
339950,0migration/32544-21kthreadcore07:57:313
339950,0migration/325273-21aten_r1power_fr11:47:203
339950,0migration/324725-21taskset11:27:093
339950,0migration/324486-21kthreadcore07:27:253
339950,0migration/32419-21fschecks_time06:47:273
339950,0migration/323539-21kthreadcore10:02:293
339950,0migration/322543-21munin-node09:42:273
339950,0migration/321744-21latency_hist07:02:103
339950,0migration/31520-21gltestperf11:32:253
339950,0migration/315125-21missed_timers10:57:273
339950,0migration/310378-21munin-node08:02:343
28415050,0irq/27-eth0-tx-9504-21munin-node07:17:270
28415050,0irq/27-eth0-tx-6372-21iceccd07:47:490
28415050,0irq/27-eth0-tx-10591-21munin-node07:42:340
28415050,0irq/27-eth0-tx-0-21swapper/211:07:262
28415050,0irq/27-eth0-tx-0-21swapper/210:23:122
28415050,0irq/27-eth0-tx-0-21swapper/210:23:122
28415050,0irq/27-eth0-tx-0-21swapper/209:57:212
28415050,0irq/27-eth0-tx-0-21swapper/209:32:442
28415050,0irq/27-eth0-tx-0-21swapper/209:22:302
28415050,0irq/27-eth0-tx-0-21swapper/209:05:322
28415050,0irq/27-eth0-tx-0-21swapper/208:50:172
28415050,0irq/27-eth0-tx-0-21swapper/007:33:470
28415050,0irq/27-eth0-tx-0-21swapper/006:57:250
28405050,0irq/26-eth0-rx-8093-21ssh11:54:573
28405050,0irq/26-eth0-rx-7381-21cpu09:12:243
28405050,0irq/26-eth0-rx-20966-21kthreadcore10:22:273
28405050,0irq/26-eth0-rx-20966-21kthreadcore10:22:273
28405050,0irq/26-eth0-rx-0-21swapper/312:09:493
28405050,0irq/26-eth0-rx-0-21swapper/311:45:013
28405050,0irq/26-eth0-rx-0-21swapper/311:39:103
28405050,0irq/26-eth0-rx-0-21swapper/311:29:553
28405050,0irq/26-eth0-rx-0-21swapper/311:14:153
28405050,0irq/26-eth0-rx-0-21swapper/310:52:213
28405050,0irq/26-eth0-rx-0-21swapper/310:37:293
28405050,0irq/26-eth0-rx-0-21swapper/310:17:553
28405050,0irq/26-eth0-rx-0-21swapper/310:07:313
28405050,0irq/26-eth0-rx-0-21swapper/309:39:343
28405050,0irq/26-eth0-rx-0-21swapper/309:19:313
28405050,0irq/26-eth0-rx-0-21swapper/309:07:323
28405050,0irq/26-eth0-rx-0-21swapper/308:57:303
28405050,0irq/26-eth0-rx-0-21swapper/308:52:463
259950,0migration/211517-21switchtime08:27:322
249950,0watchdog/24022-21cut11:32:292
249950,0watchdog/215983-21kthreadcore10:37:292
249950,0watchdog/20-21swapper/212:13:132
249950,0watchdog/20-21swapper/211:17:532
249950,0watchdog/20-21swapper/211:12:572
249950,0watchdog/20-21swapper/211:02:572
249950,0watchdog/20-21swapper/210:42:412
249950,0watchdog/20-21swapper/209:47:332
249950,0watchdog/20-21swapper/209:18:172
249950,0watchdog/20-21swapper/208:47:092
249950,0watchdog/20-21swapper/208:18:372
249950,0watchdog/20-21swapper/208:07:172
249950,0watchdog/20-21swapper/207:57:372
249950,0watchdog/20-21swapper/207:52:252
249950,0watchdog/20-21swapper/207:29:292
249950,0watchdog/20-21swapper/207:14:092
249950,0watchdog/20-21swapper/207:04:012
179950,0migration/18754-21kthreadcore09:32:331
179950,0migration/14041-21timerwakeupswit07:57:371
179950,0migration/131675-21aten_r1power_po09:07:261
179950,0migration/129987-21taskset07:53:441
179950,0migration/129775-21cpuspeed_turbos09:47:241
179950,0migration/127377-21switchtime09:42:341
179950,0migration/114776-21latency_hist09:17:111
15909954,0cyclictest4093-21kworker/1:111:18:181
15909954,0cyclictest4093-21kworker/1:110:55:541
15909954,0cyclictest4093-21kworker/1:110:44:141
15909954,0cyclictest4093-21kworker/1:110:26:261
15909954,0cyclictest4093-21kworker/1:110:26:261
15909954,0cyclictest26546-21kworker/1:208:50:281
15909950,0cyclictest0-21swapper/111:46:171
15909950,0cyclictest0-21swapper/110:13:441
15899955,0cyclictest0-21swapper/011:27:090
15899955,0cyclictest0-21swapper/010:10:030
15899955,0cyclictest0-21swapper/009:42:370
149950,0migration/09993-21munin-node11:37:260
149950,0migration/09852-21kthreadcore09:12:290
149950,0migration/08484-21kthreadcore07:37:290
149950,0migration/0824-21kthreadcore09:27:300
149950,0migration/04636-21kthreadcore11:52:280
149950,0migration/04393-21kthreadcore09:07:330
149950,0migration/04119-21kthreadcore10:52:290
149950,0migration/04105-21sh09:50:240
149950,0migration/03513-21fschecks_time10:32:260
149950,0migration/032660-21kthreadcore07:57:310
149950,0migration/028615-21irqrtprio10:27:280
149950,0migration/02822-21munin-node06:47:290
149950,0migration/027895-21kthreadcore08:42:280
149950,0migration/025625-21taskset11:47:220
149950,0migration/022041-21kthreadcore10:42:290
149950,0migration/021421-21latency_hist09:42:110
149950,0migration/020995-21kthreadcore11:02:310
149950,0migration/020506-21kthreadcore12:02:290
149950,0migration/019781-21gltestperf08:37:280
149950,0migration/01783-21kthreadcore08:47:290
149950,0migration/015680-21kthreadcore08:32:300
149950,0migration/013432-21kthreadcore10:37:280
8225040,0irq/24-0000:00:0-21swapper/207:32:452
339940,0migration/39527-21kthreadcore09:32:333
339940,0migration/35593-21latency_hist08:27:113
339940,0migration/327379-21irqcore10:47:343
339940,0migration/325347-21kthreadcore07:02:303
339940,0migration/315867-21taskset07:21:213
339940,0migration/315442-21kthreadcore08:07:293
339940,0migration/314925-21python11:17:343
28415040,0irq/27-eth0-tx-2934-21tail07:12:260
28415040,0irq/27-eth0-tx-0-21swapper/007:02:240
28405040,0irq/26-eth0-rx-22922-40yum-updatesd-he09:22:273
28405040,0irq/26-eth0-rx-0-21swapper/312:02:283
28405040,0irq/26-eth0-rx-0-21swapper/311:02:313
28405040,0irq/26-eth0-rx-0-21swapper/310:45:503
28405040,0irq/26-eth0-rx-0-21swapper/310:32:173
28405040,0irq/26-eth0-rx-0-21swapper/310:12:263
28405040,0irq/26-eth0-rx-0-21swapper/309:52:173
28405040,0irq/26-eth0-rx-0-21swapper/309:28:543
28405040,0irq/26-eth0-rx-0-21swapper/309:04:593
28405040,0irq/26-eth0-rx-0-21swapper/308:47:563
28405040,0irq/26-eth0-rx-0-21swapper/307:37:223
259940,0migration/29508-21aten_r1power_cu07:17:212
259940,0migration/23199-21kthreadcore06:47:302
249940,0watchdog/20-21swapper/208:32:132
249940,0watchdog/20-21swapper/208:22:332
249940,0watchdog/20-21swapper/207:47:292
249940,0watchdog/20-21swapper/207:42:252
249940,0watchdog/20-21swapper/207:01:132
179940,0migration/14731-21idleruntime07:37:261
179940,0migration/127794-21kthreadcore10:47:341
179940,0migration/127720-21kthreadcore08:17:301
179940,0migration/122393-21fschecks_count07:02:261
179940,0migration/12212-21kthreadcore11:32:281
179940,0migration/121245-21kthreadcore08:57:331
179940,0migration/118668-21kthreadcore11:22:271
179940,0migration/118578-21fschecks_count08:12:261
179940,0migration/110937-21irqrtprio10:57:251
169940,0watchdog/18945-21diskstats06:52:251
169940,0watchdog/131450-21unixbench_multi12:07:331
169940,0watchdog/127092-21cut11:07:291
169940,0watchdog/10-21swapper/109:57:291
15919943,0cyclictest3425-21lldpd08:39:322
15919943,0cyclictest3425-21lldpd06:52:162
15919943,0cyclictest31532-21awk09:27:282
15919943,0cyclictest3110-21irqbalance08:02:152
15919943,0cyclictest19054-21awk08:12:282
15909944,0cyclictest4093-21kworker/1:112:12:241
15909944,0cyclictest4093-21kworker/1:111:57:421
15909944,0cyclictest4093-21kworker/1:111:52:101
15909944,0cyclictest4093-21kworker/1:111:47:181
15909944,0cyclictest4093-21kworker/1:111:37:241
15909944,0cyclictest4093-21kworker/1:111:27:141
15909944,0cyclictest4093-21kworker/1:111:12:101
15909944,0cyclictest4093-21kworker/1:111:02:141
15909944,0cyclictest4093-21kworker/1:110:38:181
15909944,0cyclictest4093-21kworker/1:110:32:181
15909944,0cyclictest4093-21kworker/1:110:27:181
15909944,0cyclictest4093-21kworker/1:110:17:281
15909944,0cyclictest4093-21kworker/1:109:54:181
15909944,0cyclictest4093-21kworker/1:109:37:161
15909944,0cyclictest4093-21kworker/1:109:28:141
15909944,0cyclictest28751-21kworker/1:008:42:141
15909944,0cyclictest28751-21kworker/1:008:38:381
15909944,0cyclictest28751-21kworker/1:008:32:121
15909944,0cyclictest28751-21kworker/1:008:27:241
15909944,0cyclictest28751-21kworker/1:008:23:141
15909944,0cyclictest28751-21kworker/1:008:08:041
15909944,0cyclictest28751-21kworker/1:007:47:181
15909944,0cyclictest28751-21kworker/1:007:43:221
15909944,0cyclictest28751-21kworker/1:007:22:321
15909944,0cyclictest28751-21kworker/1:007:18:041
15909944,0cyclictest28751-21kworker/1:007:12:201
15909944,0cyclictest28751-21kworker/1:007:08:081
15909944,0cyclictest28751-21kworker/1:006:57:161
15909944,0cyclictest28751-21kworker/1:006:47:441
15909944,0cyclictest26546-21kworker/1:209:23:201
15909944,0cyclictest26546-21kworker/1:209:02:481
15909944,0cyclictest26546-21kworker/1:208:52:261
15909944,0cyclictest0-21swapper/112:04:251
15909943,0cyclictest6445-21snmpd10:09:461
15909943,0cyclictest6445-21snmpd10:02:461
15909943,0cyclictest6445-21snmpd08:02:151
15909943,0cyclictest3425-21lldpd07:32:171
15909943,0cyclictest23717-21awk07:27:231
15909943,0cyclictest15898-21awk09:17:271
15899944,0cyclictest0-21swapper/009:55:240
15899944,0cyclictest0-21swapper/009:36:290
15899943,0cyclictest6445-21snmpd10:17:160
15899943,0cyclictest6445-21snmpd08:52:310
15899943,0cyclictest20443-21awk10:22:260
15899943,0cyclictest20443-21awk10:22:260
15899943,0cyclictest19030-21awk08:12:280
15899943,0cyclictest18068-21awk11:42:240
15899943,0cyclictest13741-21awk09:57:270
149940,0migration/06039-21kthreadcore12:12:310
149940,0migration/03837-21kthreadcore11:32:290
149940,0migration/02901-21kthreadcore11:12:290
149940,0migration/025560-21kthreadcore07:27:250
149940,0migration/017404-21irqrtprio07:22:380
149940,0migration/012902-21kthreadcore08:07:280
149940,0migration/011318-21sh10:16:440
28415030,0irq/27-eth0-tx-8638-21aten2.4-expect06:52:230
28415030,0irq/27-eth0-tx-0-21swapper/008:22:210
28405030,0irq/26-eth0-rx-0-21swapper/308:37:253
28405030,0irq/26-eth0-rx-0-21swapper/308:32:113
28405030,0irq/26-eth0-rx-0-21swapper/308:13:483
28405030,0irq/26-eth0-rx-0-21swapper/307:48:063
28405030,0irq/26-eth0-rx-0-21swapper/307:42:223
28405030,0irq/26-eth0-rx-0-21swapper/306:52:243
15899933,0cyclictest24754-21awk07:52:270
15899932,0cyclictest27192-21awk10:47:330
15899932,0cyclictest26446-21tr11:27:270
15899932,0cyclictest15735-21grep09:17:260
15899932,0cyclictest14909-21sensors11:17:340
15899930,0cyclictest24133-21diskstats09:02:260
28405020,0irq/26-eth0-rx-0-21swapper/308:42:123
28405020,0irq/26-eth0-rx-0-21swapper/308:27:223
28405020,0irq/26-eth0-rx-0-21swapper/307:22:473
28405020,0irq/26-eth0-rx-0-21swapper/307:12:293
15899921,0cyclictest25424-21basename12:07:210
15899920,0cyclictest0-21swapper/011:57:230
15899920,0cyclictest0-21swapper/010:02:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional