You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 02:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Mon Apr 20, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1293324637,0sleep20-21swapper/218:53:052
1922324132,0sleep30-21swapper/318:57:023
1893523524,0sleep00-21swapper/018:53:390
1899023423,0sleep10-21swapper/118:54:201
35160,0ktimersoftd/30-21swapper/322:52:043
339960,0migration/38981-21sh22:12:353
339960,0migration/331638-21needreboot21:23:173
339960,0migration/330848-21sh23:44:453
339960,0migration/324325-21sh21:59:463
339960,0migration/322789-21sh23:18:453
339960,0migration/315918-21sh22:35:313
29725060,0irq/27-eth0-tx-17875-21munin-node21:38:152
29715060,0irq/26-eth0-rx-31648-21ssh22:23:403
29715060,0irq/26-eth0-rx-12703-21cat21:13:163
193789966,0cyclictest17496-21diskmemload21:14:392
193769960,0cyclictest29306-21ps21:23:150
149960,0migration/031872-21sh22:03:320
149960,0migration/017703-21kthreadcore20:08:170
149960,0migration/014731-21sh23:55:120
149960,0migration/012824-21kthreadcore21:13:160
8215050,0irq/24-0000:00:8472-21cat23:12:572
8215050,0irq/24-0000:00:26170-21cat19:02:592
8215050,0irq/24-0000:00:17936-21cat21:17:582
35150,0ktimersoftd/30-21swapper/321:31:013
339950,0migration/3771-21munin-node19:08:113
339950,0migration/36569-21kthreadcore21:48:163
339950,0migration/33893-21taskset20:45:213
339950,0migration/331971-21taskset00:08:043
339950,0migration/331822-21taskset23:05:503
339950,0migration/331019-21latency22:03:163
339950,0migration/328526-21kthreadcore21:43:163
339950,0migration/327579-21taskset19:48:573
339950,0migration/323923-21kthreadcore19:23:153
339950,0migration/318327-21cpuspeed_turbos21:18:113
339950,0migration/317102-21df_abs20:33:113
339950,0migration/316525-21kthreadcore19:43:173
339950,0migration/315278-21taskset23:36:033
339950,0migration/311821-21kthreadcore20:03:163
339950,0migration/310709-21taskset20:51:223
29725050,0irq/27-eth0-tx-9839-21ssh21:33:032
29725050,0irq/27-eth0-tx-8237-21ssh21:09:262
29725050,0irq/27-eth0-tx-5924-21ssh00:08:392
29725050,0irq/27-eth0-tx-3514-21snmpd00:04:072
29725050,0irq/27-eth0-tx-32761-21ssh21:24:402
29725050,0irq/27-eth0-tx-29813-21open_files23:43:182
29725050,0irq/27-eth0-tx-26650-21if_eth122:03:132
29725050,0irq/27-eth0-tx-25909-21ssh21:43:062
29725050,0irq/27-eth0-tx-25438-21munin-node22:23:132
29725050,0irq/27-eth0-tx-0-21swapper/223:27:532
29725050,0irq/27-eth0-tx-0-21swapper/223:00:552
29725050,0irq/27-eth0-tx-0-21swapper/221:58:182
29725050,0irq/27-eth0-tx-0-21swapper/200:23:172
29725050,0irq/27-eth0-tx-0-21swapper/200:01:152
29725050,0irq/27-eth0-tx-0-21swapper/020:28:100
29725050,0irq/27-eth0-tx-0-21swapper/019:28:200
29715050,0irq/26-eth0-rx-29452-21sshd20:38:473
29715050,0irq/26-eth0-rx-23831-21sort00:02:573
29715050,0irq/26-eth0-rx-17904-21head20:58:113
29715050,0irq/26-eth0-rx-10849-21kthreadcore00:13:183
29715050,0irq/26-eth0-rx-10617-21kthreadcore19:38:163
29715050,0irq/26-eth0-rx-0-21swapper/323:49:143
29715050,0irq/26-eth0-rx-0-21swapper/323:28:083
29715050,0irq/26-eth0-rx-0-21swapper/323:08:163
29715050,0irq/26-eth0-rx-0-21swapper/323:02:133
29715050,0irq/26-eth0-rx-0-21swapper/322:53:163
29715050,0irq/26-eth0-rx-0-21swapper/322:43:113
29715050,0irq/26-eth0-rx-0-21swapper/322:38:113
29715050,0irq/26-eth0-rx-0-21swapper/322:28:323
29715050,0irq/26-eth0-rx-0-21swapper/322:19:333
29715050,0irq/26-eth0-rx-0-21swapper/322:13:183
29715050,0irq/26-eth0-rx-0-21swapper/321:56:173
29715050,0irq/26-eth0-rx-0-21swapper/321:08:073
29715050,0irq/26-eth0-rx-0-21swapper/321:04:013
29715050,0irq/26-eth0-rx-0-21swapper/300:21:363
29715050,0irq/26-eth0-rx-0-21swapper/300:03:143
259950,0migration/28431-21sh22:52:422
259950,0migration/25496-21munin-node23:48:202
259950,0migration/226987-21kthreadcore23:03:132
259950,0migration/221703-21kthreadcore20:58:162
259950,0migration/215039-21sh22:53:412
259950,0migration/214888-21switchtime22:33:202
249950,0watchdog/23826-21kthreadcore22:08:162
249950,0watchdog/220535-21kthreadcore22:18:162
249950,0watchdog/215966-21ssh21:53:522
193789955,0cyclictest0-21swapper/223:54:402
193789955,0cyclictest0-21swapper/222:43:152
193789955,0cyclictest0-21swapper/200:15:582
193789950,0cyclictest0-21swapper/223:36:232
193789950,0cyclictest0-21swapper/222:39:282
193789950,0cyclictest0-21swapper/222:16:272
193779955,0cyclictest0-21swapper/123:53:381
193779955,0cyclictest0-21swapper/123:13:451
193779955,0cyclictest0-21swapper/122:00:451
193779955,0cyclictest0-21swapper/121:08:531
193779950,0cyclictest0-21swapper/123:45:261
193779950,0cyclictest0-21swapper/123:00:401
193779950,0cyclictest0-21swapper/121:18:021
193779950,0cyclictest0-21swapper/100:01:501
193769955,0cyclictest0-21swapper/023:08:150
193769950,0cyclictest0-21swapper/023:47:310
193769950,0cyclictest0-21swapper/023:25:080
193769950,0cyclictest0-21swapper/020:59:550
179950,0migration/17833-21missed_timers19:58:181
179950,0migration/131495-21sh00:27:571
179950,0migration/126030-21idleruntime-cro19:02:591
179950,0migration/123455-21sendmail_mailqu20:58:201
179950,0migration/115763-21aten_r1power_en00:18:101
179950,0migration/11327-21latency_hist21:47:581
169950,0watchdog/19828-21fschecks_time20:03:131
169950,0watchdog/12346-21munin-node21:48:121
149950,0migration/09903-21aten2.4-expect21:33:090
149950,0migration/04822-21kthreadcore00:08:160
149950,0migration/04435-21kthreadcore19:58:150
149950,0migration/04336-21kthreadcore19:33:150
149950,0migration/04260-21kthreadcore21:08:160
149950,0migration/0349-21fschecks_time23:48:130
149950,0migration/030879-21taskset00:05:340
149950,0migration/02869-21kthreadcore19:08:150
149950,0migration/028554-21kthreadcore22:23:150
149950,0migration/027749-21kthreadcore22:43:150
149950,0migration/026662-21taskset19:24:500
149950,0migration/026535-21fschecks_time21:43:120
149950,0migration/023046-21taskset00:21:440
149950,0migration/022484-21timerandwakeup20:33:210
149950,0migration/02100-21df21:28:110
149950,0migration/020479-21kthreadcore22:18:160
149950,0migration/017749-21kthreadcore23:38:140
149950,0migration/016535-21fschecks_time23:58:110
149950,0migration/015985-21irqcore19:43:160
149950,0migration/015709-21munin-node21:53:200
149950,0migration/014815-21sh20:03:210
149950,0migration/014781-21ksmtuned23:34:360
149950,0migration/013891-21kthreadcore20:53:160
339940,0migration/38598-21kthreadcore20:23:173
339940,0migration/328948-21latency_hist20:17:583
339940,0migration/328751-21kthreadcore19:28:153
339940,0migration/328537-21kthreadcore19:03:153
339940,0migration/32844-21taskset20:19:573
339940,0migration/327686-21kthreadcore00:23:173
339940,0migration/324168-21diskmemload21:38:423
339940,0migration/322206-21taskset20:12:483
339940,0migration/314901-21irqcore19:18:153
29725040,0irq/27-eth0-tx-8656-21ssh21:50:582
29725040,0irq/27-eth0-tx-15677-21ssh00:18:022
29725040,0irq/27-eth0-tx-0-21swapper/223:08:162
29725040,0irq/27-eth0-tx-0-21swapper/221:28:082
29725040,0irq/27-eth0-tx-0-21swapper/020:22:370
29725040,0irq/27-eth0-tx-0-21swapper/020:13:100
29715040,0irq/26-eth0-rx-24984-21fschecks_count23:23:113
29715040,0irq/26-eth0-rx-0-21swapper/323:53:163
29715040,0irq/26-eth0-rx-0-21swapper/323:38:023
29715040,0irq/26-eth0-rx-0-21swapper/323:13:093
29715040,0irq/26-eth0-rx-0-21swapper/321:34:203
29715040,0irq/26-eth0-rx-0-21swapper/320:28:103
29715040,0irq/26-eth0-rx-0-21swapper/319:57:593
29715040,0irq/26-eth0-rx-0-21swapper/319:53:103
259940,0migration/29269-21kthreadcore20:48:182
259940,0migration/26118-21kthreadcore22:28:162
259940,0migration/226886-21kthreadcore20:13:172
259940,0migration/217755-21kthreadcore19:43:172
259940,0migration/217005-21irqrtprio20:08:162
259940,0migration/216340-21aten_r1power_po23:38:092
259940,0migration/213847-21latency_hist19:17:592
249940,0watchdog/24881-21grep23:28:162
249940,0watchdog/231215-21grep21:03:162
249940,0watchdog/222822-21seq20:34:252
249940,0watchdog/216546-21ssh23:18:082
249940,0watchdog/215115-21grep20:53:162
249940,0watchdog/214299-21seq19:39:252
249940,0watchdog/212895-21kthreadcore20:03:172
249940,0watchdog/20-21swapper/220:43:162
249940,0watchdog/20-21swapper/220:23:172
193789944,0cyclictest29342-21cut19:53:152
193789943,0cyclictest3556-21lldpd19:59:102
193789943,0cyclictest3556-21lldpd19:02:102
193789943,0cyclictest28130-21awk19:28:142
193789943,0cyclictest24355-21awk20:38:142
193789943,0cyclictest2260-21awk19:33:122
193789943,0cyclictest22419-21awk19:48:142
193789943,0cyclictest21523-21awk19:23:132
193779944,0cyclictest17496-21diskmemload22:07:131
193779944,0cyclictest0-21swapper/123:19:451
193779943,0cyclictest3556-21lldpd22:44:091
193779943,0cyclictest3556-21lldpd22:38:391
193779943,0cyclictest3556-21lldpd22:23:081
193779943,0cyclictest3556-21lldpd21:56:091
193779943,0cyclictest3556-21lldpd21:27:091
193779943,0cyclictest3514-21snmpd23:33:101
193779943,0cyclictest3514-21snmpd23:08:101
193779943,0cyclictest3514-21snmpd19:39:091
193779943,0cyclictest3241-21irqbalance22:15:161
193779943,0cyclictest29058-21awk19:53:131
193779943,0cyclictest25705-21awk23:03:111
193779943,0cyclictest23330-21awk20:13:141
193779943,0cyclictest22370-21awk19:48:141
193779943,0cyclictest18274-21awk22:18:131
193779943,0cyclictest15717-21awk19:43:151
193779943,0cyclictest10000-21awk22:33:121
193769944,0cyclictest18452-21kworker/0:122:58:090
193769944,0cyclictest18452-21kworker/0:122:49:490
193769944,0cyclictest18452-21kworker/0:122:35:590
193769944,0cyclictest18452-21kworker/0:122:14:290
193769944,0cyclictest18452-21kworker/0:100:13:010
193769944,0cyclictest16144-21kworker/0:221:38:190
193769944,0cyclictest16144-21kworker/0:221:19:060
193769944,0cyclictest16144-21kworker/0:221:04:120
193769944,0cyclictest16144-21kworker/0:219:38:160
193769944,0cyclictest0-21swapper/020:23:110
193769944,0cyclictest0-21swapper/000:26:270
193769943,0cyclictest3556-21lldpd23:28:110
193769943,0cyclictest3556-21lldpd23:13:090
193769943,0cyclictest3514-21snmpd23:19:400
193769943,0cyclictest3514-21snmpd22:56:400
193769943,0cyclictest3514-21snmpd18:58:540
193769943,0cyclictest27218-21awk19:03:140
193769940,0cyclictest0-21swapper/019:52:580
179940,0migration/18318-21kthreadcore20:48:171
179940,0migration/15422-21kthreadcore22:28:161
179940,0migration/14527-21kthreadcore23:28:161
179940,0migration/13507-21aten2.4-expect20:23:111
179940,0migration/126042-21kthreadcore00:03:151
179940,0migration/125185-21kthreadcore20:38:151
179940,0migration/12353-21fschecks_time21:28:121
179940,0migration/116194-21runrttasks21:33:541
169940,0watchdog/13825-21kthreadcore22:08:161
169940,0watchdog/13783-21kthreadcore22:48:161
169940,0watchdog/129726-21fschecks_time20:18:131
169940,0watchdog/129357-21tr23:23:161
169940,0watchdog/121906-21tr21:38:161
169940,0watchdog/121440-21cut20:33:171
169940,0watchdog/118425-21kthreadcore19:18:171
169940,0watchdog/114574-21python22:53:201
169940,0watchdog/10-21swapper/121:03:161
169940,0watchdog/10-21swapper/120:53:161
169940,0watchdog/10-21swapper/100:08:161
149940,0migration/05137-21kthreadcore22:28:160
149940,0migration/02924-21kthreadcore22:08:160
149940,0migration/019150-21kthreadcore21:58:150
149940,0migration/017070-21runrttasks22:38:090
149940,0migration/016872-21kthreadcore19:18:160
149940,0migration/01507-21sh21:48:060
149940,0migration/01461-21kthreadcore20:43:170
149940,0migration/01253-21pmu-power19:53:190
29715030,0irq/26-eth0-rx-0-21swapper/320:53:103
29715030,0irq/26-eth0-rx-0-21swapper/319:33:093
29715030,0irq/26-eth0-rx-0-21swapper/319:13:113
29715030,0irq/26-eth0-rx-0-21swapper/318:58:093
249930,0watchdog/20-21swapper/220:28:532
249930,0watchdog/20-21swapper/220:18:132
249930,0watchdog/20-21swapper/219:13:172
193789932,0cyclictest3135-21cut19:08:162
193779932,0cyclictest28012-21awk19:28:131
193779932,0cyclictest2347-21awk19:33:131
193779932,0cyclictest17131-21awk23:38:121
193779930,0cyclictest0-21swapper/119:08:451
193769933,0cyclictest18452-21kworker/0:123:03:010
193769933,0cyclictest16144-21kworker/0:220:48:120
193769933,0cyclictest16144-21kworker/0:220:38:180
179930,0migration/16231-21sh23:48:531
179930,0migration/129889-21kthreadcore19:03:151
169930,0watchdog/117561-21kthreadcore20:08:171
169930,0watchdog/10-21swapper/121:13:001
169930,0watchdog/10-21swapper/120:43:161
169930,0watchdog/10-21swapper/119:13:171
169930,0watchdog/10-21swapper/100:13:161
149930,0migration/07557-21runrttasks19:13:120
193779921,0cyclictest10026-21cat20:27:581
169920,0watchdog/10-21swapper/119:23:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional