You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 10:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Aug 20, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450350,0irq/25-eth10-21swapper/318:42:223
1371223422,0sleep00-21swapper/018:46:070
1356223425,0sleep20-21swapper/218:44:252
735222622,0sleep10-21swapper/118:41:281
8215060,0irq/24-0000:00:1433-21munin-run22:21:212
35160,0ktimersoftd/30-21swapper/321:37:203
339960,0migration/38821-21sh22:02:343
339960,0migration/36967-21sendmail_mailqu22:41:413
339960,0migration/33246-21sh21:21:013
339960,0migration/326785-21sh20:53:033
339960,0migration/325238-21sh21:52:173
339960,0migration/311628-21munin-node23:06:353
29715060,0irq/26-eth0-rx-6653-21kthreadcore20:16:383
29715060,0irq/26-eth0-rx-23203-21ssh23:13:393
29715060,0irq/26-eth0-rx-20793-21fschecks_time21:11:353
29715060,0irq/26-eth0-rx-18106-21munin-node22:51:363
29715060,0irq/26-eth0-rx-15586-21munin-run23:31:213
149960,0migration/07764-21latency22:01:370
149960,0migration/07372-21sh23:03:410
149960,0migration/027888-21latency19:41:390
149960,0migration/015706-21munin-node22:06:400
149960,0migration/01167-21aten_r1power_en22:41:310
138039960,0cyclictest0-21swapper/122:53:571
138039960,0cyclictest0-21swapper/122:50:481
8215050,0irq/24-0000:00:27861-21awk21:16:222
8215050,0irq/24-0000:00:20465-21munin-run18:51:222
339950,0migration/38152-21idleruntime23:46:343
339950,0migration/37396-21taskset23:25:403
339950,0migration/37096-21kthreadcore19:26:393
339950,0migration/36269-21sh00:05:463
339950,0migration/330574-21munin-node19:46:383
339950,0migration/327098-21kthreadcore23:16:393
339950,0migration/326246-21munin-node22:36:343
339950,0migration/32529-21runrttasks22:21:373
339950,0migration/32373-21idleruntime19:01:363
339950,0migration/322201-21latency20:01:403
339950,0migration/322147-21sh00:15:483
339950,0migration/321199-21timerwakeupswit23:51:433
339950,0migration/319507-21taskset20:50:163
339950,0migration/318187-21kthreadcore19:11:383
339950,0migration/316900-21taskset22:08:043
339950,0migration/31434-21if_eth023:01:353
339950,0migration/313124-21munin-node21:06:383
339950,0migration/313026-21runrttasks00:07:003
339950,0migration/312863-21kthreadcore19:56:393
29725050,0irq/27-eth0-tx-9165-21ssh22:46:302
29725050,0irq/27-eth0-tx-6086-21cut22:01:372
29725050,0irq/27-eth0-tx-2548-21munin-node19:26:330
29725050,0irq/27-eth0-tx-25026-21munin-node22:56:412
29725050,0irq/27-eth0-tx-23817-21munin-node23:36:372
29725050,0irq/27-eth0-tx-15963-21munin-node19:36:480
29725050,0irq/27-eth0-tx-15020-21munin-node19:11:370
29725050,0irq/27-eth0-tx-12148-21munin-node21:06:392
29725050,0irq/27-eth0-tx-10348-21ssh21:23:192
29725050,0irq/27-eth0-tx-0-21swapper/223:56:382
29725050,0irq/27-eth0-tx-0-21swapper/223:43:522
29725050,0irq/27-eth0-tx-0-21swapper/222:24:222
29725050,0irq/27-eth0-tx-0-21swapper/221:50:382
29725050,0irq/27-eth0-tx-0-21swapper/221:31:272
29725050,0irq/27-eth0-tx-0-21swapper/220:56:452
29725050,0irq/27-eth0-tx-0-21swapper/020:48:090
29725050,0irq/27-eth0-tx-0-21swapper/020:35:520
29725050,0irq/27-eth0-tx-0-21swapper/019:34:090
29715050,0irq/26-eth0-rx-5153-21kthreadcore21:41:383
29715050,0irq/26-eth0-rx-30017-21kthreadcore21:56:383
29715050,0irq/26-eth0-rx-28366-21tail20:56:343
29715050,0irq/26-eth0-rx-18346-21fschecks_time22:31:353
29715050,0irq/26-eth0-rx-0-21swapper/323:56:253
29715050,0irq/26-eth0-rx-0-21swapper/323:44:143
29715050,0irq/26-eth0-rx-0-21swapper/323:31:323
29715050,0irq/26-eth0-rx-0-21swapper/323:01:093
29715050,0irq/26-eth0-rx-0-21swapper/322:48:283
29715050,0irq/26-eth0-rx-0-21swapper/322:26:383
29715050,0irq/26-eth0-rx-0-21swapper/322:18:323
29715050,0irq/26-eth0-rx-0-21swapper/322:15:563
29715050,0irq/26-eth0-rx-0-21swapper/321:21:303
27150,0ktimersoftd/20-21swapper/221:37:202
259950,0migration/29832-21kthreadcore23:06:342
259950,0migration/231091-21kthreadcore18:56:392
259950,0migration/217188-21ntp_offset21:26:412
259950,0migration/214630-21latency_hist00:11:212
259950,0migration/211313-21switchtime20:41:432
259950,0migration/211313-21switchtime20:41:432
249950,0watchdog/230368-21cat20:31:442
249950,0watchdog/22214-21missed_timers19:46:402
249950,0watchdog/218773-21fschecks_count22:11:352
19150,0ktimersoftd/10-21swapper/121:44:261
179950,0migration/1528-21irqrtprio23:41:391
179950,0migration/14545-21fschecks_count21:01:351
179950,0migration/129446-21kthreadcore21:56:381
179950,0migration/12927-21vmstat19:46:451
179950,0migration/12318-21sh21:18:141
149950,0migration/09121-21sh22:46:220
149950,0migration/09034-21sh22:25:090
149950,0migration/08783-21kthreadcore00:06:390
149950,0migration/07919-21fschecks_time23:46:330
149950,0migration/05868-21sh23:42:510
149950,0migration/032149-21kthreadcore19:21:380
149950,0migration/030925-21kthreadcore21:36:390
149950,0migration/03012-21sh21:41:300
149950,0migration/029883-21kthreadcore22:36:380
149950,0migration/028176-21kthreadcore23:36:410
149950,0migration/027781-21fschecks_count18:56:340
149950,0migration/025114-21sh22:13:540
149950,0migration/024322-21missed_timers21:51:390
149950,0migration/022552-21missed_timers22:51:380
149950,0migration/022513-21sh23:55:070
149950,0migration/021054-21idleruntime21:11:360
149950,0migration/018591-21diskmemload21:51:050
149950,0migration/01581-21kthreadcore19:46:390
149950,0migration/010443-21kthreadcore19:06:370
149950,0migration/010154-21kthreadcore23:06:340
149950,0migration/010150-21proc_pri20:16:420
138049950,0cyclictest0-21swapper/223:21:532
138049950,0cyclictest0-21swapper/223:03:352
138049950,0cyclictest0-21swapper/221:53:172
138049950,0cyclictest0-21swapper/221:17:202
138039955,0cyclictest0-21swapper/123:33:321
138039955,0cyclictest0-21swapper/120:49:241
138039955,0cyclictest0-21swapper/118:49:451
138039950,0cyclictest0-21swapper/123:25:291
138039950,0cyclictest0-21swapper/121:12:541
138039950,0cyclictest0-21swapper/120:57:361
138039950,0cyclictest0-21swapper/119:29:471
138029950,0cyclictest0-21swapper/022:29:440
8215040,0irq/24-0000:00:612-21ssh21:57:032
8215040,0irq/24-0000:00:30034-21idleruntime-cro20:11:222
339940,0migration/3550-21switchtime18:56:433
339940,0migration/33596-21taskset19:51:273
339940,0migration/327176-21kthreadcore19:41:383
339940,0migration/321443-21gltestperf18:51:353
339940,0migration/31438-21kthreadcore20:36:383
339940,0migration/311485-21kthreadcore19:31:363
29725040,0irq/27-eth0-tx-3015-21ssh21:41:312
29725040,0irq/27-eth0-tx-23596-21kthreadcore20:51:382
29725040,0irq/27-eth0-tx-12023-21diskmemload00:06:292
29725040,0irq/27-eth0-tx-10361-21telnet19:56:320
29725040,0irq/27-eth0-tx-0-21swapper/223:46:422
29725040,0irq/27-eth0-tx-0-21swapper/223:32:402
29725040,0irq/27-eth0-tx-0-21swapper/223:16:382
29725040,0irq/27-eth0-tx-0-21swapper/223:11:312
29725040,0irq/27-eth0-tx-0-21swapper/222:52:162
29725040,0irq/27-eth0-tx-0-21swapper/222:41:432
29725040,0irq/27-eth0-tx-0-21swapper/222:31:242
29725040,0irq/27-eth0-tx-0-21swapper/221:02:072
29725040,0irq/27-eth0-tx-0-21swapper/019:51:410
29725040,0irq/27-eth0-tx-0-21swapper/018:54:400
29715040,0irq/26-eth0-rx-0-21swapper/323:36:353
29715040,0irq/26-eth0-rx-0-21swapper/321:46:303
29715040,0irq/26-eth0-rx-0-21swapper/321:34:543
29715040,0irq/26-eth0-rx-0-21swapper/321:26:393
29715040,0irq/26-eth0-rx-0-21swapper/321:02:233
29715040,0irq/26-eth0-rx-0-21swapper/320:44:033
29715040,0irq/26-eth0-rx-0-21swapper/320:44:033
29715040,0irq/26-eth0-rx-0-21swapper/320:21:363
29715040,0irq/26-eth0-rx-0-21swapper/320:11:323
29715040,0irq/26-eth0-rx-0-21swapper/319:06:323
259940,0migration/24534-21runrttasks20:16:222
259940,0migration/228579-21cpuspeed_turbos19:21:332
259940,0migration/227500-21timerwakeupswit19:16:432
259940,0migration/21773-21kthreadcore00:01:382
259940,0migration/216601-21kernelversion23:51:362
259940,0migration/216480-21runrttasks19:58:352
259940,0migration/214904-21kthreadcore20:21:372
259940,0migration/214483-21kthreadcore22:06:392
259940,0migration/212394-21kthreadcore23:26:382
259940,0migration/211120-21kthreadcore19:31:362
249940,0watchdog/231233-21if_eth1.320:11:362
249940,0watchdog/223919-21users20:26:442
249940,0watchdog/218067-21switchtime20:46:442
249940,0watchdog/2162312sleep222:27:512
249940,0watchdog/20-21swapper/220:01:402
179940,0migration/19211-21kthreadcore00:06:391
179940,0migration/17671-21df_abs23:46:321
179940,0migration/17249-21kthreadcore19:51:391
179940,0migration/17073-21kthreadcore20:16:381
179940,0migration/16219-21fschecks_time20:41:341
179940,0migration/16219-21fschecks_time20:41:341
179940,0migration/131315-21aten_r1power_fr20:36:331
179940,0migration/131080-21kthreadcore21:36:391
179940,0migration/130082-21kthreadcore19:21:381
179940,0migration/128878-21kthreadcore22:36:381
179940,0migration/125889-21sh21:32:211
179940,0migration/125609-21fschecks_time22:56:331
179940,0migration/123176-21fschecks_count19:41:341
179940,0migration/121215-21kthreadcore21:51:381
179940,0migration/120787-21kthreadcore20:26:371
179940,0migration/119969-21kthreadcore19:36:411
179940,0migration/119394-21kthreadcore22:31:381
179940,0migration/119365-21latency_hist21:31:211
179940,0migration/114222-21kthreadcore19:31:371
179940,0migration/112927-21kthreadcore22:06:381
179940,0migration/112873-21kthreadcore22:26:381
179940,0migration/111083-21hwlatdetect19:56:351
179940,0migration/110707-21kthreadcore19:06:371
169940,0watchdog/18642-21ssh22:23:551
169940,0watchdog/16042-21kthreadcore21:21:361
169940,0watchdog/129292-21kthreadcore22:16:351
169940,0watchdog/128532-21kthreadcore23:16:391
169940,0watchdog/127739-21ssh20:55:441
169940,0watchdog/115590-21if_err_eth1.200:11:351
169940,0watchdog/10-21swapper/123:51:351
169940,0watchdog/10-21swapper/122:11:351
169940,0watchdog/10-21swapper/120:01:401
149940,0migration/07575-21kthreadcore20:41:380
149940,0migration/07575-21kthreadcore20:41:380
149940,0migration/031300-21aten_r1power_fr20:36:330
149940,0migration/02791-21sh21:19:410
149940,0migration/026628-21kthreadcore23:16:380
149940,0migration/022657-21kthreadcore20:26:380
149940,0migration/019432-21latency_hist21:31:210
138049943,0cyclictest3556-21lldpd22:37:122
138049943,0cyclictest3556-21lldpd19:06:402
138049943,0cyclictest3514-21snmpd20:36:572
138049943,0cyclictest3514-21snmpd19:52:112
138049943,0cyclictest16917-21awk19:36:392
138039944,0cyclictest0-21swapper/123:38:251
138039943,0cyclictest3556-21lldpd18:59:101
138039943,0cyclictest3556-21lldpd18:51:401
138039943,0cyclictest3514-21snmpd23:02:121
138039943,0cyclictest3514-21snmpd22:42:121
138039943,0cyclictest3514-21snmpd20:14:121
138039943,0cyclictest3514-21snmpd00:03:131
138039943,0cyclictest3333-21awk22:01:351
138039943,0cyclictest24494-21awk20:06:351
138039943,0cyclictest22530-21awk19:16:351
138039943,0cyclictest16000-21awk19:11:371
138029943,0cyclictest8884-21awk23:26:360
138029943,0cyclictest4559-21awk21:21:340
138029943,0cyclictest4449-21kworker/0:023:33:360
138029943,0cyclictest3556-21lldpd20:26:110
138029943,0cyclictest3514-21snmpd22:33:120
138029943,0cyclictest3514-21snmpd20:02:120
138029943,0cyclictest25897-21awk22:56:340
138029943,0cyclictest21464-21kworker/0:120:55:130
138029943,0cyclictest15610-21awk00:11:350
138029943,0cyclictest13125-21awk21:06:380
138029940,0cyclictest0-21swapper/019:01:450
339930,0migration/324355-21idleruntime-cro20:31:223
29715030,0irq/26-eth0-rx-0-21swapper/320:31:353
29715030,0irq/26-eth0-rx-0-21swapper/320:06:383
29715030,0irq/26-eth0-rx-0-21swapper/319:36:363
29715030,0irq/26-eth0-rx-0-21swapper/319:21:323
29715030,0irq/26-eth0-rx-0-21swapper/319:16:323
29715030,0irq/26-eth0-rx-0-21swapper/318:46:333
259930,0migration/24358-21kthreadcore19:26:382
259930,0migration/226826-21runrttasks18:53:072
259930,0migration/215920-21if_eth019:11:362
169930,0watchdog/10-21swapper/123:57:311
169930,0watchdog/10-21swapper/123:11:311
169930,0watchdog/10-21swapper/121:06:401
149930,0migration/04604-21perl21:01:350
138049932,0cyclictest2544-21awk19:01:372
138049932,0cyclictest23607-21awk19:41:362
138039932,0cyclictest9356-21awk23:06:321
138039932,0cyclictest3514-21snmpd23:27:421
138039932,0cyclictest3514-21snmpd20:22:421
138039932,0cyclictest25533-21awk20:31:371
138039932,0cyclictest2465-21awk19:01:371
138039932,0cyclictest11865-21awk21:46:361
138029933,0cyclictest4449-21kworker/0:023:11:240
138029933,0cyclictest4449-21kworker/0:022:16:410
138029933,0cyclictest4449-21kworker/0:021:31:230
138029933,0cyclictest30392-21kworker/0:218:46:310
138029933,0cyclictest21464-21kworker/0:120:56:250
138029933,0cyclictest21464-21kworker/0:120:11:270
138029933,0cyclictest21464-21kworker/0:119:16:230
138029932,0cyclictest781-21awk23:21:360
138029932,0cyclictest4612-21cat00:01:430
138029932,0cyclictest29536-21tail20:06:430
138029932,0cyclictest24172-21awk23:56:360
138029930,0cyclictest27244-21diskstats21:56:330
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional