You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-28 - 10:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sat Mar 28, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3219924837,0sleep00-21swapper/018:55:430
3253824132,0sleep10-21swapper/118:59:411
304450350,0irq/25-eth10-21swapper/318:56:393
3230423423,0sleep20-21swapper/218:56:582
8215060,0irq/24-0000:00:14112-21idleruntime-cro21:50:082
339960,0migration/35332-21sh22:03:513
339960,0migration/34804-21sh23:05:093
339960,0migration/34758-21sh22:44:213
339960,0migration/314785-21smart_sda19:55:303
339960,0migration/312642-21sh21:06:113
326469960,0cyclictest30852-21diskmemload00:03:001
326459966,0cyclictest0-21swapper/000:22:000
29715060,0irq/26-eth0-rx-31280-21diskstats21:20:213
29715060,0irq/26-eth0-rx-29678-21forks23:00:213
29715060,0irq/26-eth0-rx-27843-21cut20:30:233
29715060,0irq/26-eth0-rx-22718-21perl22:15:213
29715060,0irq/26-eth0-rx-20166-21cut20:25:253
29715060,0irq/26-eth0-rx-14878-21irqrtprio22:30:213
29715060,0irq/26-eth0-rx-13829-21ls19:10:093
149960,0migration/04183-21sh22:21:330
8215050,0irq/24-0000:00:11714-21latency_hist23:50:072
4150,0ktimersoftd/00-21swapper/022:39:410
339950,0migration/36910-21idleruntime22:05:213
339950,0migration/36411-21kthreadcore23:25:243
339950,0migration/33250-21kthreadcore19:00:253
339950,0migration/332350-21kthreadcore00:00:243
339950,0migration/331170-21kthreadcore23:20:263
339950,0migration/328569-21taskset20:06:593
339950,0migration/328313-21aten2.4-expect19:45:213
339950,0migration/325034-21diskmemload22:35:253
339950,0migration/32443-21munin-node20:10:303
339950,0migration/32335-21munin-node19:25:243
339950,0migration/322751-21aten_r1power_fr21:15:183
339950,0migration/320820-21munin-node19:35:313
339950,0migration/319818-21munin-node22:10:303
339950,0migration/317942-21cpuspeed_turbos20:50:203
339950,0migration/315886-21kthreadcore21:50:253
339950,0migration/315854-21kthreadcore23:10:243
339950,0migration/313081-21taskset21:47:083
339950,0migration/31300-21latency_hist19:25:093
326479955,0cyclictest4441-21ssh23:25:182
326479950,0cyclictest0-21swapper/223:24:312
326479950,0cyclictest0-21swapper/222:09:102
326469950,0cyclictest0-21swapper/123:18:511
326469950,0cyclictest0-21swapper/119:20:281
326459955,0cyclictest0-21swapper/023:03:580
326459955,0cyclictest0-21swapper/022:15:430
326459955,0cyclictest0-21swapper/022:04:310
326459950,0cyclictest0-21swapper/021:22:160
29725050,0irq/27-eth0-tx-31035-21rm22:20:232
29725050,0irq/27-eth0-tx-23963-21irqstats21:15:232
29725050,0irq/27-eth0-tx-21148-21df_abs23:15:182
29725050,0irq/27-eth0-tx-19237-21ssh23:32:412
29725050,0irq/27-eth0-tx-16717-21munin-node20:25:320
29725050,0irq/27-eth0-tx-14566-21if_err_eth1.222:30:192
29725050,0irq/27-eth0-tx-0-21swapper/223:55:302
29725050,0irq/27-eth0-tx-0-21swapper/223:40:582
29725050,0irq/27-eth0-tx-0-21swapper/222:02:592
29725050,0irq/27-eth0-tx-0-21swapper/221:53:192
29725050,0irq/27-eth0-tx-0-21swapper/221:25:302
29725050,0irq/27-eth0-tx-0-21swapper/221:20:362
29725050,0irq/27-eth0-tx-0-21swapper/221:05:242
29725050,0irq/27-eth0-tx-0-21swapper/200:14:082
29725050,0irq/27-eth0-tx-0-21swapper/021:00:120
29725050,0irq/27-eth0-tx-0-21swapper/020:45:120
29725050,0irq/27-eth0-tx-0-21swapper/020:19:270
29715050,0irq/26-eth0-rx-9580-21open_files23:45:263
29715050,0irq/26-eth0-rx-6559-21runrttasks22:28:203
29715050,0irq/26-eth0-rx-30993-21munin-node22:20:233
29715050,0irq/26-eth0-rx-22209-21perf20:05:083
29715050,0irq/26-eth0-rx-13858-21fschecks_time22:50:213
29715050,0irq/26-eth0-rx-116982sleep322:46:103
29715050,0irq/26-eth0-rx-0-21swapper/323:57:333
29715050,0irq/26-eth0-rx-0-21swapper/323:50:323
29715050,0irq/26-eth0-rx-0-21swapper/323:34:483
29715050,0irq/26-eth0-rx-0-21swapper/322:55:173
29715050,0irq/26-eth0-rx-0-21swapper/321:40:193
29715050,0irq/26-eth0-rx-0-21swapper/321:10:543
29715050,0irq/26-eth0-rx-0-21swapper/321:00:123
29715050,0irq/26-eth0-rx-0-21swapper/320:35:303
29715050,0irq/26-eth0-rx-0-21swapper/319:05:283
29715050,0irq/26-eth0-rx-0-21swapper/300:10:563
29715050,0irq/26-eth0-rx-0-21swapper/300:07:343
259950,0migration/228708-21taskset22:17:072
259950,0migration/227501-21aten_r1power_cu00:20:182
259950,0migration/226565-21kthreadcore21:35:252
259950,0migration/224325-21if_eth1.320:30:212
259950,0migration/223468-21kthreadcore22:35:242
259950,0migration/222553-21sh20:05:192
259950,0migration/221002-21fschecks_time23:35:212
259950,0migration/21754-21sh00:00:442
259950,0migration/215729-21munin-node21:10:232
259950,0migration/210455-21kthreadcore22:25:242
249950,0watchdog/227879-21ssh22:56:502
19150,0ktimersoftd/10-21swapper/121:06:161
179950,0migration/13915-21taskset23:03:121
179950,0migration/129951-21latency_hist20:35:081
179950,0migration/129474-21df_inode20:10:211
179950,0migration/11923-21kthreadcore22:40:261
179950,0migration/114499-21aten_r1power_fr21:10:181
179950,0migration/113439-21latency19:30:261
149950,0migration/0929-21kthreadcore22:40:260
149950,0migration/04758-21munin-node23:45:200
149950,0migration/03268-21idleruntime-cro00:05:070
149950,0migration/029440-21kernelversion19:45:260
149950,0migration/027726-21kthreadcore21:55:260
149950,0migration/026098-21kthreadcore20:55:230
149950,0migration/025962-21kthreadcore21:35:250
149950,0migration/023930-21kthreadcore20:05:240
149950,0migration/020725-21taskset21:10:420
149950,0migration/019627-21kthreadcore20:00:250
149950,0migration/019611-21aten_r1power_en00:15:190
149950,0migration/019341-21latency_hist00:15:070
149950,0migration/018822-21kthreadcore19:35:250
149950,0migration/016484-21kthreadcore21:30:220
149950,0migration/015319-21kthreadcore23:30:230
149950,0migration/010212-21kthreadcore22:45:270
339940,0migration/35702-21kthreadcore20:40:243
339940,0migration/34272-21kthreadcore19:50:243
339940,0migration/322254-21hddtemp_smartct19:40:223
339940,0migration/313422-21kthreadcore20:20:253
326479943,0cyclictest5178-21awk20:40:232
326479943,0cyclictest3514-21snmpd22:45:192
326479943,0cyclictest3514-21snmpd20:55:192
326479943,0cyclictest3514-21snmpd20:45:342
326479943,0cyclictest3514-21snmpd20:25:482
326479943,0cyclictest3514-21snmpd20:15:182
326479943,0cyclictest3514-21snmpd20:00:182
326479943,0cyclictest3514-21snmpd19:54:182
326479943,0cyclictest3514-21snmpd19:47:182
326479943,0cyclictest3241-21irqbalance22:11:162
326479943,0cyclictest2271-21awk19:25:242
326479943,0cyclictest15645-21awk21:30:212
326469944,0cyclictest0-21swapper/123:08:551
326469944,0cyclictest0-21swapper/122:50:101
326469944,0cyclictest0-21swapper/122:07:541
326469944,0cyclictest0-21swapper/121:50:151
326469944,0cyclictest0-21swapper/121:16:021
326469944,0cyclictest0-21swapper/100:13:531
326469943,0cyclictest4836-21awk23:45:211
326469943,0cyclictest3556-21lldpd22:55:111
326469943,0cyclictest3556-21lldpd22:18:121
326469943,0cyclictest3556-21lldpd22:00:121
326469943,0cyclictest3556-21lldpd21:01:421
326469943,0cyclictest3556-21lldpd20:45:121
326469943,0cyclictest3556-21lldpd20:00:421
326469943,0cyclictest3514-21snmpd23:50:191
326469943,0cyclictest3514-21snmpd23:25:191
326469943,0cyclictest3514-21snmpd20:27:181
326469943,0cyclictest3514-21snmpd19:35:181
326469943,0cyclictest3241-21irqbalance23:57:361
326469943,0cyclictest3241-21irqbalance21:30:161
326469943,0cyclictest31691-21awk21:40:231
326469943,0cyclictest30943-21awk22:20:231
326469943,0cyclictest2239-21awk19:25:231
326469943,0cyclictest21180-21awk23:35:221
326459944,0cyclictest0-21swapper/023:53:190
326459944,0cyclictest0-21swapper/021:41:490
326459944,0cyclictest0-21swapper/021:09:210
326459944,0cyclictest0-21swapper/020:40:200
326459943,0cyclictest7599-21grep21:45:230
326459943,0cyclictest4556-21awk00:05:210
326459943,0cyclictest3556-21lldpd22:13:410
326459943,0cyclictest3514-21snmpd23:21:340
326459943,0cyclictest3514-21snmpd19:02:030
326459943,0cyclictest14549-21awk22:30:190
326459943,0cyclictest14179-21awk22:50:220
29725040,0irq/27-eth0-tx-3955-21ssh23:45:112
29725040,0irq/27-eth0-tx-30025-21munin-node20:35:210
29725040,0irq/27-eth0-tx-22391-21ssh21:55:122
29725040,0irq/27-eth0-tx-0-21swapper/223:10:152
29725040,0irq/27-eth0-tx-0-21swapper/223:05:402
29725040,0irq/27-eth0-tx-0-21swapper/223:00:242
29725040,0irq/27-eth0-tx-0-21swapper/222:50:212
29725040,0irq/27-eth0-tx-0-21swapper/200:26:242
29725040,0irq/27-eth0-tx-0-21swapper/020:10:300
29725040,0irq/27-eth0-tx-0-21swapper/019:55:300
29725040,0irq/27-eth0-tx-0-21swapper/019:40:290
29725040,0irq/27-eth0-tx-0-21swapper/019:10:270
29725040,0irq/27-eth0-tx-0-21swapper/019:05:230
29715040,0irq/26-eth0-rx-0-21swapper/323:42:013
29715040,0irq/26-eth0-rx-0-21swapper/323:35:553
29715040,0irq/26-eth0-rx-0-21swapper/323:17:023
29715040,0irq/26-eth0-rx-0-21swapper/321:55:133
29715040,0irq/26-eth0-rx-0-21swapper/321:35:313
29715040,0irq/26-eth0-rx-0-21swapper/321:30:173
29715040,0irq/26-eth0-rx-0-21swapper/321:25:573
29715040,0irq/26-eth0-rx-0-21swapper/320:15:213
29715040,0irq/26-eth0-rx-0-21swapper/300:25:293
29715040,0irq/26-eth0-rx-0-21swapper/300:20:363
29715040,0irq/26-eth0-rx-0-21swapper/300:15:483
259940,0migration/26929-21runrttasks19:04:192
259940,0migration/24271-21gltestperf00:05:192
259940,0migration/232449-21kthreadcore20:10:252
259940,0migration/230720-21fschecks_time20:35:232
259940,0migration/223288-21kthreadcore19:40:252
259940,0migration/222230-21kthreadcore19:15:242
259940,0migration/21971-21kthreadcore21:40:262
259940,0migration/218273-21fschecks_time20:50:212
259940,0migration/215835-21kthreadcore19:10:262
249940,0watchdog/229636-21ssh22:40:102
249940,0watchdog/227591-21tune2fs19:20:232
249940,0watchdog/214892-21aten_r1power_en19:35:192
249940,0watchdog/21234-21kthreadcore21:00:262
249940,0watchdog/20-21swapper/200:15:262
19140,0ktimersoftd/10-21swapper/100:18:171
179940,0migration/19954-21taskset00:06:061
179940,0migration/19135-21taskset19:55:141
179940,0migration/17910-21kthreadcore20:40:251
179940,0migration/16328-21kthreadcore20:15:261
179940,0migration/131610-21kthreadcore19:45:261
179940,0migration/13062-21kthreadcore19:00:251
179940,0migration/129841-21sh21:38:201
179940,0migration/12820-21taskset23:42:391
179940,0migration/118495-21kthreadcore22:10:261
179940,0migration/11748-21sh00:21:561
179940,0migration/113939-21cpuspeed_turbos22:30:171
179940,0migration/113911-21runrttasks21:28:301
179940,0migration/113131-21gltestperf23:30:191
179940,0migration/11136-21kthreadcore23:20:271
179940,0migration/110391-21df20:20:201
169940,0watchdog/15287-21ssh22:45:141
169940,0watchdog/13514-21snmpd20:59:341
169940,0watchdog/13076-21ssh00:25:101
169940,0watchdog/112519-21memory19:05:271
149940,0migration/028272-21kthreadcore20:30:230
149940,0migration/027270-21timerwakeupswit22:55:310
149940,0migration/025057-21kthreadcore23:15:230
326479932,0cyclictest8081-21awk19:05:232
326479932,0cyclictest3514-21snmpd20:20:182
326479932,0cyclictest3514-21snmpd19:58:332
326479932,0cyclictest3514-21snmpd19:31:332
326469932,0cyclictest6820-21forks21:45:201
326469932,0cyclictest6644-21awk22:25:221
326469932,0cyclictest3514-21snmpd19:15:181
326469932,0cyclictest3241-21irqbalance21:20:161
326469932,0cyclictest24253-21awk20:30:211
326469932,0cyclictest23334-21awk21:55:231
326469932,0cyclictest22468-21aten_r1power_en20:05:181
326469932,0cyclictest14760-21awk19:10:241
326469932,0cyclictest13723-21awk23:10:221
326469930,0cyclictest26627-21cut19:40:261
326459933,0cyclictest1223-21kworker/0:023:55:190
326459933,0cyclictest1223-21kworker/0:023:41:070
326459933,0cyclictest1223-21kworker/0:023:35:090
326459933,0cyclictest1223-21kworker/0:023:05:210
326459933,0cyclictest1223-21kworker/0:022:25:190
326459933,0cyclictest1223-21kworker/0:022:05:130
326459933,0cyclictest1223-21kworker/0:021:50:150
326459933,0cyclictest1223-21kworker/0:021:25:110
326459933,0cyclictest1223-21kworker/0:021:15:110
326459933,0cyclictest1223-21kworker/0:020:50:120
326459933,0cyclictest1223-21kworker/0:020:20:160
326459933,0cyclictest1223-21kworker/0:019:50:120
326459933,0cyclictest1223-21kworker/0:019:30:160
326459933,0cyclictest1223-21kworker/0:019:20:140
326459933,0cyclictest1223-21kworker/0:019:15:120
326459933,0cyclictest1223-21kworker/0:000:25:110
326459932,0cyclictest13676-21awk23:10:220
326459932,0cyclictest10039-21grep23:25:280
326459930,0cyclictest0-21swapper/019:25:090
29715030,0irq/26-eth0-rx-0-21swapper/320:45:193
29715030,0irq/26-eth0-rx-0-21swapper/319:15:193
169930,0watchdog/10-21swapper/122:36:221
169930,0watchdog/10-21swapper/120:50:221
169930,0watchdog/10-21swapper/119:54:351
339920,0migration/325395-21iostat_ios20:55:223
29715020,0irq/26-eth0-rx-0-21swapper/319:30:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional