You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-17 - 01:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Wed Jun 17, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3098224636,0sleep30-21swapper/318:50:133
3098523930,0sleep10-21swapper/118:50:151
3109023423,0sleep00-21swapper/018:51:300
3085823425,0sleep20-21swapper/218:48:442
312449974,0cyclictest32257-21munin-run19:17:290
35160,0ktimersoftd/30-21swapper/321:24:093
339960,0migration/37963-21kthreadcore22:37:463
339960,0migration/35335-21sh21:16:393
339960,0migration/33164-21pmu-power21:52:463
339960,0migration/331269-21munin-node19:12:483
339960,0migration/320599-21munin-node19:32:413
339960,0migration/319568-21sh00:05:183
339960,0migration/313177-21sh22:22:223
339960,0migration/311301-21sh00:19:523
312479960,0cyclictest20578-21latency_hist23:47:272
312459960,0cyclictest0-21swapper/121:35:321
29725060,0irq/27-eth0-tx-19134-21munin-node19:07:500
29715060,0irq/26-eth0-rx-28788-21ssh23:12:033
29715060,0irq/26-eth0-rx-21413-21cpuspeed_turbos19:57:403
29715060,0irq/26-eth0-rx-11439-21ssh22:58:243
149960,0migration/04197-21sh23:16:250
149960,0migration/028533-21sh21:09:160
149960,0migration/027629-21pmu-power20:22:500
149960,0migration/020221-21sh23:05:310
149960,0migration/012663-21sh22:00:190
149960,0migration/012104-21sh00:01:530
8215050,0irq/24-0000:00:0-21swapper/219:12:292
4150,0ktimersoftd/00-21swapper/022:04:210
4150,0ktimersoftd/00-21swapper/021:24:090
35150,0ktimersoftd/30-21swapper/321:22:203
339950,0migration/39701-21kthreadcore19:22:453
339950,0migration/36517-21kthreadcore23:37:453
339950,0migration/36373-21munin-node20:57:453
339950,0migration/332390-21if_eth118:52:443
339950,0migration/332322-21kthreadcore21:32:453
339950,0migration/332053-21munin-node19:37:473
339950,0migration/330401-21kthreadcore20:27:443
339950,0migration/329837-21munin-node22:12:403
339950,0migration/329789-21aten_r1power_cu20:52:393
339950,0migration/328518-21ksmtuned22:09:403
339950,0migration/327595-21unixbench_multi22:27:523
339950,0migration/32696-21sh23:53:133
339950,0migration/320219-21taskset22:45:003
339950,0migration/319883-21unixbench-2d21:42:543
339950,0migration/316052-21kthreadcore21:02:453
339950,0migration/314242-21kthreadcore23:22:443
339950,0migration/313768-21munin-node22:22:413
339950,0migration/31136-21kthreadcore23:12:463
339950,0migration/310999-21kthreadcore19:47:483
339950,0migration/310124-21kthreadcore21:37:453
312479955,0cyclictest0-21swapper/221:47:572
312479950,0cyclictest0-21swapper/223:18:362
312479950,0cyclictest0-21swapper/222:53:422
312479950,0cyclictest0-21swapper/221:38:102
312459955,0cyclictest29532-21diskmemload23:43:041
312459955,0cyclictest29532-21diskmemload00:05:181
312459955,0cyclictest0-21swapper/123:20:131
312459955,0cyclictest0-21swapper/123:12:291
312459950,0cyclictest0-21swapper/122:10:511
312459950,0cyclictest0-21swapper/122:05:571
312459950,0cyclictest0-21swapper/121:22:551
312459950,0cyclictest0-21swapper/121:00:511
312449955,0cyclictest0-21swapper/020:37:500
312449955,0cyclictest0-21swapper/018:52:500
29725050,0irq/27-eth0-tx-5766-21munin-node18:57:500
29725050,0irq/27-eth0-tx-4384-21munin-node00:17:392
29725050,0irq/27-eth0-tx-32750-21kthreadcore21:12:462
29725050,0irq/27-eth0-tx-29792-21munin-node22:12:502
29725050,0irq/27-eth0-tx-29532-21diskmemload23:28:532
29725050,0irq/27-eth0-tx-21645-21ssh22:27:402
29725050,0irq/27-eth0-tx-16430-21munin-node20:42:440
29725050,0irq/27-eth0-tx-14044-21munin-node21:22:462
29725050,0irq/27-eth0-tx-11678-21ssh20:58:122
29725050,0irq/27-eth0-tx-0-21swapper/223:52:452
29725050,0irq/27-eth0-tx-0-21swapper/222:49:482
29725050,0irq/27-eth0-tx-0-21swapper/222:37:422
29725050,0irq/27-eth0-tx-0-21swapper/222:32:282
29725050,0irq/27-eth0-tx-0-21swapper/221:31:022
29725050,0irq/27-eth0-tx-0-21swapper/221:07:542
29725050,0irq/27-eth0-tx-0-21swapper/200:06:352
29725050,0irq/27-eth0-tx-0-21swapper/019:42:490
29715050,0irq/26-eth0-rx-6968-21grep23:17:443
29715050,0irq/26-eth0-rx-6559-21runrttasks23:02:323
29715050,0irq/26-eth0-rx-6559-21runrttasks00:17:093
29715050,0irq/26-eth0-rx-27461-21latency21:27:463
29715050,0irq/26-eth0-rx-1841-21munin-run20:07:283
29715050,0irq/26-eth0-rx-0-21swapper/323:50:223
29715050,0irq/26-eth0-rx-0-21swapper/323:29:013
29715050,0irq/26-eth0-rx-0-21swapper/322:55:223
29715050,0irq/26-eth0-rx-0-21swapper/322:51:563
29715050,0irq/26-eth0-rx-0-21swapper/322:32:463
29715050,0irq/26-eth0-rx-0-21swapper/322:04:243
29715050,0irq/26-eth0-rx-0-21swapper/322:01:533
29715050,0irq/26-eth0-rx-0-21swapper/321:08:073
29715050,0irq/26-eth0-rx-0-21swapper/320:17:433
29715050,0irq/26-eth0-rx-0-21swapper/300:00:323
27150,0ktimersoftd/20-21swapper/223:00:192
259950,0migration/23528-21sendmail_mailqu21:32:502
259950,0migration/224614-21kthreadcore19:32:452
259950,0migration/22399-21munin-node00:12:592
259950,0migration/22170-21munin-node23:12:462
259950,0migration/220294-21sh23:05:432
259950,0migration/214283-21munin-node21:42:412
259950,0migration/213792-21idleruntime-cro21:22:282
249950,0watchdog/228688-21ssh23:11:462
249950,0watchdog/20-21swapper/222:17:462
179950,0migration/17701-21kthreadcore20:32:471
179950,0migration/14617-21munin-node23:57:391
179950,0migration/14502-21latency_hist23:37:271
179950,0migration/132373-21rt-features19:37:501
179950,0migration/126787-21munin-node19:57:481
179950,0migration/126422-21kthreadcore20:22:471
179950,0migration/125646-21latency_hist19:12:291
179950,0migration/122682-21kthreadcore23:27:451
179950,0migration/114092-21munin-node20:12:471
179950,0migration/112962-21sh22:21:441
169950,0watchdog/16810-21cut21:57:461
149950,0migration/09935-21kthreadcore19:22:460
149950,0migration/0718-21missed_timers20:02:470
149950,0migration/06986-21kthreadcore20:32:460
149950,0migration/05495-21latency_hist21:57:280
149950,0migration/0544-21fschecks_count19:17:420
149950,0migration/03411-21sh23:55:120
149950,0migration/032454-21kthreadcore00:12:480
149950,0migration/029983-21latency_hist21:12:290
149950,0migration/029051-21latency_hist23:12:270
149950,0migration/026591-21kthreadcore22:47:480
149950,0migration/026526-21missed_timers19:57:460
149950,0migration/024155-21kthreadcore23:27:450
149950,0migration/023543-21kthreadcore00:07:460
149950,0migration/022063-21munin-node22:27:420
149950,0migration/020942-21cpu23:47:420
149950,0migration/020214-21aten_r1power_po19:32:390
149950,0migration/019730-21taskset19:30:460
149950,0migration/018352-21kthreadcore20:17:450
149950,0migration/015538-21kthreadcore00:02:460
149950,0migration/014956-21kthreadcore23:22:440
149950,0migration/014345-21sh22:42:450
149950,0migration/013982-21df_abs21:02:410
149950,0migration/011733-21taskset22:38:440
149950,0migration/011282-21sh23:18:410
339940,0migration/34396-21kthreadcore20:32:453
339940,0migration/330787-21kthreadcore23:32:443
339940,0migration/327130-21diskmemload00:08:503
339940,0migration/323564-21kthreadcore19:07:473
339940,0migration/322125-21sh20:42:493
312479943,0cyclictest3556-21lldpd19:45:042
312479943,0cyclictest3514-21snmpd20:44:092
312479943,0cyclictest3514-21snmpd20:19:092
312479943,0cyclictest3514-21snmpd20:13:542
312479943,0cyclictest3514-21snmpd19:30:092
312479943,0cyclictest3514-21snmpd19:25:242
312479943,0cyclictest3514-21snmpd19:14:242
312479943,0cyclictest30655-21awk20:52:432
312479943,0cyclictest2952-21awk20:07:432
312479943,0cyclictest28709-21awk20:02:432
312479943,0cyclictest27618-21awk19:37:442
312479943,0cyclictest21555-21awk00:07:432
312479943,0cyclictest1004-21awk19:17:442
312459943,0cyclictest6674-21awk18:57:451
312459943,0cyclictest3556-21lldpd23:22:341
312459943,0cyclictest3556-21lldpd23:10:041
312459943,0cyclictest3556-21lldpd23:02:341
312459943,0cyclictest3556-21lldpd22:33:341
312459943,0cyclictest3556-21lldpd21:50:351
312459943,0cyclictest3556-21lldpd20:47:351
312459943,0cyclictest3556-21lldpd20:09:051
312459943,0cyclictest3556-21lldpd00:11:041
312459943,0cyclictest3514-21snmpd23:32:541
312459943,0cyclictest3514-21snmpd22:40:541
312459943,0cyclictest3514-21snmpd21:10:241
312459943,0cyclictest32299-21awk18:52:441
312459943,0cyclictest28757-21awk20:02:431
312449944,0cyclictest0-21swapper/000:20:460
312449943,0cyclictest8560-21sadc20:12:280
312449943,0cyclictest6853-21awk21:17:430
312449943,0cyclictest3556-21lldpd20:12:340
312449943,0cyclictest3514-21snmpd23:47:240
312449943,0cyclictest30408-21awk22:52:430
312449943,0cyclictest29734-21awk20:27:430
312449943,0cyclictest2544-21grep23:32:500
312449943,0cyclictest13325-21sadc21:02:280
29725040,0irq/27-eth0-tx-5269-21ssh23:37:412
29725040,0irq/27-eth0-tx-14784-21irqcore22:02:442
29725040,0irq/27-eth0-tx-13961-21df21:02:412
29725040,0irq/27-eth0-tx-10930-21ssh23:58:342
29725040,0irq/27-eth0-tx-0-21swapper/223:47:332
29725040,0irq/27-eth0-tx-0-21swapper/223:32:372
29725040,0irq/27-eth0-tx-0-21swapper/222:42:372
29725040,0irq/27-eth0-tx-0-21swapper/222:07:582
29725040,0irq/27-eth0-tx-0-21swapper/221:57:352
29725040,0irq/27-eth0-tx-0-21swapper/221:52:362
29725040,0irq/27-eth0-tx-0-21swapper/020:52:400
29725040,0irq/27-eth0-tx-0-21swapper/020:49:050
29725040,0irq/27-eth0-tx-0-21swapper/019:52:480
29725040,0irq/27-eth0-tx-0-21swapper/019:37:470
29715040,0irq/26-eth0-rx-27283-21kthreadcore20:47:463
29715040,0irq/26-eth0-rx-0-21swapper/323:42:313
29715040,0irq/26-eth0-rx-0-21swapper/321:50:193
29715040,0irq/26-eth0-rx-0-21swapper/319:57:293
29715040,0irq/26-eth0-rx-0-21swapper/319:02:383
259940,0migration/223151-21runrttasks20:47:392
259940,0migration/221064-21latency_hist19:57:292
259940,0migration/22037-21sh20:27:502
259940,0migration/218534-21kthreadcore22:22:472
259940,0migration/214327-21kthreadcore23:22:442
259940,0migration/211054-21kthreadcore19:47:482
249940,0watchdog/231458-21ps18:52:392
249940,0watchdog/225620-21grep20:22:472
249940,0watchdog/214526-21tr19:02:432
249940,0watchdog/214024-21kthreadcore20:37:472
249940,0watchdog/20-21swapper/220:32:472
249940,0watchdog/20-21swapper/218:57:472
179940,0migration/17722-21kthreadcore21:17:451
179940,0migration/13807-21taskset00:16:371
179940,0migration/132137-21kthreadcore22:12:451
179940,0migration/131074-21kernelversion20:52:441
179940,0migration/130783-21fschecks_time21:12:421
179940,0migration/123835-21kthreadcore23:47:481
179940,0migration/122192-21switchtime20:42:491
179940,0migration/121251-21timerandwakeup20:17:511
179940,0migration/117214-21kthreadcore19:27:471
179940,0migration/116039-21kthreadcore21:42:471
179940,0migration/114811-21kthreadcore19:02:431
179940,0migration/114570-21irqcore22:22:451
179940,0migration/110822-21kthreadcore19:22:461
169940,0watchdog/16133-21cut19:42:471
169940,0watchdog/14716-21ssh21:55:381
169940,0watchdog/129391-21rm21:31:101
169940,0watchdog/12935-21ssh23:53:541
169940,0watchdog/122065-21grep22:27:421
169940,0watchdog/121785-21diskstats22:47:421
169940,0watchdog/118575-21kthreadcore19:52:471
169940,0watchdog/111081-21sed22:57:501
169940,0watchdog/10-21swapper/122:43:141
169940,0watchdog/10-21swapper/121:37:461
149940,0migration/06876-21iostat21:37:430
149940,0migration/06444-21if_eth122:17:440
149940,0migration/024565-21kthreadcore22:07:460
149940,0migration/018135-21sensors_temp19:02:480
312479932,0cyclictest14699-21grep19:52:412
312459932,0cyclictest935-21awk19:17:431
312459932,0cyclictest8656-21awk19:47:451
312459932,0cyclictest3556-21lldpd21:02:351
312459932,0cyclictest3556-21lldpd20:40:051
312459932,0cyclictest3556-21lldpd19:14:341
312459932,0cyclictest3556-21lldpd00:17:341
312459932,0cyclictest3514-21snmpd22:52:351
312459932,0cyclictest20156-21aten_r1power_en19:32:391
312449933,0cyclictest13610-21kworker/0:119:47:380
312449933,0cyclictest11769-21kworker/0:123:37:330
312449933,0cyclictest11769-21kworker/0:122:33:050
312449933,0cyclictest11769-21kworker/0:122:22:430
312449933,0cyclictest11769-21kworker/0:122:12:350
312449933,0cyclictest11769-21kworker/0:121:47:290
312449932,0cyclictest5401-21tr22:57:390
312449932,0cyclictest31134-21awk21:32:440
312449932,0cyclictest13867-21munin-node21:42:410
312449932,0cyclictest11769-21kworker/0:121:27:400
29715030,0irq/26-eth0-rx-0-21swapper/320:22:413
29715030,0irq/26-eth0-rx-0-21swapper/320:15:103
29715030,0irq/26-eth0-rx-0-21swapper/319:42:403
29715030,0irq/26-eth0-rx-0-21swapper/319:00:563
312459921,0cyclictest28715-21cat20:27:291
29715020,0irq/26-eth0-rx-0-21swapper/320:37:313
29715020,0irq/26-eth0-rx-0-21swapper/320:07:303
29715020,0irq/26-eth0-rx-0-21swapper/319:27:403
29715020,0irq/26-eth0-rx-0-21swapper/319:17:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional