You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 02:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sun May 03, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450560,0irq/25-eth10-21swapper/318:52:063
1751123824,0sleep00-21swapper/018:51:540
2368623324,0sleep10-21swapper/118:54:361
2350923329,0sleep20-21swapper/218:52:322
35160,0ktimersoftd/30-21swapper/323:14:443
339960,0migration/32927-21taskset00:03:073
339960,0migration/320159-21munin-node22:12:063
29725060,0irq/27-eth0-tx-11393-21python23:07:052
29715060,0irq/26-eth0-rx-5337-21ssh22:25:223
29715060,0irq/26-eth0-rx-3947-21port_sshpriv22:02:063
29715060,0irq/26-eth0-rx-20127-21ntpq21:12:033
29715060,0irq/26-eth0-rx-1327-21tr21:22:033
29715060,0irq/26-eth0-rx-0-21swapper/322:17:063
27160,0ktimersoftd/20-21swapper/222:57:012
239459960,0cyclictest22083-21diskmemload21:43:042
239459960,0cyclictest0-21swapper/221:37:522
239419966,0cyclictest22083-21diskmemload23:52:191
239419966,0cyclictest22083-21diskmemload21:33:251
239409960,0cyclictest0-21swapper/021:20:470
149960,0migration/021049-21sh22:55:080
149960,0migration/020640-21sh23:35:520
8215050,0irq/24-0000:00:6890-21latency_hist19:56:452
4150,0ktimersoftd/00-21swapper/022:23:510
35150,0ktimersoftd/30-21swapper/323:37:083
35150,0ktimersoftd/30-21swapper/321:42:513
339950,0migration/38361-21munin-node20:21:573
339950,0migration/38018-21kthreadcore22:07:013
339950,0migration/36971-21kthreadcore23:47:013
339950,0migration/36221-21aten_r1power_po22:26:543
339950,0migration/35006-21latency_hist19:06:453
339950,0migration/331190-21df_abs19:01:583
339950,0migration/329952-21taskset21:01:103
339950,0migration/329581-21munin-node23:21:553
339950,0migration/327662-21munin-node22:57:063
339950,0migration/326193-21kthreadcore22:37:013
339950,0migration/321712-21taskset20:52:383
339950,0migration/320463-21irqcore19:42:013
339950,0migration/320009-21missed_timers21:52:033
339950,0migration/318416-21munin-node23:52:043
339950,0migration/316316-21kthreadcore20:27:003
339950,0migration/315329-21irqrtprio22:32:003
339950,0migration/313143-21fschecks_time00:11:573
339950,0migration/312942-21irqrtprio19:12:003
339950,0migration/312428-21taskset19:35:083
29725050,0irq/27-eth0-tx-6701-21if_err_eth1.222:46:572
29725050,0irq/27-eth0-tx-6672-21aten2.4-expect21:46:552
29725050,0irq/27-eth0-tx-4962-21ssh23:05:452
29725050,0irq/27-eth0-tx-31019-21munin-node21:22:032
29725050,0irq/27-eth0-tx-30470-21munin-node21:02:072
29725050,0irq/27-eth0-tx-28756-21latency_hist00:01:442
29725050,0irq/27-eth0-tx-27003-21memory23:17:022
29725050,0irq/27-eth0-tx-27003-21memory23:17:022
29725050,0irq/27-eth0-tx-23652-21ssh20:57:010
29725050,0irq/27-eth0-tx-15187-21munin-node21:31:572
29725050,0irq/27-eth0-tx-14815-21if_eth122:51:592
29725050,0irq/27-eth0-tx-13782-21munin-node20:02:000
29725050,0irq/27-eth0-tx-0-21swapper/223:24:182
29725050,0irq/27-eth0-tx-0-21swapper/222:46:122
29725050,0irq/27-eth0-tx-0-21swapper/221:58:432
29725050,0irq/27-eth0-tx-0-21swapper/200:17:002
29725050,0irq/27-eth0-tx-0-21swapper/019:31:570
29725050,0irq/27-eth0-tx-0-21swapper/019:06:580
29715050,0irq/26-eth0-rx-3904-21python21:02:073
29715050,0irq/26-eth0-rx-23273-21fschecks_count21:16:573
29715050,0irq/26-eth0-rx-21406-21fschecks_time23:56:573
29715050,0irq/26-eth0-rx-16349-21kthreadcore23:32:003
29715050,0irq/26-eth0-rx-1580-21kthreadcore23:42:023
29715050,0irq/26-eth0-rx-11404-21tr21:27:013
29715050,0irq/26-eth0-rx-0-21swapper/323:31:263
29715050,0irq/26-eth0-rx-0-21swapper/323:17:013
29715050,0irq/26-eth0-rx-0-21swapper/323:17:013
29715050,0irq/26-eth0-rx-0-21swapper/323:06:503
29715050,0irq/26-eth0-rx-0-21swapper/322:46:453
29715050,0irq/26-eth0-rx-0-21swapper/322:43:113
29715050,0irq/26-eth0-rx-0-21swapper/321:57:373
29715050,0irq/26-eth0-rx-0-21swapper/321:37:033
29715050,0irq/26-eth0-rx-0-21swapper/321:07:093
29715050,0irq/26-eth0-rx-0-21swapper/300:21:473
29715050,0irq/26-eth0-rx-0-21swapper/300:08:433
259950,0migration/24663-21kthreadcore20:42:012
259950,0migration/231641-21irqrtprio22:22:002
259950,0migration/227454-21munin-node20:11:552
259950,0migration/222850-21aten_r1power_en21:16:552
259950,0migration/219495-21kthreadcore21:52:022
259950,0migration/21629-21runrttasks20:37:132
259950,0migration/214195-21sh21:11:332
249950,0watchdog/229276-21sort00:21:582
249950,0watchdog/229188-21munin-node23:42:062
249950,0watchdog/225035-21cut20:32:032
249950,0watchdog/22274-21kthreadcore22:02:032
239459955,0cyclictest0-21swapper/221:31:342
239459950,0cyclictest0-21swapper/223:37:132
239459950,0cyclictest0-21swapper/223:16:152
239459950,0cyclictest0-21swapper/220:58:372
239459950,0cyclictest0-21swapper/220:16:592
239419954,0cyclictest5192-21ssh22:04:191
239419954,0cyclictest15544-21tr22:12:001
239419950,0cyclictest0-21swapper/123:32:271
239419950,0cyclictest0-21swapper/123:21:341
239419950,0cyclictest0-21swapper/123:21:341
239419950,0cyclictest0-21swapper/122:54:181
239419950,0cyclictest0-21swapper/121:01:341
239419950,0cyclictest0-21swapper/120:32:051
239419950,0cyclictest0-21swapper/119:57:051
239409955,0cyclictest29976-21kworker/0:223:52:450
239409955,0cyclictest11602-21kworker/0:123:15:200
239409955,0cyclictest11602-21kworker/0:122:10:320
239409955,0cyclictest11602-21kworker/0:120:51:120
239409955,0cyclictest11602-21kworker/0:120:35:520
239409955,0cyclictest11602-21kworker/0:120:24:480
239409955,0cyclictest11602-21kworker/0:120:20:540
239409955,0cyclictest11602-21kworker/0:120:10:580
239409955,0cyclictest11602-21kworker/0:119:57:580
239409955,0cyclictest11602-21kworker/0:119:51:200
239409955,0cyclictest11602-21kworker/0:119:45:120
239409955,0cyclictest11602-21kworker/0:119:29:080
239409955,0cyclictest11602-21kworker/0:119:02:330
239409955,0cyclictest11602-21kworker/0:118:58:450
239409955,0cyclictest0-21swapper/021:41:540
239409955,0cyclictest0-21swapper/020:36:570
239409950,0cyclictest0-21swapper/022:18:480
19150,0ktimersoftd/10-21swapper/121:54:141
19150,0ktimersoftd/10-21swapper/121:42:511
19150,0ktimersoftd/10-21swapper/121:05:591
179950,0migration/15197-21df_abs23:46:561
179950,0migration/124628-21kthreadcore00:17:011
179950,0migration/124032-21sh22:37:001
179950,0migration/121284-21munin-node20:52:061
179950,0migration/111979-21munin-node21:47:021
149950,0migration/07186-21irqcore22:26:590
149950,0migration/0699-21munin-node19:51:570
149950,0migration/06943-21kthreadcore00:07:010
149950,0migration/06295-21irqcore23:26:590
149950,0migration/04156-21taskset00:06:370
149950,0migration/0357-21kthreadcore21:02:020
149950,0migration/03360-21taskset00:25:360
149950,0migration/032435-21needreboot20:12:000
149950,0migration/028205-21runrttasks23:39:420
149950,0migration/021484-21munin-node23:56:570
149950,0migration/020403-21switchtime20:27:050
149950,0migration/019319-21taskset19:17:000
149950,0migration/019044-21taskset19:39:590
149950,0migration/018260-21kthreadcore22:12:010
149950,0migration/018158-21taskset19:16:250
149950,0migration/017609-21kthreadcore00:12:020
149950,0migration/016445-21gltestperf20:51:580
149950,0migration/015801-21kthreadcore22:32:010
149950,0migration/014047-21sh21:11:080
149950,0migration/010757-21kthreadcore23:07:010
339940,0migration/328450-21kthreadcore20:11:583
339940,0migration/322700-21kthreadcore00:17:013
339940,0migration/321087-21runrttasks21:32:383
339940,0migration/316551-21kthreadcore22:52:013
339940,0migration/310809-21kthreadcore19:57:013
339940,0migration/31053-21ntp_offset20:37:043
29725040,0irq/27-eth0-tx-15684-21kthreadcore22:12:002
29725040,0irq/27-eth0-tx-13205-21ssh23:31:462
29725040,0irq/27-eth0-tx-0-21swapper/223:52:302
29725040,0irq/27-eth0-tx-0-21swapper/223:49:312
29725040,0irq/27-eth0-tx-0-21swapper/223:28:462
29725040,0irq/27-eth0-tx-0-21swapper/222:27:002
29725040,0irq/27-eth0-tx-0-21swapper/200:06:512
29715040,0irq/26-eth0-rx-0-21swapper/323:01:483
29715040,0irq/26-eth0-rx-0-21swapper/320:42:013
29715040,0irq/26-eth0-rx-0-21swapper/320:16:573
29715040,0irq/26-eth0-rx-0-21swapper/319:47:023
29715040,0irq/26-eth0-rx-0-21swapper/319:27:023
29715040,0irq/26-eth0-rx-0-21swapper/319:21:563
29715040,0irq/26-eth0-rx-0-21swapper/319:16:453
259940,0migration/28624-21fschecks_time20:21:582
259940,0migration/25119-21aten2.4-expect19:06:562
259940,0migration/224989-21kthreadcore22:17:012
259940,0migration/224380-21kthreadcore22:37:002
259940,0migration/213710-21irqrtprio00:12:002
259940,0migration/212962-21irqrtprio19:12:002
259940,0migration/212661-21munin-node19:57:042
249940,0watchdog/231767-21munin-node19:27:032
249940,0watchdog/221625-21hwlatdetect23:56:582
249940,0watchdog/20-21swapper/222:07:032
249940,0watchdog/20-21swapper/220:47:032
249940,0watchdog/20-21swapper/219:32:032
249940,0watchdog/20-21swapper/219:16:592
239459943,0cyclictest3556-21lldpd22:31:562
239459943,0cyclictest3556-21lldpd19:36:542
239459943,0cyclictest27003-21awk19:46:592
239459943,0cyclictest16664-21awk20:52:002
239459943,0cyclictest15521-21awk21:11:592
239419944,0cyclictest0-21swapper/122:47:121
239419944,0cyclictest0-21swapper/121:17:301
239419944,0cyclictest0-21swapper/119:12:091
239419944,0cyclictest0-21swapper/100:07:461
239419943,0cyclictest3556-21lldpd22:56:551
239419943,0cyclictest3556-21lldpd21:26:541
239419943,0cyclictest3556-21lldpd20:46:531
239419943,0cyclictest3556-21lldpd19:01:241
239419943,0cyclictest3556-21lldpd00:04:261
239419943,0cyclictest3241-21irqbalance23:31:121
239419943,0cyclictest31887-21awk21:22:011
239419943,0cyclictest31312-21awk22:21:591
239419943,0cyclictest26022-21awk19:22:001
239419943,0cyclictest2159-21awk20:17:001
239419943,0cyclictest14537-21awk23:11:571
239419943,0cyclictest13787-21awk19:37:001
239409944,0cyclictest11602-21kworker/0:121:51:320
239409944,0cyclictest0-21swapper/022:57:350
239409944,0cyclictest0-21swapper/022:50:290
239409944,0cyclictest0-21swapper/021:21:530
239409943,0cyclictest3514-21snmpd21:33:300
239409943,0cyclictest31594-21awk22:02:000
239409943,0cyclictest31060-21awk22:42:000
239409943,0cyclictest11602-21kworker/0:123:04:300
239409943,0cyclictest11602-21kworker/0:121:57:560
239409943,0cyclictest11602-21kworker/0:121:13:160
239409940,0cyclictest0-21swapper/023:43:370
179940,0migration/18557-21kthreadcore21:07:001
179940,0migration/16826-21fschecks_time22:26:571
179940,0migration/16541-21runrttasks19:52:581
179940,0migration/14616-21taskset19:04:011
179940,0migration/130986-21runrttasks22:41:591
179940,0migration/128177-21munin-node21:57:041
179940,0migration/122092-21kthreadcore20:07:001
179940,0migration/121730-21fschecks_count23:36:581
179940,0migration/121712-21kthreadcore19:17:021
179940,0migration/11890-21kthreadcore23:22:021
179940,0migration/118856-21taskset00:12:551
179940,0migration/118752-21kthreadcore20:02:011
179940,0migration/111203-21kthreadcore19:32:041
179940,0migration/110412-21kthreadcore22:07:021
169940,0watchdog/13215-21if_eth120:41:591
169940,0watchdog/130243-21ssh21:40:471
169940,0watchdog/121993-21ssh21:15:311
169940,0watchdog/120787-21ssh22:33:551
169940,0watchdog/10-21swapper/123:09:381
169940,0watchdog/10-21swapper/120:11:591
169940,0watchdog/10-21swapper/100:21:581
149940,0migration/03147-21if_err_eth1.220:41:590
149940,0migration/027289-21kthreadcore19:22:020
149940,0migration/024667-21kthreadcore22:37:000
149940,0migration/019096-21kthreadcore21:52:020
339930,0migration/326653-21sshd20:09:493
339930,0migration/315061-21runrttasks20:47:313
29715030,0irq/26-eth0-rx-0-21swapper/321:46:483
29715030,0irq/26-eth0-rx-0-21swapper/320:32:373
29715030,0irq/26-eth0-rx-0-21swapper/320:02:093
29715030,0irq/26-eth0-rx-0-21swapper/319:36:503
29715030,0irq/26-eth0-rx-0-21swapper/319:00:573
239459933,0cyclictest3556-21lldpd20:06:532
239459933,0cyclictest3556-21lldpd18:57:542
239459932,0cyclictest3556-21lldpd20:26:532
239459932,0cyclictest31788-21awk19:02:012
239459932,0cyclictest25983-21awk19:22:002
239459932,0cyclictest20290-21awk19:42:002
239459932,0cyclictest19310-21proc_pri20:02:042
239419933,0cyclictest20393-21awk19:42:001
239419932,0cyclictest8834-21sort20:21:591
239419932,0cyclictest3514-21snmpd22:19:001
239419932,0cyclictest2570-21sensors23:42:061
239419930,0cyclictest0-21swapper/120:41:331
239409933,0cyclictest29976-21kworker/0:223:46:530
239409933,0cyclictest29976-21kworker/0:200:16:450
239409933,0cyclictest11602-21kworker/0:123:16:580
239409933,0cyclictest11602-21kworker/0:123:16:580
239409933,0cyclictest11602-21kworker/0:121:27:000
239409933,0cyclictest0-21swapper/023:21:520
239409932,0cyclictest22769-21sshd21:36:500
179930,0migration/11957-21kthreadcore19:27:011
169930,0watchdog/10-21swapper/123:56:581
169930,0watchdog/10-21swapper/123:02:101
169930,0watchdog/10-21swapper/120:28:551
169930,0watchdog/10-21swapper/119:47:071
169930,0watchdog/10-21swapper/119:07:031
339920,0migration/31379-21iostat_ios19:52:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional