You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-21 - 14:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu May 21, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450550,0irq/25-eth10-21swapper/306:51:243
1232923423,0sleep00-21swapper/006:50:450
1195723324,0sleep20-21swapper/206:50:232
1259323127,0sleep10-21swapper/106:53:501
339960,0migration/326806-21sh10:24:173
339960,0migration/320255-21munin-node09:20:113
339960,0migration/318905-21sh08:59:163
339960,0migration/31005-21sh10:45:383
29725060,0irq/27-eth0-tx-23619-21munin-node08:40:110
29725060,0irq/27-eth0-tx-20640-21munin-node07:25:120
29715060,0irq/26-eth0-rx-19262-21irqrtprio12:20:143
29715060,0irq/26-eth0-rx-1179-21ssh09:25:173
27160,0ktimersoftd/20-21swapper/209:42:002
259960,0migration/232260-21needreboot11:25:172
149960,0migration/0516-21sh11:46:120
149960,0migration/028685-21cpuspeed_turbos07:55:120
149960,0migration/026001-21sh11:03:120
149960,0migration/025799-21sh12:04:180
149960,0migration/02525-21sh10:49:580
127639966,0cyclictest0-21swapper/210:01:142
127639965,0cyclictest30505-21sed08:20:152
127639960,0cyclictest26379-21ssh10:43:292
127639960,0cyclictest13637-21fschecks_time06:55:122
127629960,0cyclictest0-21swapper/111:20:011
127619960,0cyclictest0-21swapper/009:10:200
4150,0ktimersoftd/00-21swapper/011:58:480
4150,0ktimersoftd/00-21swapper/011:24:300
4150,0ktimersoftd/00-21swapper/010:02:020
4150,0ktimersoftd/00-21swapper/008:59:080
339950,0migration/3777-21cpuspeed_turbos07:10:133
339950,0migration/3777-21cpuspeed_turbos07:10:133
339950,0migration/37640-21kthreadcore08:00:183
339950,0migration/37408-21kthreadcore11:10:173
339950,0migration/35716-21kthreadcore10:50:153
339950,0migration/34785-21kthreadcore10:30:153
339950,0migration/332293-21kthreadcore10:05:183
339950,0migration/331410-21kthreadcore07:55:163
339950,0migration/330219-21munin-node08:20:143
339950,0migration/32991-21kthreadcore08:45:173
339950,0migration/328839-21irqcore09:45:173
339950,0migration/328533-21idleruntime09:05:133
339950,0migration/328221-21munin-node10:25:123
339950,0migration/32793-21df_abs11:30:123
339950,0migration/327366-21munin-node12:05:143
339950,0migration/325072-21timerandwakeup10:40:213
339950,0migration/323257-21taskset08:38:363
339950,0migration/322130-21taskset07:50:123
339950,0migration/321241-21kthreadcore09:00:173
339950,0migration/319409-21munin-node07:20:163
339950,0migration/318463-21latency_hist11:39:593
339950,0migration/317605-21taskset12:19:253
339950,0migration/317296-21sh11:57:563
339950,0migration/315817-21kthreadcore10:35:173
339950,0migration/314653-21kthreadcore11:15:153
339950,0migration/312984-21kthreadcore09:55:153
339950,0migration/311775-21kthreadcore07:15:173
339950,0migration/310862-21taskset10:55:003
339950,0migration/310720-21kthreadcore07:40:173
339950,0migration/310562-21sh09:53:073
29725050,0irq/27-eth0-tx-3960-21ssh09:50:142
29725050,0irq/27-eth0-tx-25039-21ssh11:41:282
29725050,0irq/27-eth0-tx-17231-21taskset11:36:592
29725050,0irq/27-eth0-tx-0-21swapper/008:00:390
29725050,0irq/27-eth0-tx-0-21swapper/007:50:090
29725050,0irq/27-eth0-tx-0-21swapper/007:40:130
29725050,0irq/27-eth0-tx-0-21swapper/007:17:300
29715050,0irq/26-eth0-rx-9517-21ssh09:30:233
29715050,0irq/26-eth0-rx-7022-21kthreadcore12:10:153
29715050,0irq/26-eth0-rx-3297-21ssh10:10:103
29715050,0irq/26-eth0-rx-27985-21memory08:15:173
29715050,0irq/26-eth0-rx-27340-21ssh09:44:333
29715050,0irq/26-eth0-rx-26973-21fschecks_count11:45:113
29715050,0irq/26-eth0-rx-19189-21tune2fs11:20:133
29715050,0irq/26-eth0-rx-18456-21sh06:55:183
29715050,0irq/26-eth0-rx-18298-21ssh10:17:543
29715050,0irq/26-eth0-rx-0-21swapper/312:00:133
29715050,0irq/26-eth0-rx-0-21swapper/311:50:083
29715050,0irq/26-eth0-rx-0-21swapper/311:35:023
29715050,0irq/26-eth0-rx-0-21swapper/311:05:183
29715050,0irq/26-eth0-rx-0-21swapper/309:38:493
29715050,0irq/26-eth0-rx-0-21swapper/309:18:173
29715050,0irq/26-eth0-rx-0-21swapper/308:50:273
29715050,0irq/26-eth0-rx-0-21swapper/308:10:173
29715050,0irq/26-eth0-rx-0-21swapper/307:25:133
27150,0ktimersoftd/20-21swapper/208:39:062
19150,0ktimersoftd/10-21swapper/110:02:021
179950,0migration/18459-21munin-node07:40:121
169950,0watchdog/14409-21kthreadcore07:10:181
169950,0watchdog/14409-21kthreadcore07:10:181
169950,0watchdog/115849-21kthreadcore08:55:181
149950,0migration/08698-21kthreadcore09:30:190
149950,0migration/06072-21kthreadcore07:35:160
149950,0migration/05921-21kthreadcore10:10:150
149950,0migration/04512-21irqcore09:50:160
149950,0migration/03296-21kthreadcore07:10:180
149950,0migration/03296-21kthreadcore07:10:180
149950,0migration/032556-21kthreadcore10:25:150
149950,0migration/032177-21kthreadcore11:05:170
149950,0migration/031362-21irqcore08:45:150
149950,0migration/031060-21kthreadcore08:20:160
149950,0migration/030882-21kthreadcore07:05:170
149950,0migration/030772-21kthreadcore10:05:180
149950,0migration/025499-21sendmail_mailqu09:40:190
149950,0migration/02459-21taskset09:48:050
149950,0migration/021011-21kthreadcore12:20:150
149950,0migration/019623-21irqcore11:40:150
149950,0migration/01803-21latency_hist12:09:590
149950,0migration/017642-21kthreadcore07:45:170
149950,0migration/017265-21cpuspeed_turbos08:35:120
149950,0migration/014141-21aten2.4-expect07:20:090
149950,0migration/012811-21munin-node09:35:170
149950,0migration/012608-21kthreadcore08:30:170
149950,0migration/012196-21fschecks_time09:55:120
149950,0migration/010925-21taskset08:53:390
127639955,0cyclictest0-21swapper/212:21:432
127639955,0cyclictest0-21swapper/209:30:392
127639955,0cyclictest0-21swapper/209:22:182
127639955,0cyclictest0-21swapper/207:55:162
127639950,0cyclictest0-21swapper/212:14:562
127639950,0cyclictest0-21swapper/211:10:552
127639950,0cyclictest0-21swapper/210:14:482
127629955,0cyclictest11058-21diskmemload10:29:151
127629955,0cyclictest0-21swapper/112:11:341
127629955,0cyclictest0-21swapper/111:29:191
127629955,0cyclictest0-21swapper/109:51:531
127629955,0cyclictest0-21swapper/109:31:521
127629955,0cyclictest0-21swapper/109:18:001
127629950,0cyclictest0-21swapper/111:14:401
127629950,0cyclictest0-21swapper/110:45:481
127629950,0cyclictest0-21swapper/110:10:111
127629950,0cyclictest0-21swapper/109:21:181
127619950,0cyclictest0-21swapper/010:42:150
127619950,0cyclictest0-21swapper/009:08:070
339940,0migration/325775-21runrttasks07:01:393
339940,0migration/315764-21taskset08:09:233
29725040,0irq/27-eth0-tx-2965-21ssh11:50:122
29725040,0irq/27-eth0-tx-27182-21munin-node10:45:122
29725040,0irq/27-eth0-tx-18243-21ssh12:00:012
29725040,0irq/27-eth0-tx-17146-21munin-node09:35:192
29725040,0irq/27-eth0-tx-17026-21ssh11:15:562
29725040,0irq/27-eth0-tx-1-21init09:15:082
29725040,0irq/27-eth0-tx-0-21swapper/211:45:082
29725040,0irq/27-eth0-tx-0-21swapper/211:04:592
29725040,0irq/27-eth0-tx-0-21swapper/210:55:102
29725040,0irq/27-eth0-tx-0-21swapper/210:35:162
29725040,0irq/27-eth0-tx-0-21swapper/210:31:022
29725040,0irq/27-eth0-tx-0-21swapper/209:25:102
29725040,0irq/27-eth0-tx-0-21swapper/209:10:032
29725040,0irq/27-eth0-tx-0-21swapper/008:25:110
29725040,0irq/27-eth0-tx-0-21swapper/008:15:180
29725040,0irq/27-eth0-tx-0-21swapper/007:31:090
29715040,0irq/26-eth0-rx-0-21swapper/311:25:093
29715040,0irq/26-eth0-rx-0-21swapper/311:01:443
29715040,0irq/26-eth0-rx-0-21swapper/310:00:013
29715040,0irq/26-eth0-rx-0-21swapper/309:10:123
259940,0migration/28633-21kthreadcore08:25:162
259940,0migration/24204-21sh10:50:152
259940,0migration/227815-21cpuspeed_turbos10:05:132
259940,0migration/225421-21kthreadcore07:50:172
259940,0migration/223880-21cpuspeed_turbos08:40:112
259940,0migration/219633-21latency_hist07:00:002
259940,0migration/211978-21fschecks_count09:55:122
259940,0migration/210554-21fschecks_count12:15:122
249940,0watchdog/229092-21kthreadcore10:25:132
249940,0watchdog/226057-21netstat07:25:182
249940,0watchdog/225102-21python10:20:212
249940,0watchdog/222814-21grep11:20:172
249940,0watchdog/215848-21kthreadcore08:55:182
249940,0watchdog/20-21swapper/211:30:132
249940,0watchdog/20-21swapper/209:45:182
249940,0watchdog/20-21swapper/208:30:182
179940,0migration/18865-21kthreadcore09:10:171
179940,0migration/17627-21latency11:50:161
179940,0migration/13963-21kthreadcore11:30:181
179940,0migration/132210-21kthreadcore09:25:161
179940,0migration/127457-21switchtime07:50:221
179940,0migration/126223-21latency_hist12:04:591
179940,0migration/124810-21timerandwakeup11:00:231
179940,0migration/124310-21kthreadcore08:15:161
179940,0migration/119533-21munin-node07:20:181
179940,0migration/119205-21latency_hist10:19:591
179940,0migration/117451-21kthreadcore07:45:171
179940,0migration/113091-21timerandwakeup07:15:221
179940,0migration/111304-21if_eth1.208:30:161
169940,0watchdog/129529-21grep09:45:181
169940,0watchdog/12795-21ps10:50:051
169940,0watchdog/123582-21kthreadcore09:00:181
169940,0watchdog/119901-21ssh09:40:101
169940,0watchdog/116374-21ssh12:15:491
169940,0watchdog/10-21swapper/111:17:131
169940,0watchdog/10-21swapper/108:00:181
169940,0watchdog/10-21swapper/107:00:141
149940,0migration/024924-21missed_timers09:00:190
149940,0migration/020109-21taskset10:20:140
149940,0migration/018095-21sh10:58:290
149940,0migration/015917-21kthreadcore10:15:170
127639943,0cyclictest8238-21awk07:15:142
127639943,0cyclictest3556-21lldpd09:05:102
127639943,0cyclictest3556-21lldpd08:49:402
127639943,0cyclictest3556-21lldpd08:05:092
127639943,0cyclictest3556-21lldpd08:00:092
127639943,0cyclictest3556-21lldpd07:07:082
127639943,0cyclictest3514-21snmpd08:54:222
127639943,0cyclictest3514-21snmpd08:19:222
127639943,0cyclictest3514-21snmpd08:11:222
127639943,0cyclictest3514-21snmpd07:46:222
127639943,0cyclictest3514-21snmpd07:40:222
127639943,0cyclictest3514-21snmpd07:36:222
127639943,0cyclictest3514-21snmpd07:33:522
127639943,0cyclictest3514-21snmpd07:20:522
127639943,0cyclictest3241-21irqbalance11:55:072
127639943,0cyclictest3241-21irqbalance11:00:082
127639943,0cyclictest3241-21irqbalance09:00:082
127639943,0cyclictest27213-21awk12:05:132
127639943,0cyclictest12192-21awk10:15:152
127629944,0cyclictest0-21swapper/111:55:301
127629944,0cyclictest0-21swapper/111:07:531
127629944,0cyclictest0-21swapper/110:09:041
127629943,0cyclictest3556-21lldpd12:21:111
127629943,0cyclictest3556-21lldpd12:01:411
127629943,0cyclictest3556-21lldpd10:57:401
127629943,0cyclictest3556-21lldpd10:33:101
127629943,0cyclictest3556-21lldpd09:38:391
127629943,0cyclictest3556-21lldpd08:26:101
127629943,0cyclictest3556-21lldpd07:31:091
127629943,0cyclictest3556-21lldpd07:26:391
127629943,0cyclictest3556-21lldpd07:08:381
127629943,0cyclictest3556-21lldpd06:56:081
127629943,0cyclictest3241-21irqbalance11:45:171
127629943,0cyclictest3241-21irqbalance11:42:071
127629943,0cyclictest3241-21irqbalance09:56:181
127629943,0cyclictest3241-21irqbalance08:50:081
127629943,0cyclictest3241-21irqbalance08:45:181
127629943,0cyclictest3241-21irqbalance08:40:081
127629943,0cyclictest3241-21irqbalance08:35:281
127629943,0cyclictest3241-21irqbalance08:20:481
127629943,0cyclictest3241-21irqbalance08:10:081
127629943,0cyclictest3241-21irqbalance08:05:381
127629943,0cyclictest28694-21awk09:05:151
127629943,0cyclictest2437-21awk07:35:131
127629943,0cyclictest20119-21awk10:40:131
127629943,0cyclictest12029-21awk10:35:151
127629943,0cyclictest11415-21awk11:35:121
127619943,0cyclictest3514-21snmpd10:35:070
127619943,0cyclictest3514-21snmpd09:15:220
127619943,0cyclictest3241-21irqbalance11:16:470
127619943,0cyclictest3241-21irqbalance11:10:170
127619943,0cyclictest27694-21awk11:25:130
127619943,0cyclictest11515-21awk11:35:120
35130,0ktimersoftd/30-21swapper/307:36:453
29715030,0irq/26-eth0-rx-0-21swapper/308:42:193
29715030,0irq/26-eth0-rx-0-21swapper/308:30:173
29715030,0irq/26-eth0-rx-0-21swapper/308:25:103
29715030,0irq/26-eth0-rx-0-21swapper/307:48:083
29715030,0irq/26-eth0-rx-0-21swapper/307:30:183
29715030,0irq/26-eth0-rx-0-21swapper/307:05:113
249930,0watchdog/20-21swapper/207:10:142
249930,0watchdog/20-21swapper/207:10:142
127629932,0cyclictest29365-21awk07:55:151
127629932,0cyclictest16068-21grep10:15:171
127619933,0cyclictest308-21kworker/0:206:55:050
127619933,0cyclictest20526-21kworker/0:212:15:320
127619933,0cyclictest20526-21kworker/0:212:05:000
127619933,0cyclictest20526-21kworker/0:211:50:040
127619933,0cyclictest20526-21kworker/0:210:50:170
127619933,0cyclictest18673-21kworker/0:109:25:070
127619933,0cyclictest18673-21kworker/0:109:20:010
127619933,0cyclictest18673-21kworker/0:108:10:030
127619933,0cyclictest18673-21kworker/0:108:05:030
127619933,0cyclictest18673-21kworker/0:107:00:030
127619932,0cyclictest2420-21munin-node11:30:120
127619931,0cyclictest3791-21hddtemp_smartct10:30:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional