You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-27 - 00:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Thu Mar 26, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3157723727,0sleep20-21swapper/207:00:092
3120923627,0sleep30-21swapper/306:55:503
3158023524,0sleep00-21swapper/007:00:110
3131823430,0sleep10-21swapper/106:57:081
339960,0migration/39123-21sh12:27:293
339960,0migration/325520-21latency11:15:343
339960,0migration/319359-21sh11:13:343
339960,0migration/317425-21sh12:11:523
316599964,0cyclictest0-21swapper/208:15:322
316599960,0cyclictest0-21swapper/210:55:152
29725060,0irq/27-eth0-tx-1399-21munin-node07:50:280
29715060,0irq/26-eth0-rx-27710-21mdstat09:35:363
29715060,0irq/26-eth0-rx-25898-21sh12:18:343
29715060,0irq/26-eth0-rx-19694-21sh11:35:233
149960,0migration/04506-21sh09:21:130
149960,0migration/010198-21sh12:09:010
8215050,0irq/24-0000:00:5178-21latency_hist10:05:162
8215050,0irq/24-0000:00:26477-21dump-pmu-power12:20:162
8215050,0irq/24-0000:00:0-21swapper/212:05:162
4150,0ktimersoftd/00-21swapper/011:53:220
339950,0migration/35644-21kthreadcore11:25:323
339950,0migration/35138-21fschecks_count10:45:283
339950,0migration/332535-21kthreadcore09:00:333
339950,0migration/330411-21kthreadcore12:00:333
339950,0migration/328414-21sh09:56:533
339950,0migration/32593-21taskset11:44:063
339950,0migration/323701-21aten_r1power_vo08:55:283
339950,0migration/320829-21sh09:31:443
339950,0migration/320608-21aten_r1power_vo07:40:283
339950,0migration/319554-21aten_r1power_cu07:15:273
339950,0migration/316483-21kthreadcore07:10:333
339950,0migration/31539-21sh11:20:383
339950,0migration/312468-21latency_hist10:50:163
339950,0migration/310147-21munin-node08:20:323
316599955,0cyclictest29947-21diskmemload10:43:352
316599955,0cyclictest15731-21kthreadcore09:10:342
316599954,0cyclictest20640-21grep11:15:292
316599950,0cyclictest0-21swapper/209:58:202
316599950,0cyclictest0-21swapper/209:54:522
316589955,0cyclictest29947-21diskmemload12:24:081
316589955,0cyclictest29947-21diskmemload09:24:441
316589955,0cyclictest0-21swapper/111:06:341
316589955,0cyclictest0-21swapper/109:48:041
316589950,0cyclictest0-21swapper/111:22:111
316589950,0cyclictest0-21swapper/110:34:441
316579955,0cyclictest2229-21kworker/0:111:22:170
316579955,0cyclictest2229-21kworker/0:109:02:100
316579955,0cyclictest2229-21kworker/0:107:30:320
316579955,0cyclictest2229-21kworker/0:107:05:340
316579950,0cyclictest0-21swapper/009:46:120
29725050,0irq/27-eth0-tx-26761-21munin-node12:20:362
29725050,0irq/27-eth0-tx-2427-21ssh12:05:022
29725050,0irq/27-eth0-tx-22367-21kthreadcore10:35:342
29725050,0irq/27-eth0-tx-12200-21munin-node11:10:322
29725050,0irq/27-eth0-tx-11244-21sendmail_mailtr10:05:382
29725050,0irq/27-eth0-tx-0-21swapper/210:48:172
29725050,0irq/27-eth0-tx-0-21swapper/210:13:322
29725050,0irq/27-eth0-tx-0-21swapper/209:31:262
29715050,0irq/26-eth0-rx-9625-21load11:05:343
29715050,0irq/26-eth0-rx-4752-21tr11:45:313
29715050,0irq/26-eth0-rx-32582-21cut11:00:333
29715050,0irq/26-eth0-rx-32458-21kthreadcore08:35:333
29715050,0irq/26-eth0-rx-29299-21sh09:18:063
29715050,0irq/26-eth0-rx-21594-21kthreadcore08:50:333
29715050,0irq/26-eth0-rx-20753-21cpu10:55:273
29715050,0irq/26-eth0-rx-18012-21ssh11:52:133
29715050,0irq/26-eth0-rx-11744-21swap09:05:403
29715050,0irq/26-eth0-rx-0-21swapper/312:20:323
29715050,0irq/26-eth0-rx-0-21swapper/311:32:413
29715050,0irq/26-eth0-rx-0-21swapper/310:40:173
29715050,0irq/26-eth0-rx-0-21swapper/310:36:103
29715050,0irq/26-eth0-rx-0-21swapper/310:00:283
29715050,0irq/26-eth0-rx-0-21swapper/309:50:283
29715050,0irq/26-eth0-rx-0-21swapper/309:25:213
29715050,0irq/26-eth0-rx-0-21swapper/309:10:343
27150,0ktimersoftd/20-21swapper/211:53:222
259950,0migration/28723-21latency11:45:332
259950,0migration/26187-21sh09:25:222
259950,0migration/23658-21switchtime09:00:382
259950,0migration/229201-21fschecks_time11:00:292
259950,0migration/225136-21kthreadcore08:30:322
259950,0migration/222411-21munin-node08:50:362
259950,0migration/22143-21kthreadcore10:20:342
259950,0migration/212218-21kernelversion12:10:302
249950,0watchdog/211727-21munin-node11:30:262
179950,0migration/19914-21taskset08:44:481
179950,0migration/130758-21fschecks_time09:00:291
179950,0migration/12786-21taskset12:05:231
179950,0migration/12378-21latency10:20:341
179950,0migration/123594-21munin-node08:55:271
169950,0watchdog/126413-21swap10:55:421
169950,0watchdog/118553-21timerwakeupswit10:50:381
149950,0migration/07232-21kthreadcore11:45:320
149950,0migration/030253-21irqrtprio10:40:330
149950,0migration/030159-21kthreadcore12:00:330
149950,0migration/029915-21kthreadcore11:00:320
149950,0migration/029657-21cpuspeed_turbos10:20:290
149950,0migration/029240-21kthreadcore11:40:310
149950,0migration/029092-21taskset09:58:550
149950,0migration/027056-21munin-node07:20:310
149950,0migration/025336-21kthreadcore08:30:320
149950,0migration/024938-21kthreadcore10:15:320
149950,0migration/022824-21fschecks_count09:35:310
149950,0migration/022642-21kthreadcore12:15:330
149950,0migration/021336-21kthreadcore11:55:320
149950,0migration/021092-21aten_r1power_en10:35:280
149950,0migration/019378-21proc_pri07:35:370
149950,0migration/019324-21latency_hist07:15:160
149950,0migration/018400-21latency10:30:340
149950,0migration/01831-21taskset12:24:390
149950,0migration/017353-21kthreadcore11:10:330
149950,0migration/017338-21kthreadcore10:10:340
149950,0migration/016086-21irqrtprio08:00:320
149950,0migration/013251-21kthreadcore08:45:330
149950,0migration/012743-21taskset11:30:300
149950,0migration/012288-21sh10:28:510
149950,0migration/012210-21kernelversion12:10:300
149950,0migration/011712-21munin-node09:05:390
149950,0migration/011491-21sh11:09:010
149950,0migration/011338-21kthreadcore08:20:330
149950,0migration/011202-21kthreadcore07:55:320
149950,0migration/01029-21latency08:10:350
149950,0migration/01001-21kthreadcore07:00:320
339940,0migration/39418-21unixbench-2d08:40:393
339940,0migration/38957-21kthreadcore07:05:343
339940,0migration/36991-21irqcore09:45:333
339940,0migration/34264-21diskmemload09:41:093
339940,0migration/331859-21swap07:20:383
339940,0migration/325893-21kthreadcore10:15:333
339940,0migration/32323-21kthreadcore10:20:343
339940,0migration/319303-21diskmemload10:30:593
339940,0migration/313889-21aten_r1power_en07:35:283
339940,0migration/313532-21aten2.4-expect10:10:273
339940,0migration/31298-21latency_hist07:50:163
339940,0migration/310303-21kthreadcore10:25:343
316599943,0cyclictest4743-21awk11:25:302
316599943,0cyclictest3514-21snmpd09:05:322
316599943,0cyclictest3514-21snmpd07:10:322
316599943,0cyclictest2332-21awk07:50:312
316599943,0cyclictest21326-21awk07:40:312
316599943,0cyclictest1254-21awk07:25:312
316599943,0cyclictest10058-21awk08:20:312
316589943,0cyclictest9023-21awk07:55:301
316589943,0cyclictest8037-21awk07:30:301
316589943,0cyclictest3556-21lldpd10:45:281
316589943,0cyclictest3556-21lldpd10:09:581
316589943,0cyclictest3556-21lldpd09:50:571
316589943,0cyclictest3514-21snmpd11:56:171
316589943,0cyclictest3514-21snmpd10:19:321
316589943,0cyclictest3514-21snmpd08:50:321
316589943,0cyclictest3300-21awk08:15:311
316589943,0cyclictest3241-21irqbalance12:12:541
316589943,0cyclictest3241-21irqbalance11:51:041
316589943,0cyclictest3241-21irqbalance11:42:441
316589943,0cyclictest3241-21irqbalance11:19:541
316589943,0cyclictest30683-21awk10:00:301
316589943,0cyclictest29544-21awk11:00:311
316589943,0cyclictest23157-21awk09:15:291
316589943,0cyclictest21909-21awk10:35:321
316589943,0cyclictest15689-21awk08:00:301
316589943,0cyclictest14390-21awk10:10:301
316579944,0cyclictest29947-21diskmemload10:01:390
316579944,0cyclictest2229-21kworker/0:111:36:510
316579944,0cyclictest2229-21kworker/0:110:55:370
316579944,0cyclictest2229-21kworker/0:110:50:500
316579944,0cyclictest2229-21kworker/0:109:50:440
316579944,0cyclictest2229-21kworker/0:109:31:080
316579944,0cyclictest2229-21kworker/0:109:10:340
316579944,0cyclictest2229-21kworker/0:108:40:280
316579944,0cyclictest2229-21kworker/0:107:45:300
316579944,0cyclictest0-21swapper/010:47:040
316579944,0cyclictest0-21swapper/010:06:420
316579944,0cyclictest0-21swapper/009:15:180
316579943,0cyclictest3556-21lldpd12:25:280
316579943,0cyclictest3514-21snmpd09:25:320
316579940,0cyclictest0-21swapper/009:42:440
29725040,0irq/27-eth0-tx-4918-21latency_hist10:25:162
29725040,0irq/27-eth0-tx-24612-21kthreadcore11:55:332
29725040,0irq/27-eth0-tx-0-21swapper/212:25:562
29725040,0irq/27-eth0-tx-0-21swapper/211:40:282
29725040,0irq/27-eth0-tx-0-21swapper/211:20:292
29725040,0irq/27-eth0-tx-0-21swapper/211:05:272
29725040,0irq/27-eth0-tx-0-21swapper/210:30:222
29725040,0irq/27-eth0-tx-0-21swapper/209:46:212
29725040,0irq/27-eth0-tx-0-21swapper/209:40:162
29725040,0irq/27-eth0-tx-0-21swapper/008:50:370
29725040,0irq/27-eth0-tx-0-21swapper/008:35:270
29715040,0irq/26-eth0-rx-20700-21kthreadcore11:55:323
29715040,0irq/26-eth0-rx-0-21swapper/312:06:553
29715040,0irq/26-eth0-rx-0-21swapper/310:05:193
29715040,0irq/26-eth0-rx-0-21swapper/309:21:313
29715040,0irq/26-eth0-rx-0-21swapper/308:25:333
29715040,0irq/26-eth0-rx-0-21swapper/308:00:273
29715040,0irq/26-eth0-rx-0-21swapper/307:30:273
259940,0migration/28639-21kthreadcore07:30:322
259940,0migration/28543-21kthreadcore08:40:342
259940,0migration/232653-21munin-node07:00:302
259940,0migration/232090-21kthreadcore08:35:332
259940,0migration/227918-21turbostat08:10:162
259940,0migration/226305-21kthreadcore09:35:352
259940,0migration/224130-21kthreadcore09:15:312
259940,0migration/223894-21kthreadcore10:15:322
259940,0migration/223195-21kthreadcore11:35:342
259940,0migration/220690-21munin-node07:15:322
259940,0migration/220524-21latency_hist10:55:162
259940,0migration/218122-21kthreadcore08:00:332
259940,0migration/215785-21latency_hist08:25:162
259940,0migration/210836-21fschecks_time08:45:292
249940,0watchdog/26978-21hddtemp_smartct07:05:312
249940,0watchdog/232143-21kthreadcore09:20:352
179940,0migration/19635-21timerandwakeup11:25:381
179940,0migration/19454-21kthreadcore07:05:351
179940,0migration/15785-21taskset07:03:291
179940,0migration/13891-21kthreadcore12:25:311
179940,0migration/129467-21fschecks_count10:40:301
179940,0migration/125555-21kthreadcore09:55:331
179940,0migration/122114-21kthreadcore11:35:331
179940,0migration/121732-21kthreadcore07:15:331
179940,0migration/120645-21kthreadcore12:15:321
179940,0migration/117627-21munin-node11:10:341
179940,0migration/114575-21kthreadcore08:45:331
169940,0watchdog/19485-21users11:45:381
169940,0watchdog/16473-21fschecks_time09:05:311
169940,0watchdog/125078-21grep09:35:351
169940,0watchdog/118242-21kthreadcore09:30:351
169940,0watchdog/110880-21python10:25:381
169940,0watchdog/10-21swapper/109:25:471
169940,0watchdog/10-21swapper/109:10:351
169940,0watchdog/10-21swapper/108:20:311
149940,0migration/03380-21if_eth108:15:320
149940,0migration/025326-21kthreadcore08:55:320
149940,0migration/025220-21kthreadcore11:15:340
149940,0migration/02237-21kthreadcore07:25:330
149940,0migration/021840-21kthreadcore07:15:330
316599932,0cyclictest3241-21irqbalance07:20:352
316599932,0cyclictest29274-21cut07:45:322
316599932,0cyclictest24387-21awk08:55:302
316599930,0cyclictest0-21swapper/208:10:582
316589933,0cyclictest14728-21awk07:35:321
316589933,0cyclictest0-21swapper/108:05:441
316589932,0cyclictest7168-21sensors07:50:371
316589932,0cyclictest28179-21awk12:00:311
316589932,0cyclictest27987-21awk07:45:301
316589932,0cyclictest26489-21users07:40:401
316589932,0cyclictest1299-21awk07:25:321
316589932,0cyclictest12871-21munin-node07:10:281
316589930,0cyclictest666-21kthreadcore08:35:331
316589930,0cyclictest31769-21kthreadcore09:40:331
316589930,0cyclictest30952-21kthreadcore08:10:341
316589930,0cyclictest27440-21irqstats07:20:321
316579933,0cyclictest2229-21kworker/0:111:25:250
316579933,0cyclictest2229-21kworker/0:108:25:240
316579933,0cyclictest2229-21kworker/0:108:05:320
316579933,0cyclictest2229-21kworker/0:107:40:180
29715030,0irq/26-eth0-rx-0-21swapper/308:46:203
29715030,0irq/26-eth0-rx-0-21swapper/308:15:193
29715030,0irq/26-eth0-rx-0-21swapper/308:05:493
29715030,0irq/26-eth0-rx-0-21swapper/307:55:273
29715030,0irq/26-eth0-rx-0-21swapper/307:50:403
29715030,0irq/26-eth0-rx-0-21swapper/307:25:283
29715030,0irq/26-eth0-rx-0-21swapper/307:00:273
249930,0watchdog/20-21swapper/207:55:312
169930,0watchdog/10-21swapper/111:30:261
169930,0watchdog/10-21swapper/108:30:271
169930,0watchdog/10-21swapper/108:25:311
316599920,0cyclictest13792-21latency_hist07:35:172
29715020,0irq/26-eth0-rx-0-21swapper/308:30:303
29715020,0irq/26-eth0-rx-0-21swapper/308:10:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional