You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-28 - 09:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Tue Apr 28, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1437423925,0sleep00-21swapper/018:55:040
304450350,0irq/25-eth10-21swapper/318:52:463
1437123526,0sleep20-21swapper/218:55:022
1442123424,0sleep10-21swapper/118:55:411
339960,0migration/328347-21sh22:05:433
339960,0migration/32705-21ksmtuned22:47:593
339960,0migration/327010-21sh23:45:093
339960,0migration/324063-21taskset20:16:133
339960,0migration/312701-21sh22:37:203
29715060,0irq/26-eth0-rx-28391-21perl00:07:283
29715060,0irq/26-eth0-rx-24187-21cut23:22:303
29715060,0irq/26-eth0-rx-1730-21grep22:27:303
146289965,0cyclictest26501-21sed19:52:311
146289960,0cyclictest12836-21diskmemload21:05:341
149960,0migration/08065-21kthreadcore21:52:290
149960,0migration/028030-21sh22:04:490
149960,0migration/019992-21sh22:40:480
149960,0migration/019913-21sh21:59:270
4150,0ktimersoftd/00-21swapper/021:16:190
35150,0ktimersoftd/30-21swapper/321:36:113
339950,0migration/38915-21munin-node19:37:333
339950,0migration/34975-21latency_hist21:52:133
339950,0migration/330306-21kthreadcore23:27:283
339950,0migration/330188-21munin-node21:47:283
339950,0migration/328359-21kthreadcore19:52:313
339950,0migration/328003-21sh22:45:463
339950,0migration/326771-21runrttasks00:26:123
339950,0migration/322121-21kthreadcore00:02:313
339950,0migration/321955-21kthreadcore23:02:303
339950,0migration/320008-21munin-node20:37:283
339950,0migration/319864-21diskmemload21:18:273
339950,0migration/317062-21munin-node19:22:293
339950,0migration/316246-21kthreadcore20:32:313
339950,0migration/315173-21kthreadcore20:07:293
339950,0migration/314003-21kthreadcore22:57:293
29725050,0irq/27-eth0-tx-6716-21cut23:52:302
29725050,0irq/27-eth0-tx-5634-21diskstats21:12:242
29725050,0irq/27-eth0-tx-3514-21snmpd19:19:580
29725050,0irq/27-eth0-tx-28499-21ssh22:06:092
29725050,0irq/27-eth0-tx-23921-21ssh21:42:302
29725050,0irq/27-eth0-tx-20635-21perl23:42:242
29725050,0irq/27-eth0-tx-20578-21munin-node23:02:252
29725050,0irq/27-eth0-tx-20283-21ssh21:39:532
29725050,0irq/27-eth0-tx-19937-21ssh22:40:392
29725050,0irq/27-eth0-tx-14307-21irqrtprio20:57:300
29725050,0irq/27-eth0-tx-12282-21munin-node23:17:292
29725050,0irq/27-eth0-tx-1195-21munin-node23:47:342
29725050,0irq/27-eth0-tx-0-21swapper/223:26:142
29725050,0irq/27-eth0-tx-0-21swapper/222:32:172
29725050,0irq/27-eth0-tx-0-21swapper/200:15:012
29725050,0irq/27-eth0-tx-0-21swapper/020:02:380
29725050,0irq/27-eth0-tx-0-21swapper/019:31:530
29725050,0irq/27-eth0-tx-0-21swapper/019:06:490
29715050,0irq/26-eth0-rx-7276-21kthreadcore23:52:303
29715050,0irq/26-eth0-rx-4254-21ssh23:32:263
29715050,0irq/26-eth0-rx-30895-21kthreadcore23:07:293
29715050,0irq/26-eth0-rx-29829-21grep21:07:253
29715050,0irq/26-eth0-rx-28604-21fschecks_time23:47:273
29715050,0irq/26-eth0-rx-27625-21ssh21:42:563
29715050,0irq/26-eth0-rx-2554-21cat21:27:313
29715050,0irq/26-eth0-rx-20179-21ssh21:00:213
29715050,0irq/26-eth0-rx-18686-21ntp_offset21:57:313
29715050,0irq/26-eth0-rx-0-21swapper/323:57:303
29715050,0irq/26-eth0-rx-0-21swapper/321:22:303
29715050,0irq/26-eth0-rx-0-21swapper/321:12:403
29715050,0irq/26-eth0-rx-0-21swapper/300:17:243
29715050,0irq/26-eth0-rx-0-21swapper/300:14:283
27150,0ktimersoftd/20-21swapper/221:36:112
259950,0migration/27252-21kthreadcore21:52:292
259950,0migration/27177-21kthreadcore22:12:282
259950,0migration/2613-21latency00:07:322
259950,0migration/25957-21kthreadcore23:32:312
259950,0migration/229414-21cpuspeed_turbos22:07:252
259950,0migration/229414-21cpuspeed_turbos22:07:252
259950,0migration/22904-21fschecks_count19:12:262
259950,0migration/223890-21kthreadcore19:27:302
259950,0migration/221487-21cpuspeed_turbos21:02:262
259950,0migration/22120-21munin-node22:27:322
259950,0migration/217139-21kthreadcore21:57:292
259950,0migration/214245-21kthreadcore22:17:292
259950,0migration/214121-21kthreadcore23:57:302
259950,0migration/213569-21fschecks_count20:57:272
249950,0watchdog/217155-21munin-node00:17:342
249950,0watchdog/215895-21kthreadcore20:32:312
19150,0ktimersoftd/10-21swapper/121:44:311
179950,0migration/17512-21kthreadcore21:32:311
179950,0migration/123582-21kthreadcore23:22:301
179950,0migration/12048-21sh00:09:211
179950,0migration/119496-21unixbench_singl21:17:371
179950,0migration/110677-21ksmtuned00:15:591
179950,0migration/110224-21munin-node19:42:251
169950,0watchdog/12060-21kthreadcore21:27:311
146299955,0cyclictest0-21swapper/221:22:162
146299955,0cyclictest0-21swapper/221:11:322
146299954,0cyclictest3236-21taskset22:49:272
146299950,0cyclictest0-21swapper/223:15:372
146299950,0cyclictest0-21swapper/222:59:512
146299950,0cyclictest0-21swapper/220:27:432
146289955,0cyclictest0-21swapper/123:32:501
146289955,0cyclictest0-21swapper/122:15:151
146289955,0cyclictest0-21swapper/121:13:081
146289950,0cyclictest0-21swapper/100:21:421
146279950,0cyclictest0-21swapper/021:41:230
146279950,0cyclictest0-21swapper/000:23:560
149950,0migration/09198-21kthreadcore22:12:290
149950,0migration/08570-21kthreadcore22:32:330
149950,0migration/05058-21missed_timers20:47:290
149950,0migration/04358-21aten_r1power_po23:12:240
149950,0migration/04255-21fschecks_count23:52:260
149950,0migration/03663-21taskset21:28:290
149950,0migration/03497-21taskset22:09:170
149950,0migration/03497-21taskset22:09:170
149950,0migration/031707-21kthreadcore19:57:300
149950,0migration/031356-21unixbench_singl20:42:360
149950,0migration/029379-21fschecks_time22:47:260
149950,0migration/028905-21hddtemp_smartct19:07:260
149950,0migration/027937-21kthreadcore20:17:300
149950,0migration/025041-21kthreadcore22:22:280
149950,0migration/023952-21kthreadcore00:02:320
149950,0migration/02393-21aten_r1power_en19:12:240
149950,0migration/023823-21taskset20:13:270
149950,0migration/022956-21taskset19:49:450
149950,0migration/020883-21kthreadcore20:37:280
149950,0migration/020649-21latency_hist22:42:130
149950,0migration/019261-21sh00:01:350
149950,0migration/017018-21kthreadcore21:17:310
149950,0migration/01643-21kthreadcore21:47:310
149950,0migration/012502-21taskset21:35:420
149950,0migration/011118-21munin-node20:27:360
149950,0migration/010920-21ksmtuned23:34:590
339940,0migration/38789-21kthreadcore20:27:313
339940,0migration/33903-21kthreadcore20:22:303
339940,0migration/331384-21kthreadcore19:07:303
339940,0migration/327677-21runrttasks19:04:423
339940,0migration/326421-21kthreadcore20:17:303
339940,0migration/320802-21kthreadcore19:47:313
339940,0migration/31777-21kthreadcore19:32:323
339940,0migration/316199-21runrttasks19:43:243
339940,0migration/315962-21irqcore18:57:303
339940,0migration/313079-21munin-node23:17:243
29725040,0irq/27-eth0-tx-19764-21munin-node00:02:332
29725040,0irq/27-eth0-tx-0-21swapper/223:27:432
29725040,0irq/27-eth0-tx-0-21swapper/222:52:202
29725040,0irq/27-eth0-tx-0-21swapper/222:22:272
29725040,0irq/27-eth0-tx-0-21swapper/200:22:172
29715040,0irq/26-eth0-rx-8022-21kthreadcore22:32:323
29715040,0irq/26-eth0-rx-0-21swapper/323:39:023
29715040,0irq/26-eth0-rx-0-21swapper/323:12:293
29715040,0irq/26-eth0-rx-0-21swapper/322:52:233
29715040,0irq/26-eth0-rx-0-21swapper/322:24:203
29715040,0irq/26-eth0-rx-0-21swapper/322:19:303
29715040,0irq/26-eth0-rx-0-21swapper/322:14:523
29715040,0irq/26-eth0-rx-0-21swapper/322:07:243
29715040,0irq/26-eth0-rx-0-21swapper/322:07:243
29715040,0irq/26-eth0-rx-0-21swapper/321:38:203
29715040,0irq/26-eth0-rx-0-21swapper/321:02:323
29715040,0irq/26-eth0-rx-0-21swapper/320:47:233
29715040,0irq/26-eth0-rx-0-21swapper/319:31:533
259940,0migration/28390-21kthreadcore20:52:302
259940,0migration/25143-21gltestperf20:02:282
259940,0migration/2369-21kthreadcore20:22:292
259940,0migration/230920-21kthreadcore21:47:302
259940,0migration/230885-21irqrtprio21:27:302
259940,0migration/226872-21kthreadcore20:42:292
259940,0migration/223294-21kthreadcore20:37:292
259940,0migration/21629-21kthreadcore19:57:312
249940,0watchdog/22340-21sensors19:32:352
179940,0migration/17244-21idleruntime20:52:271
179940,0migration/17016-21kthreadcore23:12:301
179940,0migration/129145-21irqcore19:07:281
179940,0migration/129080-21fschecks_count23:07:261
179940,0migration/127279-21runrttasks22:43:401
179940,0migration/126396-21kthreadcore19:02:331
179940,0migration/125730-21kthreadcore21:22:301
179940,0migration/123413-21kthreadcore00:02:321
179940,0migration/122865-21taskset19:48:421
179940,0migration/114544-21kthreadcore22:37:301
179940,0migration/111174-21aten_r1power_fr20:07:231
169940,0watchdog/1788-21kthreadcore21:47:311
169940,0watchdog/17356-21grep20:02:311
169940,0watchdog/14440-21timerandwakeup20:22:351
169940,0watchdog/130268-21if_eth021:07:271
169940,0watchdog/129810-21basename22:27:271
169940,0watchdog/129123-21ssh22:07:191
169940,0watchdog/129123-21ssh22:07:191
169940,0watchdog/127292-21ssh23:45:581
169940,0watchdog/122972-21cut22:22:271
169940,0watchdog/12271-21ssh23:28:271
169940,0watchdog/12043-21kthreadcore20:47:271
169940,0watchdog/118950-21kthreadcore18:57:321
169940,0watchdog/118620-21ssh23:39:071
169940,0watchdog/113359-21hwlatdetect22:57:271
169940,0watchdog/10-21swapper/120:57:311
169940,0watchdog/10-21swapper/120:32:271
146299944,0cyclictest0-21swapper/219:02:162
146299943,0cyclictest3514-21snmpd21:17:422
146299943,0cyclictest29496-21awk23:07:272
146299943,0cyclictest29042-21awk19:07:282
146299943,0cyclictest20771-21munin-node22:42:292
146299943,0cyclictest16720-21awk19:22:282
146299943,0cyclictest15707-21awk18:57:292
146299943,0cyclictest12947-21awk23:37:272
146299943,0cyclictest12063-21awk20:07:272
146299943,0cyclictest10000-21awk19:17:272
146289944,0cyclictest0-21swapper/123:57:251
146289944,0cyclictest0-21swapper/119:37:361
146289943,0cyclictest6037-21awk21:52:271
146289943,0cyclictest3514-21snmpd23:52:421
146289943,0cyclictest3514-21snmpd23:17:271
146289943,0cyclictest3514-21snmpd22:07:121
146289943,0cyclictest3514-21snmpd21:57:571
146289943,0cyclictest3514-21snmpd20:37:271
146289943,0cyclictest3514-21snmpd20:27:421
146289943,0cyclictest3514-21snmpd00:22:571
146289943,0cyclictest3250-21awk19:12:271
146289943,0cyclictest3241-21irqbalance22:18:211
146289943,0cyclictest3241-21irqbalance21:37:511
146289943,0cyclictest29663-21awk22:47:271
146289943,0cyclictest28864-21awk23:47:281
146289943,0cyclictest25273-21awk20:17:281
146289943,0cyclictest23351-21awk19:27:291
146289943,0cyclictest16745-21awk19:22:281
146279944,0cyclictest0-21swapper/023:48:090
146279944,0cyclictest0-21swapper/022:32:090
146279943,0cyclictest3556-21lldpd19:00:390
146279943,0cyclictest3241-21irqbalance19:43:520
146279943,0cyclictest30300-21awk21:07:270
146279943,0cyclictest22344-21awk21:42:290
146279943,0cyclictest22042-21awk21:02:290
146279943,0cyclictest13152-21awk23:17:250
149940,0migration/0967-21kthreadcore20:22:300
149940,0migration/05843-21kthreadcore22:52:300
149940,0migration/030803-21kthreadcore23:27:290
149940,0migration/025592-21kthreadcore23:02:310
149940,0migration/021365-21irqcore23:22:290
149940,0migration/016692-21if_eth019:22:280
149940,0migration/015126-21kthreadcore23:37:280
35130,0ktimersoftd/314805-21smart_sdb19:17:343
339930,0migration/331443-21taskset20:43:003
29715030,0irq/26-eth0-rx-0-21swapper/320:52:293
29715030,0irq/26-eth0-rx-0-21swapper/319:57:253
29715030,0irq/26-eth0-rx-0-21swapper/319:13:303
249930,0watchdog/20-21swapper/220:47:232
169930,0watchdog/10-21swapper/123:02:311
169930,0watchdog/10-21swapper/119:57:271
169930,0watchdog/10-21swapper/119:32:311
146299933,0cyclictest4269-21awk19:37:282
146299932,0cyclictest23887-21grep19:52:272
146299932,0cyclictest23400-21sensors20:12:362
146299932,0cyclictest17700-21awk19:47:292
146299932,0cyclictest10044-21cat19:42:132
146289932,0cyclictest9890-21tail19:17:261
146289932,0cyclictest7380-21cut22:52:301
146289932,0cyclictest3514-21snmpd22:32:261
146289932,0cyclictest26322-21awk20:42:281
146289932,0cyclictest18668-21awk20:12:281
146279933,0cyclictest6573-21kworker/0:121:22:140
146279933,0cyclictest6573-21kworker/0:120:52:160
146279933,0cyclictest6573-21kworker/0:120:32:160
146279933,0cyclictest6573-21kworker/0:120:07:150
146279933,0cyclictest6573-21kworker/0:119:52:150
146279933,0cyclictest6573-21kworker/0:119:37:150
146279933,0cyclictest6573-21kworker/0:119:32:150
146279933,0cyclictest4509-21kworker/0:200:17:140
146279933,0cyclictest4509-21kworker/0:200:12:220
146279933,0cyclictest4509-21kworker/0:200:07:160
146279933,0cyclictest26036-21kworker/0:023:42:140
146279933,0cyclictest21558-21kworker/0:123:07:140
146279933,0cyclictest21558-21kworker/0:122:57:140
146279932,0cyclictest13738-21awk22:17:270
29715020,0irq/26-eth0-rx-0-21swapper/320:02:283
249920,0watchdog/20-21swapper/220:17:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional