You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-07 - 03:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Tue Apr 07, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1962323625,0sleep20-21swapper/218:54:392
304450350,0irq/25-eth10-21swapper/318:54:193
1984523526,0sleep10-21swapper/118:57:181
1979923531,0sleep00-21swapper/018:56:410
339960,0migration/36366-21munin-node23:29:293
339960,0migration/331412-21munin-node23:24:353
339960,0migration/325457-21sh22:01:353
339960,0migration/325389-21sh21:40:233
339960,0migration/318642-21kernelversion23:19:283
29715060,0irq/26-eth0-rx-3932-21cat20:44:293
29715060,0irq/26-eth0-rx-21331-21tr23:59:293
29715060,0irq/26-eth0-rx-11274-21forks21:14:253
200829966,0cyclictest18441-21diskmemload21:15:272
200829965,0cyclictest25455-21grep23:44:242
200829960,0cyclictest0-21swapper/222:58:152
200819966,0cyclictest0-21swapper/121:44:261
149960,0migration/09516-21sh22:12:490
149960,0migration/0512-21sh21:04:400
149960,0migration/032375-21taskset22:24:360
149960,0migration/031812-21timerandwakeup22:44:330
149960,0migration/030191-21sendmail_mailtr00:24:310
149960,0migration/025756-21sh21:21:090
149960,0migration/02509-21sh21:09:150
149960,0migration/017053-21idleruntime-cro23:19:110
149960,0migration/015754-21sh00:18:020
149960,0migration/012417-21kthreadcore21:14:290
339950,0migration/39009-21taskset21:50:123
339950,0migration/33975-21kthreadcore23:09:273
339950,0migration/328311-21kthreadcore23:04:293
339950,0migration/327394-21kthreadcore20:39:283
339950,0migration/327370-21taskset19:24:433
339950,0migration/326847-21runrttasks21:04:223
339950,0migration/326697-21taskset22:04:213
339950,0migration/324819-21taskset22:41:533
339950,0migration/32468-21munin-node22:49:273
339950,0migration/324587-21timerandwakeup20:59:343
339950,0migration/322311-21taskset19:49:133
339950,0migration/315434-21hwlatdetect19:19:263
339950,0migration/312438-21kthreadcore19:39:293
339950,0migration/311663-21kthreadcore23:54:283
339950,0migration/310783-21fschecks_time20:04:253
29725050,0irq/27-eth0-tx-990-21ssh21:05:512
29725050,0irq/27-eth0-tx-9792-21ssh21:31:392
29725050,0irq/27-eth0-tx-7655-21seq23:31:102
29725050,0irq/27-eth0-tx-26250-21munin-node22:24:272
29725050,0irq/27-eth0-tx-23398-21munin-node20:14:320
29725050,0irq/27-eth0-tx-0-21swapper/223:55:052
29725050,0irq/27-eth0-tx-0-21swapper/223:28:082
29725050,0irq/27-eth0-tx-0-21swapper/020:44:220
29725050,0irq/27-eth0-tx-0-21swapper/020:29:520
29725050,0irq/27-eth0-tx-0-21swapper/019:23:560
29715050,0irq/26-eth0-rx-7206-21kthreadcore21:09:323
29715050,0irq/26-eth0-rx-6840-21kthreadcore19:34:293
29715050,0irq/26-eth0-rx-649-21aten_r1power_po23:49:223
29715050,0irq/26-eth0-rx-32463-40yum-updatesd-he21:44:343
29715050,0irq/26-eth0-rx-27288-21ssh21:24:193
29715050,0irq/26-eth0-rx-25960-21irqrtprio00:04:273
29715050,0irq/26-eth0-rx-12402-21cut23:14:293
29715050,0irq/26-eth0-rx-11076-21idleruntime22:14:253
29715050,0irq/26-eth0-rx-0-21swapper/323:39:313
29715050,0irq/26-eth0-rx-0-21swapper/322:46:323
29715050,0irq/26-eth0-rx-0-21swapper/322:36:573
29715050,0irq/26-eth0-rx-0-21swapper/322:29:223
29715050,0irq/26-eth0-rx-0-21swapper/322:19:293
29715050,0irq/26-eth0-rx-0-21swapper/321:34:263
29715050,0irq/26-eth0-rx-0-21swapper/321:33:313
29715050,0irq/26-eth0-rx-0-21swapper/320:34:293
29715050,0irq/26-eth0-rx-0-21swapper/300:24:273
29715050,0irq/26-eth0-rx-0-21swapper/300:23:233
27150,0ktimersoftd/20-21swapper/200:02:422
259950,0migration/227143-21latency_hist21:24:122
259950,0migration/217096-21sh22:16:422
259950,0migration/211398-21fschecks_count21:34:232
249950,0watchdog/214189-21kthreadcore21:54:302
200829955,0cyclictest18441-21diskmemload22:50:592
200829955,0cyclictest0-21swapper/223:08:102
200829955,0cyclictest0-21swapper/222:34:172
200829955,0cyclictest0-21swapper/222:06:512
200829955,0cyclictest0-21swapper/221:50:182
200829955,0cyclictest0-21swapper/220:59:322
200829950,0cyclictest0-21swapper/223:50:352
200819950,0cyclictest0-21swapper/123:51:301
200819950,0cyclictest0-21swapper/123:40:091
200819950,0cyclictest0-21swapper/123:29:071
200809955,0cyclictest0-21swapper/020:24:320
19150,0ktimersoftd/10-21swapper/121:35:171
179950,0migration/13314-21kthreadcore23:29:271
179950,0migration/122684-21kthreadcore19:24:291
179950,0migration/117545-21idleruntime20:09:271
179950,0migration/115284-21sh23:55:521
149950,0migration/0999-21sh21:45:280
149950,0migration/0835-21kthreadcore19:54:300
149950,0migration/08149-21aten_r1power_vo19:14:230
149950,0migration/06528-21irqrtprio20:49:290
149950,0migration/05320-21kthreadcore23:49:290
149950,0migration/032467-21kthreadcore21:24:260
149950,0migration/030247-21munin-node00:04:300
149950,0migration/024785-21kthreadcore18:59:280
149950,0migration/020614-21kthreadcore23:59:290
149950,0migration/018993-21kthreadcore00:19:260
149950,0migration/018299-21kthreadcore20:09:300
149950,0migration/017336-21taskset21:35:190
149950,0migration/016109-21sh22:34:530
149950,0migration/015769-21kthreadcore20:54:300
149950,0migration/014425-21kthreadcore21:54:300
149950,0migration/010488-21kthreadcore19:39:280
339940,0migration/38051-21kthreadcore20:49:303
339940,0migration/36442-21kthreadcore19:59:283
339940,0migration/34792-21kthreadcore22:09:293
339940,0migration/332327-21kthreadcore20:19:273
339940,0migration/331469-21sh23:45:323
339940,0migration/318414-21kthreadcore20:09:303
29725040,0irq/27-eth0-tx-4468-21cut22:09:292
29725040,0irq/27-eth0-tx-3514-21snmpd22:39:332
29725040,0irq/27-eth0-tx-27556-21ps00:24:272
29725040,0irq/27-eth0-tx-14912-21pmu-power23:14:322
29725040,0irq/27-eth0-tx-0-21swapper/223:41:512
29725040,0irq/27-eth0-tx-0-21swapper/222:59:162
29725040,0irq/27-eth0-tx-0-21swapper/221:44:252
29725040,0irq/27-eth0-tx-0-21swapper/221:39:282
29725040,0irq/27-eth0-tx-0-21swapper/200:19:262
29725040,0irq/27-eth0-tx-0-21swapper/200:14:362
29725040,0irq/27-eth0-tx-0-21swapper/200:10:002
29725040,0irq/27-eth0-tx-0-21swapper/200:05:172
29725040,0irq/27-eth0-tx-0-21swapper/019:49:310
29725040,0irq/27-eth0-tx-0-21swapper/019:34:230
29725040,0irq/27-eth0-tx-0-21swapper/019:33:580
29725040,0irq/27-eth0-tx-0-21swapper/019:09:230
29715040,0irq/26-eth0-rx-27452-21munin-node19:04:253
29715040,0irq/26-eth0-rx-0-21swapper/323:34:273
29715040,0irq/26-eth0-rx-0-21swapper/323:01:053
29715040,0irq/26-eth0-rx-0-21swapper/322:55:073
29715040,0irq/26-eth0-rx-0-21swapper/322:28:233
29715040,0irq/26-eth0-rx-0-21swapper/321:57:033
29715040,0irq/26-eth0-rx-0-21swapper/321:19:223
29715040,0irq/26-eth0-rx-0-21swapper/320:54:303
29715040,0irq/26-eth0-rx-0-21swapper/320:29:283
29715040,0irq/26-eth0-rx-0-21swapper/320:24:223
29715040,0irq/26-eth0-rx-0-21swapper/300:16:223
29715040,0irq/26-eth0-rx-0-21swapper/300:13:313
259940,0migration/29360-21kthreadcore20:24:292
259940,0migration/26351-21kthreadcore22:29:282
259940,0migration/26150-21kthreadcore19:34:292
259940,0migration/232415-21kthreadcore20:44:282
259940,0migration/23070-21kthreadcore19:09:282
259940,0migration/225278-21kthreadcore19:24:292
259940,0migration/224730-21kthreadcore19:49:292
259940,0migration/222124-21kthreadcore21:19:282
259940,0migration/222047-21kthreadcore18:59:282
259940,0migration/217310-21fschecks_time20:09:262
259940,0migration/216889-21timerandwakeup20:29:332
259940,0migration/213172-21kthreadcore20:04:282
259940,0migration/211775-21runrttasks20:53:562
259940,0migration/210961-21kthreadcore19:39:282
249940,0watchdog/24330-21if_eth1.319:59:262
249940,0watchdog/22881-21cpuspeed_turbos21:09:262
249940,0watchdog/20-21swapper/223:09:222
249940,0watchdog/20-21swapper/221:59:262
249940,0watchdog/20-21swapper/220:54:302
249940,0watchdog/20-21swapper/220:14:262
200829943,0cyclictest9835-21awk23:34:252
200829943,0cyclictest3514-21snmpd23:20:432
200829943,0cyclictest3514-21snmpd22:19:132
200829943,0cyclictest3514-21snmpd20:21:282
200829943,0cyclictest29953-21awk19:54:272
200829943,0cyclictest26842-21awk22:44:262
200819944,0cyclictest0-21swapper/122:06:021
200819944,0cyclictest0-21swapper/122:02:341
200819944,0cyclictest0-21swapper/121:32:181
200819943,0cyclictest9969-21awk19:39:271
200819943,0cyclictest5284-21awk20:24:261
200819943,0cyclictest3556-21lldpd23:14:231
200819943,0cyclictest3556-21lldpd22:41:521
200819943,0cyclictest3556-21lldpd22:31:221
200819943,0cyclictest3556-21lldpd22:13:231
200819943,0cyclictest28110-21awk21:24:231
200819943,0cyclictest27880-21awk19:04:261
200819943,0cyclictest25411-21awk00:24:251
200819943,0cyclictest25326-21awk20:39:261
200819943,0cyclictest19163-21awk22:19:271
200819943,0cyclictest17278-21awk00:19:241
200819943,0cyclictest15600-21awk19:19:271
200819943,0cyclictest11554-21awk21:54:271
200819940,0cyclictest0-21swapper/122:27:191
200809944,0cyclictest0-21swapper/022:59:120
200809944,0cyclictest0-21swapper/022:04:480
200809943,0cyclictest3556-21lldpd23:05:230
200809943,0cyclictest3514-21snmpd19:24:130
200809943,0cyclictest3241-21irqbalance21:02:000
200809943,0cyclictest30892-21awk20:19:250
200809943,0cyclictest19824-21awk21:39:260
200809943,0cyclictest16426-21kworker/0:020:34:330
200809943,0cyclictest11714-21kworker/0:122:20:490
200809943,0cyclictest10102-21kworker/0:023:38:290
200809940,0cyclictest0-21swapper/021:52:020
179940,0migration/14654-21kthreadcore22:49:291
179940,0migration/131439-21sh23:45:271
179940,0migration/130272-21sh00:04:301
179940,0migration/127756-21kthreadcore22:44:271
179940,0migration/12739-21aten_r1power_vo21:49:231
179940,0migration/124309-21kthreadcore19:49:291
179940,0migration/122303-21kthreadcore21:39:281
179940,0migration/120758-21kthreadcore23:59:291
179940,0migration/119077-21kthreadcore22:59:271
179940,0migration/116307-21kthreadcore20:29:281
179940,0migration/116024-21taskset22:55:251
179940,0migration/11282-21aten_r1power_en19:09:231
179940,0migration/111446-21munin-node22:14:271
179940,0migration/111401-21kthreadcore19:14:291
179940,0migration/110866-21hwlatdetect20:04:251
169940,0watchdog/16239-21cat00:09:331
169940,0watchdog/15767-21fschecks_count20:49:261
169940,0watchdog/12880-21tr21:09:261
169940,0watchdog/119029-21aten_r1power_en21:19:221
169940,0watchdog/117318-21ssh23:19:221
169940,0watchdog/114271-21cat00:14:331
169940,0watchdog/10-21swapper/123:09:261
169940,0watchdog/10-21swapper/122:34:421
169940,0watchdog/10-21swapper/119:59:261
169940,0watchdog/10-21swapper/119:34:221
149940,0migration/07822-21sh23:10:410
149940,0migration/032472-21runrttasks23:27:400
149940,0migration/030458-21kthreadcore19:04:290
149940,0migration/025736-21fschecks_time23:44:240
149940,0migration/019294-21kthreadcore19:44:280
149940,0migration/018566-21kthreadcore23:39:270
149940,0migration/01659-21hwlatdetect23:29:250
149940,0migration/010687-21fschecks_time20:04:250
29715030,0irq/26-eth0-rx-0-21swapper/320:14:183
29715030,0irq/26-eth0-rx-0-21swapper/319:10:393
29715030,0irq/26-eth0-rx-0-21swapper/318:59:313
259930,0migration/231872-21kthreadcore19:29:282
249930,0watchdog/20-21swapper/219:44:222
200829932,0cyclictest3514-21snmpd19:23:282
200829932,0cyclictest27826-21awk19:04:262
200829932,0cyclictest25224-21awk20:39:262
200829932,0cyclictest13640-21sensors19:14:332
200829931,0cyclictest18222-21fschecks_count20:34:252
200819932,0cyclictest3556-21lldpd23:05:531
200819932,0cyclictest3514-21snmpd20:59:281
200819932,0cyclictest3514-21snmpd20:54:141
200819932,0cyclictest27702-21awk21:04:271
200819932,0cyclictest21181-21awk18:59:261
200819932,0cyclictest16578-21sensors_fan21:14:341
200819932,0cyclictest14780-21sensors23:34:321
200819932,0cyclictest1253-21sendmail_mailtr19:29:331
200809933,0cyclictest31685-21kworker/0:222:54:170
200809933,0cyclictest31685-21kworker/0:222:49:170
200809933,0cyclictest24409-21kworker/0:100:09:150
200809933,0cyclictest16426-21kworker/0:021:29:150
200809933,0cyclictest16426-21kworker/0:020:39:190
200809933,0cyclictest16426-21kworker/0:019:59:130
200809933,0cyclictest11714-21kworker/0:122:39:130
200809933,0cyclictest11714-21kworker/0:122:29:130
200809933,0cyclictest11714-21kworker/0:122:14:150
200809933,0cyclictest11714-21kworker/0:121:59:230
200809933,0cyclictest10102-21kworker/0:023:54:170
200809933,0cyclictest10102-21kworker/0:023:19:130
179930,0migration/117787-21aten_r1power_en20:34:231
169930,0watchdog/13876-21runrttasks20:21:261
169930,0watchdog/10-21swapper/120:14:261
169930,0watchdog/10-21swapper/119:44:221
29715020,0irq/26-eth0-rx-0-21swapper/319:54:243
29715020,0irq/26-eth0-rx-0-21swapper/319:44:263
29715020,0irq/26-eth0-rx-0-21swapper/319:29:233
29715020,0irq/26-eth0-rx-0-21swapper/319:14:203
200819921,0cyclictest29001-21latency_hist19:54:121
200819920,0cyclictest0-21swapper/120:44:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional