You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 06:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sat Dec 06, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
291350970,0irq/25-eth10-21swapper/318:35:593
1235523824,0sleep00-21swapper/018:34:120
1844423324,0sleep20-21swapper/218:36:032
1859923026,0sleep10-21swapper/118:37:501
187779976,0cyclictest0-21swapper/122:39:231
149970,0migration/021417-21sh00:04:240
35160,0ktimersoftd/30-21swapper/318:53:243
339960,0migration/35125-21sh23:32:513
339960,0migration/329407-21kthreadcore21:04:243
339960,0migration/327796-21sh23:44:563
339960,0migration/31116-21sh20:49:123
28415060,0irq/27-eth0-tx-0-21swapper/223:44:412
28415060,0irq/27-eth0-tx-0-21swapper/223:35:112
28415060,0irq/27-eth0-tx-0-21swapper/223:28:402
28415060,0irq/27-eth0-tx-0-21swapper/223:14:062
28415060,0irq/27-eth0-tx-0-21swapper/223:00:062
28415060,0irq/27-eth0-tx-0-21swapper/222:53:502
28415060,0irq/27-eth0-tx-0-21swapper/222:43:232
28415060,0irq/27-eth0-tx-0-21swapper/222:32:242
28415060,0irq/27-eth0-tx-0-21swapper/222:16:192
28415060,0irq/27-eth0-tx-0-21swapper/222:08:582
28415060,0irq/27-eth0-tx-0-21swapper/222:00:412
28415060,0irq/27-eth0-tx-0-21swapper/221:57:482
28415060,0irq/27-eth0-tx-0-21swapper/221:40:222
28415060,0irq/27-eth0-tx-0-21swapper/221:34:442
28415060,0irq/27-eth0-tx-0-21swapper/221:30:392
28415060,0irq/27-eth0-tx-0-21swapper/221:23:502
28415060,0irq/27-eth0-tx-0-21swapper/221:04:372
28415060,0irq/27-eth0-tx-0-21swapper/220:55:442
28415060,0irq/27-eth0-tx-0-21swapper/220:44:552
28415060,0irq/27-eth0-tx-0-21swapper/220:43:052
28415060,0irq/27-eth0-tx-0-21swapper/020:19:190
28415060,0irq/27-eth0-tx-0-21swapper/019:20:320
28405060,0irq/26-eth0-rx-15616-21awk22:39:063
187779960,0cyclictest0-21swapper/122:13:041
187779960,0cyclictest0-21swapper/120:51:241
187769960,0cyclictest0-21swapper/022:43:280
169960,0watchdog/164262sleep122:52:371
149960,0migration/023649-21sh22:00:220
149960,0migration/014371-21sh22:57:460
8225050,0irq/24-0000:00:322-21latency_hist22:09:062
4150,0ktimersoftd/00-21swapper/023:38:580
339950,0migration/39501-21aten_r1power_fr21:54:213
339950,0migration/38746-21aten_r1power_en22:14:173
339950,0migration/38481-21munin-node19:19:213
339950,0migration/35839-21munin-node23:34:183
339950,0migration/35147-21kthreadcore21:29:243
339950,0migration/33221-21kthreadcore19:14:243
339950,0migration/331371-21kthreadcore23:49:223
339950,0migration/327969-21kthreadcore22:44:253
339950,0migration/326030-21kthreadcore23:04:243
339950,0migration/325802-21taskset18:44:183
339950,0migration/323988-21kthreadcore19:54:253
339950,0migration/320549-21latency19:49:253
339950,0migration/319640-21kthreadcore23:19:233
339950,0migration/319223-21munin-node23:39:263
339950,0migration/318306-21kthreadcore22:59:233
339950,0migration/317329-21aten2.4-expect21:59:183
339950,0migration/313320-21kthreadcore20:54:273
28415050,0irq/27-eth0-tx-6646-21munin-node18:54:180
28415050,0irq/27-eth0-tx-3663-21python23:29:302
28415050,0irq/27-eth0-tx-32280-21munin-node18:49:180
28415050,0irq/27-eth0-tx-30712-21kernelversion23:49:222
28415050,0irq/27-eth0-tx-29579-21munin-node20:24:270
28415050,0irq/27-eth0-tx-26686-21gltestperf21:24:202
28415050,0irq/27-eth0-tx-2038-21munin-node20:49:212
28415050,0irq/27-eth0-tx-0-21swapper/223:56:382
28415050,0irq/27-eth0-tx-0-21swapper/223:39:352
28415050,0irq/27-eth0-tx-0-21swapper/222:54:102
28415050,0irq/27-eth0-tx-0-21swapper/222:34:512
28415050,0irq/27-eth0-tx-0-21swapper/222:24:352
28415050,0irq/27-eth0-tx-0-21swapper/221:50:132
28415050,0irq/27-eth0-tx-0-21swapper/221:45:052
28415050,0irq/27-eth0-tx-0-21swapper/221:09:122
28415050,0irq/27-eth0-tx-0-21swapper/221:01:012
28415050,0irq/27-eth0-tx-0-21swapper/200:07:222
28415050,0irq/27-eth0-tx-0-21swapper/200:00:032
28415050,0irq/27-eth0-tx-0-21swapper/020:39:150
28415050,0irq/27-eth0-tx-0-21swapper/019:50:320
28415050,0irq/27-eth0-tx-0-21swapper/019:31:010
28415050,0irq/27-eth0-tx-0-21swapper/019:14:280
28415050,0irq/27-eth0-tx-0-21swapper/019:08:010
28405050,0irq/26-eth0-rx-31579-21ssh20:44:443
28405050,0irq/26-eth0-rx-3143-21cut21:09:213
28405050,0irq/26-eth0-rx-29771-21grep22:04:243
28405050,0irq/26-eth0-rx-29661-21kthreadcore21:24:233
28405050,0irq/26-eth0-rx-26533-21gltestperf21:44:233
28405050,0irq/26-eth0-rx-24563-21ssh20:42:443
28405050,0irq/26-eth0-rx-23557-21grep19:04:243
28405050,0irq/26-eth0-rx-22764-21forks23:24:223
28405050,0irq/26-eth0-rx-13579-21kthreadcore20:34:253
28405050,0irq/26-eth0-rx-12785-21cpu23:59:223
28405050,0irq/26-eth0-rx-0-21swapper/323:17:343
28405050,0irq/26-eth0-rx-0-21swapper/322:55:583
28405050,0irq/26-eth0-rx-0-21swapper/322:49:293
28405050,0irq/26-eth0-rx-0-21swapper/322:24:083
28405050,0irq/26-eth0-rx-0-21swapper/322:19:273
28405050,0irq/26-eth0-rx-0-21swapper/322:11:493
28405050,0irq/26-eth0-rx-0-21swapper/321:42:043
28405050,0irq/26-eth0-rx-0-21swapper/321:18:153
28405050,0irq/26-eth0-rx-0-21swapper/320:59:193
28405050,0irq/26-eth0-rx-0-21swapper/319:59:303
259950,0migration/29396-21munin-node20:29:272
259950,0migration/2716-21latency_hist19:14:072
259950,0migration/230480-21sh22:46:182
259950,0migration/216150-21latency_hist20:14:072
259950,0migration/215056-21fschecks_time23:19:192
259950,0migration/211863-21kthreadcore21:14:232
249950,0watchdog/20-21swapper/219:29:292
19150,0ktimersoftd/10-21swapper/121:46:341
187779955,0cyclictest17053-21diskmemload23:37:061
187779950,0cyclictest1397-21kthreadcore22:29:231
187779950,0cyclictest0-21swapper/122:44:361
187779950,0cyclictest0-21swapper/122:34:111
187769955,0cyclictest0-21swapper/020:53:060
187769950,0cyclictest0-21swapper/023:39:170
187769950,0cyclictest0-21swapper/021:50:490
187769950,0cyclictest0-21swapper/021:40:240
187769950,0cyclictest0-21swapper/019:59:210
179950,0migration/127612-21munin-node19:54:301
179950,0migration/127515-21kthreadcore21:04:231
179950,0migration/121580-21kthreadcore19:04:231
179950,0migration/113200-21sh23:16:211
179950,0migration/111670-21kthreadcore22:54:261
179950,0migration/11000-21kthreadcore23:29:251
169950,0watchdog/16547-21runrttasks18:43:411
169950,0watchdog/129768-21ssh23:28:331
169950,0watchdog/126995-21grep23:04:251
169950,0watchdog/116671-21fschecks_count20:14:171
169950,0watchdog/116477-21ssh20:56:451
169950,0watchdog/10-21swapper/123:43:051
169950,0watchdog/10-21swapper/122:25:051
169950,0watchdog/10-21swapper/122:19:251
169950,0watchdog/10-21swapper/122:16:571
169950,0watchdog/10-21swapper/121:42:531
169950,0watchdog/10-21swapper/121:01:491
169950,0watchdog/10-21swapper/120:34:251
169950,0watchdog/10-21swapper/120:06:371
149950,0migration/09874-21irqcore22:14:230
149950,0migration/09246-21mdstat20:29:260
149950,0migration/0700-21kthreadcore23:49:220
149950,0migration/06247-21taskset22:29:310
149950,0migration/0438-21latency_hist22:09:060
149950,0migration/028650-21sh23:25:040
149950,0migration/02857-21irqcore21:09:210
149950,0migration/027636-21kthreadcore21:44:260
149950,0migration/026990-21kthreadcore20:44:240
149950,0migration/026533-21kthreadcore23:04:250
149950,0migration/025429-21kthreadcore23:44:240
149950,0migration/02042-21fschecks_count21:29:200
149950,0migration/019891-21sh00:03:560
149950,0migration/018142-21kthreadcore18:59:240
149950,0migration/017025-21hddtemp_smartct20:14:180
149950,0migration/013290-21kthreadcore21:54:280
149950,0migration/012562-21kthreadcore21:34:240
149950,0migration/012234-21kthreadcore21:14:230
149950,0migration/011836-21irqrtprio20:34:240
149950,0migration/010215-21fschecks_time20:09:210
8225040,0irq/24-0000:00:0-21swapper/218:44:512
339940,0migration/39788-21aten2.4-expect21:34:183
339940,0migration/39259-21kthreadcore18:54:223
339940,0migration/331539-21kthreadcore20:24:243
339940,0migration/329960-21kthreadcore19:34:223
339940,0migration/327321-21fschecks_time19:09:203
339940,0migration/322919-21kthreadcore18:39:243
339940,0migration/318645-21kthreadcore22:39:233
339940,0migration/316935-21kthreadcore19:24:233
339940,0migration/310897-21kthreadcore19:44:243
28415040,0irq/27-eth0-tx-0-21swapper/223:04:202
28415040,0irq/27-eth0-tx-0-21swapper/019:26:460
28405040,0irq/26-eth0-rx-0-21swapper/323:54:273
28405040,0irq/26-eth0-rx-0-21swapper/323:09:093
28405040,0irq/26-eth0-rx-0-21swapper/322:29:233
28405040,0irq/26-eth0-rx-0-21swapper/321:50:393
28405040,0irq/26-eth0-rx-0-21swapper/321:19:523
28405040,0irq/26-eth0-rx-0-21swapper/320:04:183
28405040,0irq/26-eth0-rx-0-21swapper/300:04:273
259940,0migration/24204-21kthreadcore18:49:232
259940,0migration/222348-21kthreadcore19:04:232
259940,0migration/212133-21kthreadcore19:44:242
249940,0watchdog/219946-21kthreadcore22:19:252
249940,0watchdog/214801-21cut20:34:252
249940,0watchdog/20-21swapper/220:19:252
249940,0watchdog/20-21swapper/220:14:172
187789943,0cyclictest7483-21awk23:14:232
187789943,0cyclictest6445-21snmpd19:17:542
187789943,0cyclictest3425-21lldpd20:28:342
187789943,0cyclictest3425-21lldpd19:51:022
187789943,0cyclictest3425-21lldpd18:41:012
187789943,0cyclictest29493-21awk19:59:242
187789943,0cyclictest28641-21awk19:34:202
187789943,0cyclictest20751-21sadc19:29:072
187789943,0cyclictest14266-21awk18:59:222
187789940,0cyclictest0-21swapper/219:19:292
187779943,0cyclictest9664-21awk19:44:221
187779943,0cyclictest6445-21snmpd19:26:391
187779943,0cyclictest6445-21snmpd19:01:091
187779943,0cyclictest6445-21snmpd18:53:091
187779943,0cyclictest3425-21lldpd19:23:021
187779943,0cyclictest3425-21lldpd19:15:021
187779943,0cyclictest3425-21lldpd19:14:021
187779943,0cyclictest3110-21irqbalance00:04:301
187779943,0cyclictest30517-21awk20:24:231
187779943,0cyclictest26556-21awk18:44:211
187779943,0cyclictest18090-21kworker/1:020:41:121
187779943,0cyclictest16299-21awk19:49:221
187779943,0cyclictest1204-21kworker/1:121:36:361
187779943,0cyclictest10498-21awk20:09:231
187769944,0cyclictest0-21swapper/023:20:220
187769944,0cyclictest0-21swapper/021:24:480
187769943,0cyclictest6445-21snmpd22:28:400
187769943,0cyclictest22821-21awk19:54:230
187769943,0cyclictest1605-21awk22:09:240
179940,0migration/1903-21sendmail_mailqu19:34:261
179940,0migration/131998-21irqcore23:09:221
179940,0migration/128428-21kthreadcore21:24:231
179940,0migration/126165-21kthreadcore20:19:251
179940,0migration/12578-21fschecks_time19:39:241
179940,0migration/121589-21fschecks_time19:29:201
179940,0migration/119813-21missed_timers23:19:241
179940,0migration/116793-21kthreadcore22:59:231
169940,0watchdog/130474-21sed22:04:251
169940,0watchdog/128797-21ssh23:48:051
169940,0watchdog/123013-21python21:59:291
169940,0watchdog/10-21swapper/123:55:051
169940,0watchdog/10-21swapper/123:49:371
169940,0watchdog/10-21swapper/121:54:211
169940,0watchdog/10-21swapper/121:49:131
169940,0watchdog/10-21swapper/121:20:011
169940,0watchdog/10-21swapper/121:14:211
169940,0watchdog/10-21swapper/121:12:211
169940,0watchdog/10-21swapper/120:48:531
169940,0watchdog/10-21swapper/120:29:211
149940,0migration/031581-21fschecks_time23:09:200
149940,0migration/019508-21kthreadcore22:19:250
149940,0migration/017468-21kthreadcore22:59:230
149940,0migration/014307-21taskset22:35:080
149940,0migration/013617-21kthreadcore20:54:270
149940,0migration/013125-21sh23:16:070
339930,0migration/35139-21kthreadcore19:39:273
28415030,0irq/27-eth0-tx-0-21swapper/019:09:180
28405030,0irq/26-eth0-rx-0-21swapper/320:19:343
28405030,0irq/26-eth0-rx-0-21swapper/320:14:153
28405030,0irq/26-eth0-rx-0-21swapper/320:09:183
28405030,0irq/26-eth0-rx-0-21swapper/319:29:203
28405030,0irq/26-eth0-rx-0-21swapper/319:01:203
187789932,0cyclictest7583-21awk18:54:212
187789931,0cyclictest271ktimersoftd/219:54:252
187789930,0cyclictest2829-21latency_hist20:04:072
187789930,0cyclictest2335-21diskstats19:39:232
187779933,0cyclictest5699-21kworker/1:023:59:101
187779933,0cyclictest28154-21kworker/1:118:54:101
187779933,0cyclictest10275-21kworker/1:021:29:081
187769932,0cyclictest8616-21sensors20:04:270
187769932,0cyclictest32576-21awk22:49:240
187769932,0cyclictest31133-21awk23:29:230
187769932,0cyclictest30999-21ssh22:47:570
187769932,0cyclictest28479-21hwlatdetect19:34:190
187769932,0cyclictest2806-21awk19:39:250
187769932,0cyclictest18624-21awk20:59:220
187769932,0cyclictest18281-21df_abs21:19:190
187769930,0cyclictest41ktimersoftd/023:54:230
187769930,0cyclictest19629-21perl18:39:210
187769930,0cyclictest10992sleep021:08:160
187769930,0cyclictest0-21swapper/019:44:220
187769930,0cyclictest0-21swapper/018:44:190
169930,0watchdog/10-21swapper/119:59:091
28405020,0irq/26-eth0-rx-0-21swapper/320:29:073
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional