You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-01 - 05:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Wed Jul 01, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2404824735,0sleep10-21swapper/118:47:241
304450460,0irq/25-eth10-21swapper/318:49:343
2406223420,0sleep00-21swapper/018:47:350
2405923324,0sleep20-21swapper/218:47:332
339960,0migration/35903-21taskset00:03:333
339960,0migration/35329-21sh21:16:393
339960,0migration/330149-21aten_r1power_cu23:16:193
339960,0migration/329288-21sh22:33:053
29715060,0irq/26-eth0-rx-15646-21iostat22:26:243
27160,0ktimersoftd/20-21swapper/221:21:242
244429966,0cyclictest22537-21diskmemload21:33:332
244429960,0cyclictest0-21swapper/222:55:312
244429960,0cyclictest0-21swapper/200:20:212
244419960,0cyclictest0-21swapper/122:34:251
149960,0migration/014545-21sh21:45:240
149960,0migration/013270-21sh21:02:460
4150,0ktimersoftd/00-21swapper/022:17:360
339950,0migration/39593-21kthreadcore21:21:243
339950,0migration/38175-21fschecks_time19:51:223
339950,0migration/330447-21fschecks_time23:36:183
339950,0migration/326268-21ntp_offset20:01:293
339950,0migration/325731-21kthreadcore23:56:243
339950,0migration/324924-21munin-node19:11:313
339950,0migration/320418-21kthreadcore00:11:253
339950,0migration/319756-21mdstat22:46:253
339950,0migration/318985-21kthreadcore21:26:263
339950,0migration/314790-21taskset00:10:553
339950,0migration/312907-21taskset23:03:343
339950,0migration/312357-21munin-node22:21:283
29725050,0irq/27-eth0-tx-7510-21munin-node19:51:210
29725050,0irq/27-eth0-tx-5938-21ssh22:41:002
29725050,0irq/27-eth0-tx-30046-21ssh20:55:052
29725050,0irq/27-eth0-tx-26525-21munin-node19:41:290
29725050,0irq/27-eth0-tx-21300-21ssh21:46:452
29725050,0irq/27-eth0-tx-21049-21vmstat22:06:322
29725050,0irq/27-eth0-tx-19547-21sendmail_mailqu23:26:292
29725050,0irq/27-eth0-tx-15477-21munin-node23:51:142
29725050,0irq/27-eth0-tx-15333-21munin-node21:26:212
29725050,0irq/27-eth0-tx-14098-21perf22:46:072
29725050,0irq/27-eth0-tx-0-21swapper/019:08:070
29715050,0irq/26-eth0-rx-8102-21kthreadcore21:01:243
29715050,0irq/26-eth0-rx-30674-21cut19:16:253
29715050,0irq/26-eth0-rx-30595-21cpuspeed_turbos22:56:203
29715050,0irq/26-eth0-rx-27888-21nfs_client22:51:253
29715050,0irq/26-eth0-rx-2402-21kthreadcore18:56:253
29715050,0irq/26-eth0-rx-22940-21aten_r1power_cu22:11:183
29715050,0irq/26-eth0-rx-20491-21fschecks_time19:36:213
29715050,0irq/26-eth0-rx-16938-21kthreadcore23:26:263
29715050,0irq/26-eth0-rx-15392-21kthreadcore23:06:233
29715050,0irq/26-eth0-rx-0-21swapper/323:54:183
29715050,0irq/26-eth0-rx-0-21swapper/323:33:043
29715050,0irq/26-eth0-rx-0-21swapper/323:11:243
29715050,0irq/26-eth0-rx-0-21swapper/322:41:103
29715050,0irq/26-eth0-rx-0-21swapper/322:39:193
29715050,0irq/26-eth0-rx-0-21swapper/322:06:253
29715050,0irq/26-eth0-rx-0-21swapper/322:00:073
29715050,0irq/26-eth0-rx-0-21swapper/321:32:523
29715050,0irq/26-eth0-rx-0-21swapper/321:11:313
29715050,0irq/26-eth0-rx-0-21swapper/321:09:343
29715050,0irq/26-eth0-rx-0-21swapper/320:56:213
259950,0migration/229668-21sh23:56:422
259950,0migration/222850-21irqcore23:31:232
259950,0migration/219980-21aten2.4-expect19:36:192
259950,0migration/211727-21munin-node23:01:272
244429955,0cyclictest0-21swapper/223:42:492
244429955,0cyclictest0-21swapper/223:20:452
244429955,0cyclictest0-21swapper/222:28:432
244429955,0cyclictest0-21swapper/221:01:132
244429955,0cyclictest0-21swapper/200:02:032
244429950,0cyclictest0-21swapper/219:16:322
244419955,0cyclictest0-21swapper/122:46:331
244419950,0cyclictest0-21swapper/123:27:521
244419950,0cyclictest0-21swapper/122:12:301
244409955,0cyclictest0-21swapper/022:22:580
244409950,0cyclictest0-21swapper/023:47:180
244409950,0cyclictest0-21swapper/022:34:280
244409950,0cyclictest0-21swapper/021:35:020
249950,0watchdog/222537-21diskmemload23:11:252
19150,0ktimersoftd/10-21swapper/123:23:361
179950,0migration/19284-21idleruntime20:16:221
179950,0migration/16949-21latency_hist00:06:071
179950,0migration/131932-21irqrtprio20:56:241
179950,0migration/124512-21kthreadcore19:36:251
179950,0migration/122760-21sh23:54:531
179950,0migration/118610-21needreboot19:31:271
179950,0migration/115368-21munin-node22:26:221
179950,0migration/11285-21sh21:36:251
169950,0watchdog/110976-21kthreadcore21:41:261
149950,0migration/0982-21kthreadcore18:56:240
149950,0migration/08958-21taskset20:38:080
149950,0migration/08435-21kthreadcore23:01:240
149950,0migration/08046-21kthreadcore23:21:230
149950,0migration/07384-21aten_r1power_po21:21:180
149950,0migration/05878-21kthreadcore19:46:250
149950,0migration/05680-21aten_r1power_fr19:01:200
149950,0migration/0557-21idleruntime19:21:220
149950,0migration/05500-21sh21:37:380
149950,0migration/031543-21kthreadcore20:06:250
149950,0migration/030815-21fschecks_time23:16:210
149950,0migration/030696-21taskset00:20:410
149950,0migration/030169-21taskset23:58:100
149950,0migration/028036-21kthreadcore19:16:240
149950,0migration/027695-21kthreadcore21:11:260
149950,0migration/027574-21kthreadcore22:51:240
149950,0migration/027092-21kthreadcore22:11:240
149950,0migration/026423-21kthreadcore21:51:250
149950,0migration/025403-21kthreadcore20:01:270
149950,0migration/022593-21sh21:10:220
149950,0migration/022295-21taskset00:14:330
149950,0migration/021936-21timerwakeupswit20:46:320
149950,0migration/021530-21sh23:10:260
149950,0migration/018665-21kthreadcore23:51:190
149950,0migration/018627-21kthreadcore22:06:250
149950,0migration/017101-21kthreadcore19:56:240
149950,0migration/016993-21kthreadcore21:46:240
149950,0migration/015458-21kthreadcore19:31:250
149950,0migration/015263-21aten_r1power_po20:21:200
149950,0migration/012937-21taskset22:42:500
149950,0migration/010435-21kthreadcore19:26:250
8215040,0irq/24-0000:00:8439-21cat20:16:072
8215040,0irq/24-0000:00:26286-21crond19:41:072
339940,0migration/39431-21kthreadcore19:01:263
339940,0migration/39062-21kthreadcore23:41:243
339940,0migration/38896-21kthreadcore19:26:243
339940,0migration/38268-21irqrtprio21:41:243
339940,0migration/36379-21kthreadcore20:11:253
339940,0migration/331601-21kthreadcore19:41:253
339940,0migration/329514-21diskmemload00:17:123
339940,0migration/327822-21kthreadcore20:51:263
339940,0migration/327409-21kthreadcore18:51:263
339940,0migration/324201-21irqcore21:51:233
339940,0migration/318520-21kthreadcore21:46:253
339940,0migration/312768-21kthreadcore20:41:243
339940,0migration/31046-21irqrtprio19:21:243
29725040,0irq/27-eth0-tx-4963-21ssh22:17:222
29725040,0irq/27-eth0-tx-31344-21ssh21:36:162
29725040,0irq/27-eth0-tx-30101-21grep23:36:172
29725040,0irq/27-eth0-tx-15147-21munin-node00:11:222
29725040,0irq/27-eth0-tx-0-21swapper/223:47:002
29725040,0irq/27-eth0-tx-0-21swapper/223:21:072
29725040,0irq/27-eth0-tx-0-21swapper/223:06:092
29725040,0irq/27-eth0-tx-0-21swapper/222:56:242
29725040,0irq/27-eth0-tx-0-21swapper/222:46:102
29725040,0irq/27-eth0-tx-0-21swapper/222:24:532
29725040,0irq/27-eth0-tx-0-21swapper/222:11:382
29725040,0irq/27-eth0-tx-0-21swapper/222:01:092
29725040,0irq/27-eth0-tx-0-21swapper/221:56:172
29725040,0irq/27-eth0-tx-0-21swapper/221:41:202
29725040,0irq/27-eth0-tx-0-21swapper/221:06:272
29725040,0irq/27-eth0-tx-0-21swapper/200:06:082
29725040,0irq/27-eth0-tx-0-21swapper/020:51:510
29715040,0irq/26-eth0-rx-0-21swapper/323:46:403
29715040,0irq/26-eth0-rx-0-21swapper/323:21:123
29715040,0irq/26-eth0-rx-0-21swapper/322:16:203
29715040,0irq/26-eth0-rx-0-21swapper/322:01:073
29715040,0irq/26-eth0-rx-0-21swapper/321:36:263
29715040,0irq/26-eth0-rx-0-21swapper/319:56:073
259940,0migration/2738-21aten_r1power_cu19:46:192
259940,0migration/27266-21kthreadcore20:36:252
259940,0migration/231395-21latency_hist21:16:072
259940,0migration/22508-21kthreadcore18:56:252
259940,0migration/223637-21fschecks_count21:11:212
259940,0migration/220161-21kthreadcore20:46:262
259940,0migration/217961-21kthreadcore19:56:242
259940,0migration/213693-21kthreadcore20:41:242
244429943,0cyclictest632-21awk19:21:232
244429943,0cyclictest3556-21lldpd20:36:012
244429943,0cyclictest3556-21lldpd19:54:592
244429943,0cyclictest3556-21lldpd19:13:272
244429943,0cyclictest3514-21snmpd20:59:452
244429943,0cyclictest3514-21snmpd19:33:152
244429943,0cyclictest28430-21awk20:06:222
244429943,0cyclictest25824-21seq22:31:262
244429943,0cyclictest24028-21awk21:51:222
244419943,0cyclictest7440-21awk21:01:221
244419943,0cyclictest3556-21lldpd23:39:301
244419943,0cyclictest3514-21snmpd20:22:001
244419943,0cyclictest3514-21snmpd19:24:301
244419943,0cyclictest3241-21irqbalance22:22:361
244419943,0cyclictest3241-21irqbalance21:47:261
244419943,0cyclictest31271-21awk22:56:231
244419943,0cyclictest29274-21awk20:31:221
244419943,0cyclictest2707-21awk20:11:231
244419943,0cyclictest24052-21awk21:51:221
244419943,0cyclictest1631-21awk19:46:221
244419943,0cyclictest15812-21awk22:06:231
244419943,0cyclictest15144-21awk19:56:221
244419940,0cyclictest0-21swapper/123:42:121
244409944,0cyclictest0-21swapper/022:30:130
244409944,0cyclictest0-21swapper/021:59:190
244409944,0cyclictest0-21swapper/000:01:200
244409943,0cyclictest25227-21ps23:11:250
244409943,0cyclictest23893-21kworker/0:022:03:490
244409943,0cyclictest21527-21sadc23:31:070
249940,0watchdog/221216-21diskstats20:01:222
249940,0watchdog/215813-21fschecks_time20:21:222
249940,0watchdog/212291-21switchtime19:26:302
249940,0watchdog/20-21swapper/218:51:262
179940,0migration/17998-21kthreadcore19:01:251
179940,0migration/15751-21kthreadcore20:36:251
179940,0migration/14277-21munin-node22:16:291
179940,0migration/132572-21runrttasks19:41:581
179940,0migration/132121-21taskset18:56:221
179940,0migration/1310-21kthreadcore21:56:221
179940,0migration/130708-21kthreadcore20:06:241
179940,0migration/127090-21kthreadcore21:11:251
179940,0migration/124962-21kthreadcore22:51:231
179940,0migration/122337-21cpuspeed_turbos23:11:201
179940,0migration/122052-21runrttasks21:09:051
179940,0migration/117045-21kthreadcore19:06:251
179940,0migration/115648-21fschecks_count00:11:201
179940,0migration/110591-21kthreadcore22:01:261
179940,0migration/110162-21kthreadcore00:06:251
169940,0watchdog/130309-21ssh00:19:331
169940,0watchdog/129222-21kthreadcore18:51:261
169940,0watchdog/128079-21ssh23:31:491
169940,0watchdog/127117-21kthreadcore20:51:261
169940,0watchdog/126445-21kthreadcore21:31:261
169940,0watchdog/121646-21ssh21:27:061
169940,0watchdog/117432-21kthreadcore20:46:261
169940,0watchdog/113632-21ssh21:22:061
169940,0watchdog/112312-21cat19:26:301
169940,0watchdog/10-21swapper/123:56:211
169940,0watchdog/10-21swapper/120:41:301
169940,0watchdog/10-21swapper/119:51:261
149940,0migration/031746-21cpuspeed_turbos21:16:180
149940,0migration/031253-21fschecks_time22:36:200
149940,0migration/019412-21kthreadcore21:26:260
29725030,0irq/27-eth0-tx-20008-21aten2.4-expect19:36:200
29725030,0irq/27-eth0-tx-0-21swapper/020:16:190
29715030,0irq/26-eth0-rx-2771-21aten_r1power_en20:36:193
29715030,0irq/26-eth0-rx-0-21swapper/320:46:423
29715030,0irq/26-eth0-rx-0-21swapper/320:33:373
29715030,0irq/26-eth0-rx-0-21swapper/320:26:193
29715030,0irq/26-eth0-rx-0-21swapper/320:16:193
29715030,0irq/26-eth0-rx-0-21swapper/320:06:293
29715030,0irq/26-eth0-rx-0-21swapper/319:48:173
29715030,0irq/26-eth0-rx-0-21swapper/319:31:193
29715030,0irq/26-eth0-rx-0-21swapper/319:06:243
259930,0migration/21676-21latency_hist20:11:072
244429932,0cyclictest5683-21basename19:01:202
244429932,0cyclictest22684-21awk20:26:222
244429930,0cyclictest12952-21perl19:06:202
244419933,0cyclictest32407-21awk21:16:211
244419933,0cyclictest0-21swapper/123:46:471
244419932,0cyclictest3514-21snmpd19:15:301
244419932,0cyclictest31308-21fschecks_time22:36:201
244419932,0cyclictest31075-21awk23:16:231
244419932,0cyclictest27959-21kthreadcore19:16:241
244419932,0cyclictest14012-21sh23:06:091
244409933,0cyclictest32003-21kworker/0:120:56:210
244409933,0cyclictest32003-21kworker/0:120:31:250
244409933,0cyclictest32003-21kworker/0:120:11:210
244409933,0cyclictest23893-21kworker/0:023:41:100
244409933,0cyclictest23893-21kworker/0:023:36:160
244409933,0cyclictest23893-21kworker/0:023:31:240
244409933,0cyclictest23893-21kworker/0:000:06:160
244409933,0cyclictest19885-21kworker/0:219:11:250
244409932,0cyclictest734-21kthreadcore22:56:250
244409932,0cyclictest22660-21awk20:26:220
244409932,0cyclictest15214-21awk22:46:210
244409930,0cyclictest0-21swapper/020:42:410
179930,0migration/16010-21sh23:01:101
169930,0watchdog/10-21swapper/122:41:211
169930,0watchdog/10-21swapper/120:01:261
339920,0migration/315548-21diskstats20:21:213
244409922,0cyclictest19885-21kworker/0:218:51:090
169920,0watchdog/10-21swapper/120:26:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional