You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 13:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Fri Dec 05, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
291350530,0irq/25-eth10-21swapper/306:37:223
2815923425,0sleep20-21swapper/206:36:442
2807223228,0sleep00-21swapper/006:35:420
2807323122,0sleep10-21swapper/106:35:431
28433991212,0cyclictest16356-21sensors09:09:292
35160,0ktimersoftd/30-21swapper/310:57:373
339960,0migration/3819-21sh11:48:543
339960,0migration/37614-21munin-node11:29:233
339960,0migration/332308-21sh12:07:263
339960,0migration/319178-21sh11:19:103
339960,0migration/317422-21sh09:34:173
339960,0migration/317170-21dump-pmu-power09:34:083
339960,0migration/314175-21munin-node09:49:283
284319960,0cyclictest0-21swapper/009:53:160
28415060,0irq/27-eth0-tx-23336-21ssh09:57:442
28415060,0irq/27-eth0-tx-0-21swapper/212:05:202
28415060,0irq/27-eth0-tx-0-21swapper/211:59:092
28415060,0irq/27-eth0-tx-0-21swapper/211:23:142
28415060,0irq/27-eth0-tx-0-21swapper/210:56:432
28415060,0irq/27-eth0-tx-0-21swapper/210:45:442
28415060,0irq/27-eth0-tx-0-21swapper/210:37:162
28415060,0irq/27-eth0-tx-0-21swapper/210:30:032
28415060,0irq/27-eth0-tx-0-21swapper/210:23:432
28415060,0irq/27-eth0-tx-0-21swapper/210:02:072
28415060,0irq/27-eth0-tx-0-21swapper/209:40:382
28415060,0irq/27-eth0-tx-0-21swapper/209:37:112
28415060,0irq/27-eth0-tx-0-21swapper/209:30:412
28415060,0irq/27-eth0-tx-0-21swapper/209:26:042
28415060,0irq/27-eth0-tx-0-21swapper/209:21:292
28415060,0irq/27-eth0-tx-0-21swapper/209:06:502
28415060,0irq/27-eth0-tx-0-21swapper/209:00:352
28415060,0irq/27-eth0-tx-0-21swapper/208:46:072
28415060,0irq/27-eth0-tx-0-21swapper/008:33:400
28405060,0irq/26-eth0-rx-25380-21munin-run11:44:073
28405060,0irq/26-eth0-rx-0-21swapper/310:20:493
149960,0migration/024433-21munin-node09:59:230
149960,0migration/010555-21munin-node11:54:250
8225050,0irq/24-0000:00:0-21swapper/207:29:082
339950,0migration/3843-21sh09:22:343
339950,0migration/38053-21kthreadcore10:49:253
339950,0migration/35326-21kthreadcore08:44:253
339950,0migration/34603-21kthreadcore09:24:263
339950,0migration/33540-21kthreadcore11:49:243
339950,0migration/332115-21kthreadcore11:24:253
339950,0migration/331374-21kthreadcore08:39:223
339950,0migration/330987-21kthreadcore08:59:263
339950,0migration/330860-21aten_r1power_vo10:24:283
339950,0migration/328223-21aten2.4-expect07:39:183
339950,0migration/326184-21taskset10:59:453
339950,0migration/325797-21fschecks_time09:39:213
339950,0migration/318577-21taskset08:15:163
339950,0migration/317447-21kthreadcore07:09:243
339950,0migration/317413-21sh11:38:463
339950,0migration/316938-21switchtime08:49:283
339950,0migration/315708-21kthreadcore10:34:253
339950,0migration/315504-21sh11:54:493
339950,0migration/314836-21taskset10:13:453
339950,0migration/313021-21kthreadcore09:09:253
339950,0migration/31067-21kthreadcore10:04:253
284339955,0cyclictest0-21swapper/209:52:502
284339950,0cyclictest0-21swapper/211:34:322
284339950,0cyclictest0-21swapper/209:44:292
284329953,0cyclictest1534-21awk09:44:221
284319955,0cyclictest0-21swapper/011:30:420
284319955,0cyclictest0-21swapper/011:13:030
284319955,0cyclictest0-21swapper/010:27:480
284319950,0cyclictest0-21swapper/009:29:470
284319950,0cyclictest0-21swapper/008:48:480
28415050,0irq/27-eth0-tx-6096-21ssh10:06:142
28415050,0irq/27-eth0-tx-5680-21ssh10:28:502
28415050,0irq/27-eth0-tx-26066-21ssh11:44:212
28415050,0irq/27-eth0-tx-24123-21kthreadcore10:39:272
28415050,0irq/27-eth0-tx-2005-21ssh11:05:572
28415050,0irq/27-eth0-tx-14023-21ssh10:11:162
28415050,0irq/27-eth0-tx-11560-21ssh08:49:192
28415050,0irq/27-eth0-tx-0-21swapper/211:49:592
28415050,0irq/27-eth0-tx-0-21swapper/211:25:332
28415050,0irq/27-eth0-tx-0-21swapper/211:11:382
28415050,0irq/27-eth0-tx-0-21swapper/210:52:012
28415050,0irq/27-eth0-tx-0-21swapper/210:16:162
28415050,0irq/27-eth0-tx-0-21swapper/209:16:342
28415050,0irq/27-eth0-tx-0-21swapper/209:16:342
28415050,0irq/27-eth0-tx-0-21swapper/208:57:032
28415050,0irq/27-eth0-tx-0-21swapper/208:42:352
28415050,0irq/27-eth0-tx-0-21swapper/007:56:220
28415050,0irq/27-eth0-tx-0-21swapper/007:02:040
28405050,0irq/26-eth0-rx-3716-21df_abs11:09:193
28405050,0irq/26-eth0-rx-24187-21aten_r1power_vo09:59:233
28405050,0irq/26-eth0-rx-22852-21grep10:39:273
28405050,0irq/26-eth0-rx-22164-21kthreadcore11:59:263
28405050,0irq/26-eth0-rx-0-21swapper/311:17:333
28405050,0irq/26-eth0-rx-0-21swapper/310:29:593
28405050,0irq/26-eth0-rx-0-21swapper/309:57:313
28405050,0irq/26-eth0-rx-0-21swapper/309:15:263
28405050,0irq/26-eth0-rx-0-21swapper/309:15:263
28405050,0irq/26-eth0-rx-0-21swapper/309:08:333
28405050,0irq/26-eth0-rx-0-21swapper/308:59:003
259950,0migration/24297-21aten_r1power_en06:54:182
259950,0migration/225983-21unixbench-2d10:59:322
259950,0migration/215942-21sh11:56:082
259950,0migration/212178-21latency_hist07:09:082
249950,0watchdog/221646-21kthreadcore08:24:222
249950,0watchdog/20-21swapper/211:30:142
179950,0migration/17207-21kthreadcore07:54:241
179950,0migration/128079-21irqcore11:24:241
179950,0migration/118458-21fschecks_time11:39:221
179950,0migration/113743-21kthreadcore09:49:251
169950,0watchdog/131308-21cut10:24:301
169950,0watchdog/127560-21diskmemload11:53:301
169950,0watchdog/10-21swapper/108:49:221
169950,0watchdog/10-21swapper/108:29:581
169950,0watchdog/10-21swapper/107:39:261
169950,0watchdog/10-21swapper/106:52:381
149950,0migration/032434-21sh09:43:180
149950,0migration/027304-21kthreadcore10:19:250
149950,0migration/026268-21latency_hist08:29:080
149950,0migration/02380-21sh08:42:130
149950,0migration/017514-21munin-node09:54:250
149950,0migration/01536-21switchtime07:39:310
149950,0migration/012973-21kthreadcore09:09:250
149950,0migration/01158-21sh09:23:290
339940,0migration/38772-21kthreadcore07:54:243
339940,0migration/319204-21runrttasks08:19:583
339940,0migration/317786-21unixbench_multi08:09:353
284339944,0cyclictest0-21swapper/211:14:592
284339943,0cyclictest6445-21snmpd08:18:532
284339943,0cyclictest6445-21snmpd08:06:532
284339943,0cyclictest6445-21snmpd07:46:082
284339943,0cyclictest6445-21snmpd07:37:082
284339943,0cyclictest6445-21snmpd07:32:532
284339943,0cyclictest6445-21snmpd07:17:382
284339943,0cyclictest6445-21snmpd06:45:072
284339943,0cyclictest3425-21lldpd08:19:202
284339943,0cyclictest27556-21sadc08:39:082
284329944,0cyclictest0-21swapper/111:04:251
284329943,0cyclictest6445-21snmpd10:04:381
284329943,0cyclictest6445-21snmpd09:07:381
284329943,0cyclictest6445-21snmpd08:36:381
284329943,0cyclictest6445-21snmpd08:16:081
284329943,0cyclictest6445-21snmpd08:05:381
284329943,0cyclictest6445-21snmpd08:00:381
284329943,0cyclictest6445-21snmpd07:50:381
284329943,0cyclictest6445-21snmpd07:36:381
284329943,0cyclictest6445-21snmpd07:16:381
284329943,0cyclictest6445-21snmpd07:03:381
284329943,0cyclictest6445-21snmpd06:45:371
284329943,0cyclictest26170-21awk12:04:201
284329943,0cyclictest2547-21awk09:24:231
284329943,0cyclictest21900-21kworker/1:111:19:251
284329943,0cyclictest21900-21kworker/1:110:41:571
284329943,0cyclictest20072-21awk08:54:241
284329943,0cyclictest18211-21awk11:59:241
284329943,0cyclictest14695-21awk10:34:231
284329943,0cyclictest12371-21irqcore11:14:251
284329943,0cyclictest10559-21awk09:29:261
284319944,0cyclictest0-21swapper/010:53:390
284319944,0cyclictest0-21swapper/010:44:210
284319944,0cyclictest0-21swapper/010:41:220
284319944,0cyclictest0-21swapper/009:37:210
284319944,0cyclictest0-21swapper/008:57:470
284319944,0cyclictest0-21swapper/008:19:090
284319943,0cyclictest9296-21sadc09:29:080
284319943,0cyclictest8148-21awk10:09:200
284319943,0cyclictest3425-21lldpd10:30:310
284319943,0cyclictest3110-21irqbalance11:00:020
284319943,0cyclictest2099-21awk11:49:220
284319943,0cyclictest20119-21awk11:19:240
284319943,0cyclictest14671-21awk10:34:230
284319943,0cyclictest12666-21awk08:09:270
284319943,0cyclictest12518-21irqrtprio11:14:250
284319943,0cyclictest12034-21awk08:49:210
284319940,0cyclictest0-21swapper/007:16:430
284319940,0cyclictest0-21swapper/006:48:390
28415040,0irq/27-eth0-tx-0-21swapper/211:39:212
28415040,0irq/27-eth0-tx-0-21swapper/008:02:070
28415040,0irq/27-eth0-tx-0-21swapper/007:22:490
28405040,0irq/26-eth0-rx-27833-21aten_r1power_cu11:04:163
28405040,0irq/26-eth0-rx-0-21swapper/310:46:233
28405040,0irq/26-eth0-rx-0-21swapper/310:16:043
28405040,0irq/26-eth0-rx-0-21swapper/309:44:503
259940,0migration/21469-21latency06:39:252
249940,0watchdog/2676-21kthreadcore07:39:262
249940,0watchdog/212512-21fschecks_time08:09:262
249940,0watchdog/20-21swapper/207:56:342
19140,0ktimersoftd/122489-21ssh09:55:001
179940,0migration/19866-21latency_hist11:34:071
179940,0migration/16915-21idleruntime10:29:231
179940,0migration/14602-21kernelversion08:44:251
179940,0migration/131440-21sh11:44:311
179940,0migration/129976-21kthreadcore08:39:221
179940,0migration/129325-21kthreadcore09:39:251
179940,0migration/127657-21gltestperf08:59:231
179940,0migration/119693-21latency_hist08:24:081
179940,0migration/115977-21hwlatdetect10:14:231
179940,0migration/111824-21aten_r1power_cu08:09:181
179940,0migration/110411-21timerandwakeup10:49:301
169940,0watchdog/14202-21grep11:29:221
169940,0watchdog/132247-21kthreadcore10:44:261
169940,0watchdog/120658-21perl10:59:221
169940,0watchdog/120208-21munin-node07:24:261
169940,0watchdog/120196-21df_inode08:24:181
169940,0watchdog/10-21swapper/111:54:261
169940,0watchdog/10-21swapper/111:12:021
169940,0watchdog/10-21swapper/110:19:261
169940,0watchdog/10-21swapper/110:09:261
169940,0watchdog/10-21swapper/110:00:301
169940,0watchdog/10-21swapper/109:34:261
169940,0watchdog/10-21swapper/109:23:461
169940,0watchdog/10-21swapper/106:43:181
149940,0migration/032132-21kthreadcore11:04:230
149940,0migration/031445-21sh12:04:470
149940,0migration/023662-21kthreadcore09:14:260
149940,0migration/023662-21kthreadcore09:14:260
149940,0migration/023026-21kthreadcore11:39:270
149940,0migration/02298-21kthreadcore10:04:250
149940,0migration/016497-21kthreadcore10:54:240
149940,0migration/016277-21irqcore10:14:250
149940,0migration/012070-21latency_hist07:09:080
339930,0migration/326640-21ksmtuned08:31:133
284339933,0cyclictest3425-21lldpd07:06:032
284339932,0cyclictest6445-21snmpd08:01:232
284339932,0cyclictest6445-21snmpd07:52:082
284339932,0cyclictest6445-21snmpd07:22:532
284339932,0cyclictest6445-21snmpd07:02:532
284339932,0cyclictest6445-21snmpd06:49:372
284339932,0cyclictest26227-21latency_hist08:29:082
284329933,0cyclictest21900-21kworker/1:109:09:171
284329933,0cyclictest21900-21kworker/1:107:44:131
284329930,0cyclictest12066-21latency_hist10:54:071
284329930,0cyclictest0-21swapper/109:14:241
284329930,0cyclictest0-21swapper/109:14:241
284329930,0cyclictest0-21swapper/107:04:371
284319933,0cyclictest3425-21lldpd07:36:340
284319933,0cyclictest3425-21lldpd06:53:330
284319933,0cyclictest0-21swapper/007:51:150
284319932,0cyclictest6445-21snmpd12:01:080
284319932,0cyclictest3425-21lldpd08:05:370
284319932,0cyclictest27758-21awk08:59:230
284319932,0cyclictest26538-21awk11:44:230
284319932,0cyclictest25863-21sensors07:24:270
284319932,0cyclictest1870-21sensors06:39:280
284319932,0cyclictest11802-21cut11:34:260
284319932,0cyclictest10307-21munin-run09:09:080
284319931,0cyclictest0-21swapper/008:35:360
284319930,0cyclictest852-21df09:44:190
284319930,0cyclictest4641-21diskstats06:54:200
284319930,0cyclictest27614-21perl11:24:210
28415030,0irq/27-eth0-tx-0-21swapper/008:14:360
28415030,0irq/27-eth0-tx-0-21swapper/007:30:380
28415030,0irq/27-eth0-tx-0-21swapper/007:09:180
28405030,0irq/26-eth0-rx-0-21swapper/308:24:273
28405030,0irq/26-eth0-rx-0-21swapper/307:51:213
28405030,0irq/26-eth0-rx-0-21swapper/307:46:203
28405030,0irq/26-eth0-rx-0-21swapper/307:38:303
28405030,0irq/26-eth0-rx-0-21swapper/307:29:533
28405030,0irq/26-eth0-rx-0-21swapper/307:26:133
28405030,0irq/26-eth0-rx-0-21swapper/307:14:503
28405030,0irq/26-eth0-rx-0-21swapper/307:00:373
28405030,0irq/26-eth0-rx-0-21swapper/306:56:283
28405030,0irq/26-eth0-rx-0-21swapper/306:49:283
28405030,0irq/26-eth0-rx-0-21swapper/306:45:003
28405030,0irq/26-eth0-rx-0-21swapper/306:42:483
179930,0migration/119476-21latency_hist07:19:081
169930,0watchdog/10-21swapper/107:29:101
169930,0watchdog/10-21swapper/107:09:141
169930,0watchdog/10-21swapper/106:54:301
284319920,0cyclictest2418-21latency_hist07:44:080
28405020,0irq/26-eth0-rx-0-21swapper/308:34:213
28405020,0irq/26-eth0-rx-0-21swapper/308:04:133
28405020,0irq/26-eth0-rx-0-21swapper/308:01:343
28405020,0irq/26-eth0-rx-0-21swapper/307:19:173
28405020,0irq/26-eth0-rx-0-21swapper/307:04:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional