You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 18:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Wed Nov 26, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3166827170,0sleep00-21swapper/006:35:200
3192525142,0sleep30-21swapper/306:38:193
3175124738,0sleep10-21swapper/106:36:171
3163223826,0sleep20-21swapper/206:34:582
8225060,0irq/24-0000:00:6460-21latency_hist09:09:272
339960,0migration/39492-21sh10:10:463
339960,0migration/329852-21sh09:24:363
339960,0migration/32946-21sh09:47:423
339960,0migration/326884-21sh10:03:263
339960,0migration/321009-21sendmail_mailst11:39:483
339960,0migration/320838-21sh12:00:083
339960,0migration/315855-21sh10:56:553
321109960,0cyclictest0-21swapper/209:26:172
321049966,0cyclictest0-21swapper/109:38:011
321049960,0cyclictest0-21swapper/107:39:471
28415060,0irq/27-eth0-tx-30909-21ssh08:40:122
28415060,0irq/27-eth0-tx-28387-21munin-node09:44:492
28415060,0irq/27-eth0-tx-25835-21ssh10:22:462
28415060,0irq/27-eth0-tx-19588-21munin-node09:59:482
28415060,0irq/27-eth0-tx-14293-21munin-node07:14:440
28415060,0irq/27-eth0-tx-0-21swapper/212:00:092
28415060,0irq/27-eth0-tx-0-21swapper/211:57:472
28415060,0irq/27-eth0-tx-0-21swapper/211:49:372
28415060,0irq/27-eth0-tx-0-21swapper/211:44:382
28415060,0irq/27-eth0-tx-0-21swapper/211:27:282
28415060,0irq/27-eth0-tx-0-21swapper/211:10:042
28415060,0irq/27-eth0-tx-0-21swapper/211:04:512
28415060,0irq/27-eth0-tx-0-21swapper/210:38:082
28415060,0irq/27-eth0-tx-0-21swapper/210:17:252
28415060,0irq/27-eth0-tx-0-21swapper/210:08:352
28415060,0irq/27-eth0-tx-0-21swapper/209:30:012
28415060,0irq/27-eth0-tx-0-21swapper/209:14:592
28415060,0irq/27-eth0-tx-0-21swapper/008:31:210
28415060,0irq/27-eth0-tx-0-21swapper/008:11:180
28415060,0irq/27-eth0-tx-0-21swapper/007:34:410
28405060,0irq/26-eth0-rx-31227-21cut10:04:453
28405060,0irq/26-eth0-rx-30470-21python08:39:513
249960,0watchdog/24315-21irqstats10:09:462
249960,0watchdog/219216-21cut11:19:462
169960,0watchdog/12889-21sed07:54:501
8225050,0irq/24-0000:00:0-21swapper/209:09:592
8225050,0irq/24-0000:00:0-21swapper/207:59:462
339950,0migration/39598-21timerwakeupswit07:59:513
339950,0migration/37966-21irqcore11:54:433
339950,0migration/36500-21runrttasks08:45:253
339950,0migration/331546-21runrttasks11:29:303
339950,0migration/331152-21taskset11:49:283
339950,0migration/330253-21kthreadcore10:24:453
339950,0migration/325221-21kthreadcore11:44:453
339950,0migration/324265-21sh10:40:453
339950,0migration/323830-21aten_r1power_po08:14:393
339950,0migration/323595-21taskset08:59:423
339950,0migration/323319-21munin-node12:04:403
339950,0migration/320710-21hwlatdetect06:54:393
339950,0migration/318102-21sh09:55:393
339950,0migration/317742-21munin-node10:59:413
339950,0migration/317253-21taskset10:15:313
339950,0migration/31665-21kthreadcore07:54:443
339950,0migration/31433-21munin-node07:29:473
339950,0migration/313324-21kthreadcore08:04:433
339950,0migration/311166-21taskset09:53:483
339950,0migration/310402-21munin-node08:24:503
339950,0migration/310217-21munin-node07:39:413
321109955,0cyclictest0-21swapper/211:37:372
321109955,0cyclictest0-21swapper/209:20:092
321109950,0cyclictest0-21swapper/210:24:282
321049955,0cyclictest0-21swapper/110:04:591
321049955,0cyclictest0-21swapper/108:46:001
321049954,0cyclictest18028-21awk08:09:451
321049950,0cyclictest0-21swapper/111:12:561
321049950,0cyclictest0-21swapper/109:09:391
320939954,0cyclictest16983-21awk07:44:420
28415050,0irq/27-eth0-tx-20615-21munin-node09:39:502
28415050,0irq/27-eth0-tx-18643-21proc_pri09:34:472
28415050,0irq/27-eth0-tx-0-21swapper/212:06:072
28415050,0irq/27-eth0-tx-0-21swapper/211:29:322
28415050,0irq/27-eth0-tx-0-21swapper/211:02:552
28415050,0irq/27-eth0-tx-0-21swapper/210:50:112
28415050,0irq/27-eth0-tx-0-21swapper/210:44:392
28415050,0irq/27-eth0-tx-0-21swapper/210:40:092
28415050,0irq/27-eth0-tx-0-21swapper/210:31:242
28415050,0irq/27-eth0-tx-0-21swapper/209:54:302
28415050,0irq/27-eth0-tx-0-21swapper/209:52:282
28415050,0irq/27-eth0-tx-0-21swapper/208:57:592
28415050,0irq/27-eth0-tx-0-21swapper/208:50:522
28415050,0irq/27-eth0-tx-0-21swapper/208:44:582
28415050,0irq/27-eth0-tx-0-21swapper/006:49:410
28405050,0irq/26-eth0-rx-6556-21kthreadcore09:29:433
28405050,0irq/26-eth0-rx-4574-21tr10:29:473
28405050,0irq/26-eth0-rx-3072-21df_abs07:34:413
28405050,0irq/26-eth0-rx-28491-21cut11:04:453
28405050,0irq/26-eth0-rx-27686-21open_inodes09:19:483
28405050,0irq/26-eth0-rx-2287-21cut09:04:453
28405050,0irq/26-eth0-rx-20568-21cut08:54:493
28405050,0irq/26-eth0-rx-18790-21munin-node10:19:403
28405050,0irq/26-eth0-rx-16269-21ssh10:35:143
28405050,0irq/26-eth0-rx-0-21swapper/311:35:303
28405050,0irq/26-eth0-rx-0-21swapper/311:25:183
28405050,0irq/26-eth0-rx-0-21swapper/311:14:443
28405050,0irq/26-eth0-rx-0-21swapper/311:10:553
28405050,0irq/26-eth0-rx-0-21swapper/310:50:263
28405050,0irq/26-eth0-rx-0-21swapper/310:44:473
28405050,0irq/26-eth0-rx-0-21swapper/309:40:063
28405050,0irq/26-eth0-rx-0-21swapper/309:14:463
259950,0migration/230400-21sh09:02:142
259950,0migration/217583-21kthreadcore07:44:442
249950,0watchdog/20-21swapper/210:54:422
19150,0ktimersoftd/10-21swapper/109:29:591
179950,0migration/15603-21kthreadcore07:04:451
179950,0migration/127129-21kthreadcore11:24:451
179950,0migration/125167-21kthreadcore08:59:521
179950,0migration/124410-21kthreadcore07:49:451
179950,0migration/115853-21kthreadcore09:14:461
169950,0watchdog/10-21swapper/111:15:021
169950,0watchdog/10-21swapper/110:21:421
169950,0watchdog/10-21swapper/110:18:141
169950,0watchdog/10-21swapper/110:13:221
169950,0watchdog/10-21swapper/109:55:421
169950,0watchdog/10-21swapper/108:50:341
169950,0watchdog/10-21swapper/108:41:141
149950,0migration/09972-21kthreadcore07:09:440
149950,0migration/09273-21kthreadcore11:34:440
149950,0migration/08015-21aten_r1power_en08:49:380
149950,0migration/05716-21kthreadcore07:59:440
149950,0migration/032702-21munin-node06:39:420
149950,0migration/031116-21munin-node10:44:470
149950,0migration/031006-21kthreadcore09:44:440
149950,0migration/030491-21kthreadcore07:29:450
149950,0migration/030206-21kthreadcore06:59:440
149950,0migration/02860-21kthreadcore11:49:450
149950,0migration/027468-21idleruntime10:24:420
149950,0migration/026966-21kthreadcore08:39:450
149950,0migration/026185-21taskset10:01:150
149950,0migration/024812-21kthreadcore09:19:450
149950,0migration/023534-21hwlatdetect07:49:420
149950,0migration/023455-21kthreadcore10:19:450
149950,0migration/021781-21irqcore09:39:440
149950,0migration/021260-21taskset07:21:200
149950,0migration/019937-21missed_timers09:14:480
149950,0migration/019920-21sh09:38:010
149950,0migration/018207-21kthreadcore11:39:440
149950,0migration/01631-21kthreadcore08:19:440
149950,0migration/01485-21kthreadcore09:04:440
149950,0migration/013708-21munin-node11:14:470
149950,0migration/012958-21taskset06:48:540
149950,0migration/011661-21kthreadcore10:34:430
149950,0migration/01088-21cpuspeed_turbos07:04:390
149950,0migration/010846-21latency_hist10:14:270
149950,0migration/010544-21latency09:29:440
149950,0migration/010115-21kthreadcore11:54:440
149950,0migration/010088-21irqcore10:54:440
339940,0migration/327024-21fschecks_count06:59:403
339940,0migration/324674-21kthreadcore07:49:453
339940,0migration/318623-21kthreadcore07:44:443
339940,0migration/311234-21kthreadcore06:44:473
339940,0migration/310684-21kthreadcore07:09:443
321109944,0cyclictest0-21swapper/207:04:522
321109943,0cyclictest856-21awk06:39:442
321109943,0cyclictest6445-21snmpd08:10:572
321109943,0cyclictest6445-21snmpd07:57:272
321109943,0cyclictest6445-21snmpd06:45:562
321109943,0cyclictest3425-21lldpd08:04:282
321109943,0cyclictest3425-21lldpd07:20:092
321109943,0cyclictest3110-21irqbalance11:39:312
321109943,0cyclictest3110-21irqbalance08:29:322
321109943,0cyclictest3110-21irqbalance08:25:122
321109943,0cyclictest3110-21irqbalance07:35:122
321109943,0cyclictest3110-21irqbalance07:09:322
321109943,0cyclictest3110-21irqbalance07:02:522
321109943,0cyclictest3110-21irqbalance06:54:322
321109943,0cyclictest29331-21awk07:29:432
321109943,0cyclictest14214-21awk06:49:452
321109943,0cyclictest0-21swapper/207:24:442
321049943,0cyclictest6445-21snmpd11:57:431
321049943,0cyclictest6445-21snmpd09:27:421
321049943,0cyclictest3425-21lldpd10:00:231
321049943,0cyclictest3425-21lldpd07:29:431
321049943,0cyclictest3425-21lldpd06:52:591
321049943,0cyclictest3110-21irqbalance09:49:411
321049943,0cyclictest3110-21irqbalance07:59:421
321049943,0cyclictest3110-21irqbalance06:41:021
321049943,0cyclictest3086-21awk10:29:461
321049943,0cyclictest27546-21awk10:24:421
321049943,0cyclictest24459-21sadc08:39:271
321049943,0cyclictest24059-21awk11:44:431
321049943,0cyclictest22945-21awk09:19:431
321049943,0cyclictest17007-21awk07:44:421
321049943,0cyclictest16690-21awk08:54:471
321049943,0cyclictest15131-21awk07:14:431
321049943,0cyclictest11340-21awk08:04:411
320939943,0cyclictest6445-21snmpd12:05:430
320939943,0cyclictest6445-21snmpd12:02:280
320939943,0cyclictest6445-21snmpd10:07:120
320939943,0cyclictest3964-21awk10:09:450
320939943,0cyclictest3425-21lldpd10:29:360
320939943,0cyclictest31315-21sadc11:29:270
320939943,0cyclictest3110-21irqbalance10:50:410
320939943,0cyclictest3110-21irqbalance09:00:220
320939943,0cyclictest18873-21awk08:34:410
320939940,0cyclictest0-21swapper/010:40:520
28415040,0irq/27-eth0-tx-0-21swapper/008:17:280
28405040,0irq/26-eth0-rx-0-21swapper/311:19:433
28405040,0irq/26-eth0-rx-0-21swapper/309:35:333
28405040,0irq/26-eth0-rx-0-21swapper/309:10:383
28405040,0irq/26-eth0-rx-0-21swapper/308:49:443
259940,0migration/23639-21rt-features08:19:472
259940,0migration/227892-21kthreadcore07:49:462
259940,0migration/224608-21munin-node08:14:422
259940,0migration/221087-21kthreadcore08:34:442
259940,0migration/212155-21kthreadcore07:39:442
249940,0watchdog/20-21swapper/211:14:502
249940,0watchdog/20-21swapper/207:14:422
179940,0migration/1682-21kthreadcore08:19:431
179940,0migration/15190-21fschecks_count08:24:411
179940,0migration/14041-21irqrtprio07:34:441
179940,0migration/131645-21sh09:04:401
179940,0migration/127020-21sh10:44:451
179940,0migration/123059-21taskset10:59:591
179940,0migration/122572-21kthreadcore06:54:421
179940,0migration/121502-21kthreadcore10:39:441
179940,0migration/118638-21kthreadcore11:39:441
179940,0migration/116142-21cpuspeed_turbos11:19:411
179940,0migration/114634-21kthreadcore10:34:441
169940,0watchdog/129927-21runrttasks12:08:251
169940,0watchdog/121229-21fschecks_time09:39:421
169940,0watchdog/11545-21diskstats10:49:461
169940,0watchdog/10-21swapper/111:34:491
169940,0watchdog/10-21swapper/111:31:341
169940,0watchdog/10-21swapper/111:04:421
169940,0watchdog/10-21swapper/110:54:421
169940,0watchdog/10-21swapper/109:45:461
169940,0watchdog/10-21swapper/106:44:461
149940,0migration/03260-21sh09:24:510
149940,0migration/027801-21kthreadcore11:04:450
149940,0migration/023471-21kthreadcore06:54:420
149940,0migration/01335-21kthreadcore11:29:430
149940,0migration/011766-21diskmemload09:54:390
339930,0migration/316883-21kthreadcore06:49:473
321049933,0cyclictest3425-21lldpd07:21:391
321049932,0cyclictest24704-21awk08:14:431
321049932,0cyclictest11298-21cat08:29:271
321049930,0cyclictest15216-21perl11:59:401
320939932,0cyclictest9708-21sensors09:50:150
320939932,0cyclictest6445-21snmpd09:09:420
320939932,0cyclictest4456-21tr08:44:460
320939932,0cyclictest21769-21munin-node07:24:490
320939932,0cyclictest17756-21awk10:59:410
320939932,0cyclictest10702-21grep08:24:510
320939932,0cyclictest10324-21awk07:39:410
320939930,0cyclictest543-21df_inode11:09:410
320939930,0cyclictest24855-21grep11:44:450
320939930,0cyclictest0-21swapper/011:23:000
320939930,0cyclictest0-21swapper/008:04:390
28415030,0irq/27-eth0-tx-0-21swapper/007:54:380
28405030,0irq/26-eth0-rx-0-21swapper/308:19:423
28405030,0irq/26-eth0-rx-0-21swapper/308:09:423
28405030,0irq/26-eth0-rx-0-21swapper/307:14:403
28405030,0irq/26-eth0-rx-0-21swapper/307:04:383
28405030,0irq/26-eth0-rx-0-21swapper/306:39:413
179930,0migration/121700-21latency_hist07:24:271
169930,0watchdog/10-21swapper/111:50:171
169930,0watchdog/10-21swapper/107:09:461
169930,0watchdog/10-21swapper/106:59:421
320939920,0cyclictest15620-21cat08:54:270
329920,0watchdog/30-21swapper/308:34:383
329920,0watchdog/30-21swapper/307:24:303
28405020,0irq/26-eth0-rx-0-21swapper/308:29:303
28405020,0irq/26-eth0-rx-0-21swapper/307:19:333
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional