You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-07 - 14:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Mon Sep 07, 2026 00:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
290049910491048,1cyclictest28085-21kworker/3:1+events22:46:293
290049910491045,4cyclictest10674-21turbostat19:49:553
290019910491047,2cyclictest5585-21kworker/2:1+events22:36:262
290019910491047,2cyclictest5585-21kworker/2:1+events22:36:262
289969910491046,3cyclictest6593-21kworker/1:0+events19:39:261
290049910481046,2cyclictest21173-21turbostat20:14:553
290049910481046,2cyclictest18728-21kworker/3:0+events22:55:453
289969910481045,3cyclictest23634-21kworker/1:1+events19:49:161
289969910481045,3cyclictest23634-21kworker/1:1+events19:41:291
289969910441042,2cyclictest23634-21kworker/1:1+events23:12:341
289929910431042,1cyclictest7534-21kworker/0:2+events21:27:360
290019910411038,3cyclictest18786-21kworker/2:1+events21:10:222
290049910401036,2cyclictest8950-21kworker/3:0+events23:41:473
290049910391038,1cyclictest1113-21kworker/3:0+events00:39:243
290049910331032,1cyclictest30205-21kworker/3:1+events23:04:343
290049910331031,2cyclictest19421-21kworker/3:2+events20:08:403
290019910311029,2cyclictest18786-21kworker/2:1+events21:00:232
290049910301029,1cyclictest19421-21kworker/3:2+events20:21:583
290019910301028,2cyclictest22375-21kworker/2:0+events22:23:502
290019910291026,3cyclictest13729-21kworker/2:2+events21:17:562
290019910281025,3cyclictest13729-21kworker/2:2+events21:24:192
290019910271024,3cyclictest20972-21kworker/2:2+events19:55:232
290019910261023,3cyclictest9624-21kworker/2:0+events21:06:072
289929910251024,1cyclictest14738-21kworker/0:0+events23:15:530
289929910241022,2cyclictest7534-21kworker/0:2+events21:38:410
290019910211019,2cyclictest18786-21kworker/2:1+events20:02:212
290049910201017,3cyclictest4480-21turbostat23:09:553
290049910191018,1cyclictest16108-21kworker/3:1+events23:46:483
289929910191017,0cyclictest7534-21kworker/0:2+events21:33:070
290019910171015,2cyclictest855-21kworker/2:0+events23:37:092
289929910171017,0cyclictest6642-21kworker/0:0+events21:43:130
290049910161014,1cyclictest3362-21kworker/3:2+events23:28:413
290049910141011,3cyclictest29611-21turbostat21:29:553
290049910131012,1cyclictest16108-21kworker/3:1+events23:59:253
289929910101009,1cyclictest7534-21kworker/0:2+events20:55:220
289929910101007,2cyclictest27219-21turbostat22:29:550
290049910081006,2cyclictest21587-21kworker/3:2+events00:16:163
289929910081007,1cyclictest23456-21kworker/0:0+events20:45:220
289929910071006,1cyclictest6642-21kworker/0:0+events22:01:050
290049910061005,1cyclictest8950-21kworker/3:0+events23:34:143
290049910061005,1cyclictest21587-21kworker/3:2+events00:09:413
290049910061004,2cyclictest21587-21kworker/3:2+events23:54:233
289929910061005,1cyclictest6642-21kworker/0:0+events21:46:060
289929910061005,1cyclictest31189-21kworker/0:2+events19:17:120
289929910061005,1cyclictest2737-21kworker/0:1+events19:11:280
290049910051004,1cyclictest21587-21kworker/3:2+events00:00:443
290049910041002,2cyclictest16960-21kworker/3:1+events00:24:333
289929910041003,1cyclictest9187-21kworker/0:1+events22:12:540
290049910031002,1cyclictest3362-21kworker/3:2+events23:23:193
290049910031002,1cyclictest21587-21kworker/3:2+events00:29:253
290049910021001,1cyclictest21587-21kworker/3:2+events00:34:043
290049910021000,2cyclictest27911-21kworker/3:2+events22:28:303
290049910011000,1cyclictest5851-21kworker/3:0+events00:12:243
290049910011000,1cyclictest19421-21kworker/3:2+events20:25:213
289929910011001,0cyclictest30929-21kworker/0:2+events21:59:030
289929910011000,1cyclictest32011-21kworker/0:2+events20:39:230
29004991000999,1cyclictest6328-21kworker/3:1+events19:30:213
28992991000998,1cyclictest12653-21kworker/0:1+events20:18:200
2899299999998,1cyclictest2737-21kworker/0:1+events19:26:070
2899299999998,1cyclictest23456-21kworker/0:0+events20:50:430
2899299998997,1cyclictest12653-21kworker/0:1+events19:53:280
2900499997996,1cyclictest18728-21kworker/3:0+events22:50:433
2900499996995,1cyclictest19421-21kworker/3:2+events20:30:223
2899299996995,1cyclictest32011-21kworker/0:2+events20:43:200
2899299996995,1cyclictest31189-21kworker/0:2+events19:23:180
2899299994993,1cyclictest30929-21kworker/0:2+events22:08:170
2899299994993,1cyclictest14738-21kworker/0:0+events22:19:150
2900499993993,0cyclictest28085-21kworker/3:1+events22:41:083
2899299993991,1cyclictest11944-21kworker/0:1+events21:50:220
2899299993990,3cyclictest17123-21turbostat20:04:550
2900199991989,2cyclictest2563-21turbostat00:09:542
2900499990989,1cyclictest27911-21kworker/3:2+events22:34:363
2900499970967,3cyclictest12681-21turbostat19:54:553
2899699963961,2cyclictest14119-21turbostat23:49:541
2900499948945,3cyclictest9669-21turbostat23:14:553
2899699945943,2cyclictest7844-21turbostat20:59:561
2899299939937,2cyclictest22031-21turbostat22:24:550
2899699937934,3cyclictest29685-21turbostat20:34:551
2900499936933,3cyclictest24377-21turbostat21:24:553
2899699929925,3cyclictest15313-21turbostat23:19:551
2899699924922,2cyclictest30908-21turbostat23:34:541
2899299924923,1cyclictest18171-21turbostat00:24:540
2900499688687,1cyclictest22031-21turbostat22:24:553
2900499684681,3cyclictest5235-21turbostat22:39:553
2900499684681,3cyclictest5235-21turbostat22:39:553
2900499677674,3cyclictest2582-21turbostat19:29:553
2900199674670,4cyclictest7844-21turbostat20:59:562
2900499653650,3cyclictest23198-21turbostat20:19:563
2900199652650,2cyclictest30908-21turbostat23:34:542
2899299636635,1cyclictest29685-21turbostat20:34:550
2899699633630,3cyclictest4480-21turbostat23:09:551
2900499622620,2cyclictest15313-21turbostat23:19:553
2899699585583,2cyclictest4598-21turbostat19:34:551
2900199565561,4cyclictest16813-21turbostat22:19:552
2900199561558,3cyclictest32424-21turbostat22:34:562
2900199556553,3cyclictest12681-21turbostat19:54:552
2900499540539,1cyclictest17123-21turbostat20:04:553
28992995130,512cyclictest17515-21runrttasks21:12:450
2899299506505,1cyclictest9669-21turbostat23:14:550
2900499495493,2cyclictest3703-21turbostat23:39:553
2899299494493,1cyclictest24377-21turbostat21:24:550
2899299493492,1cyclictest10674-21turbostat19:49:550
2899299407406,1cyclictest21173-21turbostat20:14:550
29004991919,0cyclictest67750irq/123-lan21:34:483
29004991918,1cyclictest67750irq/123-lan21:50:203
29004991918,1cyclictest67750irq/123-lan21:16:553
29004991918,1cyclictest67750irq/123-lan21:10:183
29004991818,0cyclictest67750irq/123-lan22:05:153
29004991818,0cyclictest67750irq/123-lan21:56:453
29004991818,0cyclictest67750irq/123-lan21:43:493
29004991818,0cyclictest67750irq/123-lan20:40:083
29004991817,1cyclictest67750irq/123-lan22:12:583
29004991817,1cyclictest67750irq/123-lan21:45:133
29004991817,1cyclictest67750irq/123-lan21:00:223
29004991817,1cyclictest67750irq/123-lan19:20:103
29004991717,0cyclictest67750irq/123-lan22:00:033
29004991716,1cyclictest67750irq/123-lan22:15:083
28992991717,0cyclictest784-21snmpd23:38:120
28992991717,0cyclictest784-21snmpd20:25:500
28992991717,0cyclictest22142-21cat22:50:170
29004991616,0cyclictest67750irq/123-lan19:10:213
29004991615,1cyclictest67750irq/123-lan21:38:303
29004991615,1cyclictest67750irq/123-lan20:55:123
29004991615,1cyclictest67750irq/123-lan20:49:093
29004991615,1cyclictest67750irq/123-lan19:40:143
28992991616,0cyclictest784-21snmpd23:34:060
28992991616,0cyclictest784-21snmpd22:32:170
28992991616,0cyclictest784-21snmpd21:19:230
28992991616,0cyclictest784-21snmpd19:57:190
29004991515,0cyclictest67750irq/123-lan20:54:593
29004991514,1cyclictest67750irq/123-lan19:35:203
28992991515,0cyclictest784-21snmpd23:48:210
28992991515,0cyclictest784-21snmpd19:38:040
28992991515,0cyclictest784-21snmpd00:26:470
28992991515,0cyclictest5324-21cat23:05:160
28992991515,0cyclictest19859-21cat20:05:150
29004991414,0cyclictest67750irq/123-lan20:35:213
28992991414,0cyclictest784-21snmpd22:43:160
29004991313,0cyclictest67750irq/123-lan19:15:183
29004991312,1cyclictest67750irq/123-lan19:55:163
29004991010,0cyclictest6328-21kworker/3:1+events_freezable19:10:013
290049999,0cyclictest28085-21kworker/3:1+events_freezable21:05:153
290019998,1cyclictest526-21avahi-daemon21:48:552
289969998,1cyclictest782-21lldpd21:17:251
289929998,1cyclictest782-21lldpd00:39:220
289929996,3cyclictest762-21ntpd20:20:380
290019988,0cyclictest784-21snmpd21:31:262
290019988,0cyclictest782-21lldpd23:10:192
290019988,0cyclictest782-21lldpd21:50:222
290019987,1cyclictest784-21snmpd20:23:052
290019987,1cyclictest526-21avahi-daemon23:22:282
290019987,1cyclictest507-21systemd-resolve22:25:142
290019987,1cyclictest30796-21ntp_kernel_pll_21:25:172
290019986,2cyclictest784-21snmpd22:07:452
290019986,2cyclictest784-21snmpd20:44:212
289969988,0cyclictest784-21snmpd22:37:581
289969988,0cyclictest784-21snmpd22:37:581
289969988,0cyclictest784-21snmpd21:46:111
289969988,0cyclictest784-21snmpd20:26:051
289969988,0cyclictest784-21snmpd00:30:351
289969988,0cyclictest782-21lldpd23:21:171
289969988,0cyclictest782-21lldpd22:45:081
289969988,0cyclictest782-21lldpd22:33:021
289969988,0cyclictest782-21lldpd21:37:591
289969988,0cyclictest782-21lldpd21:14:241
289969988,0cyclictest782-21lldpd21:03:501
289969987,1cyclictest782-21lldpd00:22:341
289969986,2cyclictest784-21snmpd22:10:061
289969986,2cyclictest784-21snmpd00:07:521
289929988,0cyclictest782-21lldpd21:02:460
289929988,0cyclictest782-21lldpd19:44:300
289929986,2cyclictest784-21snmpd23:41:130
289929986,2cyclictest784-21snmpd23:22:390
289929986,2cyclictest784-21snmpd00:19:020
289929986,2cyclictest762-21ntpd19:32:450
289929980,2cyclictest21119-21latency_hist22:50:010
290019977,0cyclictest784-21snmpd23:01:242
290019977,0cyclictest784-21snmpd19:11:002
290019977,0cyclictest782-21lldpd22:40:212
290019977,0cyclictest782-21lldpd20:18:382
290019976,1cyclictest784-21snmpd20:35:522
290019976,1cyclictest784-21snmpd00:16:232
290019976,1cyclictest507-21systemd-resolve21:58:092
290019976,1cyclictest507-21systemd-resolve00:29:322
290019976,1cyclictest2681-21cat00:35:162
290019975,2cyclictest784-21snmpd23:19:232
290019973,3cyclictest21200-21ssh00:21:582
290019973,3cyclictest14619-21ssh22:12:002
290019973,3cyclictest12639-21ssh23:42:562
289969977,0cyclictest784-21snmpd22:06:151
289969977,0cyclictest784-21snmpd22:02:571
289969977,0cyclictest782-21lldpd22:23:391
289969977,0cyclictest782-21lldpd21:28:381
289969977,0cyclictest782-21lldpd20:42:071
289969977,0cyclictest782-21lldpd20:05:551
289969977,0cyclictest782-21lldpd00:36:561
289969977,0cyclictest526-21avahi-daemon20:14:441
289969976,1cyclictest784-21snmpd23:43:401
289969976,1cyclictest784-21snmpd23:00:181
289969976,1cyclictest784-21snmpd00:01:551
289969976,1cyclictest782-21lldpd20:19:411
289969976,1cyclictest572-21in:imuxsock22:26:461
289969976,1cyclictest526-21avahi-daemon21:41:051
289969976,1cyclictest25139-21fschecks_count23:55:151
289969976,1cyclictest23985-21kworker/u16:1+nfsiod21:50:171
289969976,1cyclictest17751-21gltestperf20:00:151
289969975,2cyclictest784-21snmpd23:25:181
289969975,2cyclictest784-21snmpd19:51:101
289929977,0cyclictest782-21lldpd23:04:030
289929977,0cyclictest782-21lldpd00:05:000
289929976,1cyclictest507-21systemd-resolve23:53:130
289929975,2cyclictest12599-21cat21:05:140
290019966,0cyclictest782-21lldpd22:59:152
290019966,0cyclictest782-21lldpd20:53:352
290019966,0cyclictest782-21lldpd19:48:392
290019966,0cyclictest782-21lldpd19:28:222
290019966,0cyclictest526-21avahi-daemon20:27:542
290019965,1cyclictest9678-21dump-pmu-power23:10:002
290019965,1cyclictest784-21snmpd23:28:302
290019965,1cyclictest762-21ntpd00:01:232
290019963,2cyclictest15302-21ssh23:45:152
290019962,3cyclictest9852-21ssh00:10:402
290019962,3cyclictest4107-21ssh22:01:492
290019962,3cyclictest16242-21ssh21:42:342
290019961,5cyclictest547-21NetworkManager19:43:252
289969966,0cyclictest782-21lldpd22:53:051
289969966,0cyclictest782-21lldpd21:05:201
289969966,0cyclictest526-21avahi-daemon20:51:581
289969965,1cyclictest784-21snmpd19:22:001
289969965,1cyclictest784-21snmpd00:25:171
289969965,1cyclictest784-21snmpd00:12:431
289969964,2cyclictest784-21snmpd00:16:081
289969963,2cyclictest574-21rs:main1
289969963,2cyclictest245-20systemd-journal21:59:031
289969962,3cyclictest31814-21sed23:00:001
289969962,3cyclictest13164-21ssh22:41:341
289969961,4cyclictest5958-21cat20:50:001
289929966,0cyclictest784-21snmpd22:37:130
289929966,0cyclictest784-21snmpd22:37:130
289929966,0cyclictest782-21lldpd00:10:560
289929965,1cyclictest29674-21cut00:30:150
289929963,0cyclictest28879-21ssh22:56:090
290019955,0cyclictest784-21snmpd22:50:222
290019955,0cyclictest784-21snmpd19:30:482
290019953,2cyclictest837-21http19:20:102
290019953,2cyclictest784-21snmpd19:16:302
290019952,3cyclictest547-21NetworkManager20:10:202
290019952,2cyclictest25682-21ld23:55:182
290019952,2cyclictest22681-21ssh23:52:272
290019952,2cyclictest18316-21sh22:46:292
290019951,3cyclictest9913-21sh21:36:012
290019951,3cyclictest4640-21memory20:45:152
289969955,0cyclictest526-21avahi-daemon20:35:371
289969954,1cyclictest784-21snmpd19:11:391
289969952,3cyclictest4421-21grep23:35:131
289969952,2cyclictest3399-21cat19:25:151
289969952,2cyclictest245-20systemd-journal20:22:481
289969952,2cyclictest21968-21ssh23:51:271
289969952,2cyclictest17838-21ssh22:15:151
289969951,3cyclictest32700-21seq19:17:051
289929955,0cyclictest782-21lldpd23:57:390
289929953,2cyclictest528-21dbus-daemon00:00:030
289929952,2cyclictest26057-21sh23:25:120
290019942,2cyclictest784-21snmpd19:35:082
290019942,2cyclictest30097-21grep20:30:122
290019940,4cyclictest20226-21ntpq20:05:172
289969944,0cyclictest0-21swapper/121:20:121
289969941,2cyclictest15720-21grep19:55:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional