You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 01:20
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot2.osadl.org (updated Thu Apr 02, 2026 00:43:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
188812126150,7sleep50-21swapper/519:00:505
188853999490,47cyclictest2421733-21grep23:07:367
188806424939,7sleep40-21swapper/419:00:004
1888535994845,2cyclictest2218010-21kworker/u16:2+events_unbound22:17:396
1888535994644,2cyclictest2469824-21kworker/u16:4+events_unbound23:37:396
1888539994439,4cyclictest2589828-21latency_hist00:17:367
1888535994341,2cyclictest2312383-21kworker/u16:1+events_unbound23:25:166
1888535994341,2cyclictest2312383-21kworker/u16:1+events_unbound23:15:246
1888535994341,2cyclictest2218010-21kworker/u16:2+events_unbound22:02:316
1888535994240,2cyclictest2601895-21kworker/u16:3+events_unbound00:30:276
1888535994139,2cyclictest2361765-21kworker/u16:4+events_unbound23:04:346
1888535994139,2cyclictest2312383-21kworker/u16:1+events_unbound22:29:156
1888535994139,2cyclictest2168136-21kworker/u16:0+events_unbound21:20:046
1888535994139,2cyclictest2146081-21kworker/u16:4+events_unbound21:56:516
1888535994139,2cyclictest2054659-21kworker/u16:0+events_unbound21:06:596
188853999400,38cyclictest2385824-21latency_hist22:52:377
1888535994038,2cyclictest2588217-21kworker/u16:2+events_unbound00:16:116
1888535994038,2cyclictest2422153-21kworker/u16:0+events_unbound23:21:556
1888535994038,2cyclictest2312383-21kworker/u16:1+events_unbound22:54:246
1888535994038,2cyclictest2026350-21kworker/u16:1+events_unbound21:28:196
18880042404,31sleep70-21swapper/718:59:077
1888535993937,2cyclictest2588217-21kworker/u16:2+events_unbound00:23:156
1888535993937,2cyclictest2588217-21kworker/u16:2+events_unbound00:23:156
1888535993937,2cyclictest2564816-21kworker/u16:0+events_unbound00:11:596
1888535993937,2cyclictest2530209-21kworker/u16:2+events_unbound23:56:406
1888535993937,2cyclictest2253691-21kworker/u16:0+events_unbound22:06:046
1888535993937,2cyclictest2146081-21kworker/u16:4+events_unbound22:12:156
1888535993836,2cyclictest2530209-21kworker/u16:2+events_unbound00:03:116
1888535993836,2cyclictest2146081-21kworker/u16:4+events_unbound21:44:126
1888535993836,2cyclictest2146081-21kworker/u16:4+events_unbound21:25:116
1888535993735,2cyclictest2564816-21kworker/u16:0+events_unbound00:18:306
1888535993735,2cyclictest2545266-21kworker/u16:3+events_unbound23:59:566
1888535993735,2cyclictest2443572-21kworker/u16:2+events_unbound23:31:206
1888535993735,2cyclictest2361765-21kworker/u16:4+events_unbound23:10:596
1888535993735,2cyclictest2326102-21kworker/u16:0+events_unbound22:46:006
1888535993735,2cyclictest2218010-21kworker/u16:2+events_unbound22:26:476
1888535993735,2cyclictest2218010-21kworker/u16:2+events_unbound22:26:476
1888535993735,2cyclictest2026350-21kworker/u16:1+events_unbound21:48:246
1888535993735,2cyclictest1879132-21kworker/u16:2+flush-8:020:27:376
1888535993634,2cyclictest2503491-21kworker/u16:0+events_unbound23:46:076
1888535993634,2cyclictest2469824-21kworker/u16:4+events_unbound23:35:246
1888535993634,2cyclictest2326102-21kworker/u16:0+events_unbound22:48:366
1888535993634,2cyclictest2326102-21kworker/u16:0+events_unbound22:41:286
1888535993634,2cyclictest2312383-21kworker/u16:1+events_unbound22:58:556
1888535993634,2cyclictest2146081-21kworker/u16:4+events_unbound21:41:036
1888535993634,2cyclictest2146081-21kworker/u16:4+events_unbound21:14:246
1888535993634,2cyclictest2026350-21kworker/u16:1+events_unbound21:34:486
1888535993533,2cyclictest2326102-21kworker/u16:0+events_unbound22:36:196
1888535993533,2cyclictest2218010-21kworker/u16:2+events_unbound22:16:196
1888535993533,2cyclictest2054659-21kworker/u16:0+events_unbound21:12:096
188798323525,7sleep60-21swapper/618:58:506
1888535993229,2cyclictest2503491-21kworker/u16:0+flush-8:023:47:376
1888530993230,2cyclictest2361765-21kworker/u16:4+flush-8:023:12:525
1888535993129,2cyclictest1879132-21kworker/u16:2+events_unbound20:53:376
1888535993028,2cyclictest1835010-21kworker/u16:0+events_unbound19:16:376
1888530993028,2cyclictest2218010-21kworker/u16:2+flush-8:021:57:535
1888535992927,2cyclictest1948476-21kworker/u16:1+events_unbound19:30:326
1888535992826,2cyclictest2054659-21kworker/u16:0+events_unbound21:00:046
1888535992826,2cyclictest2054659-21kworker/u16:0+events_unbound20:38:076
1888535992826,2cyclictest2026350-21kworker/u16:1+flush-8:020:17:476
1888535992825,2cyclictest1879132-21kworker/u16:2+flush-8:019:42:476
1888530992826,2cyclictest2326102-21kworker/u16:0+events_unbound22:42:525
1888530992826,2cyclictest2179333-21kworker/u16:3+events_unbound21:33:165
1888526992825,2cyclictest2146081-21kworker/u16:4+flush-8:022:22:414
1888526992825,2cyclictest2146081-21kworker/u16:4+flush-8:022:22:414
1888539992722,4cyclictest1958976-21latency_hist19:37:377
1888535992725,2cyclictest1958204-21kworker/u16:0+events_unbound20:09:086
1888530992725,2cyclictest2483478-21kworker/u16:5+flush-8:023:43:245
1888530992725,2cyclictest2312383-21kworker/u16:1+flush-8:023:02:425
1888530992725,2cyclictest2218010-21kworker/u16:2+flush-8:022:19:485
1888530992725,2cyclictest1973080-21kworker/u16:4+events_unbound20:37:595
1888530992725,2cyclictest1879132-21kworker/u16:2+events_unbound19:46:515
1888530992724,2cyclictest2146081-21kworker/u16:4+events_unbound22:03:005
1888526992724,2cyclictest1879132-21kworker/u16:2+flush-8:019:38:034
1888526992724,2cyclictest1879132-21kworker/u16:2+flush-8:019:38:034
1888539992624,1cyclictest0-21swapper/721:27:597
1888539992620,5cyclictest1997279-21latency_hist19:57:377
1888535992624,2cyclictest1958204-21kworker/u16:0+events_unbound20:04:036
1888535992624,2cyclictest1879132-21kworker/u16:2+events_unbound20:49:046
1888530992624,2cyclictest2146081-21kworker/u16:4+events_unbound21:37:555
1888530992624,2cyclictest1958204-21kworker/u16:0+events_unbound19:54:035
1888530992624,2cyclictest1879132-21kworker/u16:2+flush-8:020:27:565
1888539992524,0cyclictest0-21swapper/719:03:047
1888535992523,2cyclictest1973080-21kworker/u16:4+events_unbound19:58:116
1888535992523,2cyclictest1835010-21kworker/u16:0+flush-8:019:07:476
1888530992523,2cyclictest1958204-21kworker/u16:0+flush-8:020:02:595
1888539992423,0cyclictest0-21swapper/721:07:577
1888530992422,2cyclictest2253691-21kworker/u16:0+events_unbound22:08:015
1888530992422,2cyclictest1879132-21kworker/u16:2+flush-8:019:47:565
1888530992422,2cyclictest1879132-21kworker/u16:2+events_unbound20:48:035
1888526992422,2cyclictest2146081-21kworker/u16:4+events_unbound22:02:154
1888539992318,4cyclictest650-21systemd-journal19:32:327
1888539992317,5cyclictest2253666-21latency_hist21:57:377
1888535992321,2cyclictest1948476-21kworker/u16:1+flush-8:019:37:476
1888535992321,2cyclictest1948476-21kworker/u16:1+flush-8:019:37:476
1888530992321,2cyclictest2179333-21kworker/u16:3+flush-8:021:28:025
1888530992321,2cyclictest1879132-21kworker/u16:2+flush-8:019:57:525
1888530992321,2cyclictest1879132-21kworker/u16:2+events_unbound20:59:275
1888530992321,2cyclictest1879132-21kworker/u16:2+events_unbound20:15:075
1888539992221,0cyclictest0-21swapper/723:12:507
1888539992217,4cyclictest2625783-21latency_hist00:32:367
1888535992220,2cyclictest1973080-21kworker/u16:4+events_unbound20:29:086
1888535992220,2cyclictest1879132-21kworker/u16:2+flush-8:020:37:326
1888535992220,2cyclictest1879132-21kworker/u16:2+events_unbound19:57:326
1888530992220,2cyclictest2218010-21kworker/u16:2+events_unbound22:27:515
1888530992220,2cyclictest2026350-21kworker/u16:1+flush-8:020:17:525
1888530992220,2cyclictest1948476-21kworker/u16:1+events_unbound19:40:105
1888530992220,2cyclictest1948476-21kworker/u16:1+events_unbound19:40:105
1888530992220,2cyclictest1879132-21kworker/u16:2+events_unbound20:33:025
1888539992120,0cyclictest0-21swapper/721:17:597
1888539992117,3cyclictest2016466-21awk20:07:377
1888530992119,2cyclictest2545266-21kworker/u16:3+flush-8:000:07:365
1888530992119,2cyclictest1879132-21kworker/u16:2+events_unbound19:34:505
24898932200,1sleep70-21swapper/723:32:567
1888539992020,0cyclictest0-21swapper/721:33:007
1888539992014,5cyclictest2045089-21latency_hist20:22:377
1888535992018,2cyclictest1879132-21kworker/u16:2+events_unbound19:34:036
1888535992018,2cyclictest1835010-21kworker/u16:0+events_unbound19:05:046
1888535992017,2cyclictest1835010-21kworker/u16:0+events_unbound19:22:396
1888530992018,2cyclictest2312383-21kworker/u16:1+events_unbound22:32:525
1888530992018,2cyclictest1879132-21kworker/u16:2+events_unbound19:22:595
1888530992017,2cyclictest2588217-21kworker/u16:2+events_unbound00:28:045
1888530992017,2cyclictest2026350-21kworker/u16:1+events_unbound21:02:435
1888526992016,3cyclictest2541807-21updatedb23:57:364
188853999198,10cyclictest0-21swapper/722:43:007
1888539991919,0cyclictest0-21swapper/720:33:037
1888535991917,2cyclictest1879132-21kworker/u16:2+events_unbound20:16:206
1888530991917,2cyclictest2564816-21kworker/u16:0+events_unbound00:23:115
1888530991917,2cyclictest2564816-21kworker/u16:0+events_unbound00:23:115
1888530991917,2cyclictest2349760-21kworker/u16:3+flush-8:022:47:425
1888530991917,2cyclictest2026350-21kworker/u16:1+events_unbound21:43:025
1888530991917,2cyclictest1835010-21kworker/u16:0+events_unbound19:19:355
188853999188,8cyclictest0-21swapper/723:52:597
1888539991815,2cyclictest96-21ksoftirqd/723:42:547
1888539991813,4cyclictest2481663-21latency_hist23:32:377
1888535991816,2cyclictest1958204-21kworker/u16:0+events_unbound19:50:036
1888530991816,2cyclictest2530209-21kworker/u16:2+flush-8:023:57:585
1888530991816,2cyclictest2026350-21kworker/u16:1+events_unbound20:46:575
1888530991816,2cyclictest1973080-21kworker/u16:4+events_unbound20:24:185
1888530991816,2cyclictest1835010-21kworker/u16:0+events_unbound19:15:115
1888526991814,3cyclictest1879132-21kworker/u16:2+flush-8:020:53:034
188853999171,2cyclictest2154663-21ssh21:13:017
1888535991715,2cyclictest1879132-21kworker/u16:2+events_unbound19:22:046
1888530991715,2cyclictest1879132-21kworker/u16:2+flush-8:019:07:535
1888530991715,2cyclictest1835010-21kworker/u16:0+events_unbound19:04:305
188852699170,16cyclictest1995441-21tr19:52:584
1888539991614,1cyclictest0-21swapper/719:23:037
1888530991614,2cyclictest2483478-21kworker/u16:5+flush-8:023:47:525
1888530991614,2cyclictest2483478-21kworker/u16:5+events_unbound23:33:075
1888530991614,2cyclictest1958204-21kworker/u16:0+events_unbound20:09:275
1888530991614,2cyclictest1879132-21kworker/u16:2+events_unbound21:13:025
1888526991612,3cyclictest2326102-21kworker/u16:0+flush-8:022:42:544
188853999158,6cyclictest0-21swapper/721:23:037
188853999157,7cyclictest0-21swapper/723:17:567
188853999157,7cyclictest0-21swapper/722:12:567
188853999156,8cyclictest0-21swapper/722:58:047
188853999156,8cyclictest0-21swapper/700:12:377
188853999156,8cyclictest0-21swapper/700:02:587
188853999156,7cyclictest0-21swapper/723:37:557
188853999156,7cyclictest0-21swapper/723:22:507
1888539991511,3cyclictest1267-21snmpd23:50:267
1888539991511,3cyclictest1267-21snmpd21:43:417
1888535991513,2cyclictest2026350-21kworker/u16:1+events_unbound20:46:086
1888530991513,2cyclictest2588217-21kworker/u16:2+events_unbound00:15:325
1888530991513,2cyclictest2146081-21kworker/u16:4+events_unbound21:24:055
1888530991512,2cyclictest2312383-21kworker/u16:1+events_unbound23:37:475
1888526991514,1cyclictest0-21swapper/400:12:564
188853999147,6cyclictest0-21swapper/722:38:007
188853999147,6cyclictest0-21swapper/722:28:037
188853999147,6cyclictest0-21swapper/722:26:117
188853999147,6cyclictest0-21swapper/722:26:117
188853999147,6cyclictest0-21swapper/721:06:047
188853999146,7cyclictest0-21swapper/722:32:597
188853999146,7cyclictest0-21swapper/721:38:007
188853999146,7cyclictest0-21swapper/700:17:587
188853999145,8cyclictest0-21swapper/723:57:537
188853999145,8cyclictest0-21swapper/722:56:237
188853999144,9cyclictest2434028-21cut23:12:377
1888539991410,3cyclictest2313712-21latency_hist22:22:367
188853999140,13cyclictest1987721-21systemd-run19:52:377
1888530991413,1cyclictest0-21swapper/521:53:145
1888530991412,2cyclictest2564816-21kworker/u16:0+events_unbound00:12:035
1888530991412,2cyclictest2361765-21kworker/u16:4+events_unbound22:55:075
1888530991412,2cyclictest2168136-21kworker/u16:0+events_unbound21:20:285
1888530991412,2cyclictest2054659-21kworker/u16:0+events_unbound21:09:235
1888530991412,2cyclictest1879132-21kworker/u16:2+events_unbound19:28:035
188853099140,14cyclictest0-21swapper/522:59:455
1888526991413,1cyclictest0-21swapper/423:23:354
1888526991413,1cyclictest0-21swapper/422:53:484
1888526991413,1cyclictest0-21swapper/420:07:574
1888526991411,2cyclictest1938956-21ntp_states19:23:014
188852699140,1cyclictest0-21swapper/421:31:024
188852699140,1cyclictest0-21swapper/421:10:324
188852699140,14cyclictest0-21swapper/422:37:244
188852699140,14cyclictest0-21swapper/422:21:384
188852699140,14cyclictest0-21swapper/422:11:514
188852699140,14cyclictest0-21swapper/421:51:124
188852699140,14cyclictest0-21swapper/421:20:184
188853999136,6cyclictest0-21swapper/721:47:527
188853999135,7cyclictest0-21swapper/722:04:317
188853999135,7cyclictest0-21swapper/700:22:507
188853999135,7cyclictest0-21swapper/700:22:507
1888539991310,2cyclictest2286897-21ssh22:08:047
1888539991310,2cyclictest0-21swapper/721:57:377
188853999130,11cyclictest2005173-21kthreadcore19:58:007
1888530991311,2cyclictest2312383-21kworker/u16:1+events_unbound23:08:205
1888530991311,2cyclictest2146081-21kworker/u16:4+events_unbound22:13:045
1888530991311,2cyclictest2026350-21kworker/u16:1+flush-8:021:47:585
1888530991311,2cyclictest2026350-21kworker/u16:1+events_unbound20:52:525
1888526991312,1cyclictest0-21swapper/423:21:324
1888526991312,1cyclictest0-21swapper/400:26:014
1888526991312,1cyclictest0-21swapper/400:26:014
1888526991310,2cyclictest1879132-21kworker/u16:2+flush-8:020:18:344
188852699130,13cyclictest0-21swapper/423:46:554
188852699130,13cyclictest0-21swapper/423:29:544
188852699130,13cyclictest0-21swapper/419:29:474
188853999128,3cyclictest1267-21snmpd20:50:457
188853999128,3cyclictest1267-21snmpd20:24:357
188853999128,3cyclictest1267-21snmpd20:09:117
188853999128,3cyclictest1267-21snmpd19:19:157
188853999127,4cyclictest2111797-21sshd-session20:55:037
188853999120,11cyclictest1267-21snmpd20:39:547
188853999120,11cyclictest1096-21ntpd19:11:227
188853999120,11cyclictest0-21swapper/719:38:007
188853999120,11cyclictest0-21swapper/719:38:007
188853999120,10cyclictest2112375-21/usr/sbin/munin20:57:537
188853099128,3cyclictest2350949-21cat22:37:515
1888530991210,2cyclictest2530209-21kworker/u16:2+events_unbound23:53:355
1888530991210,2cyclictest2312383-21kworker/u16:1+events_unbound23:22:505
1888530991210,2cyclictest2218010-21kworker/u16:2+flush-8:022:23:055
1888530991210,2cyclictest2218010-21kworker/u16:2+flush-8:022:23:055
188852699128,3cyclictest2146905-21cat21:12:524
188852699128,3cyclictest1998449-21cat19:57:534
1888526991211,1cyclictest0-21swapper/423:57:394
1888526991211,1cyclictest0-21swapper/423:15:144
1888526991211,1cyclictest0-21swapper/422:39:174
1888526991211,1cyclictest0-21swapper/419:16:234
188852699120,1cyclictest0-21swapper/419:06:464
188853999117,3cyclictest5944-21gmain19:12:597
188853999114,6cyclictest6184-21gvfs-afc-volume20:12:567
188853999110,2cyclictest0-21swapper/719:43:007
188853999110,10cyclictest0-21swapper/720:42:527
1888530991110,1cyclictest0-21swapper/523:31:555
1888526991110,1cyclictest0-21swapper/423:34:004
1888526991110,1cyclictest0-21swapper/421:41:544
1888526991110,1cyclictest0-21swapper/421:36:234
1888526991110,1cyclictest0-21swapper/421:06:354
1888526991110,1cyclictest0-21swapper/420:23:334
188852699110,11cyclictest0-21swapper/420:30:014
188852699110,11cyclictest0-21swapper/420:16:564
188853999100,2cyclictest2056220-21irqstats20:27:547
188853099108,2cyclictest2589467-21kworker/u16:4+events_unbound00:17:475
188853099108,2cyclictest2443572-21kworker/u16:2+events_unbound23:19:035
188852699109,1cyclictest0-21swapper/423:04:404
188852699109,1cyclictest0-21swapper/422:49:234
188852699109,1cyclictest0-21swapper/421:27:244
188852699109,1cyclictest0-21swapper/420:36:004
188852699109,1cyclictest0-21swapper/419:18:364
188852699106,3cyclictest2591098-21cat00:17:494
18885269998,1cyclictest0-21swapper/423:51:114
18885269998,1cyclictest0-21swapper/422:31:244
18885269998,1cyclictest0-21swapper/420:50:124
18885269998,1cyclictest0-21swapper/420:44:034
18885269998,1cyclictest0-21swapper/419:48:144
18885269998,1cyclictest0-21swapper/419:11:304
18885269995,3cyclictest2616288-21kthreadcore00:27:534
18885269995,3cyclictest2494982-21cat23:37:504
18885269995,3cyclictest2242770-21tune2fs21:52:504
18885269995,3cyclictest2074928-21cat20:37:514
18885269994,4cyclictest2562799-21ntpq00:02:584
18885269994,4cyclictest2398250-21/usr/sbin/munin22:57:514
18885269990,8cyclictest1969659-21gltestperf19:42:534
18885269987,1cyclictest0-21swapper/421:00:364
18885269987,1cyclictest0-21swapper/420:03:324
18885269987,1cyclictest0-21swapper/400:08:314
18885269984,3cyclictest2269098-21cut22:02:554
18885269984,3cyclictest2218349-21cpuspeed_turbos21:42:494
18885269983,4cyclictest1952899-21cut19:32:544
18885269975,2cyclictest2146081-21kworker/u16:4+events_unbound22:12:524
18885269973,3cyclictest2422484-21expr23:07:504
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional