You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 18:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Thu Jul 09, 2026 12:44:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
57995599145134,8cyclictest684957-21latency_hist12:35:020
57995599145134,8cyclictest684957-21latency_hist12:35:020
57995599145131,5cyclictest592673-21kworker/u8:1+events_unbound08:20:000
57995599131125,5cyclictest683415-21kworker/u8:1+events_unbound12:40:000
57995999127122,3cyclictest605918-21kworker/u8:3+events_unbound08:50:251
57995599126116,7cyclictest600640-21latency_hist08:15:010
57995599126112,10cyclictest1213033-21systemd-journal10:00:000
57995599122117,4cyclictest630122-21kworker/u8:3+events_unbound09:50:250
57995599122117,4cyclictest630122-21kworker/u8:3+events_unbound09:50:250
57995999121106,12cyclictest1213033-21systemd-journal11:10:001
57995999119107,9cyclictest621884-21apt-get09:20:001
57995599117108,7cyclictest670674-21kworker/u8:0+events_unbound12:25:250
57995599116110,4cyclictest617320-21kworker/u8:1+events_unbound09:05:240
57995599115107,3cyclictest636689-21kworker/u8:0+events_unbound10:15:240
57995599115104,9cyclictest643146-21cat10:25:010
57995599114106,6cyclictest567199-21kworker/u8:2+events_unbound07:15:230
5799559911399,7cyclictest592673-21kworker/u8:1+events_unbound08:20:180
57995599113108,4cyclictest620250-21kworker/u8:0+events_unbound09:30:240
57995999110105,4cyclictest636689-21kworker/u8:0+events_unbound10:10:241
57995599110104,4cyclictest653484-21kworker/u8:2+flush-179:010:55:240
57995599110100,7cyclictest599054-21latency_hist08:10:010
57995999109104,4cyclictest644706-21kworker/u8:3+events_unbound11:00:241
57995999109103,4cyclictest652590-21kworker/u8:1+events_unbound10:55:201
57995999106101,4cyclictest612081-21kworker/u8:2+flush-179:008:55:241
57995599106102,3cyclictest600622-21kworker/u8:0+flush-179:008:35:250
57995599106102,3cyclictest600622-21kworker/u8:0+flush-179:008:35:250
57995599106101,4cyclictest644706-21kworker/u8:3+events_unbound11:20:230
57995999105101,3cyclictest630059-21kworker/u8:1+events_unbound10:00:241
57995599105100,4cyclictest644706-21kworker/u8:3+events_unbound10:40:250
5799559910498,4cyclictest644706-21kworker/u8:3+flush-179:011:50:210
5799559910498,4cyclictest620250-21kworker/u8:0+flush-179:009:40:240
5799559910496,5cyclictest592725-21kworker/u8:3+events_unbound08:00:150
5799599910398,4cyclictest670674-21kworker/u8:0+events_unbound12:15:251
5799559910393,7cyclictest510-21rs:main0
5799559910194,5cyclictest605937-21apt-get08:30:110
5799599910096,3cyclictest644706-21kworker/u8:3+events_unbound11:20:221
5799599910090,7cyclictest678589-21latency_hist12:15:021
5799599910089,9cyclictest630334-21apt-get09:45:131
5799599910089,9cyclictest630334-21apt-get09:45:131
5799559910091,6cyclictest609112-21kworker/u8:1+events_unbound08:45:140
579955999991,6cyclictest636689-21kworker/u8:0+events_unbound10:05:140
579955999887,9cyclictest1213033-21systemd-journal11:05:000
579955999793,3cyclictest644706-21kworker/u8:3+flush-179:011:05:230
579959999691,4cyclictest636689-21kworker/u8:0+flush-179:010:30:201
579959999688,6cyclictest617084-21latency_hist09:05:011
579959999688,6cyclictest617084-21latency_hist09:05:011
579959999687,7cyclictest667513-21apt-key11:40:021
579959999686,7cyclictest633508-21latency_hist09:55:011
579959999686,7cyclictest633508-21latency_hist09:55:011
579955999692,3cyclictest644706-21kworker/u8:3+events_unbound11:15:240
579955999691,4cyclictest670674-21kworker/u8:0+events_unbound12:10:230
579955999684,5cyclictest657624-21kworker/u8:1+events_unbound11:10:220
579959999590,4cyclictest600622-21kworker/u8:0+flush-179:008:35:281
579959999590,4cyclictest600622-21kworker/u8:0+flush-179:008:35:271
579955999587,6cyclictest612081-21kworker/u8:2+events_unbound09:20:230
579959999490,3cyclictest596876-21kworker/u8:2+events_unbound08:15:241
579955999490,3cyclictest670674-21kworker/u8:0+events_unbound12:00:240
579959999389,3cyclictest618657-21kworker/u8:3+flush-179:009:10:151
579959999387,4cyclictest670674-21kworker/u8:0+flush-179:012:20:141
579959999387,4cyclictest567199-21kworker/u8:2+events_unbound07:10:131
579955999387,5cyclictest630059-21kworker/u8:1+flush-179:010:00:200
579955999386,5cyclictest573959-21kworker/u8:1+flush-179:007:20:210
579959999287,4cyclictest592725-21kworker/u8:3+events_unbound08:00:191
579955999288,3cyclictest636689-21kworker/u8:0+flush-179:010:30:230
579959999186,4cyclictest636689-21kworker/u8:0+flush-179:010:35:221
579959999179,9cyclictest665345-21latency_hist11:35:001
579955999186,4cyclictest592673-21kworker/u8:1+events_unbound07:55:230
579955999185,4cyclictest630059-21kworker/u8:1+flush-179:009:45:220
579955999185,4cyclictest630059-21kworker/u8:1+flush-179:009:45:220
579959999085,4cyclictest596876-21kworker/u8:2+events_unbound08:25:151
579959999079,8cyclictest670639-21latency_hist11:50:011
579955999084,4cyclictest600622-21kworker/u8:0+events_unbound08:40:150
579955998984,4cyclictest573959-21kworker/u8:1+events_unbound07:25:250
579959998884,3cyclictest670674-21kworker/u8:0+events_unbound12:00:201
579959998877,4cyclictest567199-21kworker/u8:2+events_unbound07:20:141
579955998882,4cyclictest644706-21kworker/u8:3+flush-179:011:40:200
579959998780,6cyclictest596876-21kworker/u8:2+events_unbound08:25:001
579959998775,9cyclictest1213033-21systemd-journal11:54:591
579955998782,4cyclictest612081-21kworker/u8:2+events_unbound08:55:210
579959998681,4cyclictest620250-21kworker/u8:0+flush-179:009:55:271
579955998680,5cyclictest644706-21kworker/u8:3+flush-179:010:50:250
579959998580,4cyclictest630059-21kworker/u8:1+events_unbound10:25:161
579959998479,4cyclictest636689-21kworker/u8:0+flush-179:010:15:241
579959998474,8cyclictest510-21rs:main1
579959998470,11cyclictest1213033-21systemd-journal07:45:171
579959998272,4cyclictest573959-21kworker/u8:1+events_unbound07:25:201
579955998277,4cyclictest620250-21kworker/u8:0+events_unbound09:35:140
579955998276,4cyclictest605918-21kworker/u8:3+events_unbound08:50:190
579959998176,4cyclictest636740-21kworker/u8:2+events_unbound10:05:231
579959998069,9cyclictest1213033-21systemd-journal07:55:011
579955998074,4cyclictest620250-21kworker/u8:0+flush-179:009:15:230
579955998064,7cyclictest644656-21latency_hist10:30:010
579959997975,3cyclictest584695-21kworker/u8:3+events_unbound07:35:221
579959997968,9cyclictest1213033-21systemd-journal12:00:011
579955997975,3cyclictest657624-21kworker/u8:1+events_unbound11:35:210
579955997974,4cyclictest647813-21kworker/u8:1+flush-179:010:45:270
579955997974,4cyclictest615461-21kworker/u8:0+flush-179:009:00:260
579955997974,4cyclictest615461-21kworker/u8:0+flush-179:009:00:260
579959997869,7cyclictest600622-21kworker/u8:0+events_unbound08:45:001
579959997868,8cyclictest589776-21apt-get07:40:111
579955997874,3cyclictest600622-21kworker/u8:0+flush-179:008:25:270
579955997871,5cyclictest591085-21kworker/u8:0+events_unbound07:45:150
579959997772,4cyclictest612081-21kworker/u8:2+flush-179:009:35:261
579955997772,4cyclictest567199-21kworker/u8:2+flush-179:007:10:200
579955997771,5cyclictest670674-21kworker/u8:0+flush-179:011:59:240
579955997672,3cyclictest676422-21kworker/u8:1+flush-179:012:05:230
579955997671,4cyclictest670674-21kworker/u8:0+flush-179:012:20:240
579964997567,5cyclictest608870-21turbostat08:40:003
579964997567,5cyclictest608870-21turbostat08:40:003
579959997571,3cyclictest612081-21kworker/u8:2+events_unbound09:20:271
579955997570,4cyclictest657624-21kworker/u8:1+events_unbound11:45:260
579959997470,3cyclictest592673-21kworker/u8:1+events_unbound08:05:241
579959997470,3cyclictest567199-21kworker/u8:2+events_unbound07:15:261
579959997461,10cyclictest684974-21apt-get12:35:011
579959997461,10cyclictest684974-21apt-get12:35:001
579959997368,4cyclictest644706-21kworker/u8:3+flush-179:011:15:241
579959997366,5cyclictest599309-21apt-get08:10:101
579955997368,4cyclictest636689-21kworker/u8:0+flush-179:010:10:210
579955997365,6cyclictest586492-21apt-get07:30:110
579959997267,4cyclictest620250-21kworker/u8:0+events_unbound09:25:231
579959997267,4cyclictest620250-21kworker/u8:0+events_unbound09:25:231
579955997260,4cyclictest584695-21kworker/u8:3+events_unbound07:35:190
579959997163,5cyclictest651027-21latency_hist10:50:011
579959997065,4cyclictest657624-21kworker/u8:1+flush-179:011:10:261
579955997065,4cyclictest657624-21kworker/u8:1+flush-179:011:30:260
579955996858,3cyclictest591085-21kworker/u8:0+events_unbound07:50:220
579959996763,3cyclictest657624-21kworker/u8:1+flush-179:011:35:241
579959996763,3cyclictest644706-21kworker/u8:3+flush-179:011:29:561
579959996762,4cyclictest592673-21kworker/u8:1+events_unbound07:55:241
579959996758,7cyclictest647813-21kworker/u8:1+flush-179:010:45:001
579959996662,3cyclictest617320-21kworker/u8:1+flush-179:009:05:271
579959996658,6cyclictest674068-21kworker/u8:2+flush-179:012:05:141
579955996556,6cyclictest591105-21latency_hist07:45:020
579959996358,4cyclictest612081-21kworker/u8:2+events_unbound09:40:121
579964996260,1cyclictest0-21swapper/309:22:593
579959996257,3cyclictest584695-21kworker/u8:3+events_unbound07:30:021
579955996258,3cyclictest670674-21kworker/u8:0+events_unbound12:15:270
579955996257,4cyclictest660550-21kworker/u8:0+flush-179:011:25:530
579959996156,4cyclictest670674-21kworker/u8:0+events_unbound12:35:191
579959996152,7cyclictest630059-21kworker/u8:1+flush-179:010:20:231
579959995955,3cyclictest644706-21kworker/u8:3+flush-179:010:50:211
579964995856,1cyclictest0-21swapper/311:35:583
579955995854,3cyclictest618657-21kworker/u8:3+events_unbound09:10:140
579955995450,3cyclictest620250-21kworker/u8:0+events_unbound09:25:270
579955995450,3cyclictest620250-21kworker/u8:0+events_unbound09:25:270
4577091510,14ptp4l441ktimers/307:10:003
579959994945,3cyclictest609112-21kworker/u8:1+flush-179:008:49:551
57995999491,45cyclictest683213-21/usr/sbin/munin12:25:261
579964994240,1cyclictest0-21swapper/311:05:583
579964994239,2cyclictest0-21swapper/308:22:583
57996499420,41cyclictest0-21swapper/310:47:593
57996499410,40cyclictest0-21swapper/310:05:593
579964993934,4cyclictest613634-21turbostat08:55:003
57996499390,4cyclictest589508-21turbostat07:40:003
57996499390,38cyclictest625315-21turbostat09:30:023
57996499390,38cyclictest625315-21turbostat09:30:013
57995999390,37cyclictest714-21Xorg08:33:591
579964993833,4cyclictest644669-21turbostat10:30:003
579964993831,5cyclictest583046-21turbostat07:20:003
579964993735,1cyclictest0-21swapper/307:14:553
579964993733,3cyclictest641468-21turbostat10:19:593
579964993733,3cyclictest617098-21turbostat09:05:013
579964993733,3cyclictest617098-21turbostat09:05:013
57996499370,2cyclictest595888-21turbostat08:00:013
57996499370,1cyclictest0-21swapper/310:41:593
579961993734,2cyclictest0-21swapper/209:42:592
57996199370,35cyclictest0-21swapper/211:44:582
4577091370,6ptp4l441ktimers/311:54:593
579964993634,1cyclictest0-21swapper/311:29:593
579964993634,1cyclictest0-21swapper/311:04:563
579964993634,1cyclictest0-21swapper/308:29:553
57996499360,35cyclictest0-21swapper/312:09:003
57996499360,35cyclictest0-21swapper/310:02:583
57996499360,35cyclictest0-21swapper/309:19:593
57996499360,35cyclictest0-21swapper/309:11:593
57996499360,1cyclictest0-21swapper/311:14:553
57996499360,1cyclictest0-21swapper/309:07:593
579961993633,2cyclictest0-21swapper/210:36:592
579961993633,2cyclictest0-21swapper/210:24:002
579964993533,1cyclictest0-21swapper/307:49:553
579961993533,1cyclictest0-21swapper/209:04:592
579961993533,1cyclictest0-21swapper/209:04:592
579964993432,1cyclictest0-21swapper/309:58:593
579964993432,1cyclictest0-21swapper/308:41:003
579961993431,2cyclictest0-21swapper/209:59:562
579961993431,2cyclictest0-21swapper/208:50:002
579961993431,2cyclictest0-21swapper/207:12:002
579964993331,1cyclictest0-21swapper/312:29:593
579964993331,1cyclictest0-21swapper/312:04:553
579961993330,2cyclictest0-21swapper/209:32:592
579964993230,1cyclictest0-21swapper/309:51:593
579964993230,1cyclictest0-21swapper/309:51:593
579961993229,2cyclictest0-21swapper/210:57:582
579961993229,2cyclictest0-21swapper/207:31:002
57996199320,2cyclictest0-21swapper/211:17:592
579964993129,1cyclictest0-21swapper/312:33:593
579964993129,1cyclictest0-21swapper/312:33:583
579964993129,1cyclictest0-21swapper/309:34:563
57996499310,30cyclictest669073-21turbostat11:45:003
57996499310,29cyclictest0-21swapper/312:15:143
579961993128,2cyclictest0-21swapper/212:18:582
579964993028,1cyclictest0-21swapper/308:04:553
579964993028,1cyclictest0-21swapper/307:31:593
579964993024,5cyclictest673826-21turbostat11:59:593
579961993028,1cyclictest0-21swapper/209:49:592
579961993028,1cyclictest0-21swapper/209:49:592
579961993027,2cyclictest0-21swapper/211:25:122
579961993027,2cyclictest0-21swapper/208:25:592
579961993027,2cyclictest0-21swapper/208:17:592
579961993027,2cyclictest0-21swapper/207:56:592
579961993027,2cyclictest0-21swapper/207:41:592
4577091300,7ptp4l441ktimers/308:20:003
4577091300,6ptp4l441ktimers/308:50:003
57996499290,27cyclictest0-21swapper/312:35:023
57996499290,27cyclictest0-21swapper/308:55:013
57996499290,1cyclictest0-21swapper/310:30:123
579961992927,1cyclictest0-21swapper/212:05:122
579961992926,2cyclictest0-21swapper/212:35:022
579961992926,2cyclictest0-21swapper/211:35:032
579961992926,2cyclictest0-21swapper/210:08:592
579961992926,2cyclictest0-21swapper/209:24:592
57996199290,28cyclictest0-21swapper/212:00:102
57996199290,27cyclictest0-21swapper/212:20:142
57996199290,27cyclictest0-21swapper/211:50:142
57996199290,27cyclictest0-21swapper/211:45:122
57996199290,27cyclictest0-21swapper/210:10:132
57996199290,27cyclictest0-21swapper/209:50:022
57996199290,27cyclictest0-21swapper/209:50:022
57996199290,27cyclictest0-21swapper/209:17:592
57996199290,27cyclictest0-21swapper/209:08:592
57996199290,27cyclictest0-21swapper/208:20:122
57996199290,27cyclictest0-21swapper/208:05:132
57996199290,27cyclictest0-21swapper/207:35:582
57996199290,26cyclictest0-21swapper/210:55:002
57996199290,1cyclictest0-21swapper/210:00:132
4577091290,7ptp4l441ktimers/309:45:013
4577091290,6ptp4l441ktimers/310:55:003
4577091290,6ptp4l441ktimers/310:25:013
4577091290,6ptp4l441ktimers/309:40:003
4577091290,6ptp4l441ktimers/307:30:013
4577091290,5ptp4l441ktimers/311:50:013
231999290,16rtkit-daemon578839-21df_inode07:05:171
579964992826,1cyclictest0-21swapper/311:15:593
579964992826,1cyclictest0-21swapper/308:33:003
579964992825,2cyclictest0-21swapper/309:45:143
579964992825,2cyclictest0-21swapper/309:45:143
579964992825,2cyclictest0-21swapper/307:50:123
57996499280,2cyclictest0-21swapper/310:35:143
579961992826,1cyclictest0-21swapper/209:14:592
579961992825,2cyclictest0-21swapper/212:30:122
579961992825,2cyclictest0-21swapper/212:30:122
579961992825,2cyclictest0-21swapper/212:28:002
579961992825,2cyclictest0-21swapper/211:55:102
579961992825,2cyclictest0-21swapper/209:25:132
579961992825,2cyclictest0-21swapper/209:25:132
579961992825,2cyclictest0-21swapper/208:50:592
57996199280,2cyclictest0-21swapper/212:10:122
57996199280,27cyclictest0-21swapper/207:25:122
57996199280,26cyclictest0-21swapper/211:20:122
57996199280,26cyclictest0-21swapper/210:18:592
57996199280,26cyclictest0-21swapper/208:58:592
57996199280,26cyclictest0-21swapper/208:10:112
57996199280,26cyclictest0-21swapper/207:15:132
4577091280,6ptp4l441ktimers/311:00:013
4577091280,5ptp4l61490irq/134-eno1-TxRx-012:15:003
4577091280,5ptp4l441ktimers/312:25:013
579961992724,2cyclictest0-21swapper/211:32:592
579955992723,3cyclictest0-21swapper/007:10:020
4577091270,6ptp4l441ktimers/310:15:013
4577091270,5ptp4l61490irq/134-eno1-TxRx-011:25:003
4577091270,5ptp4l441ktimers/307:25:013
4577091270,4ptp4l441ktimers/308:10:013
579964992624,1cyclictest0-21swapper/308:14:593
579961992623,2cyclictest0-21swapper/211:14:582
579961992623,2cyclictest0-21swapper/207:23:582
4577091260,5ptp4l61490irq/134-eno1-TxRx-007:45:023
4577091260,4ptp4l61490irq/134-eno1-TxRx-011:35:003
579961992522,2cyclictest0-21swapper/208:01:002
579961992421,2cyclictest0-21swapper/210:31:592
579961992421,2cyclictest0-21swapper/207:54:552
579961992321,1cyclictest0-21swapper/208:40:122
579961992219,2cyclictest0-21swapper/211:09:552
579961992119,1cyclictest0-21swapper/211:04:592
579961992119,1cyclictest0-21swapper/207:49:002
579961991916,2cyclictest0-21swapper/209:36:592
579961991816,1cyclictest0-21swapper/210:26:592
579961991311,1cyclictest0-21swapper/210:40:132
579961991311,1cyclictest0-21swapper/208:36:002
579961991311,1cyclictest0-21swapper/208:35:592
57996199119,1cyclictest0-21swapper/208:30:122
5799619970,6cyclictest0-21swapper/210:46:082
5799619930,2cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional