You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-11 - 18:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot2.osadl.org (updated Thu Jun 11, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
234498399197129,52cyclictest171ktimers/010:10:210
234498399197129,52cyclictest171ktimers/010:10:210
234498399191173,11cyclictest171ktimers/012:05:360
234498399175148,17cyclictest171ktimers/010:35:180
234498399153143,6cyclictest81-21hwrng09:14:300
23449869914274,61cyclictest461ktimers/311:45:463
234498699141124,13cyclictest81-21hwrng07:25:583
234498699141124,13cyclictest81-21hwrng07:25:583
2344985991403,107cyclictest2436808-21/usr/sbin/munin10:16:152
2344985991403,107cyclictest2436808-21/usr/sbin/munin10:16:152
23449859913662,62cyclictest2344982-21cyclictest12:05:362
2344986991332,61cyclictest2470157-21ntpq11:01:023
234498699131116,11cyclictest81-21hwrng07:30:043
234498599130119,7cyclictest81-21hwrng11:18:252
234498599130118,8cyclictest81-21hwrng11:08:102
234498599130118,8cyclictest81-21hwrng11:08:102
234498599130117,9cyclictest0-21swapper/212:05:022
234498599130117,9cyclictest0-21swapper/212:05:022
234498699129118,6cyclictest81-21hwrng12:11:403
234498699129118,6cyclictest81-21hwrng12:11:403
23449859912836,73cyclictest0-21swapper/209:20:512
234498599128120,5cyclictest81-21hwrng11:26:362
234498699127111,8cyclictest0-21swapper/311:40:383
234498699127111,8cyclictest0-21swapper/311:40:383
234498599126118,5cyclictest81-21hwrng09:13:292
234498599126115,7cyclictest81-21hwrng11:22:302
234498599126115,7cyclictest81-21hwrng11:22:302
23449869912513,100cyclictest2486907-21apt-get11:25:163
234498699125107,13cyclictest81-21hwrng09:01:123
234498599125114,7cyclictest81-21hwrng09:53:252
234498599125114,7cyclictest81-21hwrng09:53:252
2344986991242,53cyclictest2530416-21idleruntime12:25:393
23449859912457,44cyclictest2445224-21/usr/sbin/munin10:26:082
23449859912457,44cyclictest2445224-21/usr/sbin/munin10:26:082
234498599124115,5cyclictest81-21hwrng11:13:172
234498599124114,6cyclictest81-21hwrng10:59:592
234498599124114,6cyclictest81-21hwrng10:59:592
234498699123112,7cyclictest81-21hwrng11:38:533
234498699123112,7cyclictest81-21hwrng11:38:533
2344985991224,107cyclictest2518792-21apt-get12:10:032
2344985991224,107cyclictest2518792-21apt-get12:10:032
2344983991221,64cyclictest2521398-21kworker/u16:19+rpciod12:20:310
234498599121113,5cyclictest81-21hwrng11:53:142
234498599121112,5cyclictest81-21hwrng10:06:442
234498599121107,7cyclictest81-21hwrng07:36:122
234498599121104,11cyclictest81-21hwrng08:22:172
234498599120109,7cyclictest81-21hwrng08:37:392
2344983991203,46cyclictest2361900-21store07:55:020
234498699119107,6cyclictest81-21hwrng10:46:403
234498399119102,5cyclictest81-21hwrng11:45:020
234498399119102,5cyclictest81-21hwrng11:45:020
234498699117104,9cyclictest81-21hwrng09:15:323
23449859911797,16cyclictest81-21hwrng10:12:522
23449859911797,16cyclictest81-21hwrng10:12:522
2344985991173,73cyclictest2409235-21ntpq09:36:002
2344985991173,73cyclictest2409235-21ntpq09:36:002
2344984991169,51cyclictest2380330-21kworker/u16:1+rpciod09:00:231
23449859911599,6cyclictest81-21hwrng10:22:052
234498599115107,6cyclictest81-21hwrng09:49:192
234498599115107,6cyclictest81-21hwrng09:49:192
234498599115100,12cyclictest81-21hwrng12:19:512
234498599115100,12cyclictest81-21hwrng12:19:512
23449859911499,10cyclictest81-21hwrng09:28:512
2344984991145,105cyclictest0-21swapper/110:41:091
2344984991145,105cyclictest0-21swapper/110:41:091
23449839911498,11cyclictest171ktimers/008:35:440
2344986991138,96cyclictest45-21rcuc/309:30:463
234498599113103,6cyclictest81-21hwrng12:28:032
234498699112102,6cyclictest81-21hwrng12:30:053
234498699112102,6cyclictest81-21hwrng12:30:053
23449839911299,9cyclictest81-21hwrng09:40:060
23449839911219,9cyclictest0-21swapper/010:45:570
234498399112103,5cyclictest81-21hwrng11:04:040
23449869911142,21cyclictest0-21swapper/310:00:183
23449839911182,20cyclictest171ktimers/011:21:090
23449839911182,20cyclictest171ktimers/011:21:090
2344983991112,10cyclictest0-21swapper/008:00:020
234498699110102,5cyclictest81-21hwrng09:48:183
234498699110102,5cyclictest81-21hwrng09:48:183
234498699110101,6cyclictest81-21hwrng07:38:153
23449839911065,41cyclictest0-21swapper/007:20:260
23449869910949,16cyclictest0-21swapper/310:00:003
23449869910949,16cyclictest0-21swapper/310:00:003
23449869910860,43cyclictest0-21swapper/311:05:383
23449869910860,43cyclictest0-21swapper/311:05:383
23449839910839,51cyclictest171ktimers/011:10:170
2344983991083,47cyclictest2491633-21ntp_states11:31:010
2344983991081,15cyclictest2499624-21kworker/u16:5+rpciod12:20:020
2344983991081,15cyclictest2499624-21kworker/u16:5+rpciod12:20:020
23449869910799,5cyclictest81-21hwrng10:19:013
23449869910799,5cyclictest81-21hwrng10:19:013
23449859910798,6cyclictest81-21hwrng11:31:432
23449859910798,6cyclictest81-21hwrng07:53:372
23449839910797,5cyclictest81-21hwrng10:32:200
23449839910710,15cyclictest9950irq/123-8000000.ethernet-tx112:00:190
23449839910710,15cyclictest9950irq/123-8000000.ethernet-tx112:00:190
23449869910650,12cyclictest0-21swapper/311:20:273
23449869910650,12cyclictest0-21swapper/311:20:273
23449859910698,5cyclictest81-21hwrng07:18:482
23449859910698,5cyclictest81-21hwrng07:18:482
2344984991065,88cyclictest2494070-21apt-get11:35:111
2344984991065,88cyclictest2494070-21apt-get11:35:111
2344986991052,6cyclictest2506158-21sed11:51:093
23449839910588,6cyclictest81-21hwrng10:26:110
23449839910588,6cyclictest81-21hwrng10:26:110
23449869910393,6cyclictest81-21hwrng10:43:363
23449869910393,6cyclictest81-21hwrng10:43:363
2344986991036,13cyclictest2537945-21ntpq12:35:533
23449859910393,7cyclictest81-21hwrng07:47:282
23449839910395,5cyclictest81-21hwrng12:39:180
23449869910288,11cyclictest81-21hwrng12:09:373
2344986991023,64cyclictest2413781-21sleep09:42:013
2344986991021,60cyclictest2400875-21latency_hist09:25:023
23449859910288,6cyclictest81-21hwrng07:22:532
23449869910194,5cyclictest81-21hwrng08:48:543
23449869910192,5cyclictest81-21hwrng08:50:573
23449859910180,14cyclictest37-21rcuc/208:00:242
23449859910180,14cyclictest37-21rcuc/208:00:242
23449839910189,7cyclictest81-21hwrng11:15:200
23449839910133,58cyclictest2425819-21sendmail-msp10:00:290
23449869910044,4cyclictest2476692-21tune2fs11:10:393
2344985999990,5cyclictest81-21hwrng08:06:552
2344985999990,5cyclictest81-21hwrng08:06:552
2344985999986,10cyclictest81-21hwrng09:04:162
2344986999874,21cyclictest0-21swapper/309:30:023
2344985999890,5cyclictest81-21hwrng07:12:392
2344985999885,7cyclictest81-21hwrng08:42:462
2344983999889,5cyclictest81-21hwrng11:36:510
2344983999889,5cyclictest81-21hwrng11:36:510
234498399981,86cyclictest2394958-21ntpq09:16:020
234498399981,50cyclictest2440839-21diskstats10:20:400
2344986999783,7cyclictest81-21hwrng08:34:343
2344984999786,9cyclictest301ktimers/108:05:591
2344984999786,9cyclictest301ktimers/108:05:591
2344984999780,15cyclictest0-21swapper/109:20:241
2344983999757,34cyclictest0-21swapper/007:30:130
2344986999651,11cyclictest461ktimers/308:00:323
2344986999651,11cyclictest461ktimers/308:00:323
234498699963,82cyclictest2450726-21sh10:35:543
2344986999611,62cyclictest2511656-21sendmail-msp12:00:073
2344986999611,62cyclictest2511656-21sendmail-msp12:00:073
2344983999641,40cyclictest171ktimers/009:25:540
234498699952,5cyclictest2448561-21ntpq10:30:503
2344983999563,22cyclictest0-21swapper/009:06:010
234498399955,79cyclictest2429659-21ssh10:05:120
234498399951,22cyclictest2415771-21fschecks_count09:45:360
234498399951,22cyclictest2415771-21fschecks_count09:45:360
2344986999473,19cyclictest0-21swapper/312:20:173
234498699943,73cyclictest2391362-21ntp_states09:11:013
2344985999486,5cyclictest81-21hwrng08:58:072
2344984999441,45cyclictest301ktimers/110:10:211
2344984999441,45cyclictest301ktimers/110:10:211
234498399948,4cyclictest0-21swapper/007:41:110
2344983999465,14cyclictest0-21swapper/012:10:240
2344983999465,14cyclictest0-21swapper/012:10:240
234498399943,24cyclictest0-21swapper/011:50:580
2344986999385,5cyclictest81-21hwrng07:14:423
2344983999336,42cyclictest0-21swapper/010:00:030
2344983999336,42cyclictest0-21swapper/010:00:030
234498699928,15cyclictest2523551-21/usr/sbin/munin12:15:473
234498699928,15cyclictest2523551-21/usr/sbin/munin12:15:473
2344986999278,6cyclictest81-21hwrng09:39:053
2344986999278,6cyclictest81-21hwrng09:39:053
2344985999284,5cyclictest81-21hwrng07:57:422
2344985999226,39cyclictest0-21swapper/212:30:292
2344985999226,39cyclictest0-21swapper/212:30:292
2344984999286,4cyclictest0-21swapper/112:01:141
2344984999286,4cyclictest0-21swapper/112:01:141
2344984999250,38cyclictest0-21swapper/111:30:391
2344983999282,7cyclictest81-21hwrng09:33:580
234498699915,56cyclictest0-21swapper/308:20:033
234498699915,56cyclictest0-21swapper/308:20:033
2344983999129,40cyclictest0-21swapper/009:00:350
234498399911,30cyclictest2355502-21/usr/sbin/munin07:36:050
234498599903,73cyclictest2450999-21latency_hist10:35:022
234498399906,77cyclictest2511828-21ssh12:00:030
234498399906,77cyclictest2511828-21ssh12:00:030
2344983999046,40cyclictest2352492-21latency_hist07:30:010
2344983999046,40cyclictest2352492-21latency_hist07:30:010
2344983999030,15cyclictest2371633-21/usr/sbin/munin08:20:360
2344985998981,5cyclictest81-21hwrng08:16:082
2344985998981,5cyclictest81-21hwrng08:16:082
2344984998973,13cyclictest0-21swapper/109:51:061
2344984998973,13cyclictest0-21swapper/109:51:061
2344983998963,21cyclictest0-21swapper/010:50:310
234498399894,6cyclictest2467878-21kworker/u16:5+rpciod11:00:020
234498399894,6cyclictest2467878-21kworker/u16:5+rpciod11:00:020
2344985998847,37cyclictest0-21swapper/210:50:592
2344984998882,3cyclictest10650irq/205-8000000.ethernet-rx011:00:141
2344984998881,5cyclictest0-21swapper/111:06:151
2344984998881,5cyclictest0-21swapper/111:06:151
2344984998870,14cyclictest0-21swapper/111:40:351
2344984998870,14cyclictest0-21swapper/111:40:351
2344984998862,14cyclictest0-21swapper/112:20:331
234498399885,16cyclictest2398617-21ntpq09:21:110
2344983998830,26cyclictest1-21systemd07:15:380
2344983998830,26cyclictest1-21systemd07:15:380
2344983998826,27cyclictest2384805-21diskstats08:55:330
234498399873,66cyclictest2530835-21ntpq12:25:550
2344986998677,6cyclictest81-21hwrng08:24:203
2344986998676,6cyclictest81-21hwrng10:51:473
2344985998642,40cyclictest0-21swapper/211:40:112
2344985998642,40cyclictest0-21swapper/211:40:112
2344984998664,12cyclictest0-21swapper/110:00:291
2344984998663,20cyclictest0-21swapper/110:21:041
234498399868,16cyclictest2487973-21ntpq11:25:550
2344983998630,4cyclictest2533483-21aten_r8power_cu12:30:290
2344983998630,4cyclictest2533483-21aten_r8power_cu12:30:290
2344983998629,29cyclictest2382055-21grep08:46:070
2344983998625,52cyclictest0-21swapper/007:10:020
2344983998625,52cyclictest0-21swapper/007:10:020
2344983998622,19cyclictest0-21swapper/008:05:230
2344983998622,19cyclictest0-21swapper/008:05:230
234498399861,39cyclictest2472714-21ssh11:05:250
234498399861,39cyclictest2472714-21ssh11:05:250
2344986998545,19cyclictest653-21containerd10:10:213
2344986998545,19cyclictest653-21containerd10:10:213
2344984998578,5cyclictest301ktimers/110:15:491
2344984998578,5cyclictest301ktimers/110:15:491
2344984998563,18cyclictest0-21swapper/109:20:021
234498499853,31cyclictest29-21rcuc/108:40:171
2344983998542,39cyclictest171ktimers/009:50:250
2344983998542,39cyclictest171ktimers/009:50:250
2344983998531,37cyclictest2344982-21cyclictest09:37:040
2344983998531,37cyclictest2344982-21cyclictest09:37:040
234498399851,36cyclictest0-21swapper/007:45:230
2344986998460,20cyclictest461ktimers/309:05:173
2344985998467,13cyclictest81-21hwrng12:20:522
234498399842,33cyclictest2438075-21ps10:16:080
234498399842,33cyclictest2438075-21ps10:16:080
2344983998417,18cyclictest2382961-21cut08:50:350
234498399841,53cyclictest2379745-21ntpq08:40:550
2344985998349,28cyclictest0-21swapper/207:30:152
2344984998359,20cyclictest301ktimers/107:35:151
234498399832,33cyclictest2346174-21ps07:11:060
234498699824,52cyclictest2418892-21apt-get09:50:163
234498699824,52cyclictest2418892-21apt-get09:50:163
2344985998250,28cyclictest37-21rcuc/211:45:282
2344984998235,37cyclictest0-21swapper/109:45:111
2344984998235,37cyclictest0-21swapper/109:45:111
2344984998172,6cyclictest301ktimers/110:55:181
2344984998172,6cyclictest301ktimers/110:55:181
2344984998164,15cyclictest301ktimers/111:20:491
2344984998164,15cyclictest301ktimers/111:20:491
2344984998159,19cyclictest301ktimers/107:45:151
2344984998152,27cyclictest0-21swapper/111:51:051
2344986998046,23cyclictest56-21kcompactd010:08:173
234498599803,66cyclictest39-21ksoftirqd/212:35:572
2344984998074,4cyclictest0-21swapper/109:10:551
2344983998030,9cyclictest10650irq/205-8000000.ethernet-rx010:40:370
2344983998030,9cyclictest10650irq/205-8000000.ethernet-rx010:40:370
234498599796,59cyclictest2352476-21latency_hist07:30:012
234498599796,59cyclictest2352476-21latency_hist07:30:012
2344983997930,9cyclictest0-21swapper/011:46:130
234498499786,67cyclictest2431353-21awk10:06:141
234498499783,71cyclictest2477154-21ntpq11:10:501
2344986997743,3cyclictest4650ktimers/307:55:243
2344984997757,8cyclictest301ktimers/109:55:101
2344984997757,8cyclictest301ktimers/109:55:101
2344984997750,13cyclictest0-21swapper/109:36:051
2344984997750,13cyclictest0-21swapper/109:36:051
2344984997735,39cyclictest0-21swapper/111:15:351
234498499772,46cyclictest2411467-21sendmail-msp09:40:171
234498699762,54cyclictest2480218-21fschecks_count11:15:373
2344985997639,33cyclictest0-21swapper/209:40:412
2344985997631,38cyclictest0-21swapper/210:04:262
2344984997663,11cyclictest29-21rcuc/107:25:161
2344984997663,11cyclictest29-21rcuc/107:25:161
2344984997637,33cyclictest29-21rcuc/110:46:101
234498699754,42cyclictest0-21swapper/311:55:253
234498699754,42cyclictest0-21swapper/311:55:253
2344984997553,11cyclictest0-21swapper/112:05:161
2344984997535,27cyclictest0-21swapper/107:55:141
2344983997517,19cyclictest0-21swapper/008:25:370
2344983997517,19cyclictest0-21swapper/008:25:370
2344985997460,12cyclictest0-21swapper/209:15:482
2344985997449,17cyclictest0-21swapper/211:00:542
234498499747,63cyclictest31-21ksoftirqd/111:49:121
2344984997466,6cyclictest301ktimers/109:25:561
2344984997459,8cyclictest301ktimers/108:26:001
2344984997459,8cyclictest301ktimers/108:26:001
2344984997419,43cyclictest0-21swapper/110:35:021
234498399741,31cyclictest2368597-21ntpq08:11:010
2344986997349,19cyclictest0-21swapper/310:20:153
2344984997349,21cyclictest0-21swapper/111:55:531
2344984997349,21cyclictest0-21swapper/111:55:531
2344986997239,5cyclictest2443985-21apt-get10:25:253
2344986997239,5cyclictest2443985-21apt-get10:25:253
234498599721,47cyclictest2424690-21sshd-auth09:57:202
234498599721,47cyclictest2424690-21sshd-auth09:57:202
2344984997262,7cyclictest0-21swapper/107:55:021
234498499725,57cyclictest2523971-21mailstats12:16:131
234498499725,57cyclictest2523971-21mailstats12:16:131
2344984997247,21cyclictest0-21swapper/109:30:371
2344984997236,26cyclictest0-21swapper/112:10:151
2344984997236,26cyclictest0-21swapper/112:10:151
2344984997230,34cyclictest2371241-21sendmail-msp08:20:171
2344983997210,28cyclictest2375181-21/usr/sbin/munin08:30:360
2344984997152,17cyclictest0-21swapper/112:31:001
2344984997152,17cyclictest0-21swapper/112:31:001
2344984997151,17cyclictest0-21swapper/107:20:461
2344984997134,20cyclictest2352278-21kworker/u16:25+xprtiod07:30:131
2344983997118,16cyclictest2365241-21/usr/sbin/munin08:01:140
2344983997118,16cyclictest2365241-21/usr/sbin/munin08:01:140
2344984997018,38cyclictest31-21ksoftirqd/112:36:001
234498399701,31cyclictest2370384-21ntpq08:15:510
234498399701,31cyclictest2370384-21ntpq08:15:510
2344985996951,12cyclictest0-21swapper/209:35:032
2344984996945,14cyclictest301ktimers/108:00:321
2344984996945,14cyclictest301ktimers/108:00:321
2344986996841,13cyclictest0-21swapper/308:05:243
2344986996841,13cyclictest0-21swapper/308:05:243
2344985996843,20cyclictest0-21swapper/207:10:012
2344985996843,20cyclictest0-21swapper/207:10:012
234498599681,55cyclictest2451859-21kernelversion10:35:422
2344984996853,8cyclictest301ktimers/110:50:261
2344984996835,20cyclictest301ktimers/108:15:481
2344984996835,20cyclictest301ktimers/108:15:481
2344986996623,9cyclictest0-21swapper/307:45:293
2344984996652,11cyclictest301ktimers/108:45:261
2344984996634,24cyclictest301ktimers/109:05:281
2344984996627,18cyclictest31-21ksoftirqd/110:25:051
2344984996627,18cyclictest31-21ksoftirqd/110:25:051
2344986996553,8cyclictest461ktimers/308:25:523
2344986996553,8cyclictest461ktimers/308:25:523
234498599654,49cyclictest37-21rcuc/211:55:532
234498599654,49cyclictest37-21rcuc/211:55:532
2344984996544,16cyclictest301ktimers/112:25:171
2344984996535,23cyclictest2356243-21sendmail-msp07:40:271
234498499652,36cyclictest10650irq/205-8000000.ethernet-rx010:40:031
2344986996437,19cyclictest0-21swapper/308:40:353
2344985996443,10cyclictest381ktimers/210:45:242
2344986996354,5cyclictest81-21hwrng07:23:553
2344986996345,7cyclictest2490496-21apt-get11:30:113
2344986996334,22cyclictest0-21swapper/307:50:503
2344985996347,11cyclictest381ktimers/209:05:162
2344985996340,13cyclictest381ktimers/211:35:542
2344985996340,13cyclictest381ktimers/211:35:542
2344986996231,27cyclictest0-21swapper/311:00:013
2344986996231,27cyclictest0-21swapper/311:00:013
2344984996229,29cyclictest2487009-21ssh11:25:261
234498499612,23cyclictest2377228-21sshd-auth08:35:311
2344986996023,12cyclictest2346648-21kworker/u16:2+xprtiod08:10:103
2344985996039,18cyclictest0-21swapper/208:30:362
2344984996039,10cyclictest0-21swapper/107:15:361
2344984996039,10cyclictest0-21swapper/107:15:361
2344986995953,3cyclictest10650irq/205-8000000.ethernet-rx007:40:163
2344985995936,19cyclictest2455182-21ssh10:40:422
2344985995936,19cyclictest2455182-21ssh10:40:422
2344984995951,6cyclictest301ktimers/108:55:431
2344985995840,7cyclictest0-21swapper/207:45:022
2344985995836,17cyclictest0-21swapper/208:10:362
2344984995846,8cyclictest301ktimers/107:10:241
2344984995845,9cyclictest0-21swapper/108:30:311
2344984995834,21cyclictest0-21swapper/108:10:111
2344986995744,8cyclictest81-21hwrng08:36:373
2344985995744,9cyclictest0-21swapper/208:45:342
2344984995731,19cyclictest2344982-21cyclictest08:50:191
2344986995641,12cyclictest0-21swapper/308:56:093
2344985995536,16cyclictest0-21swapper/208:50:302
2344986995331,16cyclictest0-21swapper/307:15:363
2344986995331,16cyclictest0-21swapper/307:15:363
2344985995234,15cyclictest0-21swapper/208:25:352
2344985995234,15cyclictest0-21swapper/208:25:352
2344984994932,9cyclictest0-21swapper/107:10:011
2344984994932,9cyclictest0-21swapper/107:10:011
2344986994531,6cyclictest2344996-21latency_hist07:10:013
2344986994531,6cyclictest2344996-21latency_hist07:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional