You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 07:58
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Mon May 25, 2026 00:55:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
288839981814,727cyclictest0-21swapper/521:39:385
28879997200,42cyclictest0-21swapper/123:17:491
288839971477,634cyclictest0-21swapper/523:40:195
2888399698104,591cyclictest0-21swapper/523:59:145
288849968041,131cyclictest0-21swapper/600:42:116
288829966329,465cyclictest0-21swapper/423:42:244
288839966224,550cyclictest0-21swapper/521:44:165
288839966224,550cyclictest0-21swapper/521:44:165
288799965825,380cyclictest0-21swapper/121:40:001
2887999635124,338cyclictest0-21swapper/122:52:561
2887999635124,338cyclictest0-21swapper/122:52:561
288789963235,338cyclictest0-21swapper/021:42:170
288789963235,338cyclictest0-21swapper/021:42:170
288789963182,126cyclictest0-21swapper/021:38:120
288789962834,337cyclictest10033-21ssh00:26:200
2888099627120,126cyclictest11126-21ssh21:13:182
288859961925,126cyclictest0-21swapper/721:52:427
2888299613105,126cyclictest0-21swapper/422:32:294
28880995951,126cyclictest0-21swapper/223:40:222
28879995940,42cyclictest0-21swapper/121:53:451
288839959037,295cyclictest0-21swapper/500:42:325
288829958738,382cyclictest0-21swapper/421:51:114
288859958635,126cyclictest0-21swapper/723:17:417
288809958473,508cyclictest0-21swapper/221:41:512
288809958473,508cyclictest0-21swapper/221:41:512
288839958376,340cyclictest0-21swapper/521:54:355
288839958230,464cyclictest0-21swapper/500:21:105
288789958170,126cyclictest0-21swapper/000:16:180
2887899580114,126cyclictest1623-21ssh22:52:560
2887899580114,126cyclictest1623-21ssh22:52:560
288789957929,126cyclictest24651-21runrttasks23:42:280
2888299577152,422cyclictest0-21swapper/400:26:204
288809957521,382cyclictest0-21swapper/221:38:372
288809957462,296cyclictest0-21swapper/223:57:372
288789957120,84cyclictest0-21swapper/021:53:530
288789957117,381cyclictest0-21swapper/021:27:190
2888399567104,339cyclictest10570-21ssh21:12:335
288799956717,296cyclictest0-21swapper/122:35:031
28883995531,508cyclictest0-21swapper/522:30:555
2887999549124,337cyclictest0-21swapper/123:42:281
2888499547503,41cyclictest641ktimersoftd/600:21:076
2888599545119,423cyclictest26375-1kworker/7:2H21:39:267
288839954535,507cyclictest0-21swapper/522:52:155
288839954535,507cyclictest0-21swapper/522:52:155
288859954335,169cyclictest0-21swapper/723:41:077
288839954231,507cyclictest0-21swapper/523:04:025
288799954232,253cyclictest0-21swapper/123:56:351
288789954134,43cyclictest121rcu_preempt22:35:000
288839953932,3cyclictest0-21swapper/500:10:595
288839953928,338cyclictest0-21swapper/523:17:215
288809953932,126cyclictest13293-21sh00:10:572
2887999539112,212cyclictest18491-21ssh23:37:261
288839953828,507cyclictest0-21swapper/523:11:525
2888299538116,84cyclictest0-21swapper/423:40:054
2888199538196,339cyclictest4094-21diskmemload22:33:113
288839953626,507cyclictest0-21swapper/521:26:515
2888399535112,126cyclictest0-21swapper/523:45:285
288839953428,79cyclictest0-21swapper/500:31:175
288839953425,506cyclictest0-21swapper/523:51:025
2887899534413,37cyclictest101ktimersoftd/000:00:050
288849953266,211cyclictest28876-21cyclictest22:52:566
288849953266,211cyclictest28876-21cyclictest22:52:566
288819953223,126cyclictest0-21swapper/321:41:513
288819953223,126cyclictest0-21swapper/321:41:513
288829953024,295cyclictest0-21swapper/423:02:164
288829953023,340cyclictest21298-21chrt23:59:354
288799953023,254cyclictest0-21swapper/121:42:211
288799953023,254cyclictest0-21swapper/121:42:211
288849952959,296cyclictest0-21swapper/623:58:596
2888099527100,214cyclictest4094-21diskmemload23:39:102
288809952517,84cyclictest121rcu_preempt23:18:242
288789952519,126cyclictest28876-21cyclictest21:13:070
288829952214,169cyclictest0-21swapper/421:40:584
288829952214,169cyclictest0-21swapper/421:40:584
288809952215,84cyclictest0-21swapper/221:26:442
288819952012,169cyclictest0-21swapper/322:51:403
288819952012,169cyclictest0-21swapper/322:51:403
288819951693,126cyclictest0-21swapper/321:36:193
288859951589,84cyclictest8202-21ssh00:43:227
288829951490,84cyclictest0-21swapper/423:16:274
2888099514427,85cyclictest311rcuc/222:31:062
288849951285,211cyclictest9319-21ssh23:50:476
28882995102,377cyclictest0-21swapper/400:40:504
28884995090,83cyclictest0-21swapper/623:01:576
288849950837,2cyclictest0-21swapper/600:15:586
28883995071,422cyclictest0-21swapper/523:39:505
28880995070,83cyclictest0-21swapper/223:01:572
28879995071,126cyclictest0-21swapper/123:50:581
2887899507124,380cyclictest18664-21ssh23:02:080
288799950541,298cyclictest0-21swapper/100:16:301
288789950337,84cyclictest121rcu_preempt23:18:370
288829950235,253cyclictest0-21swapper/421:37:504
2888299501163,123cyclictest121rcu_preempt21:13:024
288819950177,208cyclictest13008-21ssh23:17:173
2888599499114,211cyclictest73-21ksoftirqd/723:57:237
288829949834,380cyclictest1191-21dpkg21:45:104
288789949876,84cyclictest0-21swapper/000:30:250
288849949732,210cyclictest0-21swapper/621:52:186
2888199494110,381cyclictest0-21swapper/321:26:513
288789949223,338cyclictest0-21swapper/000:20:280
288799949127,254cyclictest0-21swapper/121:12:481
2887999491109,380cyclictest241ktimersoftd/100:26:201
288789949124,210cyclictest20685-21ssh23:40:030
2888499490105,169cyclictest12348-21ssh23:16:256
2888499489107,380cyclictest0-21swapper/621:26:516
2888499489104,211cyclictest0-21swapper/623:37:466
288809948919,211cyclictest33-21ksoftirqd/200:43:122
2888499487102,126cyclictest0-21swapper/622:33:226
288849948664,84cyclictest0-21swapper/600:30:256
288829948661,338cyclictest0-21swapper/400:16:114
288849948461,126cyclictest0-21swapper/621:38:186
2887899484105,295cyclictest0-21swapper/023:55:480
2888499479307,84cyclictest0-21swapper/623:41:466
288809947914,47cyclictest0-21swapper/223:51:002
288819947612,296cyclictest0-21swapper/321:12:423
288859947389,381cyclictest31103-21ssh21:41:517
288859947389,381cyclictest31103-21ssh21:41:517
288809947345,211cyclictest1553-21runrttasks23:45:282
288799947351,338cyclictest1651-21ntpd23:02:231
28881994724,84cyclictest41-21ksoftirqd/323:40:353
28881994704,126cyclictest0-21swapper/321:54:543
288819946841,84cyclictest0-21swapper/323:39:303
288819946740,211cyclictest41-21ksoftirqd/300:11:093
288799946681,169cyclictest25-21ksoftirqd/100:10:571
2887999466125,338cyclictest0-21swapper/100:40:501
28881994650,84cyclictest0-21swapper/321:55:053
28880994640,84cyclictest17208-21ssh21:55:052
288859946378,211cyclictest18999-21sh00:30:457
2888099463164,127cyclictest121rcu_preempt00:26:202
288789946380,253cyclictest0-21swapper/023:45:330
288859946277,211cyclictest73-21ksoftirqd/700:16:307
2888599462379,3cyclictest721ktimersoftd/722:31:197
288829946239,255cyclictest832-21ssh22:51:584
288829946239,255cyclictest832-21ssh22:51:584
288829946236,302cyclictest0-21swapper/421:55:034
2888099462163,126cyclictest121rcu_preempt21:52:382
288799946240,256cyclictest0-21swapper/100:00:131
288859946139,255cyclictest0-21swapper/700:00:137
288839946182,84cyclictest0-21swapper/521:45:135
2888299461120,338cyclictest0-21swapper/422:27:164
2887999461121,338cyclictest0-21swapper/121:45:101
2888399460120,84cyclictest0-21swapper/522:27:145
288859945980,295cyclictest0-21swapper/721:12:297
2888599456116,338cyclictest0-21swapper/723:38:547
2888099456411,42cyclictest311rcuc/200:20:502
2887899456115,338cyclictest0-21swapper/000:05:220
2887999454114,338cyclictest0-21swapper/123:45:311
288859945369,381cyclictest0-21swapper/700:26:207
2887899453113,338cyclictest0-21swapper/000:42:370
2887899451110,338cyclictest0-21swapper/023:12:050
288839945028,255cyclictest0-21swapper/500:00:135
288829944927,255cyclictest22025-21ssh00:00:094
288829944668,84cyclictest0-21swapper/423:12:424
288819944619,254cyclictest0-21swapper/321:45:273
2887999445362,3cyclictest231rcuc/100:30:451
288789944529,119cyclictest0-21swapper/023:51:120
288829944419,380cyclictest0-21swapper/421:27:024
2888399443102,339cyclictest0-21swapper/500:25:555
288819944318,294cyclictest0-21swapper/323:56:363
2888199442101,338cyclictest0-21swapper/300:26:203
288829944114,338cyclictest0-21swapper/423:45:344
288819944118,254cyclictest0-21swapper/300:21:093
288859944017,257cyclictest0-21swapper/721:45:137
288849944056,211cyclictest0-21swapper/621:45:036
2888199440141,126cyclictest121rcu_preempt23:50:543
2888299439100,84cyclictest0-21swapper/400:21:204
288799943959,296cyclictest0-21swapper/121:55:101
288789943614,177cyclictest0-21swapper/022:35:100
288849943412,84cyclictest4783-21kworker/u16:221:55:106
288829943451,380cyclictest0-21swapper/400:11:124
288819943092,83cyclictest704-21nscd22:27:163
28881994240,84cyclictest18469-21ssh23:01:573
28880994241,3cyclictest0-21swapper/222:53:262
28880994241,3cyclictest0-21swapper/222:53:262
2888499423124,126cyclictest32485-21ssh21:44:026
2888499423124,126cyclictest32485-21ssh21:44:026
28884994231,126cyclictest65-21ksoftirqd/623:12:256
288839942181,337cyclictest0-21swapper/521:55:085
288809942183,84cyclictest0-21swapper/200:00:102
288799942179,339cyclictest0-21swapper/123:12:071
288789942182,84cyclictest0-21swapper/021:45:100
288809942080,84cyclictest0-21swapper/222:35:112
288859941978,338cyclictest0-21swapper/721:27:017
2888499419121,126cyclictest121rcu_preempt21:13:236
288839941939,254cyclictest28876-21cyclictest00:15:565
288819941980,84cyclictest0-21swapper/300:00:033
288789941980,84cyclictest0-21swapper/021:55:020
288849941877,338cyclictest0-21swapper/600:11:156
288829941878,338cyclictest0-21swapper/423:50:474
288819941876,126cyclictest0-21swapper/323:12:343
288859941777,338cyclictest0-21swapper/723:50:597
288829941737,254cyclictest0-21swapper/400:31:104
288809941778,84cyclictest0-21swapper/221:45:062
288799941738,212cyclictest0-21swapper/122:27:151
288799941737,254cyclictest0-21swapper/100:20:441
288859941573,339cyclictest0-21swapper/723:45:297
288819941574,84cyclictest0-21swapper/300:05:203
2888499414115,126cyclictest121rcu_preempt23:45:296
288819941475,84cyclictest0-21swapper/300:31:123
288819941475,84cyclictest0-21swapper/300:16:303
288799941472,339cyclictest0-21swapper/122:35:081
2887899414115,126cyclictest0-21swapper/000:10:580
288859941333,84cyclictest0-21swapper/721:55:047
288839941373,338cyclictest0-21swapper/522:35:115
288809941374,84cyclictest0-21swapper/223:12:072
288819941273,84cyclictest0-21swapper/300:43:263
288849941170,126cyclictest0-21swapper/600:25:486
288799941069,338cyclictest0-21swapper/121:27:011
288859940968,338cyclictest0-21swapper/723:01:487
288809940769,84cyclictest0-21swapper/200:31:162
288799940626,212cyclictest0-21swapper/100:05:211
288859940564,338cyclictest0-21swapper/722:52:337
288859940564,338cyclictest0-21swapper/722:52:337
2888499404106,126cyclictest0-21swapper/600:05:226
288829940324,295cyclictest0-21swapper/400:05:214
288819940364,84cyclictest0-21swapper/322:35:113
288829939959,338cyclictest0-21swapper/422:35:094
288809939717,49cyclictest121rcu_preempt22:27:162
288859939554,338cyclictest0-21swapper/700:20:587
288849939295,127cyclictest22084-21apt-get00:00:136
288849939152,85cyclictest0-21swapper/622:35:116
28880993845,290cyclictest0-21swapper/200:16:262
288839937740,253cyclictest0-21swapper/500:05:215
288859937436,3cyclictest0-21swapper/700:11:067
288819937333,43cyclictest0-21swapper/323:45:203
288789937335,84cyclictest0-21swapper/022:27:140
288859937031,84cyclictest0-21swapper/723:12:377
288859936827,84cyclictest0-21swapper/722:27:167
288859936222,84cyclictest0-21swapper/722:35:117
288849935414,85cyclictest0-21swapper/622:27:166
288859935314,84cyclictest0-21swapper/700:05:227
288809933438,84cyclictest0-21swapper/200:05:212
1346921430,4sleep62888499cyclictest21:15:136
26588212095,6sleep50-21swapper/519:05:165
2820428944,41sleep00-21swapper/019:06:310
3722820,0sleep00-21swapper/021:25:000
2827328143,8sleep10-21swapper/119:07:281
2815028151,27sleep30-21swapper/319:05:463
2840028051,26sleep70-21swapper/719:09:187
2814228050,27sleep40-21swapper/419:05:394
2838427949,27sleep20-21swapper/219:09:052
2546927747,26sleep60-21swapper/619:05:016
283392740,0sleep50-21swapper/523:25:475
302462710,0sleep30-21swapper/320:05:133
213102700,0sleep40-21swapper/421:19:124
28881995422,29cyclictest4550-21sh22:55:123
2888299511,47cyclictest22017-21ssh23:05:014
2888099516,38cyclictest33-21ksoftirqd/223:05:022
28882995019,28cyclictest4256-21ssh22:55:094
2888299481,27cyclictest0-21swapper/423:20:544
159142480,4sleep115918-21unixbench-2d21:15:321
28880994513,4cyclictest15741-21ssh23:20:092
2888499431,14cyclictest121rcu_preempt23:05:046
2888199423,28cyclictest41-21ksoftirqd/323:05:053
2888099428,24cyclictest33-21ksoftirqd/221:15:132
2887899419,29cyclictest0-21swapper/023:05:090
2888099392,33cyclictest4537-21tr22:55:122
28885993729,5cyclictest4801-21expr22:55:127
28885993727,3cyclictest121rcu_preempt23:05:087
2888499371,32cyclictest4570-21sh22:55:126
2888199377,19cyclictest41-21ksoftirqd/321:15:163
28878993718,16cyclictest0-21swapper/021:15:120
2888599361,31cyclictest13371-21sh21:15:087
2888199362,30cyclictest0-21swapper/323:20:063
2887899362,29cyclictest4094-21diskmemload23:20:110
28879993531,2cyclictest0-21swapper/122:55:211
28879993529,4cyclictest0-21swapper/123:05:151
28878993528,4cyclictest0-21swapper/022:55:090
2888499341,30cyclictest0-21swapper/623:20:086
28885993325,5cyclictest16912-21head23:20:177
2888399331,2cyclictest0-21swapper/523:20:025
2888399321,3cyclictest0-21swapper/523:05:195
2888399321,29cyclictest0-21swapper/522:55:215
28882993010,19cyclictest0-21swapper/419:56:074
28883992923,4cyclictest13787-21expr21:15:145
28879992923,4cyclictest16509-21sed23:20:141
28878992910,18cyclictest0-21swapper/019:13:010
28881992810,0cyclictest0-21swapper/319:13:003
28885992727,0cyclictest0-21swapper/719:56:277
28885992710,17cyclictest0-21swapper/719:12:547
28884992727,0cyclictest0-21swapper/619:56:036
28883992727,0cyclictest0-21swapper/519:12:535
28883992710,17cyclictest0-21swapper/519:56:085
28882992727,0cyclictest0-21swapper/419:13:024
28881992727,0cyclictest0-21swapper/319:56:023
28880992727,0cyclictest0-21swapper/219:12:562
28880992710,17cyclictest0-21swapper/219:56:242
28879992710,0cyclictest0-21swapper/119:56:081
28879992710,0cyclictest0-21swapper/119:12:531
28878992727,0cyclictest0-21swapper/019:56:080
28882992612,10cyclictest28876-21cyclictest22:45:394
262442260,0sleep40-21swapper/423:25:144
18752260,1sleep10-21swapper/122:03:101
28884992525,0cyclictest0-21swapper/619:13:006
2888299256,13cyclictest3528-21tr19:20:114
2887899250,23cyclictest0-21swapper/023:25:110
2888299242,17cyclictest0-21swapper/422:40:014
191412230,1sleep60-21swapper/619:45:136
2888299221,16cyclictest0-21swapper/420:13:114
2888299220,18cyclictest0-21swapper/422:02:424
2888299220,18cyclictest0-21swapper/419:18:284
2888299220,17cyclictest0-21swapper/421:32:034
2888299220,16cyclictest10039-21ssh22:22:524
2888299220,16cyclictest0-21swapper/422:18:404
2888299220,16cyclictest0-21swapper/419:54:524
2888299220,16cyclictest0-21swapper/419:54:524
2888299210,17cyclictest9687-21rm23:33:184
2888299210,17cyclictest0-21swapper/419:35:134
2888299210,15cyclictest0-21swapper/419:45:114
28881992119,1cyclictest0-21swapper/319:15:183
2887899210,20cyclictest0-21swapper/019:15:180
134852210,1sleep513487-21fschecks_count20:30:145
28885992018,1cyclictest0-21swapper/720:35:027
2888299201,15cyclictest0-21swapper/420:36:424
2888299201,14cyclictest0-21swapper/422:06:474
2888299200,16cyclictest19899-21ssh22:11:334
2888299200,15cyclictest30756-21ssh21:23:384
2888299200,15cyclictest28019-21ssh00:35:254
2888299200,15cyclictest0-21swapper/420:20:004
2888299200,14cyclictest15369-21grep19:40:114
2887999200,1cyclictest0-21swapper/120:35:191
42722190,0sleep20-21swapper/220:15:132
2888299190,15cyclictest24365-21chrt20:47:514
2888299190,15cyclictest0-21swapper/420:56:524
2888299190,15cyclictest0-21swapper/420:25:124
2888299190,14cyclictest0-21swapper/421:07:584
2888299190,14cyclictest0-21swapper/420:44:234
2888299190,12cyclictest0-21swapper/420:33:294
28880991917,1cyclictest0-21swapper/219:15:182
2888299186,11cyclictest31007-21grep21:00:124
2888299180,13cyclictest32590-21chrt20:06:424
2888299180,13cyclictest0-21swapper/420:23:084
2888299180,13cyclictest0-21swapper/420:23:084
2888299180,12cyclictest28220-21grep20:00:134
2888299170,11cyclictest0-21swapper/419:30:224
2888299168,7cyclictest25143-21grep20:50:124
2888299160,10cyclictest0-21swapper/419:29:054
2888199163,8cyclictest41-21ksoftirqd/319:50:003
2888199152,9cyclictest41-21ksoftirqd/322:05:003
2888199151,8cyclictest41-21ksoftirqd/320:10:113
2888099153,7cyclictest121rcu_preempt22:50:012
28883991412,1cyclictest30420-21ssh00:37:025
2888199144,6cyclictest41-21ksoftirqd/321:31:453
2888199143,7cyclictest2885-21apt-get23:30:133
2888199141,8cyclictest41-21ksoftirqd/320:35:013
2888099146,5cyclictest33-21ksoftirqd/223:30:002
2887899144,9cyclictest0-21swapper/023:35:010
2887899143,6cyclictest9-21ksoftirqd/022:24:500
2887899143,6cyclictest9-21ksoftirqd/021:05:150
28885991311,1cyclictest3211-21ssh22:04:037
28885991311,1cyclictest26805-21ssh00:35:187
28885991311,1cyclictest26694-21ssh22:49:157
28885991311,1cyclictest0-21swapper/723:27:257
28885991311,1cyclictest0-21swapper/722:10:237
28885991310,2cyclictest9932-21tr20:25:127
28885991310,2cyclictest5630-21tr22:05:127
28885991310,2cyclictest15546-21cut19:40:117
2888499139,3cyclictest24152-21head20:45:236
28884991311,1cyclictest29418-21mailstats22:00:196
28884991311,1cyclictest13119-21expr20:30:126
28884991311,1cyclictest121722sleep622:08:136
28884991311,1cyclictest0-21swapper/620:35:186
28883991311,1cyclictest303-21systemd-journal19:40:005
28883991311,1cyclictest30118-21ssh21:23:135
2888199133,6cyclictest41-21ksoftirqd/323:25:203
2888199133,6cyclictest41-21ksoftirqd/300:38:513
2888199133,5cyclictest41-21ksoftirqd/322:17:373
2888199133,5cyclictest41-21ksoftirqd/320:00:133
2888199131,6cyclictest121rcu_preempt21:20:273
28881991311,1cyclictest6165-21fschecks_count22:05:143
28881991311,1cyclictest0-21swapper/322:43:133
28881991310,2cyclictest401ktimersoftd/320:40:153
28881991310,2cyclictest23895-21chrt22:14:103
28881991310,2cyclictest16113-21tr20:35:133
2888099136,2cyclictest33-21ksoftirqd/200:40:002
28880991311,1cyclictest2770-21ssh23:30:082
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional