You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-30 - 06:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Sat May 30, 2026 00:57:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
76409991873,761cyclictest0-21swapper/600:15:056
76409978521,761cyclictest6603-21chrt23:55:146
763699772138,380cyclictest7632-21cyclictest23:55:142
76409975778,676cyclictest0-21swapper/621:34:226
763499748194,507cyclictest7632-21cyclictest00:12:390
76409967277,84cyclictest0-21swapper/600:14:226
76349966876,127cyclictest0-21swapper/022:40:120
764099666115,126cyclictest0-21swapper/622:40:096
76409965922,169cyclictest0-21swapper/623:04:206
76409965922,169cyclictest0-21swapper/623:04:206
76409965720,635cyclictest0-21swapper/623:34:046
76399965155,296cyclictest0-21swapper/522:04:195
76399965155,296cyclictest0-21swapper/522:04:195
76399965155,296cyclictest0-21swapper/522:04:195
7635996438,126cyclictest0-21swapper/123:52:371
76349964148,126cyclictest0-21swapper/022:04:400
76349964148,126cyclictest0-21swapper/022:04:400
76349964148,126cyclictest0-21swapper/022:04:390
76409962937,589cyclictest118052chrt22:37:006
76409962937,589cyclictest118052chrt22:37:006
76369962979,126cyclictest4897-1kworker/u17:000:38:082
76369962772,343cyclictest121rcu_preempt23:43:512
76369962772,343cyclictest121rcu_preempt23:43:512
76349962532,84cyclictest0-21swapper/023:32:540
76399962370,380cyclictest0-21swapper/500:15:005
763599622115,126cyclictest0-21swapper/123:41:201
763599622115,126cyclictest0-21swapper/123:41:201
763899621282,337cyclictest64450irq/122-eth000:12:104
76359962130,84cyclictest0-21swapper/121:32:491
76389961764,379cyclictest698-21ptp4l23:11:304
76369961723,84cyclictest0-21swapper/222:15:272
76369961723,84cyclictest0-21swapper/222:15:272
76409961519,593cyclictest0-21swapper/621:40:266
763699615106,126cyclictest0-21swapper/222:46:572
764099608309,126cyclictest0-21swapper/622:12:526
764099608309,126cyclictest0-21swapper/622:12:526
764099608309,126cyclictest0-21swapper/622:12:526
76369960818,379cyclictest121rcu_preempt22:00:232
76369960818,379cyclictest121rcu_preempt22:00:232
76369960818,379cyclictest121rcu_preempt22:00:232
76369960799,126cyclictest0-21swapper/222:05:132
76419960496,126cyclictest0-21swapper/722:05:137
7640996048,169cyclictest0-21swapper/622:55:596
76409960310,126cyclictest0-21swapper/623:05:386
7637995950,84cyclictest0-21swapper/322:36:213
7637995950,84cyclictest0-21swapper/322:36:213
7637995940,84cyclictest0-21swapper/321:46:093
763699590124,211cyclictest0-21swapper/221:42:542
763799588204,296cyclictest0-21swapper/321:45:023
76409958772,126cyclictest0-21swapper/623:42:116
76409958772,126cyclictest0-21swapper/623:42:116
76349958737,126cyclictest121rcu_preempt21:42:480
763899585117,337cyclictest0-21swapper/422:11:374
763899585117,337cyclictest0-21swapper/422:11:374
763899585117,337cyclictest0-21swapper/422:11:374
76419958472,381cyclictest0-21swapper/723:42:597
76419958472,381cyclictest0-21swapper/723:42:597
7638995842,3cyclictest0-21swapper/400:37:084
763699584541,41cyclictest311rcuc/221:33:402
76409958271,126cyclictest0-21swapper/622:15:116
76409958271,126cyclictest0-21swapper/622:15:116
76359958170,338cyclictest0-21swapper/122:02:161
76359958170,338cyclictest0-21swapper/122:02:161
76359958170,338cyclictest0-21swapper/122:02:161
76359958170,296cyclictest0-21swapper/122:15:031
76359958170,296cyclictest0-21swapper/122:15:031
76359958170,296cyclictest0-21swapper/122:15:031
763799579110,295cyclictest0-21swapper/323:13:283
76369957925,381cyclictest0-21swapper/223:14:292
76419957867,338cyclictest0-21swapper/722:15:097
76419957867,338cyclictest0-21swapper/722:15:097
764199578238,338cyclictest15860-21rm22:02:107
764199578238,338cyclictest15860-21rm22:02:107
764199578238,338cyclictest15860-21rm22:02:097
763499576320,84cyclictest0-21swapper/022:57:190
76409957568,164cyclictest26998-21runrttasks22:46:116
76359957569,84cyclictest121rcu_preempt23:00:471
76359957569,84cyclictest121rcu_preempt23:00:471
76389957464,507cyclictest0-21swapper/422:57:194
763899574106,295cyclictest0-21swapper/423:34:534
76379957466,84cyclictest0-21swapper/323:34:533
76349957426,83cyclictest0-21swapper/022:20:410
76409957320,550cyclictest0-21swapper/623:11:206
76369957323,127cyclictest4007-21runrttasks23:15:102
763699572486,84cyclictest311rcuc/223:33:242
76359957161,337cyclictest0-21swapper/122:55:581
763999570149,41cyclictest187-21md0_raid122:15:435
763999570149,41cyclictest187-21md0_raid122:15:435
76369956815,126cyclictest0-21swapper/222:38:372
76369956815,126cyclictest0-21swapper/222:38:372
76349956814,338cyclictest0-21swapper/023:52:000
764199566143,127cyclictest121rcu_preempt22:56:147
76389956617,380cyclictest0-21swapper/422:01:094
76389956617,380cyclictest0-21swapper/422:01:094
76389956617,380cyclictest0-21swapper/422:01:094
76349956656,380cyclictest0-21swapper/022:14:050
76349956656,380cyclictest0-21swapper/022:14:050
76349956656,380cyclictest0-21swapper/022:14:050
76389956410,126cyclictest0-21swapper/422:15:404
76389956410,126cyclictest0-21swapper/422:15:404
7640995639,465cyclictest0-21swapper/600:38:066
76349956352,338cyclictest0-21swapper/022:38:030
76349956352,338cyclictest0-21swapper/022:38:020
763899560220,338cyclictest64450irq/122-eth022:38:294
763899560220,338cyclictest64450irq/122-eth022:38:294
7638995574,464cyclictest0-21swapper/423:45:154
76369955750,338cyclictest0-21swapper/223:51:042
76399955385,256cyclictest0-21swapper/521:44:585
76369955244,126cyclictest0-21swapper/222:57:192
76419955140,337cyclictest1553-21runrttasks21:33:157
76399955182,338cyclictest0-21swapper/523:03:275
76399955182,338cyclictest0-21swapper/523:03:275
7638995510,84cyclictest0-21swapper/421:44:454
7634995501,383cyclictest0-21swapper/000:37:080
76419954779,465cyclictest0-21swapper/700:12:497
763599547120,382cyclictest0-21swapper/121:44:591
763499547375,84cyclictest0-21swapper/000:20:320
76419954637,212cyclictest0-21swapper/723:45:057
76379954682,126cyclictest121rcu_preempt22:01:093
76379954682,126cyclictest121rcu_preempt22:01:093
76379954682,126cyclictest121rcu_preempt22:01:093
763599546247,296cyclictest15463-21diskmemload22:38:361
763599546247,296cyclictest15463-21diskmemload22:38:361
76409954536,506cyclictest0-21swapper/623:52:016
76379954534,380cyclictest0-21swapper/323:52:403
763999544206,126cyclictest31718-21kworker/u16:022:45:495
76349954479,126cyclictest0-21swapper/022:50:030
764199543202,338cyclictest713-21in:imuxsock23:11:217
76389954373,257cyclictest0-21swapper/422:49:434
76369954335,84cyclictest0-21swapper/223:45:272
763699543117,338cyclictest14897-21kworker/u16:223:03:552
763699543117,338cyclictest14897-21kworker/u16:223:03:552
76359954336,84cyclictest121rcu_preempt21:45:261
76389954172,338cyclictest0-21swapper/421:34:344
76359954177,42cyclictest121rcu_preempt23:11:271
763899540203,92cyclictest0-21swapper/423:04:134
763899540203,92cyclictest0-21swapper/423:04:134
76389953769,84cyclictest0-21swapper/421:45:464
76359953729,127cyclictest0-21swapper/123:55:111
76419953625,508cyclictest0-21swapper/723:34:237
76359953624,381cyclictest0-21swapper/100:37:341
76369953468,126cyclictest0-21swapper/222:14:312
76369953468,126cyclictest0-21swapper/222:14:312
76369953468,126cyclictest0-21swapper/222:14:312
764099533111,84cyclictest0-21swapper/622:02:446
764099533111,84cyclictest0-21swapper/622:02:446
764099533111,84cyclictest0-21swapper/622:02:446
76399953120,296cyclictest0-21swapper/500:38:065
76349953123,126cyclictest0-21swapper/023:43:130
76349953123,126cyclictest0-21swapper/023:43:130
764099530276,84cyclictest641ktimersoftd/623:45:056
76369953063,126cyclictest10025-21taskset00:14:402
763799529103,338cyclictest0-21swapper/323:04:023
763799529103,338cyclictest0-21swapper/323:04:023
76359952922,84cyclictest0-21swapper/122:48:011
76399952818,253cyclictest0-21swapper/522:37:545
76399952818,253cyclictest0-21swapper/522:37:545
76369952823,259cyclictest0-21swapper/222:50:082
76359952758,211cyclictest0-21swapper/100:00:201
764199526142,126cyclictest121rcu_preempt22:37:377
764199526142,126cyclictest121rcu_preempt22:37:367
764099526272,252cyclictest641ktimersoftd/621:45:346
76379952699,211cyclictest0-21swapper/300:11:213
76359952657,211cyclictest0-21swapper/123:32:331
76349952615,338cyclictest0-21swapper/023:15:020
76399952562,331cyclictest0-21swapper/523:12:155
76359952313,507cyclictest0-21swapper/122:05:131
76419952254,465cyclictest0-21swapper/723:00:347
76419952254,465cyclictest0-21swapper/723:00:337
76419952194,211cyclictest73-21ksoftirqd/721:40:327
76409952110,168cyclictest4003-21taskset23:15:096
76349952119,79cyclictest121rcu_preempt23:03:520
76349952119,79cyclictest121rcu_preempt23:03:520
76399951812,124cyclictest0-21swapper/522:57:195
76419951793,168cyclictest22860-1kworker/7:2H00:20:317
7634995166,507cyclictest0-21swapper/022:05:130
76379951587,169cyclictest41-21ksoftirqd/300:37:343
76349951450,126cyclictest0-21swapper/000:00:240
7639995090,84cyclictest0-21swapper/521:46:095
764199507254,42cyclictest0-21swapper/722:13:587
764199507254,42cyclictest0-21swapper/722:13:587
764199507254,42cyclictest0-21swapper/722:13:587
76389950782,422cyclictest0-21swapper/423:53:404
76419950435,466cyclictest0-21swapper/721:45:347
76369950234,295cyclictest121rcu_preempt21:35:042
76359950234,168cyclictest25-21ksoftirqd/100:20:161
76419950176,422cyclictest0-21swapper/723:53:157
764099500116,299cyclictest22926-21proc_pri00:20:336
76379950034,211cyclictest0-21swapper/323:44:413
76379950034,211cyclictest0-21swapper/323:44:413
76419949976,211cyclictest0-21swapper/723:35:177
76409949977,338cyclictest0-21swapper/623:35:086
763499499157,211cyclictest0-21swapper/022:46:520
76369949875,84cyclictest0-21swapper/221:45:322
76349949876,211cyclictest9-21ksoftirqd/023:05:160
763499496451,42cyclictest111rcuc/022:15:190
763499496451,42cyclictest111rcuc/022:15:190
763499495322,84cyclictest0-21swapper/021:45:080
76389949473,337cyclictest15359-21sed22:40:224
76359949367,211cyclictest25-21ksoftirqd/122:20:351
763699492108,381cyclictest0-21swapper/222:40:092
76349949165,210cyclictest9-21ksoftirqd/021:34:050
76399949062,380cyclictest0-21swapper/523:05:295
76399949026,294cyclictest0-21swapper/500:20:105
763799490149,126cyclictest7632-21cyclictest22:12:383
763799490149,126cyclictest7632-21cyclictest22:12:383
763799490149,126cyclictest7632-21cyclictest22:12:383
76379948962,211cyclictest41-21ksoftirqd/322:48:443
76379948962,211cyclictest0-21swapper/322:50:073
76359948924,84cyclictest0-21swapper/123:35:081
764199488105,380cyclictest0-21swapper/722:40:097
76389948818,253cyclictest0-21swapper/422:50:034
76379948861,211cyclictest0-21swapper/322:15:313
76379948861,211cyclictest0-21swapper/322:15:313
763799488145,211cyclictest0-21swapper/323:45:183
763699488444,41cyclictest311rcuc/200:20:492
76359948819,297cyclictest0-21swapper/100:10:541
76359948718,254cyclictest0-21swapper/121:35:031
764099486105,127cyclictest0-21swapper/622:05:136
76369948621,41cyclictest0-21swapper/223:35:252
76399948520,253cyclictest0-21swapper/523:53:075
763899485103,380cyclictest0-21swapper/423:44:014
763899485103,380cyclictest0-21swapper/423:44:014
76389948320,85cyclictest0-21swapper/423:05:334
764199482102,213cyclictest22367-21chrt00:37:087
763799479226,42cyclictest7371-1kworker/u17:221:31:263
76389947810,465cyclictest0-21swapper/422:05:134
763999476432,42cyclictest551rcuc/523:44:555
763999476432,42cyclictest551rcuc/523:44:555
76409947448,423cyclictest12959-21ssh22:20:396
76389946847,84cyclictest7632-21cyclictest00:00:254
7635994660,83cyclictest0-21swapper/123:15:091
763999465123,126cyclictest57-21ksoftirqd/522:12:455
763999465123,126cyclictest57-21ksoftirqd/522:12:455
763999465123,126cyclictest57-21ksoftirqd/522:12:455
763799465124,338cyclictest0-21swapper/300:20:303
76389946483,85cyclictest4008-21taskset23:15:104
763599464124,338cyclictest858-21/usr/sbin/munin22:15:201
763599464124,338cyclictest858-21/usr/sbin/munin22:15:201
763799463170,204cyclictest121rcu_preempt22:40:163
764099462163,126cyclictest121rcu_preempt21:35:136
76389946281,84cyclictest64450irq/122-eth000:20:134
763799462246,84cyclictest0-21swapper/323:35:263
763699462123,85cyclictest0-21swapper/200:30:192
763999461206,84cyclictest18267-1kworker/u17:021:33:475
76399946033,296cyclictest0-21swapper/523:33:585
76419945937,256cyclictest0-21swapper/723:55:097
76389945933,338cyclictest9227-21ssh23:35:274
763899458117,338cyclictest0-21swapper/419:15:154
76359945730,296cyclictest0-21swapper/123:05:351
76409945631,423cyclictest0-21swapper/600:40:136
76399945672,253cyclictest0-21swapper/523:55:125
76349945630,253cyclictest0-21swapper/023:45:160
76419945572,380cyclictest0-21swapper/722:47:527
763499455200,126cyclictest25430-21grep00:40:140
76389945473,84cyclictest0-21swapper/422:20:194
763899454113,338cyclictest0-21swapper/423:55:124
76419945326,211cyclictest31182-21chrt21:35:167
76379945368,211cyclictest0-21swapper/322:55:423
764199452111,338cyclictest0-21swapper/722:20:457
763899452111,338cyclictest0-21swapper/421:35:154
763499451111,126cyclictest7705-21ssh23:35:050
76369944821,211cyclictest0-21swapper/222:20:172
763699447109,336cyclictest0-21swapper/223:05:332
763999446105,126cyclictest0-21swapper/500:00:255
76359944624,256cyclictest0-21swapper/123:45:291
76379944422,2cyclictest0-21swapper/323:05:053
76379944361,211cyclictest264-21jbd2/md0-823:15:143
76359944321,296cyclictest0-21swapper/122:40:171
764199442100,126cyclictest1553-21runrttasks23:15:077
763599442101,84cyclictest25-21ksoftirqd/123:20:191
76349943816,2cyclictest0-21swapper/023:55:090
76359943756,219cyclictest121rcu_preempt00:40:141
76399943513,177cyclictest0-21swapper/523:45:275
7640994326,169cyclictest0-21swapper/600:00:216
76379943287,126cyclictest0-21swapper/322:20:183
763799429214,127cyclictest0-21swapper/322:05:113
7640994271,42cyclictest0-21swapper/619:15:056
76419942383,338cyclictest0-21swapper/723:05:117
76419942383,338cyclictest0-21swapper/700:15:047
76419942382,338cyclictest0-21swapper/722:50:077
76409942383,338cyclictest0-21swapper/622:50:036
76379942382,211cyclictest41-21ksoftirqd/300:00:203
76379942382,211cyclictest0-21swapper/321:35:073
763499423128,8cyclictest121rcu_preempt21:35:050
76399942283,84cyclictest0-21swapper/522:20:205
76349942282,338cyclictest0-21swapper/023:15:130
76419942180,338cyclictest1553-21runrttasks00:00:217
76359942080,338cyclictest0-21swapper/122:50:081
76399941979,210cyclictest15463-21diskmemload23:35:265
76399941939,42cyclictest0-21swapper/523:15:135
76389941879,84cyclictest0-21swapper/400:15:064
76389941779,84cyclictest64450irq/122-eth000:40:134
76379941372,338cyclictest0-21swapper/300:30:193
76369941373,338cyclictest18995-21unixbench-2d00:00:242
76399941071,84cyclictest0-21swapper/519:15:155
76399940870,126cyclictest0-21swapper/522:50:035
76399940869,84cyclictest0-21swapper/522:05:085
763799408109,126cyclictest6594-21apt-get23:55:133
76369940665,339cyclictest0-21swapper/200:15:072
76399940263,126cyclictest10203-21apt-get00:15:045
76379940221,126cyclictest0-21swapper/300:15:043
76389940121,253cyclictest0-21swapper/400:30:194
76419939818,41cyclictest0-21swapper/719:15:077
76399939860,84cyclictest0-21swapper/500:40:135
763999393307,84cyclictest551rcuc/500:25:305
763999381122,126cyclictest57-21ksoftirqd/521:35:055
763699379124,253cyclictest12703-21irqstats19:15:152
763499378122,126cyclictest9-21ksoftirqd/000:25:300
76399937738,84cyclictest0-21swapper/522:40:135
763799376208,7cyclictest0-21swapper/319:15:153
76419937534,338cyclictest0-21swapper/700:25:307
76409937232,338cyclictest0-21swapper/600:25:306
76349937235,46cyclictest0-21swapper/019:15:150
76359936928,338cyclictest0-21swapper/119:15:071
76389936728,84cyclictest64450irq/122-eth000:25:314
76369936622,170cyclictest788-21ssh00:25:302
76419936528,253cyclictest0-21swapper/723:20:187
76369936525,338cyclictest0-21swapper/200:40:132
76359936422,339cyclictest0-21swapper/100:25:301
76369936323,212cyclictest15865-21sh23:20:192
76419936220,338cyclictest0-21swapper/700:40:137
76359936220,300cyclictest0-21swapper/100:15:041
76359936022,254cyclictest0-21swapper/100:30:181
76379935819,84cyclictest0-21swapper/300:25:303
76419935310,340cyclictest0-21swapper/700:30:197
76409935012,253cyclictest0-21swapper/623:20:186
7634993412,84cyclictest9911-21ntp_states00:30:190
763899339123,126cyclictest49-21ksoftirqd/423:20:194
76349933940,169cyclictest0-21swapper/000:15:070
76349933536,211cyclictest0-21swapper/023:20:190
76379932732,2cyclictest0-21swapper/300:40:133
76399932569,253cyclictest0-21swapper/500:30:195
76409931420,210cyclictest0-21swapper/600:30:186
76379928835,3cyclictest0-21swapper/323:20:183
76399925950,128cyclictest7632-21cyclictest23:20:195
717929262,27sleep60-21swapper/619:09:376
708028454,27sleep20-21swapper/219:08:132
712628252,27sleep10-21swapper/119:08:521
703228253,26sleep00-21swapper/019:07:300
691328252,27sleep50-21swapper/519:05:515
56992820,2sleep115707-21kworker/1:121:56:191
698728151,27sleep30-21swapper/319:06:533
717127949,27sleep70-21swapper/719:09:307
706327848,27sleep40-21swapper/419:07:594
74602760,0sleep00-21swapper/020:55:120
7641992910,19cyclictest0-21swapper/719:12:397
7635992810,18cyclictest0-21swapper/119:12:331
7640992710,17cyclictest0-21swapper/619:12:316
7638992727,0cyclictest0-21swapper/419:12:324
7636992710,17cyclictest0-21swapper/219:12:562
7634992710,0cyclictest0-21swapper/019:12:320
7639992626,0cyclictest0-21swapper/519:12:305
7637992525,0cyclictest0-21swapper/319:12:343
316942250,0sleep40-21swapper/421:17:194
7639992316,2cyclictest57-21ksoftirqd/521:20:005
7639992211,7cyclictest57-21ksoftirqd/521:50:195
7639992211,6cyclictest57-21ksoftirqd/500:05:065
763999219,8cyclictest57-21ksoftirqd/523:25:125
7639992111,6cyclictest57-21ksoftirqd/519:20:125
7639992111,5cyclictest57-21ksoftirqd/521:56:305
7639992111,2cyclictest57-21ksoftirqd/520:55:005
7639992110,7cyclictest57-21ksoftirqd/522:25:205
7639992110,6cyclictest57-21ksoftirqd/521:26:085
7639992110,6cyclictest57-21ksoftirqd/519:25:175
279282210,0sleep10-21swapper/122:28:561
764199200,1cyclictest0-21swapper/722:25:167
7640992018,1cyclictest0-21swapper/622:25:156
763999209,6cyclictest57-21ksoftirqd/520:17:145
7639992018,1cyclictest561ktimersoftd/522:30:085
7639992018,1cyclictest561ktimersoftd/521:11:165
7639992017,2cyclictest561ktimersoftd/520:40:125
7639992012,5cyclictest57-21ksoftirqd/519:45:165
7639992011,6cyclictest57-21ksoftirqd/520:35:175
7639992010,8cyclictest57-21ksoftirqd/520:20:225
7639992010,7cyclictest57-21ksoftirqd/520:55:125
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional