You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-22 - 12:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Wed Apr 22, 2026 00:50:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
243279956256,124cyclictest121rcu_preempt23:16:146
243239954432,347cyclictest0-21swapper/223:45:362
243239954332,508cyclictest0-21swapper/223:30:562
243239954232,507cyclictest0-21swapper/200:23:292
243279951694,85cyclictest0-21swapper/623:45:466
2432699486145,338cyclictest24319-21cyclictest23:02:595
243219948618,84cyclictest0-21swapper/023:30:560
2432599485187,295cyclictest64450irq/122-eth023:15:534
243239948116,344cyclictest0-21swapper/219:39:112
2432199475303,42cyclictest0-21swapper/023:45:310
243249946341,2cyclictest0-21swapper/323:16:153
243219945937,256cyclictest0-21swapper/019:39:130
243239945834,387cyclictest0-21swapper/221:49:222
243219945836,256cyclictest0-21swapper/023:16:140
243229945330,256cyclictest0-21swapper/119:39:131
2432899444106,171cyclictest0-21swapper/723:46:217
243269944321,84cyclictest0-21swapper/521:28:215
243239944321,84cyclictest121rcu_preempt23:03:002
2432799439101,83cyclictest0-21swapper/623:03:056
2432299436140,84cyclictest14712-1kworker/u17:123:45:231
243229943513,256cyclictest0-21swapper/123:16:151
2432899433178,253cyclictest15850irq/123-ahci[0019:39:127
243259943395,84cyclictest1551-21snmpd19:39:124
243229942786,338cyclictest0-21swapper/123:30:501
24323994261,127cyclictest0-21swapper/223:16:152
243269942542,168cyclictest0-21swapper/523:45:345
243289942182,84cyclictest0-21swapper/723:30:577
243239942078,339cyclictest0-21swapper/200:01:192
243259941880,84cyclictest0-21swapper/423:30:534
243249941674,211cyclictest22701-21ssh00:23:203
2432499416201,84cyclictest121rcu_preempt23:46:213
243289941576,84cyclictest0-21swapper/723:16:167
243289941576,84cyclictest0-21swapper/723:03:017
243259941575,84cyclictest0-21swapper/423:45:244
243239941531,211cyclictest5453-21ssh21:28:212
243259940971,84cyclictest0-21swapper/421:49:004
2432499409199,83cyclictest0-21swapper/321:09:103
2432499409199,83cyclictest0-21swapper/321:09:093
243269940525,3cyclictest0-21swapper/519:39:125
243279940465,84cyclictest0-21swapper/623:30:566
243249940264,171cyclictest0-21swapper/319:39:133
243259940020,211cyclictest0-21swapper/423:02:594
243239940019,165cyclictest0-21swapper/221:08:572
243239940019,165cyclictest0-21swapper/221:08:572
243289939758,84cyclictest0-21swapper/721:49:017
243279939657,84cyclictest0-21swapper/621:49:016
243269939616,3cyclictest0-21swapper/523:16:165
243269939152,84cyclictest0-21swapper/523:30:505
243269939111,42cyclictest0-21swapper/500:23:105
243219938847,338cyclictest0-21swapper/021:49:150
24321993851,84cyclictest0-21swapper/023:02:590
243249938345,84cyclictest121rcu_preempt23:02:593
243249937538,2cyclictest0-21swapper/323:30:563
243229937534,208cyclictest25-21ksoftirqd/123:03:011
243219937437,253cyclictest0-21swapper/000:23:110
243229937234,84cyclictest0-21swapper/100:01:201
243229937134,253cyclictest22716-21ssh00:23:211
243229936827,338cyclictest0-21swapper/121:08:571
243229936827,338cyclictest0-21swapper/121:08:571
243279936672,210cyclictest0-21swapper/619:39:396
243229936625,168cyclictest0-21swapper/100:35:211
243259936164,84cyclictest0-21swapper/400:35:204
243289936064,211cyclictest0-21swapper/700:01:187
2432599356101,126cyclictest0-21swapper/400:23:294
243229935353,211cyclictest0-21swapper/121:28:281
2432599351141,129cyclictest704-21nscd21:28:254
24323993509,337cyclictest0-21swapper/200:35:212
243289934616,247cyclictest0-21swapper/720:29:147
24324993468,217cyclictest41-21ksoftirqd/321:28:263
24321993456,84cyclictest0-21swapper/021:08:570
24321993456,84cyclictest0-21swapper/021:08:570
24327993403,171cyclictest0-21swapper/621:28:276
2432899335167,87cyclictest721ktimersoftd/721:28:277
243219933538,171cyclictest0-21swapper/000:01:180
2432599334248,84cyclictest481ktimersoftd/400:01:194
243229933481,172cyclictest23519-21kworker/u16:221:49:161
243279933170,84cyclictest22718-21ssh00:23:226
243249933034,85cyclictest0-21swapper/321:49:233
2432499329119,86cyclictest401ktimersoftd/300:35:213
243279932872,126cyclictest0-21swapper/600:01:196
243239932731,211cyclictest0-21swapper/220:29:132
243219932770,126cyclictest0-21swapper/021:28:260
243289932630,43cyclictest0-21swapper/700:23:117
243219932126,84cyclictest0-21swapper/000:35:200
243249931814,172cyclictest0-21swapper/300:01:203
2432699310185,41cyclictest0-21swapper/521:09:105
2432699310185,41cyclictest0-21swapper/521:09:105
243269930914,85cyclictest0-21swapper/500:01:205
24326993012,169cyclictest0-21swapper/521:49:155
24322993001,125cyclictest0-21swapper/120:29:141
2432599292210,2cyclictest471rcuc/420:29:154
243279928730,85cyclictest0-21swapper/621:09:106
243279928730,85cyclictest0-21swapper/621:09:106
243269928533,2cyclictest0-21swapper/500:35:215
243259928369,43cyclictest0-21swapper/421:08:564
243259928369,43cyclictest0-21swapper/421:08:554
243289928229,44cyclictest0-21swapper/721:08:557
243289928229,44cyclictest0-21swapper/721:08:557
2432699282156,43cyclictest561ktimersoftd/520:29:155
243219927721,84cyclictest0-21swapper/020:29:140
243249927362,85cyclictest0-21swapper/320:29:143
243279927016,169cyclictest0-21swapper/620:29:136
243279923870,86cyclictest0-21swapper/600:35:216
243289923121,91cyclictest0-21swapper/700:35:217
23392210272,27sleep10-21swapper/119:05:481
2340828858,26sleep60-21swapper/619:06:026
2352828554,28sleep20-21swapper/219:07:432
2338028553,28sleep70-21swapper/719:05:387
2350628448,32sleep00-21swapper/019:07:250
2342628255,24sleep40-21swapper/419:06:164
2347928150,27sleep30-21swapper/319:07:033
2340728049,27sleep50-21swapper/519:06:015
6862740,0sleep40-21swapper/421:42:074
6862740,0sleep40-21swapper/421:42:064
190212710,0sleep30-21swapper/321:35:173
70962700,0sleep20-21swapper/223:24:002
65762700,0sleep50-21swapper/523:06:065
38642700,0sleep20-21swapper/220:20:162
125692700,0sleep10-21swapper/121:32:021
298622530,1sleep50-21swapper/522:29:525
182102490,2sleep449-21ksoftirqd/421:50:324
24323994622,20cyclictest20312-21cat00:05:152
24323994618,24cyclictest3378-21sh21:10:562
24323994617,25cyclictest27574-21sed00:25:302
2432799450,30cyclictest0-21swapper/600:05:236
24323994525,17cyclictest18513-21ssh21:50:492
2432799445,25cyclictest9588-21perl20:30:166
2432799443,28cyclictest9522-21sed21:30:206
24323994416,25cyclictest30392-21cut22:30:002
24323994317,23cyclictest24237-21expr23:50:132
24323994317,22cyclictest8038-21grep21:30:112
2432599422,10cyclictest0-21swapper/422:27:334
2432899416,28cyclictest73-21ksoftirqd/720:05:007
2432799412,24cyclictest1096-21sed21:10:176
24323994120,18cyclictest11655-21ssh00:16:432
2432899397,21cyclictest73-21ksoftirqd/700:25:017
2432599399,19cyclictest24658-21grep23:50:154
2432599397,21cyclictest49-21ksoftirqd/421:10:414
2432599395,26cyclictest49-21ksoftirqd/421:30:094
2432399398,27cyclictest4918-21missed_timers23:05:232
24323993919,16cyclictest9787-21if_err_eth020:30:182
2432199392,33cyclictest26951-21ssh22:27:140
24328993810,20cyclictest0-21swapper/723:50:057
24327993812,15cyclictest0-21swapper/623:05:516
2432699382,32cyclictest11346-21ssh00:16:245
24325993811,17cyclictest49-21ksoftirqd/400:25:344
2432199381,18cyclictest25510-21latency_hist20:05:000
2432899378,21cyclictest73-21ksoftirqd/721:30:157
2432899377,20cyclictest73-21ksoftirqd/720:05:157
24327993730,4cyclictest0-21swapper/622:26:596
2432799371,6cyclictest15587-21ssh21:50:126
2432399377,25cyclictest12816-21cat19:40:282
2432299371,32cyclictest0-21swapper/121:50:061
2432899369,8cyclictest1405-21grep23:20:277
2432899369,16cyclictest73-21ksoftirqd/720:30:317
2432899368,20cyclictest0-21swapper/721:10:177
2432899367,19cyclictest73-21ksoftirqd/700:05:257
2432899366,20cyclictest73-21ksoftirqd/700:18:417
2432899366,19cyclictest73-21ksoftirqd/722:29:167
2432599369,17cyclictest49-21ksoftirqd/400:05:194
2432599366,20cyclictest49-21ksoftirqd/423:05:434
2432499362,4cyclictest0-21swapper/322:27:133
2432299362,4cyclictest0-21swapper/100:16:101
2432299362,30cyclictest0-21swapper/119:40:111
2432299361,31cyclictest26951-21ssh22:27:141
2432899358,16cyclictest73-21ksoftirqd/723:05:437
2432899355,20cyclictest73-21ksoftirqd/722:30:207
2432599358,16cyclictest0-21swapper/400:18:454
24324993531,2cyclictest0-21swapper/300:16:313
24323993516,16cyclictest32165-21cat22:30:212
2432199352,3cyclictest15442-21ssh21:50:030
2432199351,31cyclictest0-21swapper/000:16:070
2432899347,18cyclictest73-21ksoftirqd/721:50:317
2432899341,26cyclictest73-21ksoftirqd/719:40:537
2432799341,30cyclictest0-21swapper/620:04:076
2432799341,29cyclictest0-21swapper/600:16:236
2432699342,4cyclictest10939-21/usr/sbin/munin19:40:125
2432499342,29cyclictest0-21swapper/319:40:063
2432399341,30cyclictest0-21swapper/220:03:412
24323993410,23cyclictest0-21swapper/221:25:012
24321993425,5cyclictest10818-21ssh21:30:460
2432199341,31cyclictest0-21swapper/019:40:060
24327993329,2cyclictest0-21swapper/619:40:016
24326993329,2cyclictest0-21swapper/520:03:475
2432599337,15cyclictest0-21swapper/420:05:254
2432599331,4cyclictest25401-21runrttasks20:03:454
2432499331,3cyclictest0-21swapper/320:03:483
24323993310,22cyclictest0-21swapper/221:57:592
24323993310,22cyclictest0-21swapper/220:19:052
24323993310,22cyclictest0-21swapper/220:19:052
24323993310,22cyclictest0-21swapper/219:14:482
24323993310,22cyclictest0-21swapper/219:14:482
24322993324,6cyclictest1061-21grep23:20:231
24327993225,4cyclictest316762sleep622:30:166
24327993225,4cyclictest23881-21cat23:50:126
2432599321,29cyclictest0-21swapper/419:40:044
2432499328,15cyclictest121rcu_preempt23:05:183
24324993225,4cyclictest24919-21head23:50:173
24323993226,5cyclictest0-21swapper/221:37:252
24323993225,6cyclictest0-21swapper/222:38:422
24323993222,5cyclictest27199-21cat20:05:222
24323993210,21cyclictest0-21swapper/221:18:342
24323993210,21cyclictest0-21swapper/220:38:282
2432299321,28cyclictest0-21swapper/120:03:551
24327993124,4cyclictest25349-21tr00:25:136
24326993124,4cyclictest30517-21ssh22:30:075
24326993124,4cyclictest28103-21ssh00:25:535
24326993123,5cyclictest32593-21ssh21:10:055
24326993123,5cyclictest31753-21sh23:20:095
2432599315,15cyclictest49-21ksoftirqd/422:30:244
24325993124,4cyclictest1107-21ssh23:20:234
24323993110,20cyclictest0-21swapper/219:29:232
24321993123,5cyclictest26117-21cut20:05:150
24321993123,5cyclictest24806-21tune2fs23:50:150
24321993122,5cyclictest5848-21grep23:05:320
24321993121,6cyclictest32222-21cat23:20:130
24321993121,6cyclictest20144-21grep00:05:130
24326993023,4cyclictest16310-21rm21:50:155
24326993022,5cyclictest26212-21ls20:05:155
24326993022,5cyclictest24858-21sh23:50:165
24326993022,5cyclictest20357-21expr00:05:155
24324993022,5cyclictest30876-21cut22:30:133
24322993023,4cyclictest31708-21grep22:30:171
24322993023,4cyclictest20112-21cut00:05:141
24326992922,5cyclictest9470-21cat21:30:215
2432499293,12cyclictest121rcu_preempt21:31:013
24324992923,4cyclictest516-21grep21:10:143
24324992921,5cyclictest25314-21tr00:25:133
24323992927,2cyclictest0-21swapper/220:52:282
24323992927,1cyclictest0-21swapper/222:13:032
24323992925,4cyclictest0-21swapper/219:53:512
24323992925,4cyclictest0-21swapper/219:47:362
24322992922,4cyclictest542-21cut21:10:151
24322992921,5cyclictest3266-21sh23:05:111
24322992921,5cyclictest26682-21sed00:25:201
2432299291,25cyclictest9449-21cut20:30:171
24321992922,4cyclictest32594-21ssh21:10:050
24321992922,4cyclictest25216-21tr00:25:130
24328992810,18cyclictest0-21swapper/722:38:367
24328992810,18cyclictest0-21swapper/722:13:117
24328992810,18cyclictest0-21swapper/722:06:087
24328992810,18cyclictest0-21swapper/721:37:427
24328992810,18cyclictest0-21swapper/720:45:467
24328992810,18cyclictest0-21swapper/720:18:407
24328992810,18cyclictest0-21swapper/720:18:407
24328992810,18cyclictest0-21swapper/719:15:017
24328992810,18cyclictest0-21swapper/719:14:397
24328992810,18cyclictest0-21swapper/719:14:397
24328992810,0cyclictest0-21swapper/721:58:147
24327992828,0cyclictest0-21swapper/621:37:316
24327992820,5cyclictest641ktimersoftd/623:20:126
24327992810,18cyclictest0-21swapper/621:18:356
24326992822,4cyclictest10276-21grep20:30:255
24325992821,4cyclictest10000-21sed20:30:214
24325992810,0cyclictest0-21swapper/419:14:394
24325992810,0cyclictest0-21swapper/419:14:384
24324992828,0cyclictest0-21swapper/319:23:533
24324992822,3cyclictest26293-21taskset20:05:153
24324992822,3cyclictest20396-21expr00:05:153
24324992820,5cyclictest18107-21grep21:50:313
24324992810,18cyclictest0-21swapper/323:00:013
24324992810,18cyclictest0-21swapper/322:38:443
24324992810,18cyclictest0-21swapper/322:13:143
24324992810,18cyclictest0-21swapper/322:06:213
24324992810,18cyclictest0-21swapper/321:57:543
24324992810,18cyclictest0-21swapper/321:18:143
24324992810,18cyclictest0-21swapper/320:45:183
24324992810,18cyclictest0-21swapper/320:11:063
24324992810,0cyclictest0-21swapper/319:18:563
24323992827,1cyclictest0-21swapper/219:23:292
24323992818,6cyclictest121rcu_preempt19:15:002
24323992810,17cyclictest0-21swapper/220:11:112
24323992810,0cyclictest0-21swapper/222:06:302
24322992827,1cyclictest30993-21kworker/u16:122:13:331
24322992822,4cyclictest27909-21date20:05:301
24321992821,4cyclictest30479-21sh22:30:050
24321992821,4cyclictest10258-21proc_pri20:30:250
24321992810,18cyclictest0-21swapper/022:38:350
24328992727,0cyclictest0-21swapper/720:11:087
24328992710,17cyclictest0-21swapper/719:29:197
24328992710,17cyclictest0-21swapper/719:23:507
24328992710,0cyclictest0-21swapper/721:18:067
24328992710,0cyclictest0-21swapper/720:59:207
24328992710,0cyclictest0-21swapper/720:52:257
24328992710,0cyclictest0-21swapper/720:37:567
24328992710,0cyclictest0-21swapper/719:53:487
24328992710,0cyclictest0-21swapper/719:47:187
24327992727,0cyclictest0-21swapper/622:12:536
24327992710,17cyclictest0-21swapper/622:38:356
24327992710,17cyclictest0-21swapper/622:06:156
24327992710,17cyclictest0-21swapper/621:58:286
24327992710,17cyclictest0-21swapper/619:23:316
24327992710,17cyclictest0-21swapper/619:14:416
24327992710,17cyclictest0-21swapper/619:14:416
24327992710,0cyclictest0-21swapper/620:37:556
24327992710,0cyclictest0-21swapper/620:11:056
24327992710,0cyclictest0-21swapper/619:29:296
24326992727,0cyclictest0-21swapper/520:59:175
24326992727,0cyclictest0-21swapper/520:52:285
24326992727,0cyclictest0-21swapper/520:37:595
24326992710,17cyclictest0-21swapper/522:38:345
24326992710,17cyclictest0-21swapper/522:13:175
24326992710,17cyclictest0-21swapper/521:57:575
24326992710,17cyclictest0-21swapper/521:37:365
24326992710,17cyclictest0-21swapper/520:18:335
24326992710,17cyclictest0-21swapper/520:18:335
24326992710,17cyclictest0-21swapper/519:29:315
24326992710,17cyclictest0-21swapper/519:15:045
24326992710,17cyclictest0-21swapper/519:14:515
24326992710,17cyclictest0-21swapper/519:14:505
24326992710,0cyclictest0-21swapper/522:06:345
24326992710,0cyclictest0-21swapper/521:18:185
24326992710,0cyclictest0-21swapper/520:45:445
24326992710,0cyclictest0-21swapper/520:11:075
24326992710,0cyclictest0-21swapper/519:23:535
24325992710,17cyclictest0-21swapper/422:38:364
24325992710,17cyclictest0-21swapper/422:12:594
24325992710,17cyclictest0-21swapper/422:06:094
24325992710,17cyclictest0-21swapper/421:18:244
24325992710,17cyclictest0-21swapper/420:59:154
24325992710,17cyclictest0-21swapper/420:52:254
24325992710,17cyclictest0-21swapper/420:18:344
24325992710,17cyclictest0-21swapper/420:18:344
24325992710,17cyclictest0-21swapper/419:53:554
24325992710,17cyclictest0-21swapper/419:47:204
24325992710,17cyclictest0-21swapper/419:29:204
24325992710,17cyclictest0-21swapper/419:23:474
24325992710,17cyclictest0-21swapper/419:18:574
24325992710,0cyclictest0-21swapper/421:57:454
24325992710,0cyclictest0-21swapper/421:37:284
24325992710,0cyclictest0-21swapper/420:38:044
24324992727,0cyclictest0-21swapper/320:52:323
24324992727,0cyclictest0-21swapper/319:14:383
24324992727,0cyclictest0-21swapper/319:14:373
24324992720,5cyclictest9003-21expr20:30:143
24324992720,4cyclictest32486-21sed23:20:153
24324992710,17cyclictest0-21swapper/320:59:223
24324992710,17cyclictest0-21swapper/320:38:293
24324992710,17cyclictest0-21swapper/320:24:563
24324992710,17cyclictest0-21swapper/320:18:363
24324992710,17cyclictest0-21swapper/320:18:353
24324992710,17cyclictest0-21swapper/319:47:483
24324992710,17cyclictest0-21swapper/319:29:243
24324992710,0cyclictest0-21swapper/319:53:473
24323992710,0cyclictest0-21swapper/220:59:122
24322992727,0cyclictest0-21swapper/120:11:181
24322992710,17cyclictest0-21swapper/122:06:001
24322992710,17cyclictest0-21swapper/120:45:511
24322992710,17cyclictest0-21swapper/119:15:071
24322992710,17cyclictest0-21swapper/119:14:401
24322992710,17cyclictest0-21swapper/119:14:401
24322992710,0cyclictest0-21swapper/122:59:571
24322992710,0cyclictest0-21swapper/122:38:381
24322992710,0cyclictest0-21swapper/121:57:571
24322992710,0cyclictest0-21swapper/121:37:171
24322992710,0cyclictest0-21swapper/121:18:221
24322992710,0cyclictest0-21swapper/120:59:231
24322992710,0cyclictest0-21swapper/120:52:261
24322992710,0cyclictest0-21swapper/120:37:581
24322992710,0cyclictest0-21swapper/120:18:451
24322992710,0cyclictest0-21swapper/120:18:451
24322992710,0cyclictest0-21swapper/119:53:511
24321992727,0cyclictest0-21swapper/022:13:000
24321992727,0cyclictest0-21swapper/020:59:100
24321992727,0cyclictest0-21swapper/020:52:240
24321992727,0cyclictest0-21swapper/020:25:000
24321992727,0cyclictest0-21swapper/019:29:280
24321992727,0cyclictest0-21swapper/019:15:040
24321992727,0cyclictest0-21swapper/019:14:480
24321992727,0cyclictest0-21swapper/019:14:480
24321992710,17cyclictest0-21swapper/022:59:550
24321992710,17cyclictest0-21swapper/022:06:010
24321992710,17cyclictest0-21swapper/021:18:190
24321992710,17cyclictest0-21swapper/020:45:450
24321992710,17cyclictest0-21swapper/020:38:010
24321992710,17cyclictest0-21swapper/020:11:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional